Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation

Purpose

The Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation was designed to assist in the detection of perceptual impairment in adults who have experienced brain damage caused by traumatic brain injury or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. The OSOT assesses perceptual dysfunction in areas related to basic living skills. The measure is also used to determine the degree of impairment, to monitor change, and to measure the effects of treatment and/or spontaneous recovery. The OSOT has been standardized for use with individuals aged 40-69 years.

In-Depth Review

Purpose of the measure

The Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation was designed to assist in the detection of perceptual impairment in adults who have experienced brain damage caused by traumatic brain injury or strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. The OSOT assesses perceptual dysfunction in areas related to basic living skills. The measure is also used to determine the degree of impairment, to monitor change, and to measure the effects of treatment and/or spontaneous recovery. The OSOT has been standardized for use with individuals aged 40-69 years.

Available versions

The OSOT was created in 1972, by a study group formed in Toronto to discuss perceptual dysfunction in adults with brain damage. However, Boys, Fisher, Holzberg, and Reid (1988) were the first to standardize the OSOT.

Features of the measure

Items:

The original OSOT:

This evaluation consists of 28 subtests that evaluate 6 domains:

(Source: Boys et al., 1988)

Functional Area	Test Item	Scoring
Sensation	Stereognosis Eight common objects identified by touch	4 = 8/8 correct 3 = 5-7/8 correct 2 = 2-4/8 correct 1 = 0-1/8 correct
Scanning	Scanning Cancellation task (total possible cancellations = 105)	4 = 0-3 errors 3 = 4-10 errors 2 = 11-25 errors 1 = 26 or more errors
Apraxia	Motor PlanningPlanning ability involves anticipating future events, formulating a goal or endpoint, and devising a sequence of steps or actions that will achieve the goal or endpoint" (Anderson, 2008, p. 17) Manipulate 3 wire and grommet devices	4 = 30 seconds (s) or less to complete 3 tasks 3 = 31-60s 2 = 61-90s 1 = unable to complete in less than 91s
Body awareness	Parts Recognition Identify parts of body	4 = 8/8 correct 3 = 5-7/8 correct 2 = 2-4/8 correct 1 = 0-1/8 correct
Spatial relations	Environmental Copy models of 4 pegboard designs	4 = 4/4 correct 3 = 3/4 correct 2 = 2/4 correct 1 = 0-1/4 correct
Visual agnosia	Shape Recognition Match 9 shapes to form board	4 = 9/9 correct 3 = 6-8/9 correct 2 = 2-5/9 correct 1 = 0-1/9 correct

Note: All test items have been recoded into the 4-point format. Only the test item Tactile suppression, which falls under the Sensation domain, is scored in a bivariate form of either 4 = present or 1 = absent.

The revised OSOT:

This evaluation consists of 18 subtests that evaluate 6 domains:

• Sensation
• Scanning
• Apraxia
• Body awareness
• Spatial relations
• Visual agnosia

In order to give the patient precise instructions, the evaluator must state the instructions verbatim prior to administering the OSOT. These instructions can only be repeated once. The evaluator must record all patient responses before offering help.

Scoring:

The OSOT uses a 5-point Likert scaleLikert scaling is one type of response to items in a questionnaire or tool. For example, Likert scaling would have you rate an item such as “I am satisfied with the care I received” on a scale using a 1-to-5 response scale where:
• 1 = strongly disagree
• 2 = disagree
• 3 = undecided
• 4 = agree
• 5 = strongly agree
You will find various options and scaling methods for the number of response choices (1-to-7, 1-to-9, 0-to-4). Odd-numbered scales usually have a middle value that is labelled Neutral or Undecided. Some tools used forced-choice Likert scaling with an even number of responses and no middle neutral or undecided choice. for each of the subtests, ranging from 0 = an inability to do what is asked of the patient, to 4 = normal performance. The scores obtained for each task are added to establish a total score. In the original OSOT, the maximum score that can be obtained is 112. Below is a breakdown of the scores. Each interval corresponds to a degree of severity of the global perceptual impairment:

(Source: Boys et al., 1988)

Score	Severity of impairment
110-112	Normal performance
101-109	Borderline case, requires additional testing
91-100	Mild impairment
81-90	Moderate impairment
80 or below	Severe impairment

In the revised OSOT, the maximum score that can be obtained is 72. Below is a breakdown of the scores. Each interval corresponds to a degree of severity of the global perceptual impairment:

(Source: Boys et al., 1991)

Score	Severity of impairment
70-72	Normal performance
61-69	Borderline case, requires additional testing
51-60	Mild impairment
41-50	Moderate impairment
40 or below	Severe impairment

Time:

There is no information published regarding the time it takes to complete the original OSOT, but it is anticipated that it would take longer than the revised OSOT which takes approximately 90 minutes to complete (Tremblay, Savard, Casimiro, & Tremblay, 2004).

Subscales:

The OSOT has 6 subscales or ‘domains’: Sensation; Scanning; Apraxia; Body awareness; Spatial relations; Visual agnosia.

Equipment:

• Instruction manual
• Evaluation material
• Pencil
• Chronometer
• 5″x11″ piece of paper
• Subtests and equipment for each subtest (e.g. a clock, 6 circles out of various sized cardboard, etc).

Training:

The OSOT was created to be used by occupational therapists. An adequate understanding of the instructions and procedures is needed before using the OSOT.

Alternative forms of the OSOT

The revised version of the OSOT (Boys, Fisher, & Holzberg, 1991).

In the revised version of the OSOT, 10 subtests have been eliminated from the 28 found in the original version of the measure.

Client suitability

Can be used with:

• Patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

Should not be used with:

• The OSOT has not been examined for use for patients with An alternative test is the Motor-Free Visual Perception Test, which can sometimes be used to examine the presence of visual perception impairments in patients with expressive aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury.
  An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) if they are able to understand simple verbal or non-verbal instructions and the various subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
  requirements.

In what languages is the measure available?

Validated in French (Desrosiers, Mercier, & Rochette, 1999).

Summary

What does the tool measure?	Perceptual impairment in adults (Sensation, Scanning, Apraxia, Body awareness, Spatial relations, Visual agnosia).
What types of clients can the tool be used for?	Patients who have experienced brain damage caused by traumatic brain injury or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	Assessment
Time to administer	The revised OSOT takes approximately 90 minutes to complete. There is no information published regarding the time it takes to complete the original OSOT.
Versions	Original OSOT (consists of 28 subtests); Revised OSOT (consists of 18 subtests).
Other Languages	English and French
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: Only one study has examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original OSOT and found alpha coefficients ranging from poor to excellent. Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: Only one study has examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the revised OSOT and reported excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.. Test-retest: No studies have examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society). of the OSOT. Inter-rater: One study examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. of the original OSOT and reported very high agreement (93.1%) across items for all subjects.
ValidityThe degree to which an assessment measures what it is supposed to measure.	Criterion: Concurrent: One study reported that the revised OSOT correlated adequately with the Physical Self-Maintenance Scale (PSMS), the Instrumental ActivitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function. of Daily Living Scale, and with the Mini-Mental State Examination (MMSE). Construct: Known groups: Both the original OSOT and the revised OSOT have been shown to discriminate between patients with acquired brain injury and healthy controls.
Floor/Ceiling Effects	No studies have examined the floor or ceiling effects of the OT-APST.
Does the tool detect change in patients?	Not Applicable.
Acceptability	A proxy respondent is not appropriate for this performance-based measure. The OSOT has not been examined for use for patients with aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury. An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada). An alternative test is the Motor-Free Visual Perception Test, which can sometimes be used to examine the presence of visual perception impairments in patients with expressive aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury. An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) if they are able to understand simple verbal or non-verbal instructions and the various subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society). requirements.
Feasibility	The OSOT is a lengthy measure to administer and requires an adequate understanding of the test procedures and instructions on the part of the examiner. A number of items are required as equipment for the OSOT, however all items are readily available. The OSOT is simple to score and uses a 5-point Likert scaleLikert scaling is one type of response to items in a questionnaire or tool. For example, Likert scaling would have you rate an item such as “I am satisfied with the care I received” on a scale using a 1-to-5 response scale where: • 1 = strongly disagree • 2 = disagree • 3 = undecided • 4 = agree • 5 = strongly agree You will find various options and scaling methods for the number of response choices (1-to-7, 1-to-9, 0-to-4). Odd-numbered scales usually have a middle value that is labelled Neutral or Undecided. Some tools used forced-choice Likert scaling with an even number of responses and no middle neutral or undecided choice. for each of the subtests. A breakdown of the total score is provided and each interval corresponds to a degree of severity of the global perceptual impairment (e.g. original OSOT: 80 or below represents severe impairment; revised OSOT: 40 or below represents severe impairment).
How to obtain the tool?	The OSOT can be purchased from the publisher: Nelson, A Thomson Company 1120 Birchmount Road Toronto, Ontario M1K 5G4, Canada E-mail: inquire@nelson.com Website: http://www.assess.nelson.com/pdf/assessmentcatalogue/cn-19.pdf

Psychometric Properties

Overview

The OSOT has not been well studied. To our knowledge, the creators of the OSOT have personally gathered the majority of psychometric data that are currently published on the scale. The psychometric properties of the original version of the OSOT were based on a study by Boys et al. (1988) of 80 patients with brain damage (experienced a strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain., tumor, normopressure hydrocephalus, or anoxia), and a group of 70 neurologically intact participants. The psychometric properties of the revised version of the OSOT were based on a study by Boys et al. in 1991.

Further investigation on the reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of "true" variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to "noise" in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
, validityThe degree to which an assessment measures what it is supposed to measure.
, and sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also "Specificity."
of the OSOT is required with larger numbers of subjects. For the purposes of this review, we conducted a literature search to identify all relevant publications on the psychometric properties of the OSOT.

Reliability

Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.:
Original OSOT:
Boys et al. (1988) examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original OSOT and reported alpha coefficients ranging from poor (Body Awareness – Parts recognition, alpha = 0.23) to excellent (Sensory Function – Localization, alpha = 0.95).

Revised OSOT:
Boys et al. (1991) examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the revised OSOT and reported that this version of the OSOT has an excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. (Cronbachs alpha = 0.90).

Inter-rater:
Original OSOT:
Boys et al. (1988) examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept.
of the OSOT by having both the attending therapist and one of the investigators score the performance of 46 patients independently, on separate score sheets. Agreement for this study was very high (93.1%) across items for all subjects.

Validity

Criterion:

Concurrent:
Boyd and Dawson (2000) examined the relationship between perceptual impairment and independence in basic and instrumental activitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of daily living in a sample of older adults living in an institution or in the community. In this study, the OSOT correlated adequately with the Physical Self-Maintenance Scale (PSMS) (Lawton, & Brody, 1969) (r = 0.44), the Instrumental ActivitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of Daily Living Scale (r = 0.44), and with the Mini-Mental State Evaluation (MMSE) (Folstein, Folstein, & McHugh, 1975) (r = 0.43). The results of this study suggest that perceptual impairment is related to activitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of daily living status.

Construct:

Boys et al. (1988) reported that the moderate correlations observed between scores obtained for each of the 6 domains of the original OSOT demonstrates that each domain measures different concepts, which together give a global perceptual deficit score.

Known groups:
Both the original OSOT and the revised OSOT have been shown to discriminate between patients with acquired brain injury and healthy controls (Boys et al., 1991; Boys et al.,1988). When the performance of patients and control participants were compared in the study by Boys et al. (1988), statistically significant differences were observed for all tests except for Ideational Apraxia.

Sensitivity and specificity:

Boys et al. (1988) reported that the total score of the original version of the OSOT can differentiate between individuals with neurological impairment from neurologically normal control participants. At a total score cutoff of 110, the OSOT can differentiate between individuals with functional impairment and a control group without neurological impairment with a sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of 100% and a specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of only 40%. At a cutoff of 100, the sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was 63.7% and the specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was 100%.

Boys et al. (1991) reported that the total score of the revised version of the OSOT can also differentiate between individuals with brain damage from neurologically normal control participants. The sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was 100% with a cutoff score of 70 and over, and of 58% with a cutoff of 60. The specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was 40% with a cutoff score of 70 and under, and 100% with a cutoff of 60.

References

• Boyd, A., Dawson, D. R. (2000). The relationship between perceptual impairment and self-care status in a sample of elderly persons. Physical & Occupational Therapy in Geriatrics, 17(4), 1-16.
• Boys, M., Fisher, P., & Holzberg, C. (1991). The OSOT Perceptual Evaluation Manual: Revised. Scarborough, Ont.: Nelson Canada.
• Boys, M., Fisher, P., Holzberg, C., & Reid, D. (1988). The OSOT Perceptual Evaluation: A research perspective. American Journal of Occupational Therapy. 42, 92-98.
• Desrosiers, J.,Mercier, L.,Rochette, A. (1999).Test-retest and inter-rater reliability of the French version of the Ontario Society of Occupational Therapy (OSOT)Perceptual Evaluation. Can J Occup Therapy, 66(3), 134 -139.
• Folstein, M., Folstein, S., McHugh, P. (1975). Mini-mental State: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12,189-198.
• Lawton, M. P., Brody, E. M. (1969). Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist, 9, 179-186.
• Temblay, L. E., Savard, J., Casimiro, L., Tremblay, M. (2004). Répertoire des Outils d’Évaluation en Français pour la Réadaptation (pp406-408). Ottawa, ON: Regroupement des intervenantes et intervenants francophones en santé et en services sociaux de l’Ontario: Université d’Ottawa: CFORP.

See the measure

The OSOT can be purchased from the publisher:

Publisher address:
Nelson, A Thomson Company
1120 Birchmount Road,
Toronto, Ontario M1K 5G4, Canada
Telephone: 416-752-9448 or 1 800 268-2222
Fax: 416-752-8101 or 1 800 430-4445
E-mail: inquire@nelson.com
Website: http://www.assess.nelson.com/pdf/assessmentcatalogue/cn-19.pdf

Cost: (price is subject to change)
Perceptual evaluation kit (version 2, with English manual): $859
English manual: $75
French manual: $31.80