Loewenstein Occupational Therapy Cognitive Assessment (LOTCA)

Evidence Reviewed as of before: 13-06-2011

Author(s)*: Annabel McDermott, OT

Editor(s): Nicol Korner-Bitensky, PhD OT

Content consistency: Gabriel Plumier

Purpose

The Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery was developed as a measure of basic cognitive skills and visual perception in older adults with neurological impairment. The LOTCA provides an in-depth assessment of basic cognitive abilities and can also be used in treatment planningPlanning ability involves anticipating future events, formulating a goal or endpoint, and devising a sequence of steps or actions that will achieve the goal or endpoint” (Anderson, 2008, p. 17)
and review of progress over time.

In-Depth Review

Purpose of the measure

The Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) is a cognitive battery that measures basic cognitive skills required for everyday function including orientation, visual perceptual and psychomotor abilities, problem-solving"Goal-directed cognitive activity that arises in situations for which no response is immediately apparent or available" (Luria, 1966; as cited in (Rath et al., 2004))
skills and thinking operations. Development of the battery was based on information from clinical experience and neuropsychological and developmental theories. The LOTCA is typically used in the initial phase of patient assessment but can also be used to establish therapeutic goals and to review cognitive status over time (Annes, Katz & Cermak, 1996; Zwecker et al., 2002).

Available versions

The original LOTCA was developed by Itzkovich, Averbuch, Elazar and Katz for use with individuals below the age of 70 years with neurological dysfunction and consisted of a total of 20 items within 4 areas: Orientation (2 items); Perception (6 items); Visuomotor Organization (7 items); and Thinking Operations (5 items).

The LOTCA-II was modified by separating the Perceptual area into three separate areas (Visual Perception, Spatial Perception and Motor Praxis), revising items and including an additional Thinking Operations subtest. The LOTCA-II consists of a total of 26 subtests within 6 areas: Orientation (2 items); Visual Perception (4 items); Spatial Perception (3 items); Motor Praxis (3 items); Visuomotor Organization (7 items); and Thinking Operations (7 items).

The LOTCA-II includes multi-choice questions in the Orientation area to accommodate language difficulties. The manual has been updated to provide more accurate assessment and administration guidelines (Su, Lin, Chen-Sea & Yang, 2007).

Features of the measure

Description of Tasks:

The original LOTCA contains 20 subtests in 4 areas. The LOTCA-II contains 26 subtests in 6 areas:

Orientation (2 subtests): Assesses the individual’s orientation to place and time.
Visual Perception (4 subtests): Assesses the individual’s ability to identify pictures of everyday objects, objects photographed from unusual angles, distinguish between overlapping figures, and recognize spatial relations between objects.
Spatial Perception (3 subtests): Assesses the individual’s ability to differentiate between right and left to determine spatial relationships between objects and self.
Motor Praxis (3 subtests): Asesses the individual’s ability to imitate motor actions, use objects and perform symbolic actions.
Visuomotor Organization (7 subtests): Assesses the individual’s ability to copy geometric figures, reproduce a 2D model, copy a coloured block design and a plain block design, reproduce a puzzle and complete a pegboard task, and draw a clock.
Thinking Operations (7 subtests): Assesses the individual’s ability to complete tasks including sorting, categorization, and picture and geometric sequences (Annes et al., 1996).

Scoring and Score Interpretation:

Most subtests of the LOTCA are scored from 1 to 4, where:

1 = Patient fails to perform the task
2 = Patient is able to perform part of the task
3 = Patient is able to perform most of the task
4 = Patient demonstrates good performance of the task

However, three Thinking Operations subtests (Categorisation, Risk Object Classification – ROC – Unstructured, ROC Structured subtests) are scored on a scale from 1 to 5 (Josman, Abdallah & Engel-Yegar, 2010; Zwecker et al., 2002).

Most subtests of the LOTCA-II are also scored from 1 to 4 using the scale above. However, Orientation subtests are scored on a scale from 1 to 8 and three Thinking Operations subtests (Categorization, ROC Unstructured, ROC Structured) are scored on a scale from 1 to 5. Accordingly, the overall LOTCA-II score ranges from 26 to 115 points. Task completion through trial-and-error is penalized in three subtests. Performance elements such as the number of prompts provided to the individual to assist him/her in completing the task, the individual’s attention/concentration and length of time taken to complete the assessment are also recorded.

Results are provided as a profile for each subtest, where higher scores indicate less cognitive impairment (Su et al., 2000). While summation of subtest scores is accepted, the authors warn that analysis of a total score impacts on the ability to identify the individual’s aptitude for each cognitive area (Katz, Itzkovich & Averbuch, 2002).

LOTCA scores were normed on an Israeli population of adults aged 20 – 70 years (Annes et al., 1996), and have since been deemed suitable for use with the US population (Annes et al., 1996; Cermak, Katz, McGuire, Greenbaum, Peralta & Maser-Flanagan, 1995; Katz et al., 1997). Age-related change in scores is not accounted for, as score norms are provided for one group of all individuals aged 20 – 70 years.

Time:

The LOTCA and LOTCA-II take approximately 45 minutes to administer, with a reported range from 30 to 90 minutes (Annes et al., 1996; Zwecker et al., 2002).

Equipment:

The LOTCA kit contains testing materials (card decks, coloured blocks, pegboard set and other materials) and a manual that includes definitions of the cognitive domains assessed, instructions for administration and specific scoring guidelines.

Alternative forms of the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA)

The LOTCA Geriatric version (LOTCA-G) is a modified version of the original LOTCA that was designed for use with individuals aged 70 – 91 years. The LOTCA-G was developed in response to difficulties that elderly patients experienced using the LOTCA (e.g. difficulty seeing and using small materials, and duration of assessment). Accordingly, it contains modifications to allow for age-related cognitive decline and sensorimotor difficulties including larger materials to compensate for visual and motor deficits, less visual detail, shorter subtests, multiple-choice questions, and additional memory tests not included in the original LOTCA (Cooke et al., 2006a, b; Bar-Haim Erez & Katz, 2003).

The LOTCA-G includes 23 subtests in 7 cognitive areas:

Orientation (2 subtests): Orientation to place; and orientation to time
Visual Perception (4 subtests): Object identification; shape identification; overlapping figures; and object consistency
Spatial Perception (3 subtests): On self; on examiner; and self and surroundings
Praxis (3 subtests): Motor imitation; utilization of objects; and symbolic actions
Visuomotor Organization (6 subtests): Copy geometric forms; two-dimension model pegboard construction; block design (colour); reproducation of a puzzle; and drawing a clock
Thinking Operations (2 subtests): Categorization; and pictorial sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
Memory (3 subtests): Famous personality; personal possession; and everyday objects

The Orientation subtests are scored from 1 – 8 while all other subtests of the LOTCA-G are scored on an ordinal scale from 1 to 4, where 1 indicates severe deficit and 4 indicates average performance. The LOTCA-G takes approximately 30-45 minutes to administer (Bar-Haim Erez & Katz, 2003, Cermak, et al., 1995). It is available in English and Hebrew.

Client suitability

Can be used with:

Patients following strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (Bar-Haim Erez & Katz, 2003)
Older individuals with dementia (Bar-Haim Erez & Katz, 2003)
Individuals with traumatic brain injury (Annes et al., 1996)
Individuals with CNS dysfunction (Annes et al., 1996)
Individuals with intellectual disabilities (Jang, Chern & Lin, 2009)
Individuals with mental illness (Jang et al., 2009; Josman & Katz, 2006)
An adapted version has also been developed for use with children with learning difficulties (Josman et al., 2010)
Patients with aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury.
An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) – procedures for assessing the patient with aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury.
An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) are included (Jang et al., 2009).

Should not be used in:

An individual’s culture can affect the construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed.
of the LOTCA when used with a pediatric population (Josman et al., 2010)

In what languages is the measure available?

English
Spanish
Hebrew
Chinese (Mandarin)
Taiwanese
Iranian

Summary

What does the tool measure?	Basic cognitive skills required for everyday function including orientation, visual perceptual and psychomotor abilities, problem-solving“Goal-directed cognitive activity that arises in situations for which no response is immediately apparent or available” (Luria, 1966; as cited in (Rath et al., 2004)) skills and thinking operations.
What types of clients can the tool be used for?	The LOTCA was designed for individuals aged up to 70 years with neurological deficit.
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	Assessment tool
Time to administer	The LOTCA and LOTCA-II take approximately 45 minutes to administer. The LOTCA-G takes approximately 30-45 minutes to administer.
Versions	LOTCA-II LOTCA – G (for individuals aged 70-91)
Other Languages	Spanish, Hebrew, Chinese (Mandarin), Taiwanese and Iranian.
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: One study reported excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original LOTCA. Test-retest: No studies have reported on test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society). of the LOTCA with patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. Intra-rater: No studies have reported on the intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings. of the LOTCA. Inter-rater: – One study reported excellent inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. of the LOTCA. – One study reported excellent inter-rater agreement for the LOTCA-G.
ValidityThe degree to which an assessment measures what it is supposed to measure.	Content: One study has reported on the content validityRefers to the extent to which a measure represents all aspects of a given social concept. Example: A depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension. of the original LOTCA using factor analysis. Criterion: Concurrent: – One study reported adequate correlations between areas of the LOTCA-G and the MMSE, using Spearman correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation. analysis. – One study reported poor to excellent correlations between the LOTCA-II and the OT-APST in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., using Somers’s d concordance correlations. – One study reported poor to adequate correlations between the LOTCA and functional motor outcomes (as measured by FIM motor and MRFS scores) in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., using Pearson correlations. Predictive: No studies have reported on the predictive validityA form of criterion validity that examines a measure’s ability to predict some subsequent event. Example: can the Berg Balance Scale predict falls over the following 6 weeks? The criterion standard in this example would be whether the patient fell over the next 6 weeks. of the LOTCA. Construct: Two studies report suitable construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed. of LOTCA subtests. Convergent/Discriminant: – One study reported a statistically significant difference in the time taken to complete the LOTCA and the LOTCA-G in a group of healthy adults. – Four studies have reported poor to excellent correlations between the original LOTCA and MMSE, FIM, Rivermead Perceptual Assessment Battery (RPAB), Motor-Free Visual Perception Test (MVPT), Rabideau Kitchen Evaluation – Revised or Phone Use tests. – One study reported adequate to excellent convergent validityA type of validity that is determined by hypothesizing and examining the overlap between two or more tests that presumably measure the same construct. In other words, convergent validity is used to evaluate the degree to which two or more measures that theoretically should be related to each other are, in fact, observed to be related to each other. of the LOTCA-II and the OT-APST, using Spearman’s rho correlations. Known Groups: – Two studies have reported that the LOTCA-G is able to differentiate healthy adults, adults with dementia and adults with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. – Six studies have reported that LOTCA is able to differentiate healthy young adults, healthy elderly adults, individuals with schizophrenia, patients with dementia, patients with intracerebral hemorrhage, ischemia or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., patients with unilateral spatial neglect, and patients with traumatic brain injury. – One study reported that different versions of the puzzle reproduction task are able to differentiate younger and older adults. – While one study reported that the LOTCA could differentiate between left vs. right strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., another study found no differences with regard to side of lesion. – One study reported differences between American and Israeli patients with CVA on several subtests of the LOTCA.
Floor/Ceiling Effects	One study reported ceiling effects from 23% to 96.9%, and negligible floor effects for most items of the LOTCA One study examined the floor/ceiling effects of the LOTCA-II in a group of individuals with schizophrenia and reported no ceiling effects.
SensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.” / SpecificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).	No studies have examined the sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.” or the specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative). of the LOTCA.
Does the tool detect change in patients?	No studies have reported on the responsivenessThe ability of an instrument to detect clinically important change over time. of the LOTCA.
Acceptability	The LOTCA has been described as time-consuming, although it can be completed over several sessions. The LOTCA has adaptations for use with individuals with communication difficulties.
Feasibility	The LOTCA can be time-consuming to administer. The LOTCA permits use of domain-specific scores rather than just a global score, allowing for assessment of many aspects of the client’s cognitive and perceptual abilities.
How to obtain the tool?	The LOTCA can be purchased from http://store.grovergear.com/, http://www.ot-innovations.com, or http://therapro.com.

Psychometric Properties

Overview

A literature search was conducted to identify all relevant publications on the psychometric properties of the LOTCA. While this assessment can be used with various populations, this module addresses the psychometric properties of the measure specifically when used with patients with stroke. Eleven studies were identified.

Reliability

Internal ConsistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.:
Katz, Itzkovich and Averbuch (1989) examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original LOTCA with patients with traumatic head injury (n=20), patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=28) and health adults (n=55), and reported excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. with alpha coefficients of 0.85, 0.87 and 0.95 for in the areas of Thinking Operations, Perception and Visuomotor Organization (respectively).

Test-retest:
No studies have reported on the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
of the LOTCA when used with patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

Intra-rater:
No studies have examined the intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings.
of the LOTCA.

Inter-rater:
Katz et al (1989) reported excellent inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept.
for subtests of the LOTCA, with Spearman’s rank correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases - for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases - for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients ranging from 0.82 to 0.97.

Katz, Elazer and Itzkovich (1995) reported an agreement rate of 90% between 2 raters who used the LOTCA-G to assess 5 healthy subjects and 5 patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain..

Validity

Katz et al. (1995) reported that the LOTCA-G was tested on patients and healthy volunteers during its development.

Katz et al. (1995) reported that the LOTCA-G takes only 30-45 minutes (as compared to 30-90 minutes for the LOTCA), which verifies the LOTCA-G as a useful tool for use with an older population who demonstrate slower performance and sensorimotor difficulties.

Content:
Katz et al. (1989) conducted a factor analysis to determine the construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed.
of the original LOTCA, using two groups of patients with traumatic head injury or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=96), and health adults (n=55). In the patient group, Visuomotor Organisation loaded on Factor 1 with 44% variance, Perception loaded on Factor 2 with 12% variance, and Thinking Operations on Factor 3 with 10% variance. In contrast, in the control group, Perception loaded on Factor 1 with 33% variance, Thinking Operations on Factor 2 with 23% variance, and Visuomotor Organisation on Factor 3 with 6% variance. CorrelationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients within the Perception, Visuomotor Organisation and Thinking Operations subtests of the original LOTCA ranged from 0.40 to 0.80, indicating that subtests are not equivalent and the battery should be performed in full.

Su et al. (2000) examined the strength of correlations between LOTCA subtests on a sample of patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=44), using Pearson correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients. Large correlations were found between the Visuomotor Organisation and Thinking Operations areas (r=0.66, p<0.0001); moderate correlations were found between the Orientation and Perception areas (r=0.55, p<0.0001), Thinking Operations and Orientation (r=0.47, p<0.01) and Perception (0.32 p<0.05); and small correlations were found between Visuomotor Organisation and Orientation (r=0.23) and Perception (r=0.25).

Criterion:
Concurrent:
Bar-Haim Erez & Katz (2003) reported adequate correlations (from r=0.55 to r=0.38) between the LOTCA-G areas and the MMSE total scores in patients with dementia (n=30), using Spearman correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
analysis.

Cooke et al. (2006a) examined the concurrent validityTo validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument. See also “gold standard.”
of the OT-APST in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=208) by comparing performance on 5 OT-APST subtests with performance on 5 corresponding LOTCA-II and LOTCA-G areas. Poor to excellent correlations were reported between OT-APST and LOTCA-II area scores (range d=0.27 to d=0.66); and poor to excellent correlations between OT-APST and LOTCA-G area scores (range d=0.25 to d=0.80), using Somers’s d concordance correlations.

Zwecker et al. (2002) reported on correlations between cognitive status and functional motor outcomes in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=66). Functional motor outcomes were measured according to efficacy and efficiency of FIM motor scores (isolated from total FIM scores) and Montebello Rehabilitation Factor Score (MRFS). An adequate correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
was found between LOTCA scores and MRFS efficacy (r=0.34, p<0.001) but poor correlations were reported between LOTCA scores and FIM motor efficacy (r=0.25, p<0.05), FIM efficiency (r=0.16) and MRFS efficiency (r=0.19), using Pearson correlations.

Predictive.
No studies have examined the predictive validityA form of criterion validity that examines a measure’s ability to predict some subsequent event. Example: can the Berg Balance Scale predict falls over the following 6 weeks? The criterion standard in this example would be whether the patient fell over the next 6 weeks.
of the LOTCA with patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

Construct:
Katz et al. (1989) conducted factor analysis of the original LOTCA and reported suitable construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed.
for the Perception, Visuomotor Organization and Thinking Operations areas.

Su et al. (2000) reported that the LOTCA Orientation and Perception areas demonstrate suitable construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed.
to measure two theoretically and statistically distinct theoretical constructs.

Convergent/Discriminant.
Katz et al. (1995) reported a statistically significant difference in the time taken to complete the LOTCA and the LOTCA-G in a group of healthy adults, using two-way ANOVA (F=11.26, P<.0001), with less time taken to complete the LOTCA-G than the LOTCA.

Cooke et al. (2006b) examined the convergent validityA type of validity that is determined by hypothesizing and examining the overlap between two or more tests that presumably measure the same construct. In other words, convergent validity is used to evaluate the degree to which two or more measures that theoretically should be related to each other are, in fact, observed to be related to each other.
of the LOTCA (2nd edition) and the LOTCA-G area means with the OT-APST subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
means in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., using Spearman’s rho correlations. Statistically significant correlations were found between the 5 areas of the LOTCA (2nd edition) and LOTCA-G, and the 5 corresponding subscales of the OT-APST (p<0.01). Correlations ranged from adequate (0.33 for OT-APST Apraxia subtest and LOTCA Motor Praxis area/LOTCA-G Praxis area) to excellent (0.80 for OT-APST Constructional skills subtest and LOTCA-G Visuomotor Organization area).

Zwecker et al. (2002) reported adequate correlations between the LOTCA and MMSE (r= 0.588, p<0.001) and between the LOTCA and FIM cognitive subtest (r=0.471, p<0.001) in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=66), using Pearson’s CorrelationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
.

Su et al. (2000) reported significant correlations between the LOTCA, Rivermead Perceptual Assessment Battery (RPAB), and the Motor-Free Visual Perception Test (MVPT), in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=44). Excellent correlations were reported between LOTCA Thinking Operations and MVPT (r = 0.72, p<0.0001); LOTCA Visuomotor Organization and MVPT (r=0.79, p<0.0001); and LOCTA Visuomotor Organisation and RPAB Spatial Awareness (r=0.74, p<0.0001). Adequate correlations were reported between LOTCA Orientation and RPAB Sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
(r=0.46, p<0.01) and Object Completion (r=0.38, p<0.01) subtests; LOTCA Perception and RPAB Sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
(r=0.38, p<0.01); LOTCA Visuomotor Organization and RPAB Sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
(r=0.52, p<0.01); and LOTCA Thinking Operations and RPAB Sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
(r=0.56, p<0.0001), Object completion (r=0.36, p<0.05), Figure-ground Discrimination (r=0.51, p<0.01) and Spatial Awareness (r=0.56, p<0.0001) subtests.

Katz et al. (2000) examined correlations between cognitive performance and daily function in two subgroups of adult with right hemisphere strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (n=40 vs. patients without unilateral spatial neglect, n=21), using Spearman’s correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
analysis. Cognitive skills were measured using the LOTCA at admission to and discharge from inpatient rehabilitation, and only using the Block Design, Puzzle and Clock Drawing subtests at 6-month follow-up. Functional skills were assessed using all or some of the FIM total, motor and cognitive scores, Rabideau Kitchen Evaluation – Revised (RKE-R) (sandwich and drink preparation), and Phone Use tests at these time points. In the neglect group, adequate to excellent correlations were reported between LOTCA Visuomotor Organisation and Thinking Operations areas and functional tests at admission, discharge and follow-up (range r=0.46 to 0.80). Adequate to excellent correlations were reported between LOTCA Perception and functional tests at discharge only (range r=-0.54 to 0.75). Poor correlations were found between LOTCA Orientation and functional tasks in this subgroup. The authors reported a possible confounding effect with this sub-group, given the visual/spatial demands of the LOTCA. In the non-neglect group, poor to excellent correlations were reported between FIM Cognitive and LOTCA Orientation (r=0.05), Perception (r=0.59), Visuomotor Organisation (r=0.67) and Thinking Operations (r=0.58) areas at admission. No significant correlations were reported with FIM total or FIM motor for any LOTCA area on admission. Adequate to excellent correlations were reported between LOTCA Visuomotor Organisation and Thinking Operations subtests and functional tasks at discharge and follow up (range r=0.43 to 0.62).

Known Group:
Katz et al. (1989) examined known group validityThe degree to which an assessment measures what it is supposed to measure.
of the original LOTCA among patients with traumatic brain injury (n=20), patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. (n=28), and healthy adults (n=55), using the Wilcoxon testThe Wilcoxon test is a nonparametric test that compares two paired groups. This test calculates and then analyzes the differences between the pairs. The Wilcoxon Rank Sum test is used to determine whether two scores have the same continuous distribution. The Wilcoxon Signed Rank test is suitable to use as an alternative to the paired t-test when the scores are not normally distributed.
. Significant differences among the three subgroups were found on initial assessment at the time of referral (p=0.0001).

Katz et al. (1995) compared LOTCA scores between healthy adults (n=29) and patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. (n=24), using the Wilcoxon testThe Wilcoxon test is a nonparametric test that compares two paired groups. This test calculates and then analyzes the differences between the pairs. The Wilcoxon Rank Sum test is used to determine whether two scores have the same continuous distribution. The Wilcoxon Signed Rank test is suitable to use as an alternative to the paired t-test when the scores are not normally distributed.
. Significant between group differences (range p=0.0002 to p=0.04) were reported for Orientation, Perception, Visuomotor Organization and Thinking Operations areas (excluding the Spatial Perception, Praxis, Coloured Block Design, Plain Block Design, Puzzle, Drawing a Clock, Categorization, Risk Object Classification – ROC -Unstructured and Pictorial Sequence B subtests). The group of healthy adults scored higher on all areas of the LOTCA than the group of patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain..

Katz et al. (1995) reported differences (although no statistical data were provided) between healthy adults and healthy elderly adults on the Visuomotor Organization and Thinking Operation areas of the LOTCA.

Katz et al (1997) compared performance between adults aged 18 – 30 years (n=36) and adults aged 58 – 70 years (n=36) on three versions of the LOTCA puzzle reproduction task (original direct placement version; subplacement version; and LOTCA-G version). Two-way anaylsis of variance (ANOVA) showed significant differences in terms of age (p<0.0001), puzzle version (p<0.0002) and age-by-version interaction (p<0.01). Older adults took significantly longer than younger adults to complete each version of the puzzle. While the younger group of adults took significantly longer to complete the subplacement version as compared to the other two versions (original; LOTCA-G version), the older adults were able to complete the LOTCA-G version at a significantly faster rate than the other two versions (original version; subplacement version).

Katz et al (2000) compared LOTCA performance between two subgroups of adults with right hemisphere strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. (patients with unilateral spatial neglect, n=19 vs. patients without unilateral spatial neglect, n=21) on admission to and discharge from inpatient rehabilitation. Patients with neglect were reported to score significantly worse on the Overlapping Figures subtest, Perception area total score, and all Visuomotor Organisation and Thinking Operations subtests and area total scores at admission and discharge (range p=0.02 to p=0.0001; Wilcoxon Rank Sum statistic).

Annes et al. (1996) compared LOTCA maximum scores between healthy young adults aged 17-25 years (n=49) and healthy older adults aged 40-75 years (n=49), using Fisher’s Exact Test or Yate’s chi square as appropriate. No significant differences were reported for Orientation or Perception areas. Younger adults performed significantly better on the Copying Geometric Forms (p=0.01), Plain Block Design (p=0.024) and Pictorial Sequence A (p=0.002) subtests, while older adults were reported to perform significantly better on the Geometric Sequence (p=0.046) subtest. The authors concluded that separate LOTCA norms are not required for the two age groups. While a significant difference was seen for the ROC-Unstructured subtest, it is considered that this difference was due to an error in the administration of the test.
Note: The maximum score was used rather than mean or standard deviations, due to a ceiling effectA ceiling effect occurs when test items aren't challenging enough for a group of individuals. Thus, the test score will not increase for a subsample of people who may have clinically improved because they have already reached the highest score that can be achieved on that test. In other words, because the test has a limited number of difficult items, the most highly functioning individuals will score at the highest possible score. This becomes a measurement problem when you are trying to identify changes - the person may continue to improve but the test does not capture that improvement. Example: A memory test that assesses how many words a participant can recall has a total of five words that each participant is asked to remember. Because most individuals can remember all five words, this measure has a ceiling effect. See also "floor effect." produced by the consistently high performance of subjects within both groups.

Annes et al. (1996) reported a significant difference in time taken to complete the LOTCA Visuomotor Organization area between healthy young adults aged 17-25 years (n=49) and healthy older adults aged 40-75 years (n=49), with the younger age group completing 6 of 7 subtests more quickly (Copying Geometric Forms, p<0.01; Pegboard Construction, p<0.01; Coloured Block Design, p<0.01; Reproduction of a 2D Model, p<0.05; Reproduction of a Model, p<0.05; and Drawing a Clock p<0.05; but not Plain Block Design).

Cermak et al. (1995) compared LOTCA performance between American (n=25) and Israeli (n=56) patients with CVA. Using the total sample group, they also compared performance between patients with right CVA (n=45) and patients with left CVA (n=36), using t-tests. Israeli participants were found to perform significantly better than American participants on the Orientation to Time subtest (p<0.01) in both subgroups (right CVA and left CVA); the Drawing a Clock (p<0.01) and Risk Object Classification – structured (p<0.05) subtests in the right CVA subgroup; and the Object Constancy (p<0.01) subtest in the left CVA subgroup. Comparison according to side of lesion showed that patients with right CVA performed significantly better (p<0.05) than those with left CVA on the Orientation to Time, Object Constancy and Spatial Perception subtests within the American subgroup; and on the Praxis subtest within the Israeli subgroup (p<0.01). However, patients with left CVA performed significantly better (p<0.05) than patients with right CVA on the Pegboard Construction subtest amongst both American and Israeli participants. It should be noted that Israeli participants were significantly younger than American participants in both the right CVA (mean age 58.5 vs 64.3, t = 2.25, p < 0.05) and left CVA (mean age 55.0 vs. 69.0, t=3.09, p<0.01) groups. The correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases - for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases - for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
between LOTCA subtests and age were low to moderate for most subtests, but was moderate for the Orientation to Time subtest, in which the Israeli group performed significantly better than the American group.

Su et al. (2000) compared the perceptual performance of patients with intracerebral hemorrhage (n=22) to patients with ischemia (n=22) early after strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. and found that patients with intracerebral hemorrhage performed significantly worse on the LOTCA Thinking Operations area (p=0.007). No significant differences were reported between subjects with right-sided lesions and subjects with left-sided lesions for any LOTCA area.

Josman and Katz (2006) examined the relationship between formal categorization-sorting tests and functional sorting tasks among individuals with schizophrenia (n=37), patients post strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. (n=18) and healthy adults (n=15), using the LOTCA Picture Sort subtest, Wisconsin Card Sorting Test, Short Category Test, Risk Object Classification and five functional daily tasks (sorting laundry, utensils, invoices, and two shopping lists). One way ANCOVA revealed a significant difference in the mean performance of the LOTCA Picture Sort subtest (F= 7.57, P=0.001). Post hoc Scheffe tests revealed that patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. performed worse than individuals with schizophrenia (P<0.05) and healthy adults (P<0.05).

Katz et al (1995) compared LOTCA-G scores between healthy older adults (n=43) and patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. (n=33), using the Wilcoxon sign test. Healthy older adults performed significantly better than patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. for the Orientation, Perception, Visuomotor Organization and Thinking Operations areas (range p=0.0001 to p=0.05), excluding the Praxis and Coloured Blocks Design subtests. Significant differences in the time taken to complete the LOTCA-G were also found between the 2 groups (P<0.0001), with the healthy older adults completing the assessment more quickly than patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain..

Bar-Haim Erez & Katz (2003) compared LOTCA-G scores between healthy adults (n=43) and individuals with dementia (n=30). Healthy adults performed significantly better than individuals with dementia on all subtests (p=0.000), except for the Object Identification and Shape Identification subtests. Health adults completed the test significantly quicker than individuals with dementia (p<0.000). Comparison of healthy adults and two subgroups of individuals with dementia (mild dementia, n=13; moderate dementia, n=17) by one way ANOVA showed significant F tests for all areas (p<0.000, except visual perception p<0.05). Post hoc Scheffe testing showed that individuals with mild dementia performed better than individuals with moderate dementia in the areas of Orientation, Visual Perception, Visuomotor Organisation, Thinking Operations and Memory. Mann-Whitney analysis of subtest mean scores from the two dementia subgroups further showed that individuals with mild dementia performed significantly better than individuals with moderate dementia on 10 subtests (Orientation to Place, Orientation to Time, Spatial Perception-Self and Surrounding, Motor Imitation, Utilisation of Objects, Copying Geometric Forms, Pegboard Construction, Block Design (Colour), Categorisation, Memory of a Famous Personality), indicating that the LOTCA-G is sensitive to degree of dementia.

Responsiveness

No studies have reported on the responsivenessThe ability of an instrument to detect clinically important change over time.
of the LOTCA.

References

Annes, G., Katz, N., & Cermak, S. (1996). Comparison of younger and older healthy American adults on the Loewenstein Occupational Therapy Cognitive Assessment. Occupational Therapy International, 3, 157-173.
Bar-Haim Erez, A., & Katz, N. (2003). Cognitive profiles of individuals with dementia and healthy elderly: The Loewenstein Occupational Therapy Cognitive Assessment (LOTCA-G). Physical and Occupational Therapy in Geriatrics, 22, 29-42.
Cermak, S. A., Katz, N., McGuire, E., Greenbaum, S., Peralta, C., & Flanagan, V.M. (1995). Performance of Americans and Israelis with cerebrovascular accident on the Loewenstein Occupational Therapy Cognitive Assessment. American Journal of Occupational Therapy, 49, 500-506.
Cooke, D. M., McKenna, K., Fleming, J. & Darnell, R. (2006a). Criterion validity of the Occupational Therapy Adult Percepetual Screening Test (OT-APST). Scandinavian Journal of Occupational Therapy, 13, 38-48.
Cooke, D. M., McKenna, K., Fleming, J. & Darnell, R. (2006b). Construct and ecological validity of the Occupational Therapy Adult Perceptual Screening Test (OT-APST). Scandinavian Journal of Occupational Therapy, 13, 49-61.
Jang, Y., Chern, J-S., & Lin, K-C. (2009). Validity of the Loewenstein Occupational Therapy Cognitive Assessment in people with intellectual disabilities. American Journal of Occupational Therapy, 63, 414-244.
Josman, N., Abdallah, T. M., & Engel-Yeger, B. (2010). Cultural factors affecting the differential performance of Israeli and Palestinian children on the Loewenstein Occupational Therapy Cognitive Assessment. Research in Developmental Disabilities, 31, 656-663.
Josman, N., & Katz, N. (2006). Relationships of categorization on tests and daily tasks in patients with schizophrenia, post-stroke patients and healthy controls. Psychiatry Research, 141, 15-28.
Katz, N., Champagne, D., & Cermak, S. (1997). Comparison of the performance of younger and older adults on three versions of a puzzle reproduction task. American Journal of Occupational Therapy, 51, 562-568.
Katz, N., Elazar, B., & Itzkovich, M. (1995). Construct validity of a geriatric version of the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery. Physical and Occupational Therapy in Geriatrics, 13, 31-46.
Katz, N., Hartman-Maeir, A., Ring, H., & Soroker, N. (2000). Relationships of cognitive performance and daily function of clients following right hemisphere stroke: Predictive and ecological validity of the LOTCA battery. Occupational Therapy Journal of Research, 20, 3-17.
Katz, N., Itzkovich, M., & Averbuch, S. (2002). The Loewenstein Occupational Therapy Cognitive Assessment. Archives of Physical Medicine and Rehabilitation, 83, 1179.
Katz, N., Itzkovich, M., Averbuch, S., & Elazar, B. (1989). Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery for brain-injured patients: Reliability and validity. American Journal of Occupational Therapy, 43, 184-192.
Su, C-Y., Chang, J-J., Chen, H-M., Su, C-J., Chien, T-H., & Huang, M-H. (2000). Perceptual differences between stroke patients with cerebral infarction and intracerebral hemorrhage. Archives of Physical Medicine and Rehabilitation, 81, 706-714.
Su, C-Y., Chen, W-L., Tsai, P-C., Tsai, C-Y., & Su, W-L. (2007). Psychometric properties of the Loewenstein Occupational Therapy Cognitive Assessment-Second Edition in Taiwanese persons with schizophrenia. American Journal of Occupational Therapy, 61, 108-118.
Su, C-Y., Lin, Y-H., Chen-Sea, M-J., & Yang, M-J. (2007). A confirmatory factor analysis of the Chinese version of the Loewenstein Occupational Therapy Cognitive Assessment-second edition in a Taiwanese mixed clinical sample. Occupational Therapy Journal of Research, 27, 71-80.
Zwecker, M., Levenkrohn, S., Fleisig, Y., Zeilig, G., Ohry, A., & Adunsky, A. (2002). Mini-Mental State Examination, cognitive FIM instrument, and the Loewenstein Occupational Therapy Cognitive Assessment: Relation to functional outcome of stroke patients. Archives of Physical Medicine and Rehabilitation, 83, 342-5.

See the measure

How to obtain the LOTCA?

The LOTCA can be purchased online from http://store.grovergear.com/, http://www.ot-innovations.com, or http://therapro.com.

Motor-Free Visual Perception Test (MVPT)

Evidence Reviewed as of before: 19-08-2008

Author(s)*: Lisa Zeltzer, MSc OT

Editor(s): Nicol Korner-Bitensky, PhD OT; Elissa Sitcoff, BA BSc

Purpose

The Motor-Free Visual Perception Test (MVPT) is a widely used, standardized test of visual perception. Unlike other typical visual perception measures, this measure is meant to assess visual perception independent of motor ability. It was originally developed for use with children (Colarusso & Hammill, 1972), however it has been used extensively with adults. The most recent version of the measure, the MVPT-3, can be administered to children (> 3 years), adolescents, and adults (< 95 years) (Colarusso & Hammill, 2003).

In-Depth Review

Purpose of the measure

The MVPT can be used to determine differences in visual perception across several different diagnostic groups, and is often used by occupational therapists to screen those with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. or head injury.

Available versions

Original MVPT
The original MVPT was published by Colarusso and Hammill in 1972.

MVPT – Revised Version (MVPT-R)
The MVPT-R was published by Colarusso and Hammill in 1996. In this version, four new items were added to the original MVPT version (40 items in total). Age-range norms (U.S.) were also added to the original MVPT, to include children up to the age of 12. No adult data were collected when the scale was developed, however, the MVPT-R has been used with both pediatric and adult populations (Brown, Rodger, & Davis, 2003). While the MVPT-R has been reported to have an excellent correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
with the original MVPT (r = 0.85, Colarusso and Hammill, 1996), Brown et al. (2003) caution that no other reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
and validityThe degree to which an assessment measures what it is supposed to measure.
data have been reported for this version.

MVPT – 3rd Edition (MVPT-3)
The MVPT-3 was published by Colarusso and Hammill in 2003. The MVPT-3 was a major revision of the MVPT-R, and includes additional test items that allow for the assessment of visual perception in adults and adolescents. The MVPT-3 is intended for individuals between the ages of 4-95, and takes approximately 25 minutes to administer. (http://www4.parinc.com/Products/Product.aspx?ProductId=MVPT-3)

Features of the measure

Items:
The items for the original MVPT, MVPT-R and MVPT-3 are comprised of items representing 5 visual domains:

Source: Colarusso & Hammill, 1996

Visual Discrimination	The ability to discriminate dominant features in different objects; for example, the ability to discriminate position, shapes, forms, colors and letter-like positions.
Visual Figure-Ground	The ability to distinguish an object from its background.
Visual Memory	The ability to recall dominant features of one stimulus item or to remember the sequence of several items.
Visual Closure	The ability to identify incomplete figures when only fragments are presented.
Visual Spatial	The ability to orient one’s body in space and to perceive the positions of objects in relation to oneself and to objects.

Note: Five domains do not represent different subscales or subtests and thus cannot be used to yield individual scores.

Original MVPT
Contains 36 items.

MVPT-R
Contains 40 items. Since the MVPT-R includes children up to 12 years old, four items were added to the items of the original MVPT to accommodate the increased age-range covered by the norms of the MVPT-R.

MVPT-3
Contains 65 items. Before administering the MVPT-3, the examiner must ask for the patient’s date of birth and compute their age. This will determine where in the test one should begin. Children between the ages of 4-10 begin with the first example item and complete items 1-40. Individuals between the ages of 11-95 begin with the third example item and complete items 14-65. All of the items that fall within an individual’s age group must be administered.

Each item consists of a black-and-white line drawing stimulus, along with four multiple-choice response options (A, B, C, D) from which to choose the item that matches the example. For most items, the stimulus and response choices appear on the same page. The stimulus drawing appears at the top of the page above a row of four multiple-choice options (see image below).

Below are four examples of test items and their corresponding multiple-choice response options:

Items assessing visual memory have the stimulus and multiple-choice options presented on separate pages. For these items, the stimulus page is presented for 5 seconds, removed, and the options page is then presented. Items with similar instructions are grouped together in order of increasing difficulty. The patient points to or says the letter that corresponds to the desired answer (Su et al. 2000). The examiner records each response on the recording form.

To ensure that the patient understands the task instructions, example items are presented for each new set of instructions. Examiners must ensure that the patient understands these directions before proceeding to the next domain.

Subscales:
None.

Equipment:
Original MVPT and MVPT-R
Materials for the test include the manual that describes the administration and scoring procedures, the test plate book, score sheet, stopwatch and a pencil (Brown et al., 2003).

MVPT-3
Materials for the test include the manual that describes administration and scoring procedures, a recording form to record patient responses, and a spiral-bound test plates easel.

Training:
Various health professionals, including occupational therapists, teachers, school psychologists, and optometrists, can administer all versions of the MVPT. Only individuals familiar with both the psychometric properties and the score limitations of the test should conduct interpretations (Colarusso & Hammill, 2003).

Time:
Original MVPT and MVPT-R
The test takes 10-15 minutes to administer, and 5 minutes to score (Brown et al., 2003).

MVPT-3
According to the manual, the MVPT-3 takes approximately 20 to 30 minutes to administer and approximately 10 minutes to score.

Scoring:
Original MVPT and MVPT-R
One point is given for each correct response. Raw scores are then converted to age and perceptual equivalents to allow for a comparison of the patient’s performance to that of a normative group of same-aged peers.

MVPT-3
A single raw score is formed, representing the patients overall visual perceptual ability. The raw score is calculated by subtracting the number of errors made from the number of the last item attempted. The total scores range from 55-145. Higher scores reflect fewer deficits in general visual perceptual function. The raw score can then be converted to standard scores, age equivalents, and percentile ranks using the norm tables provided in the manual, which will allow for the comparison of a patient’s performance to that of a normative group of same-aged peers.

Alternative form of the MVPT

MVPT – Vertical Version (MVPT-V) (Mercier, Hebert, Colarusso, & Hammill, 1996).
Response sets are presented in a vertical layout rather than the horizontal layout found in other versions of the MVPT. This layout allows for an accurate assessment of visual perceptual abilities in adults who have hemifield visual neglect, commonly found in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. or traumatic brain injury. These patients are unable to attend to a portion of the visual field, and may therefore miss any answer choices that are presented in one part of the visual field when they are presented horizontally. The MVPT-V contains 36 items. Mercier, Herbert, and Gauthier (1995) reported excellent test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
for the MVPT-V (ICC = 0.92).

Note: The MVPT-V removes unilateral visual neglect as a variable in test performance and therefore should not be used to assess driving ability (Mazer, Korner-Bitensky, & Sofer, 1998).

Client suitability

Can be used with:

Patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..
The MVPT can be used in patients with expressive aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury.
An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) if they are able to understand instructions and the various sub-scale requirements.

Should not be used in:

Children under the age of 4.
The MVPT calculates a global score and thus provides less information regarding specific visual dysfunction than a scale that provides domain-specific scores (Su et al., 2000). To assess various domains of visual perceptual ability, an alternative with good pscyhometric properties is the Rivermead Perceptual Assessment Battery. It has 16 different subtests assessing various aspects of visual perception. It takes between 45-50 minutes to administer and has established reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
(Bhavnani, Cockburn, Whiting & Lincoln, 1983) and validityThe degree to which an assessment measures what it is supposed to measure.
(Whiting, Lincoln, Bhavnani & Cockburn, 1985) and was designed to assess visual perception problems in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. (Whiting et al., 1985).
The is administered via direct observation of task completion and cannot be used with a proxy respondent.
The MVPT-3 should be used only as a screeningTesting for disease in people without symptoms.
instrument with 4-year-old children but can be used for diagnostic purposes in all other age groups (Colarusso & Hammill, 2003).
McCane (2006) argues that although Colarusso and Hammill (2003) state that the MVPT-3 can be used as a diagnostic tool in all age groups other than in four year olds, even more cautious interpretation is needed. This is based on the generally accepted notion that the reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
of a tool should be > 0.90 to be used for diagnostic and decision-making purposes (Sattler, 2001). Therefore, the MVPT-3 should only be used as a diagnostic tool in adolescents between 14-18 years old because this is the only age group in which reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
exceeds 0.90.
The MVPT-V removes unilateral visual neglect as a variable in test performance and therefore should not be used to assess driving ability (Mazer et al., 1998).

In what languages is the measure available?

No information is currently available regarding the languages in which the instructions to the MVPT have been translated.

Note: Because the test requires no use of verbal response by the respondent, if the clinician can determine through the use of the practice items that the individual understands the task requirements, then it is possible to use the test with minimal language use.

Summary

What does the tool measure?	Visual perception independent of motor ability.
What types of clients can the tool be used for?	The MVPT can be used to determine differences in visual perception across several different diagnostic groups, and is often used by occupational therapists to screen those with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. or head injury. The MVPT was originally developed for use with children, however it has been used extensively with adults.
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	Assessment.
Time to administer	The MVPT and MVPT-R takes 10-15 minutes to administer, and 5 minutes to score. The MVPT-3 takes approximately 20 to 30 minutes to administer and approximately 10 minutes to score.
Versions	Original MVPT; MVPT Revised version (MVPT-R); MVPT 3rd edition (MVPT-3); MVPT Vertical Version (MVPT-V).
Other Languages	No information is currently available regarding the languages in which the instructions to the MVPT have been translated. Note: Because the test requires no use of verbal response by the respondent, if the clinician can determine through the use of the practice items that the individual understands the task requirements, then it is possible to use the test with minimal language use.
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: One study examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original MVPT or MVPT-3. Test-retest: One study examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society). of the original MVPT, MVPT-R or MVPT-3.
ValidityThe degree to which an assessment measures what it is supposed to measure.	Content: One study examined the content validityRefers to the extent to which a measure represents all aspects of a given social concept. Example: A depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension. of the original MVPT. Criterion: Concurrent: Two studies examined the concurrent validityTo validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument. See also “gold standard.” of the original MVPT. Predictive: Out of three studies examined the predictive validityA form of criterion validity that examines a measure’s ability to predict some subsequent event. Example: can the Berg Balance Scale predict falls over the following 6 weeks? The criterion standard in this example would be whether the patient fell over the next 6 weeks. of the original MVPT. One of these studies found the MVPT highly predictive of on-road driving outcome in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., however an other study found that the MVPT may not be as highly predictive of driving ability in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. as previous research had indicated. One study reported that the MVPT was predictive of future at-fault motor vehicle collisions in a cohort of older drivers. Construct: Convergent/Discriminant: Two studies examined the convergent/discriminant validityThe degree to which an assessment measures what it is supposed to measure. of the original MVPT. Known Groups: – Two studies examined the known groups validityKnown groups validity is a form of construct validation in which the validity is determined by the degree to which an instrument can demonstate different scores for groups know to vary on the variables being measured. of the original MVPT. – One study examined the known groups validityKnown groups validity is a form of construct validation in which the validity is determined by the degree to which an instrument can demonstate different scores for groups know to vary on the variables being measured. of the MVPT-3.
Does the tool detect change in patients?	No studies have examined the responsivenessThe ability of an instrument to detect clinically important change over time. of the MVPT.
Acceptability	The MVPT is a short and simple measure and has been reported as well tolerated by patients. The test is administered by direct observation and is not suitable for proxy use.
Feasibility	The MVPT has standardized instructions for administration in an adult population and requires the test plates and manual. Only individuals familiar with both the psychometric properties and the score limitations of the test should conduct interpretations.
How to obtain the tool?	The MVPT can be purchased from: https://www.therapro.com/

Psychometric Properties

Overview

The reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of "true" variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to "noise" in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
and validityThe degree to which an assessment measures what it is supposed to measure.
of the MVPT has not been well studied. To our knowledge, the creators of the MVPT have personally gathered the majority of psychometric data that are currently published on the scale. In addition, the majority of the existing psychometric studies have been conducted using the original MVPT only, and few studies have examined the validity of the MVPT-R and MVPT-3. Further investigation on the reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of "true" variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to "noise" in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
and validityThe degree to which an assessment measures what it is supposed to measure.
of the original MVPT, MVPT-R and MVPT-3 is therefore recommended.

Reliability

Original MVPT

Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.:
Colarusso and Hammill (1996) calculated the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
.

Test-retest:
Colarusso and Hammill (1972) examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
, ranging from r = 0.77 to r = 0.83 at different age levels, with a mean coefficient of r = 0.81 for the total sample.

Inter-rater:
Has not been investigated.

MVPT-R

Test-retest:
Only one study evaluating the reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
of the MVPT-R has been reported in the literature. Burtner, Qualls, Ortega, Morris, and Scott (2002) administered the MVPT-R to a group of 38 children with learning disabilities and 37 children with age-appropriate development (aged 7 to 10 years) on two separate occasions within 2.5 weeks. Intraclass correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients (ICCs) for perceptual quotient scores ranged from adequate to excellent (ranging from ICC = 0.63 to ICC = 0.79). Perceptual age scores also ranged from adequate to excellent (ICC = 0.69 to ICC = 0.86). Pearson product moment correlations for perceptual quotient scores ranged from adequate to excellent (r = 0.70 to r = 0.80) and perceptual age scores were excellent, ranging from r = 0.77 to r = 0.87. These results suggest that the MVPT-R has adequate test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
with more stability in visual perceptual scores for children with learning disabilities.

Inter-rater:
Has not been investigated.

MVPT-3

Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.:
Colarusso and Hammill (2003) computed Cronbach’s coefficient alphas for each age group. Alpha coefficients ranged from poor to excellent (alpha = 0.69 to alpha = 0.90). In children aged 4, 5, and 7, alpha coefficients were 0.69, 0.76, and 0.73, respectively. ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
coefficients for all other age groups were excellent (alpha’s exceed 0.80).

Test-retest:
Colarusso and Hammill (2003) examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
(0.87 and 0.92, respectively), suggesting that the MVPT-3 is relatively stable over time.

Inter-rater:
Has not been investigated.

Validity

Content:
Only the content validityRefers to the extent to which a measure represents all aspects of a given social concept. Example: A depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension.
of the original MVPT has been provided.

The content of the MVPT was based on item analyses as well as the five visual perceptual categories proposed by Chalfant and Scheffelin (1969). The authors examined item bias, including the effects of gender, residence, and ethnicity. Performance on each item was compared for differing groups to determine any biased content. Only three items appeared to function differently based on group membership. The authors examined these items and chose not to eliminate the items based on other psychometric data.

Criterion:
Concurrent:

Original MVPT

The following information is from a review article by Brown, Rodger, and Davis (2003):

Correlations between the MVPT and the Frostig Developmental Test of Visual Perception ranged from adequate to excellent, from r = 0.38 to r = 0.60 (Frostig, Lefever, & Whittlesey, 1966).
Correlations between the MVPT and the Developmental Test of Visual Perception ranged from poor to excellent, r = 0.27 to r = 0.74 (Hammill, Pearson, & Voress, 1993).
CorrelationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
between the MVPT and the Matching subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
of Metropolitan Readiness Tests was adequate, r = 0.40 (Hildreth, Griffiths, & McGauvran, 1965).
Correlations between the MVPT and the Word Study Skills and Arithmetic subscales of Stanford Achievement Tests (Primary) were adequate, from r = 0.37 to r = 0.42 (Kelly, Madden, Gardner, & Rudman, 1964).
Correlations between the MVPT and the Durrell Analysis of Reading Difficulties were adequate, ranging from r = 0.33 to r = 0.46 (Durrell, 1955).
CorrelationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
between the MVPT and the Slosson Intelligence Test was adequate, r = 0.31 (Slosson, 1963).
CorrelationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
between the MVPT and the Pinter-Cunningham Primary Intelligence Test was adequate, r = 0.32 (Pintner & Cunningham, 1965).

Colarusso and Hammill (1996) concluded that the MVPT measures the construct of visual perception adequately because the MVPT correlated more highly with measures of visual perception (median r = 0.49) than it did with tests of intelligence (median r = 0.31) or school performance (median r = 0.38).

Predictive:

Original MVPT

Mazer et al. (1998) examined whether the MVPT could predict on-road driving outcome in 84 patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. Patients were given a pass or fail based on their driving behavior. The was found to be the most predictive of driving outcome out of a number of perceptual tests that were administered (the Complex Reaction Timer, the Single Letter Cancellation Test, the Double Letter Cancellation Test, the Money Road Map Test of Direction Sense, the Trail Making Test A and B, the Bells Test, and the Charron Test). Patients who scored < 30 on the MVPT were 8.7 times more likely to fail the on-road evaluation compared to those who scored >30 (positive predictive value = 86.1%; right sided lesions = 94%; left-sided lesions = 80%). Furthermore, patients who performed poorly on both the MVPT and the Trail Making test B (a test of visual conceptual and visuomotor tracking) were 22 times more likely to fail the on-road evaluation as compared with those who did well on both tests. However, the MVPT was not highly predictive of a pass, such that even at the highest possible scores, half of the subjects passed and half failed the on-road evaluation.

Korner-Bitensky et al. (2000) also examined whether the MVPT could predict on-road driving test outcome in 269 patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. A cutoff of < 30 was used to indicate poor visual perception and > 30 indicated good perception. A low positive predictive value of 60.9% (the proportion of people who had a low score on the MVPT and failed the test) and a low negative predictive value (the proportion of people who had a high score on the MVPT and passed the driving test) of 64.2% were found. Logistic regression revealed the best predictor of driving failure to be increased age, a right hemisphere lesion and a low score. The results of this study demonstrate that the MVPT may not be as highly predictive of driving ability in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. as previous research had indicated.

Ball et al. (2006) examined whether scores on the visual closure items of the MVPT were predictive of future at-fault motor vehicle collisions in a cohort of older drivers (over the age of 55). The MVPT was found to be predictive, such that individuals who made four or more errors on the MVPT were 2.10 times more likely to crash as those who made three or fewer errors.

Construct:
Convergent/Discriminant:

Original MVPT

Su et al. (2000) found excellent correlations between the MVPT and the Loewenstein Occupational Therapy Cognitive Assessment subscales of Visuo-motor organization and Thinking operations (r = 0.70 and 0.72). Adequate correlations between the Rivermead Perceptual Assessment Battery subscales of Sequencing“The coordination and proper ordering of the steps that comprise the task, requiring a proper allotment of attention to each step” (Lezak, 1989; as cited in (Baum, Morrison, Hahn & Edwards, 2007))
(0.39) and Figure-ground discrimination (0.41) were found. An excellent correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
between the MVPT and the Spatial awareness subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
of the Rivermead Perceptual Assessment Battery was observed (0.72).

Cate and Richards (2000) investigated the relationship between basic visual functions (acuity, visual field deficits, oculomotor skills and visual attention/scanning) and higher-level visual-perceptual processing skills (visual closure and figure-ground discrimination) in patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. using a Pearson product-moment correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
analysis. An excellent correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
of r = 0.75 was observed between vision screeningTesting for disease in people without symptoms.
scores and scores from the MVPT.

Known groups:

Original MVPT

Su et al. (2000) compared the perceptual performance of 22 patients with intracerebral hemorrhage to 22 patients with ischemia early after strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. The MVPT was not found to be discriminatively sensitive to side of lesion (left or right) or type of lesion (intracerebral hemorrhage vs. ischemic).

York and Cermak (1995) examined the performance of 45 individuals with either right cerebrovascular accident, left cerebrovascular accident, or individuals without cerebrovascular accident using the MVPT. Patients with right hemisphere lesions demonstrated poor performance on the MVPT in comparison to patients with left hemisphere lesions and a non-stroke group. However, the degree of difference between the mean scores of each group, as calculated using effect sizeEffect size (ES) is a name given to a family of indices that measure the magnitude of a treatment effect. Unlike significance tests, these indices are independent of sample size. The ES is generally measured in two ways: as the standardized difference between two means, or as the correlation between the independent variable classification and the individual scores on the dependent variable. This correlation is called the “effect size correlation”.
, (ES = 0.67 and 0.54, respectively), suggesting that the MVPT can discriminate between patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. versus individuals without strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

MVPT-3

Colarusso and Hammill (2003) examined MVPT-3 performance differences among individuals who were developmentally delayed, or who had experienced a head injury, or had a learning disability and compared their MVPT-3 performance to the general population mean MVPT-3 score of 100. It was hypothesized that each of these groups should display lower scores on the MVPT-3 when compared to the general population. Individuals classified as developmentally delayed had a mean MVPT-3 score of 69.46 which falls more than two standard deviations below the mean. Individuals with head injury had a mean score of 80.16, falling approximately 1.33 standard deviations below the mean. The group with learning disability had an average score of 88.24. The lower MVPT-3 scores for each of these three groups lends support for the construct validityReflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard , construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed.
of the test.

Responsiveness

Not applicable.

References

Ball, K. K., Roenker, D. L., Wadley, V. G., Edwards, J. D., Roth, D. L., McGwin, G., Raleigh, R., Joyce, J. J., Cissell, G. M., Dube, T. (2006). Can high-risk older drivers be identified through performance-based measures in a department of motor vehicles setting? Journal of the American Geriatrics Society, 54, 77-84.
Bhavnani, G., Cockburn, J., Whiting, S., Lincoln, N. (1983). The reliability of the Rivermead perceptual assessment. British Journal of Occupational Therapy, 52, 17-19.
Bouska, M. J., Kwatny, E. (1982). Manual for the application of the motor-free visual perception test to the adult population. Philadelphia (PA): Temple University Rehabilitation Research and Training Center.
Brown, T. G., Rodger, S., Davis, A. (2003). Motor-Free Visual Perception Test – Revised: An overview and critique. British Journal of Occupational Therapy, 66(4), 159-167.
Burtner, P. A., Qualls, C., Ortega, S. G., Morris, C. G., Scott, K. (2002). Test-retest of the Motor-Free Visual Perception Test Revised (MVPT-R) in children with and without learning disabilities. Physical and Occupational Therapy in Pediatrics, 22(3-4), 23-36.
Cate, Y., Richards, L. (2000). Relationship between performance on tests of basic visual functions and visual-perceptual processing in persons after brain injury. Am J Occup Ther, 54(3), 326-334.
Chalfant, J. C., Scheffelin, M. A. (1969). Task force III. Central processing dysfunctions in children: a review of research. Bethesda, MD: US Department of Health, Education and Welfare.
Colarusso, R. P., Hammill, D. D. (1972). Motor-free visual perception test. Novato CA: Academic Therapy Publications.
Colarusso, R. P., Hammill, D. D. (1996). Motor-free visual perception test–revised. Novato CA: Academic Therapy Publications.
Colarusso, R. P., & Hammill, D.D. (2003). The Motor Free Visual Perception Test (MVPT-3). Navato, CA: Academic Therapy Publications.
Durrell, D. (1955). Durrell Analysis of Reading Difficulty. Tarrytown, New York: Harcourt, Brace and World.
Frostig, M., Lefever, D. W., Whittlesey, J. R. B. (1966). Administration and scoring manual for the Marianne Frostig Developmental Test of Visual Perception. Palo Alto, CA: Consulting Psychologists Press.
Hammill, D. D., Pearson, N. A., Voress, J. K. (1993). Developmental Test of Visual Perception. 2nd ed. Austin, TX: Pro Ed.
Hildreth, G. H., Griffiths, N. L., McGauvran, M. E. (1965). Metropolitan Readiness Tests. New York: Harcourt, Brace and World.
Korner-Bitensky, N. A., Mazer, B. L., Sofer, S., Gelina, I., Meyer, M. B., Morrison, C., Tritch, L., Roelke, M. A., White, M. (2000). Visual testing for readiness to drive after stroke: A multiscenter study. Am J Phys Med Rehabil, 79(3), 253-259.
Kelly, T. L., Madden, R., Gardner, E. F., Rudman, H. C. (1964). Stanford Achievement Tests. New York: Harcourt, Brace and World.
Mazer, B. L., Korner-Bitensky, N., Sofer, S. (1998). Predicting ability to drive after stroke. Arch of Phys Med Rehabil, 79, 743-750.
McCane, S. J. (2006). Test review: Motor-Free Visual Perception Test.Journal of Psychoeducational Assessment, 24, 265-272.
Mercier, L., Herbert, R., Gauthier, L. (1995). Motor free visual perception test: Impact of vertical answer cards position on performance of adults with hemispatial neglect. Occup Ther J Res, 15, 223-226.
Mercier, L., Hebert, R., Colarusso, R., Hammill, D. (1996). Motor-Free Visual Perception Test – Vertical. Novato, CA: Academic Therapy Publications.
Pintner, R., Cunningham, B. V. (1965). Pintner-Cunningham Primary Test. New York: Harcourt, Brace and World.
Sattler, J.M. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego, CA: Jerome M. Sattler.
Slosson, R. L. (1963). Slosson Intelligence Test. East Aurora, NY: Slosson Educational Publications.
Su, C-Y., Charm, J-J., Chen, H-M., Su, C-J., Chien, T-H., Huang, M-H. (2000). Perceptual differences between stroke patients with cerebral infarction and intracerebral hemorrhage. Arch Phys Med Rehabil, 81, 706-714.
Whiting, S., Lincoln, N., Bhavnani, G., Cockburn, J. (1985). The Rivermead perceptual assessment battery. Windsor: NFER-Nelson.
York, C. D., Cermak, S. A. (1995). Visual perception and praxis in adults after stroke. Am J Occup Ther, 49(6), 543-550.

See the measure

The MVPT can be purchased from: https://www.therapro.com/

Occupational Therapy Adult Perceptual Screening Test (OT-APST)

Evidence Reviewed as of before: 19-08-2008

Author(s)*: Lisa Zeltzer, MSc OT

Editor(s): Nicol Korner-Bitensky, PhD OT; Elissa Sitcoff, BA BSc

Purpose

The Occupational Therapy Adult Perceptual ScreeningTesting for disease in people without symptoms.
Test (OT-APST) is a standardized screeningTesting for disease in people without symptoms.
measure that enables occupational therapists to test for the presence of impairment in visual perception across each of the major constructs of visual perception and praxis, including the problems most frequently occurring after strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

In-Depth Review

Purpose of the measure

The Occupational Therapy Adult Perceptual ScreeningTesting for disease in people without symptoms.
Test (OT-APST) is a standardized screeningTesting for disease in people without symptoms.
measure that enables occupational therapists to test for the presence of impairment in visual perception across each of the major constructs of visual perception and praxis, including the problems most frequently occurring after strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain..

Available versions

The OT-APST was developed by Deidre M. Cooke in 1992 and was first published in 1993 (Cooke, 1993). The OT-APST was revised in 2001.

Features of the measure

Items:

The OT-APST has 25 items in 7 subtest areas (Agnosia; Visuospatial Relations – Unilateral neglect and Body scheme -; Constructional skills; Apraxia; Acalculia; Functional Skills). Several of the items contribute to assessment of more than one area and are only performed once. The subtests and their respective items are organized as follows:

Agnosias (5 items)

Colour agnosia: The client is requested to name/identify six colors in an array to evaluate color recognition skills.
Object agnosia: The client is presented with a stapler that he/she must name and describe in terms of its use to demonstrate object recognition and naming.
Figure-ground: The client must recognize five items in an overlapping array to demonstrate figure-ground/perceptual closure skills.
Shape constancy: The client is presented with a mixed array of 10 shapes of varying sizes and positioned at different angles. The client must name/point on command to four common shapes to demonstrate shape recognition ability.
Reading-alexia: The client is asked to read a passage of text that has indentations of random lengths on both sides of the page to be sensitive to attention/neglect changes to both the left and right side of space.

Visuospatial relations – Unilateral neglect and Body scheme

Unilateral neglect (5 items)

Clock drawing
Copying a house diagram
Writing: The client is asked to provide a handwriting sample that consists of writing their name and address.
Reading a paragraph: The client is asked to read a passage of 12 lines to assess reading and visual scanningDuring this intervention the person with USN is encouraged to conduct voluntary eye movements toward the neglected visual field (usually the left side) by performing a task in that hemispace. The treatment often includes a visual target that the patient uses as an anchor to direct voluntary gaze control while scanning.
accuracy, to detect for the presence of neglect or the impact of visual field loss on reading, and to screen for alexia.
Telling time

Body scheme (4 items)

Body parts self: The client is asked to identify his/her own body parts.
Body parts therapist: The client is asked to identify the examiner’s body parts.
Left/right discrimination: The client is requested to differentiate sides (left and right).
Directions/position in space: The client is asked to move colored blocks to different positions in relation to each other (e.g. ‘on top of’ or ‘behind’), or to describe the color or point to the block that is ‘furthest away’ from and ‘nearest’ to him/her on the tabletop.

Constructional skills (3 items):

Graphic constructional skills: The client is asked to draw a clock and place its hands at a designated time. They must also copy a house.
Two-dimensional construction: The client is asked to reproduce colored block patterns in two-dimensional planes by copying a model provided.
Three-dimensional construction: Three-dimensional models rather than pictures of these models are copied.

Apraxia (6 items):

Smile command and copy
Wave right hand-command and copy
Wave left hand-command and copy
Stapler hold command and copy
Pen use for writing
Writing

Acalculia (1 item):

The client is asked to perform simple one- and two-digit addition and subtraction calculations that are set at a grade two level.

Functional skills (5 items):

Reading
Writing
Calculations: The client must complete simple mathematical calculations.
Telling time
Use of stapler

Futher observations of the client’s performance of other activitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of daily living such as dressing and grooming, in addition to performing a kitchen task are recommended but are not formally included in the OT-APST.

Other:

The length of time taken to complete the OT-APST in its entirety is recorded as a general indicator of the information-processing speed of the client and to reflect the client’s perceptual processing ability.

Scoring:

Scores for items in each construct or subtest area are summed to allow interpretation of patterns of impairments and to compare with normative data, but not to produce a total score for the OT-APST.

The table below outlines the subtest total scores and individual item scores of the OT-APST:

(Cooke, McKenna, Fleming, & Darnell, 2005a)

OT-APST subtests (subtest total score)	OT-APST items (item scores)
Agnosias (26)	Colour agnosia (6) Object agnosia (1) Figure-ground (5) Shape constancy (10) Reading-alexia (4)
Visuospatial relations:
Unilateral neglect (13)	Clock (3) House (4) Writing (1) Reading (4) Telling time (1)
Body scheme (22)	Body parts self (8) Body parts therapist (4) Left/right discrimination (4) Directions/position in space (6)
Constructional skills (53)	Graphic construction Clock (3) House (4) Two-dimensional construction (26) Three-dimensional construction (20)
Apraxia (10)	Smile command and copy (2) Wave right hand-command and copy (2) Wave left hand-command and copy (2) Stapler hold command and copy (2) Pen use for writing (1) Writing (1)
Acalculia (4)	Calculations-addition and subtraction (4)
Functional skills (11)	Reading (4) Writing (1) Calculations (4) Telling time (1) Use of stapler (1)

Cooke, McKenna, Fleming, and Darnell (2006c) examined the impact of age, education, and gender on OT-APST scoring based on the performance of 356 healthy Australian adults aged 16-97 years. When mean scores were compared for each subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
, the most significant differences were observed with age. Gender and level of education did not significantly impact on OT-APST performance of the healthy participants. Increasing age was significantly associated with reduced performance on all subscales but the Acalculia and Body scheme subscales. The age at which the most significant differences in OT-APST performance occurred was at age 75 years and above.

The tables below provides the cutoff scores indicating impairment for each of the subscales of the OT-APST, stratified by age group:

(Cooke, McKenna, Fleming, & Darnell, 2006c)

Age 16-74 years cut-off scores indicative of impairment
Agnosia	Body scheme	Neglect	Constructional skills	Apraxia	Acalculia	Functional skills
≤ 24	≤ 21	≤ 12	≤ 51	≤ 9	≤ 2	≤ 9

Age 75-97 years cut-off scores indicative of impairment
Agnosia	Body scheme	Neglect	Constructional skills	Apraxia	Acalculia	Functional skills
≤ 22	≤ 20	≤ 10	≤ 46	≤ 8	≤ 2	≤ 9

Time:

The OT-APST can be administered within 20 to 25 minutes (Cooke, McKenna, & Fleming, 2005a).

Subscales:

The OT-APST has 7 subscales: Agnosia; Visuospatial Relations – Unilateral neglect and Body scheme -; Constructional skills; Apraxia; Acalculia; Functional Skills.

Equipment:

All of the equipment required for completion of the OT-APST is provided in the assessment kit that can be purchased online at: http://www.functionforlife.com.au/images/OT-APSTorderformA4.pdf

Training:

No formal training is required for the OT-APST. The manual provides standard directions for the administration and scoring of the measure.

Alternative forms of the OT-APST

None.

Client suitability

Can be used with:

Patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

Should not be used with:

Completion of the OT-APST requires adequate comprehension of simple verbal instructions.
It is not suitable for individuals with severe auditory comprehension problems.
It is not suitable for individuals who are unable to use either hand for task completion.
The OT-APST is not suitable for individuals whose level of arousal or attentional capacity precludes participationAs defined by the International Classification of Functioning, Disability and Health, participation is an individual’s involvement in life situations in relation to health conditions, body functions or structures, activities, and contextual factors. Participation restrictions are problems an individual may have in the manner or extent of involvement in life situations. for the necessary time required for task completion.
The OT-APST is not suitable for individuals with receptive language problems, however, alternative methods of test administration are included for clients with expressive language problems.

In what languages is the measure available?

To our knowledge based on a review of the scientific literature, the OT-APST has not been formally translated and validated in other languages.

Summary

What does the tool measure?	Impairment in visual perception
What types of clients can the tool be used for?	Patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	ScreeningTesting for disease in people without symptoms.
Time to administer	The OT-APST takes 20-25 minutes to administer.
Versions	There are no alternative versions of the OT-APST.
Other Languages	The OT-APST has not been formally translated and validated into other languages.
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: Only one study has examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the OT-APST and reported adequate to excellent levels of internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.. Test-rest: Only one study has examined the test-rest reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest . of the OT-APST, and reported excellent test-retest. Inter-rater: Only one study has examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. of the OT-APST and reported adequate to excellent inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. . Intra-rater: Only one study has examined the intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings. of the OT-APST and reported adequate to excellent intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings. .
ValidityThe degree to which an assessment measures what it is supposed to measure.	Criterion: Concurrent: Poor to excellent correlations with the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) and the LOTCA-Geriatric version (LOTCA-G) have been reported. Construct: Convergent: Poor to excellent correlations were reported between each OT-ASPT subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society). and the corresponding Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) or LOTCA-Geriatric version (LOTCA-G) subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society). . Adequate to excellent correlations were reported between the OT-APST and the LOTCA to patients under 70 years of age and between the OT-APST and on the LOTCA-G to patients over 70 years of age. Poor to adequate correlations were reported between OT-ASPT and the Functional Independence Measure (FIM). Known groups: L’OT-APST can differentiate between patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. and healthy controls.
Does the tool detect change in patients?	Not Applicable.
Acceptability	The OT-APST is not suitable for individuals: With severe auditory comprehension problems Who are unable to use either hand for task completion, Whose level of arousal/attentional capacity precludes participationAs defined by the International Classification of Functioning, Disability and Health, participation is an individual’s involvement in life situations in relation to health conditions, body functions or structures, activities, and contextual factors. Participation restrictions are problems an individual may have in the manner or extent of involvement in life situations. for the necessary time required for task completion With receptive language problems Alternative methods of test administration are provided for clients with expressive language problems.
Feasibility	The OT-APST takes 20-25 minutes to complete and does not require any formal training. The measure is simple to score and all required materials and instructions for administration are provided in the assessment kit that can be purchased. The OT-APST is simple to score.
How to obtain the tool?	The OT-APST Assessment Kit and forms can be purchased from OT Australia Qld by visiting the following website: http://www.otqld.org.au/docs/2008%20OT-APST%20order%20form.pdf or by visiting http://www.functionforlife.com.au/#research.

Psychometric Properties

Overview

We conducted a literature search to identify all relevant publications on the psychometric properties of the OT-APST. To our knowledge, the creators of the OT-APST have personally gathered the majority of psychometric data that are currently published on the scale.

Reliability

Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.:
Cooke, McKenna, Fleming, Darnell (2006a) examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the OT-APST subscales Agnosia, Body scheme, Neglect, and Constructional skills in 208 participants with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. The internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. alpha coefficients for these subscales ranged between 0.71 and 0.83, indicating adequate to excellent levels of internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency..

Test-retest:
Cooke, McKenna, Fleming, and Darnell (2005b) examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
of the OT-APST by having one rater administer and score the OT-APST to 15 patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. and videotape their performance. The same rater then assessed the same patients at the same time of day, 2 weeks later. On 14 of the 25 items, there was 100% agreement between the first and second assessments. For the 11 remaining items, the percentage agreement ranged between 70% and 90%, with the exception of one item, Two-dimensional Constructional skills, which had only 20% agreement. Intraclass correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients (ICCs) were calculated for 6 of the 13 items, where the percentage agreement was less than 100% and the scale of the item was appropriate for this form of correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
calculation. The ICCs were considered excellent, ranging from 0.76 to 0.95.

Intra-rater:
Cooke, McKenna, Fleming, and Darnell (2005b) examined the intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings.
of the OT-APST by having 9 occupational therapy raters each score 5 patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. from video recordings. Following a time delay of 2 weeks, all of the raters viewed the videos again and scored the same five participants in a different randomized order. On 12 of the 25 items, there was 100% agreement between the scores for the first and second video scoring sessions. On the other 13 items, the percentage agreement for raters between the first and second scoring sessions ranged from 83% to 98%. ICCs ranged from adequate (ICC = 0.64) to excellent (ICC = 1.0).

Inter-rater:
Cooke, McKenna, Fleming, and Darnell (2005b) examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept.
of the OT-APST by having one rater administer and score the OT-APST to 15 patients with stroke and videotape their performance. Nine raters were then allocated to score 5 patients. Each video was scored by three separate raters and presented in random order to each rater. Intraclass correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases - for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases - for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
coefficients (ICCs) ranged from adequate (ICC = 0.66 for the Clock item) to excellent (ICC = 1.00 for the items Wave left hand-command and copy, and Two-dimensional Constructional skills). On 12 of the 25 items, there was 100% agreement between all raters and the original ratings given by one rater on 15 patients. On the other 13 items, the proportion of agreement between all raters and the original rater ranged between 83% and 99%.

Validity

Criterion:
Concurrent:
Cooke, McKenna, Fleming, Darnell (2006b) examined the concurrent validityTo validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument. See also “gold standard.”
of the OT-APST in 208 patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. by comparing patient performance on the OT-APST with their performance on the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) and the LOTCA-Geriatric version (LOTCA-G). Somer’s d was used to calculate correlations between the measures. The OT-APST was found to have a poor to excellent correlationThe extent to which two or more variables are associated with one another. A correlation can be positive (as one variable increases, the other also increases – for example height and weight typically represent a positive correlation) or negative (as one variable increases, the other decreases – for example as the cost of gasoline goes higher, the number of miles driven decreases. There are a wide variety of methods for measuring correlation including: intraclass correlation coefficients (ICC), the Pearson product-moment correlation coefficient, and the Spearman rank-order correlation.
with the LOTCA (0.27-0.64) and with the LOTCA-G (0.25-0.80).

Construct:
Convergent:
Cooke, McKenna, Fleming, Darnell (2006a) examined the convergent validityA type of validity that is determined by hypothesizing and examining the overlap between two or more tests that presumably measure the same construct. In other words, convergent validity is used to evaluate the degree to which two or more measures that theoretically should be related to each other are, in fact, observed to be related to each other.
of the OT-APST by comparing the subscales of the OT-APST to the corresponding subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
of the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) or the LOTCA-Geriatric version (LOTCA-G) using Spearman’s rho correlations. Statistically significant correlations were found between each subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
of the OT-APST and the corresponding subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
of the LOTCA or LOTCA-G. The highest correlations occurred for the subscales of Constructional skills and Neglect, with more moderate correlations occurring for subscales assessing Agnosia, Body scheme, and Apraxia. Correlations ranged from poor (0.33 for Apraxia) to excellent (0.80 for Constructional skills).

Similar to the study by Cooke et al. (2006a), Itzkovich, Elazar, Averbuch and Katz (2000) examined the convergent validityA type of validity that is determined by hypothesizing and examining the overlap between two or more tests that presumably measure the same construct. In other words, convergent validity is used to evaluate the degree to which two or more measures that theoretically should be related to each other are, in fact, observed to be related to each other.
of the OT-APST with the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) in 208 people with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. who were younger than 70 years of age. Adequate to excellent correlations were found between the OT-APST and on five of the related subscales of the reference tool of the LOTCA (r = 0.36 to r = 0.70).

Elazar, Itzkovich , and Katz (1996) examined the convergent validityA type of validity that is determined by hypothesizing and examining the overlap between two or more tests that presumably measure the same construct. In other words, convergent validity is used to evaluate the degree to which two or more measures that theoretically should be related to each other are, in fact, observed to be related to each other.
of the OT-APST with the Loewenstein Occupational Therapy Cognitive Assessment-Geriatric (LOTCA-G) version for those 70 years and over and found adequate to excellent correlations between the performance of elderly patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. on the OT-APST and on the LOTCA-G (r = 0.33 to r = 0.80).

Cooke, McKenna, Fleming, Darnell (2006a) examined whether the OT-APST correlated with the Functional Independence Measure (FIM). Significant correlations were observed between six of the seven OT-APST subscales and FIM motor scores. Spearman’s correlations ranged from poor to adequate (r = 0.26 to r = 0.41). Significant correlations were found between all seven OT-APST subscales and the FIM cognitive scores. Spearman’s correlations were adequate (r = 0.36 to r = 0.50). Significant negative correlations were also observed between the time taken by participants to complete the OT-APST and both FIM scores, indicating that more severe functional disability was associated with greater length of time to complete the OT-APST. Body scheme was the only OT-APST subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
score not significantly correlated with FIM motor scores.

Known groups:
Cooke, McKenna, Fleming, Darnell (2006a) examined whether the OT-APST was able to discriminate between patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. and healthy control participants. The patients with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. performed significantly worse than the healthy participants on all 7 subscales of the OT-APST, and took significantly longer to complete the test. All t-tests and Mann-Whitney U tests comparing the two groups were significant at p < 0.001. The OT-APST was found to correctly predict membership of the healthy group for 94.1% of the healthy participants, and correctly predicted membership of the strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. participant group for 56.7% of participants with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. The inverse figures for these comparisons are that 5.9% of the healthy participants were predicted to be members of the strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. group (predicted to have perceptual impairments), and 43.3% of participants following strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. were predicted to be members of the healthy group (no perceptual impairments).

Responsiveness

Not applicable.

References

Cooke, D. (1993). Development and standardization of an apraxia assessment and perceptual screening test for the elderly. In: Australian Association of Occupational Therapists 17th National Conference Proceedings, 1993.
Cooke, D. M., McKenna, K., Fleming, J. (2005a). Development of a standardized occupational therapy screening tool for visual perception in adults. Scandinavian Journal of Occupational Therapy, 12, 59-71.
Cooke, D. M., McKenna, K., Fleming, J., Darnell, R. (2005b). The reliability of the Occupational Therapy Adult Perceptual Screening Test (OT-APST). British Journal of Occupational Therapy, 68(11), 509-517.
Cooke, D. M., McKenna, K., Fleming, J., Darnell, R. (2006a). Construct and ecological validity of the Occupational Therapy Adult Perceptual Screening Test (OT-APST). Scandinavian Journal of Occupational Therapy. 13, 49- 61.
Cooke, D. M., McKenna, K., Fleming, J., Darnell, R. (2006b). Criterion validity of the Occupational Therapy Adult Perceptual Screening Test (OT-APST). Scandinavian Journal of Occupational Therapy. 13, 38-48.
Cooke, D. M., McKenna, K., Fleming, J., Darnell, R. (2006c). Australian normative data for the Occupational Therapy Adult Perceptual Screening Test. Australian Occupational Therapy, 53, 325-336.
Itzkovich, M., Elazar, B., Averbuch, S., Katz, N.(2000). LOTCA manual (2nd ed.). Pequannock, NJ: Maddak Inc.
Elazar, B., Itzkovich, M., Katz, N. (1996).Geriatric version: Loewenstein Occupational Therapy Cognitive Assessment (LOTCA-G) battery. Pequannock, NJ: Maddak Inc.

See the measure

How to obtain the OT-APST?

The OT-APST Assessment Kit and forms can be purchased from OT Australia Qld by visiting the following website: http://www.functionforlife.com.au/.

Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation

Evidence Reviewed as of before: 19-08-2008

Author(s)*: Lisa Zeltzer, MSc OT

Editor(s): Nicol Korner-Bitensky, PhD O; Elissa Sitcoff, BA BSc

Purpose

The Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation was designed to assist in the detection of perceptual impairment in adults who have experienced brain damage caused by traumatic brain injury or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain.. The OSOT assesses perceptual dysfunction in areas related to basic living skills. The measure is also used to determine the degree of impairment, to monitor change, and to measure the effects of treatment and/or spontaneous recovery. The OSOT has been standardized for use with individuals aged 40-69 years.

In-Depth Review

Purpose of the measure

The Ontario Society of Occupational Therapists (OSOT) Perceptual Evaluation was designed to assist in the detection of perceptual impairment in adults who have experienced brain damage caused by traumatic brain injury or strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. The OSOT assesses perceptual dysfunction in areas related to basic living skills. The measure is also used to determine the degree of impairment, to monitor change, and to measure the effects of treatment and/or spontaneous recovery. The OSOT has been standardized for use with individuals aged 40-69 years.

Available versions

The OSOT was created in 1972, by a study group formed in Toronto to discuss perceptual dysfunction in adults with brain damage. However, Boys, Fisher, Holzberg, and Reid (1988) were the first to standardize the OSOT.

Features of the measure

Items:

The original OSOT:

This evaluation consists of 28 subtests that evaluate 6 domains:

(Source: Boys et al., 1988)

Functional Area	Test Item	Scoring
Sensation	Stereognosis Eight common objects identified by touch	4 = 8/8 correct 3 = 5-7/8 correct 2 = 2-4/8 correct 1 = 0-1/8 correct
Scanning	Scanning Cancellation task (total possible cancellations = 105)	4 = 0-3 errors 3 = 4-10 errors 2 = 11-25 errors 1 = 26 or more errors
Apraxia	Motor Planning Manipulate 3 wire and grommet devices	4 = 30 seconds (s) or less to complete 3 tasks 3 = 31-60s 2 = 61-90s 1 = unable to complete in less than 91s
Body awareness	Parts Recognition Identify parts of body	4 = 8/8 correct 3 = 5-7/8 correct 2 = 2-4/8 correct 1 = 0-1/8 correct
Spatial relations	Environmental Copy models of 4 pegboard designs	4 = 4/4 correct 3 = 3/4 correct 2 = 2/4 correct 1 = 0-1/4 correct
Visual agnosia	Shape Recognition Match 9 shapes to form board	4 = 9/9 correct 3 = 6-8/9 correct 2 = 2-5/9 correct 1 = 0-1/9 correct

Note: All test items have been recoded into the 4-point format. Only the test item Tactile suppression, which falls under the Sensation domain, is scored in a bivariate form of either 4 = present or 1 = absent.

The revised OSOT:

This evaluation consists of 18 subtests that evaluate 6 domains:

Sensation
Scanning
Apraxia
Body awareness
Spatial relations
Visual agnosia

In order to give the patient precise instructions, the evaluator must state the instructions verbatim prior to administering the OSOT. These instructions can only be repeated once. The evaluator must record all patient responses before offering help.

Scoring:

The OSOT uses a 5-point Likert scaleLikert scaling is one type of response to items in a questionnaire or tool. For example, Likert scaling would have you rate an item such as “I am satisfied with the care I received” on a scale using a 1-to-5 response scale where:
• 1 = strongly disagree
• 2 = disagree
• 3 = undecided
• 4 = agree
• 5 = strongly agree
You will find various options and scaling methods for the number of response choices (1-to-7, 1-to-9, 0-to-4). Odd-numbered scales usually have a middle value that is labelled Neutral or Undecided. Some tools used forced-choice Likert scaling with an even number of responses and no middle neutral or undecided choice. for each of the subtests, ranging from 0 = an inability to do what is asked of the patient, to 4 = normal performance. The scores obtained for each task are added to establish a total score. In the original OSOT, the maximum score that can be obtained is 112. Below is a breakdown of the scores. Each interval corresponds to a degree of severity of the global perceptual impairment:

(Source: Boys et al., 1988)

Score	Severity of impairment
110-112	Normal performance
101-109	Borderline case, requires additional testing
91-100	Mild impairment
81-90	Moderate impairment
80 or below	Severe impairment

In the revised OSOT, the maximum score that can be obtained is 72. Below is a breakdown of the scores. Each interval corresponds to a degree of severity of the global perceptual impairment:

(Source: Boys et al., 1991)

Score	Severity of impairment
70-72	Normal performance
61-69	Borderline case, requires additional testing
51-60	Mild impairment
41-50	Moderate impairment
40 or below	Severe impairment

Time:

There is no information published regarding the time it takes to complete the original OSOT, but it is anticipated that it would take longer than the revised OSOT which takes approximately 90 minutes to complete (Tremblay, Savard, Casimiro, & Tremblay, 2004).

Subscales:

The OSOT has 6 subscales or ‘domains’: Sensation; Scanning; Apraxia; Body awareness; Spatial relations; Visual agnosia.

Equipment:

Instruction manual
Evaluation material
Pencil
Chronometer
5″x11″ piece of paper
Subtests and equipment for each subtest (e.g. a clock, 6 circles out of various sized cardboard, etc).

Training:

The OSOT was created to be used by occupational therapists. An adequate understanding of the instructions and procedures is needed before using the OSOT.

Alternative forms of the OSOT

The revised version of the OSOT (Boys, Fisher, & Holzberg, 1991).

In the revised version of the OSOT, 10 subtests have been eliminated from the 28 found in the original version of the measure.

Client suitability

Can be used with:

Patients with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..

Should not be used with:

The OSOT has not been examined for use for patients with An alternative test is the Motor-Free Visual Perception Test, which can sometimes be used to examine the presence of visual perception impairments in patients with expressive aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury.
An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) if they are able to understand simple verbal or non-verbal instructions and the various subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society).
requirements.

In what languages is the measure available?

Validated in French (Desrosiers, Mercier, & Rochette, 1999).

Summary

What does the tool measure?	Perceptual impairment in adults (Sensation, Scanning, Apraxia, Body awareness, Spatial relations, Visual agnosia).
What types of clients can the tool be used for?	Patients who have experienced brain damage caused by traumatic brain injury or strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	Assessment
Time to administer	The revised OSOT takes approximately 90 minutes to complete. There is no information published regarding the time it takes to complete the original OSOT.
Versions	Original OSOT (consists of 28 subtests); Revised OSOT (consists of 18 subtests).
Other Languages	English and French
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: Only one study has examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the original OSOT and found alpha coefficients ranging from poor to excellent. Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: Only one study has examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the revised OSOT and reported excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.. Test-retest: No studies have examined the test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society). of the OSOT. Inter-rater: One study examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. of the original OSOT and reported very high agreement (93.1%) across items for all subjects.
ValidityThe degree to which an assessment measures what it is supposed to measure.	Criterion: Concurrent: One study reported that the revised OSOT correlated adequately with the Physical Self-Maintenance Scale (PSMS), the Instrumental ActivitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function. of Daily Living Scale, and with the Mini-Mental State Examination (MMSE). Construct: Known groups: Both the original OSOT and the revised OSOT have been shown to discriminate between patients with acquired brain injury and healthy controls.
Floor/Ceiling Effects	No studies have examined the floor or ceiling effects of the OT-APST.
Does the tool detect change in patients?	Not Applicable.
Acceptability	A proxy respondent is not appropriate for this performance-based measure. The OSOT has not been examined for use for patients with aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person’s ability to communicate. It is most often the result of stroke or head injury. An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person’s intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada). An alternative test is the Motor-Free Visual Perception Test, which can sometimes be used to examine the presence of visual perception impairments in patients with expressive aphasiaAphasia is an acquired disorder caused by an injury to the brain and affects a person's ability to communicate. It is most often the result of stroke or head injury. An individual with aphasia may experience difficulty expressing themselves when speaking, difficulty understanding the speech of others, and difficulty reading and writing. Sadly, aphasia can mask a person's intelligence and ability to communicate feelings, thoughts and emotions. (The Aphasia Institute, Canada) if they are able to understand simple verbal or non-verbal instructions and the various subscaleMany measurement instruments are multidimensional and are designed to measure more than one construct or more than one domain of a single construct. In such instances subscales can be constructed in which the various items from a scale are grouped into subscales. Although a subscale could consist of a single item, in most cases subscales consist of multiple individual items that have been combined into a composite score (National Multiple Sclerosis Society). requirements.
Feasibility	The OSOT is a lengthy measure to administer and requires an adequate understanding of the test procedures and instructions on the part of the examiner. A number of items are required as equipment for the OSOT, however all items are readily available. The OSOT is simple to score and uses a 5-point Likert scaleLikert scaling is one type of response to items in a questionnaire or tool. For example, Likert scaling would have you rate an item such as “I am satisfied with the care I received” on a scale using a 1-to-5 response scale where: • 1 = strongly disagree • 2 = disagree • 3 = undecided • 4 = agree • 5 = strongly agree You will find various options and scaling methods for the number of response choices (1-to-7, 1-to-9, 0-to-4). Odd-numbered scales usually have a middle value that is labelled Neutral or Undecided. Some tools used forced-choice Likert scaling with an even number of responses and no middle neutral or undecided choice. for each of the subtests. A breakdown of the total score is provided and each interval corresponds to a degree of severity of the global perceptual impairment (e.g. original OSOT: 80 or below represents severe impairment; revised OSOT: 40 or below represents severe impairment).
How to obtain the tool?	The OSOT can be purchased from the publisher: Nelson, A Thomson Company 1120 Birchmount Road Toronto, Ontario M1K 5G4, Canada E-mail: inquire@nelson.com Website: http://www.assess.nelson.com/pdf/assessmentcatalogue/cn-19.pdf

Psychometric Properties

Overview

The OSOT has not been well studied. To our knowledge, the creators of the OSOT have personally gathered the majority of psychometric data that are currently published on the scale. The psychometric properties of the original version of the OSOT were based on a study by Boys et al. (1988) of 80 patients with brain damage (experienced a strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain., tumor, normopressure hydrocephalus, or anoxia), and a group of 70 neurologically intact participants. The psychometric properties of the revised version of the OSOT were based on a study by Boys et al. in 1991.

Further investigation on the reliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of "true" variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to "noise" in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .
, validity, and sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also "Specificity."
of the OSOT is required with larger numbers of subjects. For the purposes of this review, we conducted a literature search to identify all relevant publications on the psychometric properties of the OSOT.

Reliability

Revised OSOT:
Boys et al. (1991) examined the internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the revised OSOT and reported that this version of the OSOT has an excellent internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. (Cronbachs alpha = 0.90).

Inter-rater:
Original OSOT:
Boys et al. (1988) examined the inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept.
of the OSOT by having both the attending therapist and one of the investigators score the performance of 46 patients independently, on separate score sheets. Agreement for this study was very high (93.1%) across items for all subjects.

Validity

Criterion:

Concurrent:
Boyd and Dawson (2000) examined the relationship between perceptual impairment and independence in basic and instrumental activitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of daily living in a sample of older adults living in an institution or in the community. In this study, the OSOT correlated adequately with the Physical Self-Maintenance Scale (PSMS) (Lawton, & Brody, 1969) (r = 0.44), the Instrumental ActivitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of Daily Living Scale (r = 0.44), and with the Mini-Mental State Evaluation (MMSE) (Folstein, Folstein, & McHugh, 1975) (r = 0.43). The results of this study suggest that perceptual impairment is related to activitiesAs defined by the International Classification of Functioning, Disability and Health, activity is the performance of a task or action by an individual. Activity limitations are difficulties in performance of activities. These are also referred to as function.
of daily living status.

Construct:

Boys et al. (1988) reported that the moderate correlations observed between scores obtained for each of the 6 domains of the original OSOT demonstrates that each domain measures different concepts, which together give a global perceptual deficit score.

Known groups:
Both the original OSOT and the revised OSOT have been shown to discriminate between patients with acquired brain injury and healthy controls (Boys et al., 1991; Boys et al.,1988). When the performance of patients and control participants were compared in the study by Boys et al. (1988), statistically significant differences were observed for all tests except for Ideational Apraxia.

Sensitivity and specificity:

Boys et al. (1988) reported that the total score of the original version of the OSOT can differentiate between individuals with neurological impairment from neurologically normal control participants. At a total score cutoff of 110, the OSOT can differentiate between individuals with functional impairment and a control group without neurological impairment with a sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of 100% and a specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of only 40%. At a cutoff of 100, the sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was 63.7% and the specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was 100%.

Boys et al. (1991) reported that the total score of the revised version of the OSOT can also differentiate between individuals with brain damage from neurologically normal control participants. The sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was 100% with a cutoff score of 70 and over, and of 58% with a cutoff of 60. The specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was 40% with a cutoff score of 70 and under, and 100% with a cutoff of 60.

References

Boyd, A., Dawson, D. R. (2000). The relationship between perceptual impairment and self-care status in a sample of elderly persons. Physical & Occupational Therapy in Geriatrics, 17(4), 1-16.
Boys, M., Fisher, P., & Holzberg, C. (1991). The OSOT Perceptual Evaluation Manual: Revised. Scarborough, Ont.: Nelson Canada.
Boys, M., Fisher, P., Holzberg, C., & Reid, D. (1988). The OSOT Perceptual Evaluation: A research perspective. American Journal of Occupational Therapy. 42, 92-98.
Desrosiers, J.,Mercier, L.,Rochette, A. (1999).Test-retest and inter-rater reliability of the French version of the Ontario Society of Occupational Therapy (OSOT)Perceptual Evaluation. Can J Occup Therapy, 66(3), 134 -139.
Folstein, M., Folstein, S., McHugh, P. (1975). Mini-mental State: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12,189-198.
Lawton, M. P., Brody, E. M. (1969). Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist, 9, 179-186.
Temblay, L. E., Savard, J., Casimiro, L., Tremblay, M. (2004). Répertoire des Outils d’Évaluation en Français pour la Réadaptation (pp406-408). Ottawa, ON: Regroupement des intervenantes et intervenants francophones en santé et en services sociaux de l’Ontario: Université d’Ottawa: CFORP.

See the measure

The OSOT can be purchased from the publisher:

Publisher address:
Nelson, A Thomson Company
1120 Birchmount Road,
Toronto, Ontario M1K 5G4, Canada
Telephone: 416-752-9448 or 1 800 268-2222
Fax: 416-752-8101 or 1 800 430-4445
E-mail: inquire@nelson.com
Website: http://www.assess.nelson.com/pdf/assessmentcatalogue/cn-19.pdf

Cost: (price is subject to change)
Perceptual evaluation kit (version 2, with English manual): $859
English manual: $75
French manual: $31.80

Visual Impairment Screening Assessment (VISA)

Evidence Reviewed as of before: 20-01-2023

Author(s)*: Annabel McDermott, OT

Editor(s): Annie Rochette

Expert Reviewer: Fiona Rowe

Purpose

The Visual Impairment Screening Assessment (VISA) is designed to identify visual impairment following strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., to allow referral for specialist visual assessment. The VISA was developed by the VISION research unit, University of Liverpool.

In-Depth Review

Purpose of the measure

The Visual Impairment ScreeningTesting for disease in people without symptoms.
Assessment (VISA) is designed to identify visual impairment following strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. The VISA screens for common visual impairments following strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. including impaired central vision, eye movement problems, visual field deficits and visual inattention. The VISA can be used to detect ocular signs separate from reporting of vision symptoms.

Available versions

The VISA was developed from a review of strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. and vision research studies, and in collaboration with a panel of strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. specialists and patients, and validated in a clinical study.

The VISA is available in print and as a software app.

Features of the measure

Items:

The VISA comprises five sections:

Case history – to screen for visual symptoms and observed signs – in person or by proxy.
Visual acuity – to screen central vision at near (33cm) and distance (3m) using LogMAR (Logarithm of the Minimum Angle of Resolution) or Grating acuity; monocular or binocular depending on the ability of the patient.
Ocular alignment and movement – to screen presence/absence of strabismus and eye movement problems.
Visual field* – to screen peripheral and central field of vision by a guided confrontation method.
Visual perception – to screen for visual inattention/neglect using (i) line bisection task, (ii) cancellation task and (iii) clock drawing assessment.

*Visual field assessment – print version: confrontation follows a typical method with the clinician seated directly opposite the patient at a distance of 1m and following stages that involve the patient indicating when a 10mm red target is seen in the periphery of their vision, finger counting in each quadrant of the visual field and comparison of examiner facial features.

*Visual field assessment – app version: a kinetic visual field assessment is undertaken at a test distance of 30cm and a screen width of 24.6cm, allowing an assessment of the 40degree visual field. The patient is asked to fixate a static fixation point in the corner of the screen while a stimulus moves from the other edges. They are asked to tap the tablet screen when the stimulus is seen. This is repeated with the fixation target positioned at all four corners of the screen.

Scoring:

Administration of the VISA screeningTesting for disease in people without symptoms.
tool does not result in a score. Rather, the tool serves as a guide for referral for specialist visual assessment as per observations outlined in the VISA Instructions for Use booklet.

What to consider before beginning:

The VISA screens for common forms of visual impairment that occur from brain injury but does not screen for all possible visual impairments. As such a negative screen does not rule out the presence of visual impairment.

The individual may not be able to complete all sections of the VISA at one time (e.g. due to fatigue, cognitive difficulties, communication difficulties). In this instance the VISA can be completed over several visits.

The individual is permitted to wear glasses (if required) for some assessment items.

The VISA must be performed in good lighting conditions.

Time:

The VISA takes approximately 10 minutes to administer, but longer if multiple visual problems and if associated cognitive issues.

Equipment:

Equipment is outlined in the VISA Instructions for Use booklet and includes:

Pen torch
Occlusive tape
10mm red target
+3.00 power reading glasses
3 metre string/tape measure
Matching card for visual acuity check
Visual attention worksheets
Pencil
VISA recording sheet

Client suitability

Can be used with:

Individuals with acute strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.

Individuals with community/cognitive difficulties who are unable to comply with any letter test – The assessor can use a grating chart that uses a preferential looking technique.

StrokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. patients have reported that it is easier to respond using the touch screen (VISA app) than traditional pen and paper tasks when using their non-dominant hand.

Should not be used with:

The VISA may not be completed in full due to cognitive difficulty or fatigue. Information regarding the individual’s vision history can be gathered from reliable family members. VISA app cannot be used on small devices such as iPad Mini or smartphones.

Languages of the measure

English – print and app
Dutch – print
Norwegian – print

Requests for translations are welcome. The VISA researchers will work closely with translators using the WHO-recommended translation process.

Summary

What does the tool measure?	Visual impairment
What types of clients can the tool be used for?	The Visual Impairment Screening Assessment can be used with individuals with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain..
Is this a screeningTesting for disease in people without symptoms. or assessment tool?	ScreeningTesting for disease in people without symptoms.
Time to administer	10 minutes – may be longer if multiple visual problems
ICF Domain	Impairment
Versions	There is a print version and an app version of the VISA.
Languages	English Dutch Norwegian
Measurement Properties
ReliabilityReliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of “true” variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to “noise” in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal consistency , inter-rater agreement , intra-rater agreement , and test-retest .	Internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency.: No studies have reported on internal consistencyA method of measuring reliability . Internal consistency reflects the extent to which items of a test measure various aspects of the same characteristic and nothing else. Internal consistency coefficients can take on values from 0 to 1. Higher values represent higher levels of internal consistency. of the VISA. Test-retest: No studies have reported on test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society). of the VISA. Intra-rater: No studies have reported on intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings. of the VISA. Inter-rater: Two studies reported substantial agreement on inter-rater reliabilityA method of measuring reliability . Inter-rater reliability determines the extent to which two or more raters obtain the same result when using the same instrument to measure a concept. evaluation of the VISA.
ValidityThe degree to which an assessment measures what it is supposed to measure.	Content: Pilot validation of the VISA was conducted in collaboration with medical students (naïve screeners) and orthoptists. Criterion: Concurrent: No studies have reported on concurrent validityTo validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument. See also “gold standard.” of the VISA. Predictive: No studies have reported on predictive validityA form of criterion validity that examines a measure’s ability to predict some subsequent event. Example: can the Berg Balance Scale predict falls over the following 6 weeks? The criterion standard in this example would be whether the patient fell over the next 6 weeks. of the VISA. Construct: Convergent/Discriminant: Two studies reported poor to substantial test component agreement between the VISA screen and specialist vision assessments. One study reported perfect agreement between kinetic visual field test using the VISA app and formal perimetry. Known Groups: No studies have reported on known group validityThe degree to which an assessment measures what it is supposed to measure. of the VISA.
Floor/Ceiling Effects	No studies have reported on floor/ceiling effects of the VISA. Two studies noted false positives and false negatives on individual components of the tool.
Does the tool detect change?	No studies have reported on responsivenessThe ability of an instrument to detect clinically important change over time. of the VISA.
Acceptability	Two studies reported on acceptability of the VISA, from a sample of strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. patients and orthoptists.
Feasibility	The VISA is suitable for administration in various settings. The VISA requires minimal specialist equipment or training. One study noted that the VISA is time-intensive when used in the hyperacute stage with unwell patients.
How to obtain the tool?	The VISA is available in print or as an app (Medicines and Healthcare products Regulatory Agency regulatory approved): https://www.liverpool.ac.uk/population-health/research/groups/vision/visa/? Further information regarding the tool and administration guidelines can be found here: https://youtu.be/-s6i–PfXNY

Psychometric Properties

Overview

The Visual Impairment ScreeningTesting for disease in people without symptoms.
Assessment (VISA) was developed by the VISION Research Unit, University of Liverpool in consultation with an expert panel of stroke-specialist clinical orthoptists, strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain. research orthoptists, stroke-specialist occupational therapists and neuro-ophthalmologists (Rowe et al., 2018). A literature search was conducted to identify all relevant publications on the psychometric properties of the VISA pertinent to use with participants following strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. Two studies were identified.

Floor/Ceiling Effects

Floor/ceiling effects of the VISA have not been measured.

Reliability

Test-retest:
Test-retest reliabilityA way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability (National Multiple Sclerosis Society).
of the VISA has not been measured.

Intra-rater:
Intra-rater reliabilityThis is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater’s subsequent ratings are contaminated by knowledge of earlier ratings.
of the VISA has not been measured.

Inter-rater :
Rowe et al. (2018) examined inter-rater agreement of the VISA in a sample of 116 individuals with stroke, whereby each individual underwent two vision assessments: a specialist vision assessment performed by orthoptists/ophthalmologists (n=5) and the VISA screeningTesting for disease in people without symptoms.
assessment, completed by medical students (n=2) and orthoptists (n=4). Agreement regarding need to make a referral to specialist eye services due to visual impairment was measured using kappa values. Overall agreement was substantial (k=0.736, 95% CI 0.602 to 0.870). As expected, a higher rate of false positives and false negatives were found among screeners naïve to vision testing (n=2 medical students) vs. experienced screeners (n=5 orthoptists/ophthalmologists).

Rowe et al. (2020) examined inter-rater agreement of the VISA in a sample of 221 individuals with stroke, whereby each individual underwent two vision assessments: a specialist vision assessment performed by orthoptists/ophthalmologists and the VISA screeningTesting for disease in people without symptoms.
assessment. The outcome was the presence/absence of visual impairment*. Agreement was substantial for the VISA print (k=0.648, 95% CI 0.424 to 0.872) and for the VISA app (k= 0.690, 95% CI 0.528 to 0.851).

*Presence/absence of visual impairment was defined as one or more of: reduced distance vision <0.2 logMAR, reduced near vision <0.3 logMAR (equivalent to N6), deviated eye position, eye movement abnormality (incomplete eye rotations in any position of gaze), visual field loss, visual inattention with displaced line bisection, <42 score on cancellation task and/or incomplete/displaced clock drawing.

Validity

Content:

Pilot validation of the VISA was conducted in collaboration with medical students (naïve screeners) and orthoptists; independent specialist vison assessment was performed by orthoptists/ophthalmologists. Written and verbal feedback was gathered from screeners and a thematic approach to analysis of qualitative date was used. A modified grounded theory approach was adopted to revise themes iteratively as analyses continued (Rowe et al., 2018).

Criterion:

Concurrent:
Concurrent validityTo validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument. See also “gold standard.”
of the VISA has not been measured.

Predictive:
Predictive validityA form of criterion validity that examines a measure’s ability to predict some subsequent event. Example: can the Berg Balance Scale predict falls over the following 6 weeks? The criterion standard in this example would be whether the patient fell over the next 6 weeks.
of the VISA has not been measured.

Construct

Convergent/Discriminant :
Rowe et al. (2018) examined test component agreement between the VISA screen and specialist vision assessments (visual acuity, ocular alignment and movement, visual fields, visual perception) in a sample of 116 individuals with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain., using kappa values. Agreement of items ranged from poor to substantial:

Near visual acuity (k=0.682, CI 0.543 to 0.820; 10 false negatives, 7 false positives)
Distance visual acuity (k=0.785, CI 0.665 to 0.904; 8 false negatives, 3 false positives)
Ocular alignment (k=0.585, CI 0.221 to 0.949; 4 false negatives, 0 false positives)
Ocular motility (k=0.120, CI -0.071 to 0.311; 21 false negatives, 6 false positives)
Visual fields (k=0.741, CI 0.599 to 0.884; 3 false negatives, 8 false positives)
Visual inattention (k=0.361, CI 0.144 to 0.578; 1 false negative, 16 false positives).

Rowe et al. (2020) examined test component agreement of the VISA print and VISA app screeningTesting for disease in people without symptoms.
assessments with specialist vision assessments (visual acuity, ocular alignment and movement, visual fields, visual perception) in a sample of 221 individuals with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., using kappa values. Agreement of individual components between VISA print and orthoptic vision assessment ranged from poor to moderate:

Near visual acuity (k=0.236, CI 0.045 to 0.427; 23 false negatives, 12 false positives)
Distance visual acuity (k=0.565, CI 0.405 to 0.725; 9 false negatives, 13 false positives)
Ocular alignment (k=0.388, CI 0.110 to 0.667; 5 false negatives, 7 false positives)
Ocular motility (k=0.365, CI 0.181 to 0.553; 10 false negatives, 19 false positives)
Visual fields (k=0.504, CI 0.339 to 0.668; 7 false negatives, 18 false positives)
Visual inattention (k=0.500, CI 0.340 to 0.659; 7 false negative, 21 false positives).

Agreement of individual components between VISA app and orthoptic vision assessment ranged from fair to substantial:

Near visual acuity (k=0.416, CI 00.227 to 0.605; 19 false negatives, 3 false positives)
Distance visual acuity (k=0.783, CI 0.656 to 0.910; 6 false negatives, 4 false positives)
Visual fields (k=0.701, CI 0.564 to 0.838, 3 false negatives, 12 false positives)
Visual inattention (k=0.323, CI 0.108 to 0.538; 6 false negatives, 16 false positives).

Rowe et al. (2020) examined agreement between kinetic visual field test using the VISA app and formal perimetry using the binocular Esterman programme with 25 individuals with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain., using kappa values. There was perfect agreement (k=1.00) between measures.

Known Group:
Known group validityThe degree to which an assessment measures what it is supposed to measure.
of the VISA has not been measured.

Responsiveness

SensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
& SpecificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
:
Rowe et al. (2018) examined sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
and specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of the VISA in a sample of 89 individuals with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. by comparison with a binary assessment of the presence/absence of visual impairment (low vision <0.2 logMAR, visual field loss, eye movement abnormality, visual perception abnormality). SensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was defined as the proportion of patients with visual impairment who are correctly identified by the screener; sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of 90.24% was found. SpecificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was defined as the proportion of patients without visual impairment who were correctly identified by the screener; specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of 85.29% was found. The positive and negative predictive values were 93.67% and 78.36% (respectively).

Rowe et al. (2018) also compared sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
and specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of the VISA when performed by naïve screeners (n=2 medical students) vs. experienced screeners (n=5 orthoptists/ophthalmologists). When used by a naïve screener the VISA screen had a sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of 82.93% and specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of 80.95%; when used by an experienced screener the VISA screen had a sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of 97.56% and specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of 92.31%.

Rowe et al. (2020) examined sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
and specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of the VISA in a sample of 221 individuals with strokeAlso called a "brain attack" and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a "schemic stroke", or the formation of a blood clot in a vessel supplying blood to the brain.. SensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
was estimated as the proportion of patients with visual impairment as diagnosed by the gold-standard clinical examination, which are correctly identified by the screener; sensitivitySensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist in a patient (National Multiple Sclerosis Society). See also “Specificity.”
of the VISA print and VISA app was 97.67% and 88.31% (respectively). SpecificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
was estimated as the proportion of patients without visual impairment that are correctly identified by the screener; specificitySpecificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative).
of the VISA print and VISA app was 60.00% and 86.96% (respectively). The positive and negative predictive values of the VISA print were 93.33% and 81.82%. The positive and negative predictive values of the VISA app were 95.77% and 68.97% (respectively).

Acceptability:
Rowe et al. (2018) examined acceptability of the VISA tool through process evaluation of written feedback and interviews with strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. patients and strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. specialists. Qualitative data regarding number of items, clarity of questions, time and ease of testing was gathered and analysed using a thematic approach, and a modified grounded theory approach was subsequently used to revise themes as interviews and analyses progressed.

Rowe et al. (2020) examined acceptability of the VISA tool through process evaluation of clinician feedback sheets and strokeAlso called a “brain attack” and happens when brain cells die because of inadequate blood flow. 20% of cases are a hemorrhage in the brain caused by a rupture or leakage from a blood vessel. 80% of cases are also know as a “schemic stroke”, or the formation of a blood clot in a vessel supplying blood to the brain. patients interviews. Qualitative feedback regarding duration of assessment, presentation of tests on the VISA app and referral guides were received.

References

Rowe, F.J., Hepworth, L.R., Hanna, K.L., & Howard, C. (2018). Visual Impairment Screening Assessment (VISA) tool: pilot validation. BMJ Open, 8:e020562. doi:10.1136/bmjopen-2017-020562

Rowe, F.J., Hepworth, L., Howard, C., Bruce, A., Smerdon, V., Payne, T., Jimmieson, P., & Burnside, G. (2020). Vision Screening Assessment (VISA) tool: diagnostic accuracy validation of a novel screening tool in detecting visual impairment among stroke survivors. BMJ Open, 10:e033639. doi:10.1136/bmjopen-2019-033639

See the measure

How to obtain the Visual Impairment ScreeningTesting for disease in people without symptoms.
Assessment (VISA)

The VISA is available in print or as an app (Medicines and Healthcare products Regulatory Agency regulatory approved).

The VISA Instructions for Use booklet can be found here: https://www.liverpool.ac.uk/population-health/research/groups/vision/visa/?

The VISA Instructions for Use video can be found here: VISA stroke vision screening video – YouTube