ImPACT Computerized Neurocognitive Concussion Management Aid

DEN150037 · Impact Applications, Inc. · POM · Aug 22, 2016 · Neurology

Device Facts

• Record ID: DEN150037
• Device Name: ImPACT Computerized Neurocognitive Concussion Management Aid
• Applicant: Impact Applications, Inc.
• Product Code: POM · Neurology
• Decision Date: Aug 22, 2016
• Decision: DENG
• Submission Type: Direct
• Regulation: 21 CFR 882.1471
• Device Class: Class 2
• Attributes: Software as a Medical Device, Pediatric

Intended Use

ImPACT: ImPACT is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion. ImPACT is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 12-59. ImPACT Pediatric: ImPACT Pediatric is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion. ImPACT Pediatric is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 5-11.

Device Story

Computer-based neurocognitive test battery; provides objective measures of neurocognitive functioning; used by trained healthcare professionals to aid in concussion management. Input: patient performance on cognitive tasks. Processing: comparison of patient scores against normative database; statistical analysis of cognitive function. Output: reports of cognitive performance; normative data comparisons. Context: clinical setting; used post-concussive event. Benefit: provides objective data to assist clinicians in management decisions; does not replace clinical judgment or provide definitive diagnosis.

Clinical Evidence

No specific clinical data results provided in the document. The FDA order mandates that future submissions include clinical performance data evaluating device output, clinical interpretation, test-retest reliability, construct validity, and a description of the normative database construction (including inclusion/exclusion criteria and statistical methods).

Technological Characteristics

Computer-based software neurocognitive test battery. ImPACT (ages 12-59) and ImPACT Pediatric (ages 5-11). Operates on standard computing hardware (e.g., iPad 2 or above for Pediatric). Software-based assessment; rule-based validity index and RCI calculations. Data stored on secure HIPAA-compliant servers. Moderate level of concern software.

Indications for Use

Indicated for use as a neurocognitive test battery to aid in the assessment and management of concussion in individuals ages 5-59. ImPACT is for ages 12-59; ImPACT Pediatric is for ages 5-11. Not for use as a stand-alone diagnostic device.

Regulatory Classification

Identification

The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.

Special Controls

*Classification.* Class II (special controls). The special controls for this device are:(1) Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function, must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed. (2) Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must: (i) Evaluate device output and clinical interpretation. (ii) Evaluate device test-retest reliability of the device output. (iii) Evaluate construct validity of the device cognitive assessments. (iv) Describe the construction of the normative database, which includes the following: (A) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria. (B) Statistical methods and model assumptions used. (3) The labeling must include: (i) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following: (A) Device output and clinical interpretation. (B) Device test-retest reliability of the device output. (C) Construct validity of the device cognitive assessments. (D) A description of the normative database, which includes the following: ( *1* ) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.( *2* ) How normal values will be reported to the user.( *3* ) Representative screen shots and reports that will be generated to provide the user results and normative data.( *4* ) Statistical methods and model assumptions used.( *5* ) Whether or not the normative database was adjusted due to differences in age and gender.(ii) A warning that the device should only be used by health care professionals who are trained in concussion management. (iii) A warning that the device does not identify the presence or absence of concussion or other clinical diagnoses. (iv) A warning that the device is not a stand-alone diagnostic. (v) Any instructions technicians must convey to patients regarding the administration of the test and collection of cognitive test data.

In combination with the general controls of the FD&C Act, the Computerized Cognitive Assessment Aid for Concussion is subject to the following special controls:

Submission Summary (Full Text)

{0}------------------------------------------------

# DE NOVO CLASSIFICATION REQUEST FOR IMPACT AND IMPACT PEDIATRIC

# REGULATORY INFORMATION

FDA identifies this generic type of device as:

Computerized Cognitive Assessment Aid for Concussion. The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.

NEW REGULATION NUMBER: 882.1471

CLASSIFICATION: CLASS II

PRODUCT CODE: POM

### BACKGROUND

DEVICE NAME: IMPACT AND IMPACT PEDIATRIC

SUBMISSION NUMBER: DEN 150037

DATE OF DE NOVO: AUGUST 11, 2015

CONTACT: IMPACT APPLICATIONS, INC. 2000 TECHNOLOGY DRIVE, SUITE 150 PITTSBURGH, PA 15219

### INDICATIONS FOR USE

### ImPACT:

ImPACT is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

ImPACT is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 12-59.

{1}------------------------------------------------

# ImPACT Pediatric:

ImPACT Pediatric is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

ImPACT Pediatric is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 5-11.

# LIMITATIONS

For prescription use only.

The safety and effectiveness of ImPACT for individuals under the age of 12 years and over the age of 59 years has not been established. The safety and effectiveness of ImPACT Pediatric for individuals under the age of 5 years and over the age of 11 years has not been established.

ImPACT and ImPACT Pediatric are intended be used by medical professionals qualified to interpret the results of a concussion assessment examination and aid in the management of concussion.

The devices are not intended to be used as a stand-alone diagnostic device.

# PLEASE REFER TO THE LABELING FOR A MORE COMPLETE LIST OF WARNINGS, PRECAUTIONS AND CONTRAINDICATIONS.

### DEVICE DESCRIPTION

ImPACT® (Immediate Post-Concussion Assessment and Cognitive Testing) and ImPACT Pediatric are computer-based neurocognitive test batteries for use as an assessment aid in the management of concussion.

ImPACT and ImPACT Pediatric are software-based tools that allows healthcare professionals to conduct a series of neurocognitive tests that provide data related to the neurocognitive functioning of the test taker. This computerized neurocognitive test battery measures various aspects of neurocognitive functioning including reaction time, memory, attention, and spatial processing speed. It also records symptoms of concussion in the test taker.

ImPACT and ImPACT Pediatric provide healthcare professionals with a set of well-developed and researched neurocognitive tasks that have been medically accepted as state-of-the-art best practices. The devices are intended to be used as part of a multidisciplinary approach to concussion assessment and patient management.

{2}------------------------------------------------

# ImPACT

ImPACT (ages 12-59) is administered using the following sequence:

- 1. Demographics
- 2. Symptom Scale
- 3. Word Memory
- 4. Design Memory
- 5. X's and O's
- 6. Symbol Match
- 7. Color Match
- 8. Three Letters

An example of one of the tests, the Symbol Match, is shown below in Figure 1. This particular test evaluates processing speed, learning, and memory.

Figure 1: Example Symbol Match Task

Upon successful completion of the test battery, ImPACT generates a report with the following information:

Demographic Information

- Background Information and Native Language
- Education and Special Needs as reported by test taker ●
- Concussion and Sport Background ●
- . Medical Information as reported by test taker

# ImPACT Composite Scores

- Verbal Memory Composite, Visual Memory Composite, Visual Motor Speed ● Composite, Reaction Time Composite, Impulse Control Composite, Total Symptom Score, Cognitive Efficiency Index (CEI)

{3}------------------------------------------------

Test Battery Modules

- Word Memory, Design Memory, X's and O's, Symbol Match, Color Match, Three Letters
Post-Concussion Symptom Checklist

- . Individual Symptom Scores
Graphic Representation of Composite Scores and Symptoms

- . Graphs depicting: Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction Time Composite Scores, Total Symptom Score
Table 1 shows an example test score report, which identifies the overall composite score table for each cognitive area tested.

| Composite Scores | Percentile scores if available are listed in small type | | | |
|----------------------------|---------------------------------------------------------|-----|-------|-----|
| Memory composite (verbal) | 82 | 49% | 57 | <1% |
| Memory composite (visual) | 78 | 60% | 48 | 2% |
| Visual motor speed | 32.83 | 32% | 22.78 | <1% |
| Reaction time composite | 0.61 | 46% | 1.12 | <1% |
| Impulse control composite | 3 | | 4 | |
| Total Symptom Score | 0 | | 38 | |
| Cognitive Efficiency Index | 0.33 | | 0.17 | |

Table 1: Example ImPACT Score Report

The Reliable Change Index (RCI) methodology allows the clinician to reduce the adverse impact of measurement error on test interpretation. To represent clinically significant improvement, the change score should be statistically reliable. However, the converse is not true; a statistically reliable change does not necessarily guarantee a clinically meaningful change. It is also important to emphasize that an RCI score does not provide a diagnosis. ImPACT shows the score in red if it surpasses the RCI as shown in the figure above. Test score change over repeated administrations is to be expected. The issue for healthcare professionals is to determine when this change is significant and clinically meaningful. ImPACT provides RCIs for each Module and Composite. The table also shows percentile scores to the right of the composite score. These percentile scores indicate the patient's performance compared to the age- and gender- matched normative database.

The Cognitive Efficiency Index (CEI) measures the interaction between accuracy (percentage correct) and speed (reaction time) in seconds on the Symbol Match test. This score was not developed to make return-to-activity decisions but it can be helpful in determining the extent to which the individual tried to work very fast on symbol match (decreasing accuracy) or attempted to improve their accuracy by taking a more deliberate and slow approach (jeopardizing speed). A higher score indicates that the individual did well in both the speed and memory domains on the Symbol Match test. A low score (below .20) means that the individual performed poorly on both

{4}------------------------------------------------

the speed and accuracy component. If this score is a negative number, the individual performed very poorly on the reaction time component.

ImPACT also provides a validity index designed to aid in identifying invalid baseline examinations. This index is based on the following algorithm:

- X's and O's Total Incorrect > 30 OR
- Impulse Control Composite >30 OR
- Word Memory Learning Pct Correct <69% OR ●
- Design Memory Learning Pct Correct <50% OR ●
- Three Letters Total Letters Correct <8

If any of these criteria are reached for a given baseline test, the ImPACT Score Report will automatically print a sentence that identifies the test results as being of questionable validity. If this is the case, the test administrator is encouraged to repeat the baseline exam, only after discussing the test results with the patient and identifying the reasons for the invalid test (e.g. difficulty understanding one or more of the modules, not taking the test seriously, etc.).

# ImPACT Pediatric

ImPACT Pediatric is administered in a similar manner and has a similar output to ImPACT. ImPACT Pediatric is administered to children (ages 5-11) and includes the following assessments:

- 1. Demographics
- 2. Symptom Scale as answered by test taker
- 3. Symptom Scale as answered by parent or guardian
- 4. Word Memory
- 5. Design Memory
- 6. Stop and Go!
- 7. Memory Touch
- 8. Picture Match

The image in Figure 2 shows an example of the Picture Match task:

Image /page/4/Picture/19 description: The image shows a memory match game with 16 cards. Two of the cards are showing a peach. The game also shows the number of taps is 30 and the time is 27.

Figure 2: Example Picture Match Task

{5}------------------------------------------------

Upon successful completion of the test battery, ImPACT Pediatric generates a report with the following information:

Demographic Information and Score Overview

- Background Information and Native Language ●
- Concussion / Injury Background
- Test Dates
- . Factor Scores for each of four cognitive domains

Medical History

- Detailed Medical History for the Child Including Pre-existing Conditions .
Symptom History

- Symptoms Reported by Test Taker
- Symptoms Reported by Parent .

Test Battery Modules

- Word List, Design Rotation, Stop & Go, Memory Touch, Picture Match, Color ● Match
Table 2 shows an example of the score output from ImPACT Pediatric. Note that ImPACT Pediatric does not provide a composite score table like ImPACT. However, the scores for individual tests in the battery are presented, and values which surpass the Reliable Change Index (RCI) are presented in red.

{6}------------------------------------------------

| Word List | | |
|----------------------------------|--------|--------|
| Skipped this module | No | No |
| Immediate Recall Num. Correct | 5 | 1 |
| Immediate Recall Num. Incorrect | 0 | 2 |
| Immediate Recall Percent Correct | 100 | 20 |
| Delayed Recall Number Correct | 4 | 0 |
| Delayed Recall Number Incorrect | 0 | 2 |
| Delayed Recall Percent Correct | 100 | 20 |
| Del. Recognition Number Correct | 7 | 3 |
| Del. Recognition Percent Correct | 70 | 30 |
| Time Between Tests (in seconds) | 294.33 | 342.36 |
| Design Rotation | | |
| Skipped this module | No | No |
| Number Correct | 10 | 10 |
| Percent Correct | 100 | 100 |
| Average Time (in seconds) | 1.47 | 1.86 |
| Stop & Go | | |
| Skipped this module | No | No |
| Number Correct | 10 | 10 |
| Average Correct Answer Time | 0.62 | 1.05 |
| Number Incorrect Red Lights | 0 | 0 |
| Number Incorrect Green Lights | 0 | 0 |
| Number Incorrect Yellow Lights | 0 | 2 |
| Number of Omissions | 0 | 0 |

#### Table 2: Example ImPACT Pediatric Report

# SUMMARY OF NONCLINICAL/BENCH STUDIES

#### SOFTWARE

The software for ImPACT and ImPACT Pediatric are consistent with a 'MODERATE' level of concern, as discussed in the FDA document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices," issued May 11, 2005. Appropriate documentation was provided as part of the de novo request.

### SUMMARY OF CLINICAL INFORMATION

### ImPACT

The sponsor supported the safety and effectiveness of the ImPACT device with clinical data from over 250 peer-reviewed articles that have been published with more than half representing the results of independent research. An adequate representative sample of participants was included in the clinical studies to support the age range for the device. Ten of the published studies were provided by the sponsor as the main studies in support of the test-reliability, construct validity and development of the normative database as detailed below. These research publications analyze the psychometric properties of ImPACT based on studies that address test validity, reliability, clinical utility and sensitivity/specificity or provide indirect support for using the ImPACT test battery in evaluation of concussion which is synonymous with the term, mild traumatic brain injury.

ImPACT's design is based on the traditional neurocognitive testing. Traditional neurocognitive (also known as neuropsychological) testing standards are defined by the American Academy of

{7}------------------------------------------------

Clinical Neuropsychology and the National Academy of Neuropsychology. These organizations released a position paper with recommendations on appropriate standards and conventions for computerized neuropsychological assessment devices (Bauer et. al. Archives of Clinical Neuropsychology Vol. 27 (2012) pp362-373). The position paper includes standards for the psychometric development issues, especially reliability, and validity. In addition, on June 2-3. 2011, the FDA hosted a workshop co-sponsored by three clinical professional societies: Academy of Neurology, American Epilepsy Society, and National Academy of Neuropsychology, to discuss issues related to the validation and labeling of devices used to assess seizures, cognitive function, traumatic brain injury (TBI) and concussion. During the workshop, topics related to standards and conventions for computerized neuropsychological assessment devices were discussed, similar to the topics described in the position paper (Eastman P. FDA Workshop: Guidance Offered for Evaluating Devices for Seizure Detection, TBI/Concussion, and Cognitive Function. Neurology Today. 2011:11(13):42-43.). Development of the ImPACT test battery and its validation is based on the guidelines described in the position paper and discussed during the workshop.

- 1. Construct Validity of the ImPACT Battery. The sponsor provided abstracts from 4 published studies in support of the validity of the ImPACT test battery:
- Iverson GL, Lovell MR, Collins MW. Validity of ImPACT for measuring . processing speed following sports-related concussion. J Clin Exp Neuropsychol. 2005;27(6):683-689.
- Maerlender A, Flashman L, Kessler A, et al. Examination of the Construct . Validity of ImPACT computerized Test, Traditional and Experimental Neuropsychological Measures. The Clinical Neuropsychologist. 2010; 24(8):1309T1325.
- . Schatz P, Putz BO. Cross-validation of measures used for computer-based assessment of concussion. Appl Neuropsychol. 2006:13(3):151-159.
- Allen BJ, Gfeller JD. The immediate post-concussion assessment and cognitive . testing battery and traditional neuropsychological measures: A construct and concurrent validity study. Brain Inj. 2011;25(2):179-191.

Table 3 summarizes the results of the testing. which supports the validity of ImPACT.

{8}------------------------------------------------

| Table 3: Summary of Studies Used to Support Construct Validity of ImPACT | | |
|--------------------------------------------------------------------------|--|--|
| | | |

| Study | Sample Description | Inclusion/Exclusion<br>Criteria | Clinical Reference<br>Standard | Analysis Performed | Summary of Results |
|-------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Iverson,<br>Lovell &<br>Collins<br>(2005) | N=72. Athletes randomly extracted<br>from larger study. These athletes were<br>from high schools in PA.<br>Average age of 17.1 years. Average<br>education of 10.5 years. 83.8% male<br>and 16.2% female. For 59% this was<br>first documented concussion, 21% had<br>one prior concussion and 20% had 2+<br>prior concussions<br>Sample size was selected based upon<br>planned t tests and PCFA analysis. | Athletes had to have<br>successfully<br>completed a baseline<br>assessment pre-<br>participation. Had to<br>have been diagnosed<br>by a physician or<br>athletic trainer with a<br>concussion and seen<br>in clinic within 21<br>days of concussion | All subjects were<br>diagnosed with a<br>concussion using<br>then-applicable<br>clinical standards<br>including grading of<br>concussion using<br>AAN guidelines | Comparison of the<br>ImPACT to a<br>traditional<br>neuropsychological<br>measure, the Symbol<br>Digit Modalities Test<br>(SDMT)<br>Correlational analysis,<br>t-tests and Principle<br>Components Factor<br>Analysis | SDMT correlated 0.70 with<br>Processing Speed and -0.60<br>with Reaction Time. Pairwise<br>T-tests revealed strongest<br>relationships were between<br>SDMT and Processing Speed<br>and Reaction Time versus<br>Verbal, or Visual Memory or<br>Total Symptoms. The PCFA<br>confirms a three factor solution<br>of Speed/Reaction Time<br>Memory and Total Symptoms |
| Marlander,<br>Flashman,<br>(2010) | N=54. Varsity athletes randomly<br>selected from a larger study. All<br>athletes were tested at Dartmouth<br>College.<br>40 from football team and the<br>remainder from non- contact sports:<br>golf, cross country and crew.<br>42 reported no prior concussions.<br>13 reported at least 1 prior concussion.<br>Sample size was selected based upon<br>planned analysis | All subjects were<br>concussion free at<br>the time of testing<br>and agreed to take a<br>complete<br>neuropsych battery | A comprehensive<br>Neuropsychological<br>battery designed to<br>be sensitive to<br>cognitive functions<br>known to show<br>deficits or changes in<br>the context of TBI<br>(see Table 2 in the<br>article) | Correlational analysis<br>relating ImPACT<br>Composites and<br>Neuropsychological<br>domains | Overall ImPACT was shown<br>to have good convergent and<br>discriminant validity. ImPACT<br>domains correlated with<br>expected experimental<br>measures and traditional paper<br>and pencil measures of<br>memory and processing speed<br>Due to cross-loadings of some<br>subtests authors caution that<br>ImPACT is best used as a<br>screening tool. |
| Schatz &<br>Putz<br>(2006) | N=30. College students who<br>volunteered as part of course<br>requirement. All subjects were tested<br>at St. Joseph's University in<br>Philadelphia.<br>Subjects were 14 men, 16 women,<br>non- varsity athletes Ages were 18-23<br>(mean =21) and they were concussion<br>free in prior 6 months.<br>Sample size was selected based upon<br>planned analysis | All subjects were<br>concussion free at<br>time of testing and<br>were participating as<br>part of course<br>requirement | Trail Making Test A<br>& B (TMT),<br>Digit Symbol subtest<br>of the Wechsler<br>Adult Intelligence<br>Scale-Revised<br>(WAIS-R) | Correlational analysis<br>relating ImPACT<br>Composites to Trail<br>Making A-B.D-2 test<br>and Digit Symbol<br>from WAIS-R | ImPACT Reaction time was<br>significantly correlated with<br>Trails A-B (.61, .44<br>respectively) and Digit Symbol<br>from the WAIS (.46). Digit<br>Symbol was also significantly<br>correlated with ImPACT<br>Processing Speed Index -. 51. |
| Allen &<br>Gfeller<br>(2011) | N=100. Students recruited from<br>undergraduate psychology class at a<br>private university (Midwestern).<br>Subjects included 56 females, 44<br>males. 76% Caucasian, 9% Asian, 8%<br>Asian Indian, 4% African American,<br>1% Pacific Islander and 2% Other.<br>Mean age of 19.69 years with mean<br>education of 13.21.<br>Sample size selected based upon<br>planned analysis. | Excluded were<br>students who<br>reported a history of<br>ADHD, major TBI<br>within last 3 months<br>or ESL. Additionally.<br>subjects had to pass<br>a Validity Index | Neuropsychological<br>battery used by the<br>NFL. (See Table 1 in<br>the article) | Correlational analysis<br>with ImPACT and<br>NFL Battery (HVLT-<br>R, BVMT-R, Trail<br>Making Test A-B,<br>COWA and Symbol<br>Search and Digit Span<br>Forward and<br>Backward from the<br>WAIS-III | Significant correlations were<br>obtained between many of the<br>ImPACT composite scores<br>and components of the NFL<br>battery. Correlations tended to<br>be in the .3 to .45 range and<br>likely could have been<br>suppressed by a restricted<br>range in the sample |

PCFA: principal component factor analysis; AAN: American Academy of Neurology; SDMT: Symbol Digit Modalities Test; ADHD: Attention Deficit Hyperactivity Disorder; ESL: English as a second language; HVLT-R: Hopkins Verbal Learning Test - Revised; BVMT-R: Benton Visual Learning Test -Revised; COWA: Controlled Oral Word Association; WAIS-R: Wechsler Adult Intelligence Scale -Revised; WAIS-III: Wechsler Adult Intelligence Scale - 3rd edition.

{9}------------------------------------------------

- 2. Reliability. The sponsor provided five published studies, which assessed the reliability of the ImPACT test battery using different intervals between assessments ranging from 30 days to 2 years between tests:
- Schatz P. Long term test-retest reliability of baseline cognitive assessments using ● ImPACT. Am J Sports Med. 2010:38(1):47-53.
- Schatz P. Ferris CS. One-month test-retest reliability of the ImPACT test battery. Arch Clin Neuropsychol. 2013 Aug; 28(5):499-504. Epub 2013 May 23.
- Nakayama Y. Covassin T. Schatz P. Nogle S. Kovan J. Examination of the Test-● Retest Reliability of a Computerized Neurocognitive Test Battery. Am J Sports Med. 2014 Jun 6;42(8):2000-2005.
- . Cole WR, Arrieux JP, Schwab K, Ivins BJ, Oashu FM, Lewis SC. Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population. Arch Clin Neuropsychol. 2013 Nov:28(7):732-42. Epub 2013 Jul 2.
- . Elbin. Schatz. Covassin. One-Year Test-Retest Reliability of the Online Version of ImPACT in High School Athletes. Am J Sports Med. 2011 Nov: 39 (11):2319-24.

Table 4 below summarizes the published studies in support of the stability of the various subtests administered by ImPACT (e.g., test-retest reliability). The table demonstrates relatively robust test-retest reliability of the ImPACT modules over a variety of time intervals up to two years.

| Variable | Interval Between Assessments | | | | | | |
|--------------------|----------------------------------|--------------------------------|--------------------------------------|--------------------------------------|----------------------------------|---------------------------------|----------------------------------|
| | 30 days<br>Schatz (2013)<br>n=25 | 30 days<br>Cole (2013)<br>n=44 | 0-45 days<br>Nakayama (2014)<br>n=85 | 0-50 days<br>Nakayama (2014)<br>n=85 | 45-50<br>Nakayama (2014)<br>n=85 | 1 year<br>Elbin (2011)<br>n=369 | 2 years<br>Schatz (2010)<br>n=95 |
| Verbal Memory | | | | | | | |
| ICC | 0.79 | 0.6 | 0.76 | 0.65 | 0.69 | 0.62 | 0.46 |
| r | 0.66 | 0.61 | - | - | - | 0.45 | 0.3 |
| Visual Memory | | | | | | | |
| ICC | 0.6 | 0.5 | 0.72 | 0.6 | 0.69 | 0.7 | 0.65 |
| r | 0.43 | 0.49 | - | - | - | 0.55 | 0.49 |
| Visual Motor Speed | | | | | | | |
| ICC | 0.88 | 0.83 | 0.87 | 0.85 | 0.88 | 0.82 | 0.74 |
| r | 0.78 | 0.86 | - | - | - | 0.74 | 0.6 |
| Reaction Time | | | | | | | |
| ICC | 0.77 | 0.53 | 0.67 | 0.71 | 0.81 | 0.71 | 0.68 |
| r | 0.63 | 0.53 | - | - | - | 0.62 | 0.52 |

Table 4: Summary of the obtained test-retest reliability coefficients

*ICC - intraclass correlation coefficient, a measure of test-retest reliability

- 3. Standardization/Normative Database. The sponsor provided information on the standardization of the ImPACT test battery and the development of the normative database. Standardization of the current version of ImPACT was accomplished through participation of test subjects from high schools and colleges from around the country that are representative of the intended use population. Older adults were drawn from adult athlete populations or were coaches, school administrators, nurses. Although not keyed specifically to the US Census, the sample was inclusive of minorities at a rate that

{10}------------------------------------------------

reflected the composition of the school systems involved. It is important to note that published data have not indicated significant differences between minority and Caucasian athletes on the ImPACT test scores , nor in a large multi-racial South African sample .

Normative data were collected by research partners acknowledged in the user manual. All testing was completed by professionals who were specifically trained to administer the tests. These professionals consisted of Neuropsychologists, Psychologists and Neuropsychology/Psychology graduate students, Certified Athletic Trainers and Athletic Training Graduate Students and Nurses. All testing was completed in a supervised setting and data were later uploaded onto a secure HIPPA-compliant server. Data were deidentified and placed in a database for analysis. All participants in the resulting studies were English speakers and were not reported to have underlying intellectual or developmental data and were not currently concussed or suffering from any other medical condition that might affect their test performance.

The standardization sample consisted of 17,013 individuals who underwent baseline ImPACT testing. The older subjects represented teachers, coaches, school administrators, and adult athletes. Athletes who participated in the normative sample were participants in the following sports: Tackle football (males only), soccer (males and females), lacrosse (males and females), wrestling (males only), baseball (males only), softball (females only), swimming/diving (males and females), cheerleading (females only), crew/rowing (males and females), volleyball (males and females), track and field (males and females), field hockey (females only) and cross country (males and females). The specific age and gender breakdown is presented in Table 5.

| Age Range | Male | Female |
|-----------------------|--------|--------|
| 10-12 years 11 months | 321 | 129 |
| 13-15 years 11 months | 4,359 | 1,851 |
| 16-18 years 11 months | 4,804 | 1,767 |
| 19-29 years 11 months | 2,061 | 626 |
| 30-39 years 11 months | 443 | 127 |
| 40-49 years 11 months | 192 | 96 |
| 50-59 years 11 months | 153 | 82 |
| Totals | 12,335 | 4,678 |

Table 5: Age and Gender Breakdown for ImPACT Normative Sample

- 4. Reliable Change Index (RCI). The ImPACT software calculates a RCI, which provides information regarding if a change in the ImPACT score from baseline to post-injury is a change that is not due to either practice effects or the result of measurement error. The RCI method for interpreting change on neurocognitive tests is a method for determining change. This method relies on the standard error of the difference score.
¹ Kontos, A.P., Elbin, R.J., Covasin, T., Larson, E. (2010), Exploring differences in neurocognitive concussion testing in African American and White athletes. Archives of Clinical Neuropsycholog, 25(8), 734-744.

^&quot; Shuttleworth-Edwards AB1, Whitefield-Alexander VJ, Radloff SE, Taylor AM, Lovell MR. Computerized neuropsychological profiles of South African versus US athletes: a basis for commentary on cross-cultural norming issues in the sports concussion arena. Phys Sportmed. 2009:37(4),45-52.

{11}------------------------------------------------

The RCI is a statistical calculation to demonstrate the change in score is not due to expected test-retest variance. The RCI calculation provides additional information to the clinician in determining if the change in test scores is clinically meaningful and not solely due to measurement error and provides important information to the clinician both diagnostically and prognostically.

- 5. Validity Index. The ImPACT test battery provides an index designed to aid in identifying invalid baseline examinations. The validity index is based upon sub-optimal performance on the five subtests, which comprise the ImPACT test battery and include the published research cut-off values for each subtest:
- X's and O's Total Incorrect > 30 OR
- Impulse Control Composite > 30 OR
- Word Memory Learning Pct Correct < 69% OR
- Design Memory Learning Pct Correct < 50% OR ●
- Three Letters Total Letters Correct < 8

If any of these criteria are reached for a given baseline test, the ImPACT report will automatically print a sentence that identifies the test results as being of questionable validity. If this is the case, the test administrator is encouraged to repeat the baseline exam, only after discussing the test results with the patient and identifying the reasons for the invalid test (e.g., difficulty understanding one or more of the modules, not taking the test seriously, etc.).

# ImPACT Pediatric

Research data have been collected through the collaborative efforts of a number of independent organizations and institutions that volunteered to participate in various aspects of the overall research project. These organizations were recruited based on the interest in participating and their involvement with children from 5 to 12 years of age.

Specific research projects and results are presented below:

- 1. Development of the Normative Database. A large, age-stratified sample of children ages 5 through 12 years old that are representative of the intended use population were tested utilizing ImPACT Pediatric to generate normal score ranges for the test. An adequate representative sample of participants was included in the clinical studies to support the age range for the device. Means and standard deviations were calculated across age by gender and are reported within. The subjects were 915 children between the ages of 5 and 12 years as depicted in Table 6:

| Dataset | n |
|----------------------------------------------------------|-----|
| Children's Hospital of Atlanta – Atlanta, Georgia | 312 |
| Right Time Pediatrics – Annapolis, Maryland | 230 |
| Northern Michigan University – Marquette, Michigan | 199 |
| Mount Lebanon School District – Pittsburgh, Pennsylvania | 94 |
| Shift Concussion Management – Guelph, Ontario, Canada | 80 |
| Total | 915 |

| | Table 6: Clinical Sites and Numbers of Subjects | |
|--|-------------------------------------------------|--|
|--|-------------------------------------------------|--|

{12}------------------------------------------------

ImPACT Pediatric tests were administered by a researcher, clinician, or educational professional trained in the use of ImPACT Pediatric. Tests were taken on an iPad 2 or above with the device flat on the table and instructions read out loud to the participant. Testing typically takes between 10 and 20 minutes. The children were instructed to respond by touching the screen with the pointing (first) finger of the hand that they write with. No "group testing" was conducted and all testing was done on a one-on-one basis.

- a. Results. Examination of scores for Pediatric ImPACT involved standard calculation of means, standard deviations, and number of participants for each test response item, at each age group, by gender. There was the expected trend of older children performing better (faster with fewer mistakes) compared to younger children. Significant gender differences were uncovered between males and females through ttests on the Word Memory Immediate, Word Memory Delayed, and Picture Match Average Taps scores. For the Word Memory scores differences were observed for ages 7 through 12 years old (p<0.05). However, Picture Match Average Taps showed different performance by gender only from ages 9 to 12 years old (p<0.05).
- b. Development of Factor Scores. A factor analysis was conducted on 712 participants from the ImPACT Pediatric normative database to ascertain whether relevant score clusters exist and for the purpose of improving the interpretability and utility of the test. Data were derived from a subset of the normative database that had completed all specific subtests.

This analysis yielded a four-factor solution. Factors include what can be termed an attention and sequencing factor, a word memory factor, a visual memory factor and a reaction time factor. Once agreement was achieved on the resulting factors, each standardization case was rescored on these factors using the raw score to T score conversion and these data were then used to calculate the normative database. T scores were calculated with a mean of 50 and a standard deviation of 10. These standardized scores allow direct comparison of factor scores to one another, as they are all provided utilizing the same metric.

- 2. Reliable Change Index (RCI). The ImPACT Pediatric also calculates a RCI in a similar fashion to ImPACT. Reliable change data provide evidence for the stability of the test measures. Any test output that exceeds the RCI is displayed in red, indicating that the difference in scores shows a change that is not due to practice effects or measurement error. The RCI is a statistical calculation to demonstrate the change in score is not due to expected test-retest variance. The RCI calculation provides additional information to the clinician in determining if the change in test scores is clinically meaningful and not solely due to measurement error and provides important information to the clinician both diagnostically and prognostically.
- 3. Reliability. The sponsor provided the results of a clinical study (unpublished) of 100 children between the ages of 5 and 12 years (mean = 7.8 yrs) who were participants in youth soccer and hockey leagues. The test-retest reliability of ImPACT Pediatric was measured at one week and one month. Intraclass correlation coefficients were calculated

{13}------------------------------------------------

for the one week interval and revealed highly significant (p<.001) correlations for each test. Two items showed poor stability (ICC= .46, .54), five demonstrated adequate to good stability (ICC=.61 .63, .67, .71, .72) and the remaining five showed excellent stability (ICC=.79, 81, .82, .83, .89). The ICC reliability coefficients are depicted in Table 7 below:

| Test Module | ICC's |
|----------------------------------------------|-------|
| Word List Immediate Recall Number Correct | .83 |
| Word List Delayed Recall Number Correct | .82 |
| Word List Delayed Recognition Number Correct | .89 |
| Design Rotation Number Correct | .67 |
| Design Rotation Average Time | .61 |
| Memory Touch Number Correct | .81 |
| Memory Touch Total Sequences Correct | .79 |
| Memory Touch Highest Sequence | .72 |
| Stop and Go Number Correct | .63 |
| Stop and Go Average Time | .71 |
| Picture Match Average Taps | .46 |
| Picture Match Average Time | .54 |

Table 7: Test-retest reliability coefficients (ICCs)

- 4. Construct Validity. To assess construct validity of ImPACT Pediatric, 83 participants ages 5 through 12 were given a modified battery of the Wide Range Assessment of Memory and Learning-2 (WRAML-2) and Pediatric ImPACT. Significant correlations were revealed for 20 of the 24 potential test comparisons. Pearson's correlation coefficients were calculated between the ImPACT Pediatric scores, and comparable counterparts in the WRAML-2. ImPACT Pediatric correlates significantly with relevant WRAML-2 subtests. Small but significant Pearson's correlation coefficients were documented for all measures, except for: 1) Word Memory Recall and both WRAML-2 Design Recall and Recognition, and 2) Design Rotation and WRAML-2 Design Recognition. Therefore, ImPACT Pediatric appears to measure important aspects of memory. Note that negative correlations represent a relationship where one variable increases (i.e., higher/better Story Memory scores) and another decreases (i.e., lower/better Picture Match Average Time). These results are depicted in Table8 and Table9 below. These data support the concurrent validity is therefore considered to be adequate between the examined cognitive domains of these two tests.
#### Table 8: The relationship between WRAML-2 Story Memory and ImPACT Pediatric Word List test

| | Word Memory Recall | Word Memory Recognition |
|----------------------------|--------------------|-------------------------|
| WRAML-2 Story Memory | .49** | .30** |
| WRAML-2 Story Memory Del. | .52** | .42** |
| WRAML-2 Design Recall | .48 ** | .22 |
| WRAML-2 Design Recognition | .23* | .16 |

Note: ** denotes correlation is significant at the p<. 01 level, * at the p<. 05 level.

{14}------------------------------------------------

Table 9: The relationship between WRAML-2 Story memory and ImPACT Pediatric Word List test. All correlations were significant at P<0.01 with the exception between Design Memory (WRAML) and Word Memory Recognition.

| Test | Design Rotation<br>Time | Design Rotation<br>Correct | Picture Match<br>Average Taps | Picture Match<br>Average Time |
|----------------------------|-------------------------|----------------------------|-------------------------------|-------------------------------|
| WRAML-2 Story Memory | -.26* | .27* | -.33** | -.34** |
| WRAML-2 Story Memory-Delay | -.31** | .26* | -.36** | -.43 |
| WRAML-2 Design Recall | -.34** | .29* | -.22 | -.33** |
| WRAML-2 Design Recognition | -.34** | .11 | -.21* | -.24* |

Note: ** denotes correlation is significant at the p<. 01 level, * at the p<. 05 level.

# LABELING

The labeling for the ImPACT and ImPACT Pediatric meets the requirements of 21 CFR 801.109 for prescription devices. The labeling includes the following information to mitigate the risks of user discomfort and incorrect result (false positive and false negative):

- 1. A summary of the testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. This includes a description of the device output and clinical interpretation, information about device repeatability and reproducibility of the device output, construct validity of the device output, and a description of the normative database.
- A warning that the device should only be used by healthcare professionals who are 2. trained in concussion management.
- A warning that the device does not identify the presence or absence of concussion or 3. other clinical diagnoses.
- 4. A warning that the device is not a stand-alone diagnostic.
- 5. Instructions that test administrators must convey to patients regarding the administration of the test and collection of cognitive test data.

# RISKS TO HEALTH

Table 0 identifies the risks to health that may be associated with use of Computerized Cognitive Assessment Aid for Concussion and the measures necessary to mitigate these risks.

| Table 10: Risk Mitigation Table | |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|
| Identified Risk | Mitigation Measure |
| User discomfort (e.g., visual or mental fatigue) | • Labeling |
| Incorrect result, inclusive of:<br>• False positive – cognitive impairment<br>from concussion when in fact none is<br>present<br>• False negative – cognitive impairment<br>from concussion is not noted when in<br>fact cognitive impairment is present | • Clinical performance testing<br>• Software verification, validation,<br>and hazard analysis<br>• Labeling |

#### Table 10: Rick Mitigation Table

{15}------------------------------------------------

# SPECIAL CONTROLS:

In combination with the general controls of the FD&C Act, the Computerized Cognitive Assessment Aid for Concussion is subject to the following special controls:

- 1. Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed.
- 2. Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must:
- a. Evaluate device output and clinical interpretation.
- b. Evaluate device test-retest reliability of the device output.
- Evaluate construct validity of the device cognitive assessments. C.
- d. Describe the construction of the normative database, which includes the following:
- i. How the clinical work-up was completed to establish a "normal" population, including the establishment of inclusion and exclusion criteria.
- ii. Statistical methods and model assumptions used.
- 3. The labeling must include:
- a. A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following:
- i. Device output and clinical interpretation.
- ii. Device test-retest reliability of the device output.
- iii. Construct validity of the device cognitive assessments.
- iv. A description of the normative database, which includes the following:
- 1. How the clinical work-up was completed to establish a "normal" population, including the establishment of inclusion and exclusion criteria.
- 2. How normal values will be reported to the user.
- 3. Representative screen shots and reports that will be generated to provide the user results and normative data.
- 4. Statistical methods and model assumptions used.
- 5. Whether or not the normative database was adjusted due to differences in age and gender.
- b. A warning that the device should only be used by healthcare professionals who are trained in concussion management.
- A warning that the device does not identify the presence or absence of concussion or C. other clinical diagnoses.
- d. A warning that the device is not a stand-alone diagnostic.
- Any instructions technicians must convey to patients regarding the administration of e. the test and collection of cognitive test data.

{16}------------------------------------------------

# BENEFIT/RISK DETERMINATION

The probable benefits of ImPACT and ImPACT Pediatric are based on data collected in a group of studies published in literature and/or submitted in support of this de novo request as described above.

- . Psychometric properties of the ImPACT system, including adequate demonstration of testretest reliability, construct validity via comparison to traditional neuropsychological tests, calculation of the reliable change index when baseline testing is available, and a validity index to detect suboptimal test performance.
- Extensive normative database consisting of individuals broken down by age and gender for determining level of cognitive function in the absence of baseline testing.
- . Patient benefit value: Ability to have access to a non-invasive cognitive assessment battery that can be used to compare pre-injury (baseline cognitive performance) to postinjury cognitive performance. Ability to compare cognitive test performance to a large normative database in the absence of baseline testing.

The risks of the device are based on data collected from literature analysis of subjects and/or data submitted in support of this de novo request who were administered ImPACT or ImPACT Pediatric as described above.

- A false positive result may influence a clinician's decision to diagnose a patient with a . concussion when there is none. However, this risk to the patient is considered to be low as the current guidelines regarding treatment of mild traumatic brain injury (mTBI) is rest and observation. Therefore, a false positive result would support a conservative treatment approach (i.e., not allowing a patient to return to activity in which they are at risk for a second concussive head injury). In addition, a false positive result would not expose the patient to any unnecessary intervention.
- A false negative result may influence a clinician's decision that the subject has not . experienced a concussive head injury when indeed one is present. This would support a clinical decision not to treat the patient and possibly allow the individual to return to activity in which they are at risk for a second concussive head injury, introducing risk of a serious adverse event (i.e., second impact syndrome) that may have significant morbidity and mortality implications.
- The risks of false positive and false negative results are mitigated by the fact that ● ImPACT and ImPACT Pediatric are not indicated for use as a stand-alone diagnostic system. The device classification should be integrated into the clinician's decision making process alongside information from other evaluative tests.
- . Subjects may experience user discomfort due to visual or mental fatigue. This can be mitigated through proper test administration as described in the labeling.

{17}------------------------------------------------

Additional factors to be considered in determining probable risks and benefits for ImPACT and ImPACT Pediatric include:

- The clinical performance data in support of this de novo submission were not based upon . a prospective clinical validation study and rely totally upon published literature and/or other clinical data submitted in support of this de novo request. The literature and/or data provided in support of the psychometric properties of the ImPACT and ImPACT Pediatric, including test-retest reliability and construct validity are considered to be adequate and consistent with results obtained from traditional paper and pencil psychological and neuropsychological tests.
- Development of the normative database was based upon several published studies, which ● used various inclusion/exclusion criteria to determine that a subject did not have any medical conditions that might influence the result of the cognitive assessment. The normative database is considered adequate.

### Patient Perspectives

This submission did not include specific information on patient perspectives for this device.

### Benefit/Risk Conclusion

In conclusion, given the available information above, the data support that for use as an assessment aid and in the management of concussion, the probable benefits outweigh the probable risks for ImPACT and ImPACT Pediatric. The devices provide substantial benefits and the risks can be mitigated by the use of general and the identified special controls.

### CONCLUSION

The de novo request for ImPACT and ImPACT Pediatric is granted and the devices are classified under the following:

Product Code: POM Device Type: Computerized Cognitive Assessment Aid for Concussion Class: II Regulation: 21 CFR 882.1471