PARASCRIPT ACCUDETECT 6.1.0

P120004 · Parascript, LLC · MYN · Aug 22, 2013 · Radiology

Device Facts

• Record ID: P120004
• Device Name: PARASCRIPT ACCUDETECT 6.1.0
• Applicant: Parascript, LLC
• Product Code: MYN · Radiology
• Decision Date: Aug 22, 2013
• Decision: APRL
• Regulation: 21 CFR 892.2070
• Device Class: Class 2
• Attributes: Software as a Medical Device

Intended Use

The Parascript® AccuDetect® 6.1.0 is intended for use in screening mammography to identify areas suspicious for breast cancer for radiologist review after completing an initial read.

Device Story

Parascript® AccuDetect® 6.1.0 is a CAD software for FFDM images; processes digital mammograms from GE or Philips systems. Automatically analyzes images to identify potential soft tissue densities and calcifications; displays results as overlays on a computer monitor. Used in clinical settings by radiologists as a second reader after an initial unaided review. Radiologists re-examine cases using CAD output to inform final clinical assessments. Benefits include potential earlier detection of malignant lesions; assists in identifying suspicious areas (5-50mm masses; 0.1-0.8mm calcifications). Operates on standalone computing equipment; communicates via DICOM 3.0 protocol with PACS/Review Workstations.

Clinical Evidence

Pivotal retrospective reader study (n=12 radiologists, 240 cases) compared unassisted vs. CAD-assisted reading. Primary endpoint: patient-based ROC AUC. Results: AUC increased from 0.914 (unassisted) to 0.930 (CAD-assisted) (p=0.0010). Secondary endpoints: lesion-based L-ROC AUC increased from 0.817 to 0.838. Recall-based sensitivity increased from 0.919 to 0.934; recall-based specificity increased from 0.655 to 0.703. Standalone study (n=300 cases) reported 87.3% sensitivity and 0.427 false positive rate per image.

Technological Characteristics

Software-based CAD system; runs on off-the-shelf PC (Windows 7, 2GHz 4-core processor, 1GB RAM/core). Inputs: FFDM images (GE/Philips) via DICOM 3.0. Outputs: DICOM Mammography CAD Structured Report. Detection principle: automated image analysis for soft tissue densities and calcifications. Quality system: ISO 13485/21 CFR 820.

Indications for Use

Indicated for use in screening mammography to identify suspicious areas for breast cancer in female patients undergoing screening. No specific contraindications.

Regulatory Classification

Identification

Medical image analyzers, including computer-assisted/aided detection (CADe) devices for mammography breast cancer, ultrasound breast lesions, radiograph lung nodules, and radiograph dental caries detection, is a prescription device that is intended to identify, mark, highlight, or in any other manner direct the clinicians' attention to portions of a radiology image that may reveal abnormalities during interpretation of patient radiology images by the clinicians. This device incorporates pattern recognition and data analysis capabilities and operates on previously acquired medical images. This device is not intended to replace the review by a qualified radiologist, and is not intended to be used for triage, or to recommend diagnosis.

Special Controls

*Classification.* Class II (special controls). The special controls for this device are:(1) Design verification and validation must include: (i) A detailed description of the image analysis algorithms including a description of the algorithm inputs and outputs, each major component or block, and algorithm limitations. (ii) A detailed description of pre-specified performance testing methods and dataset(s) used to assess whether the device will improve reader performance as intended and to characterize the standalone device performance. Performance testing includes one or more standalone tests, side-by-side comparisons, or a reader study, as applicable. (iii) Results from performance testing that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures ( *e.g.,* receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain a sufficient number of cases from important cohorts (*e.g.,* subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Appropriate software documentation ( *e.g.,* device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results; and cybersecurity).(2) Labeling must include the following: (i) A detailed description of the patient population for which the device is indicated for use. (ii) A detailed description of the intended reading protocol. (iii) A detailed description of the intended user and user training that addresses appropriate reading protocols for the device. (iv) A detailed description of the device inputs and outputs. (v) A detailed description of compatible imaging hardware and imaging protocols. (vi) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level ( *e.g.,* poor image quality or for certain subpopulations), as applicable.(vii) Device operating instructions. (viii) A detailed summary of the performance testing, including: test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.

Submission Summary (Full Text)

{0}

SUMMARY OF SAFETY AND EFFECTIVENESS DATA (SSED)

I. GENERAL INFORMATION

Device Generic Name: Computer Aided Detection Software for Mammography
Device Trade Name: Parascript® AccuDetect® 6.1.0
Device Procode: MYN
Applicant’s Name and Address: Parascript, LLC
6273 Monarch Park Place
Longmont, Colorado 80503
Date(s) of Panel Recommendation: None
Premarket Approval Application (PMA) Number: P120004
Date of FDA Notice of Approval: August 22, 2013
Expedited: Not Applicable

II. INDICATIONS FOR USE

The Parascript® AccuDetect® 6.1.0 is intended for use in screening mammography to identify areas suspicious for breast cancer for radiologist review after completing an initial read.

III. CONTRAINDICATIONS

None

IV. WARNINGS AND PRECAUTIONS

The warnings and precautions can be found in the Parascript® AccuDetect® 6.1.0 labeling.

V. DEVICE DESCRIPTION

Parascript® AccuDetect® 6.1.0 is a Computer-Aided Detection (CAD) software device intended to aid radiologists reading mammograms. It is a proprietary software application designed to process FFDM images. The digital mammography images are automatically analyzed to identify areas suspicious for possible soft tissue densities and/or calcifications.

Results are displayed on a computer monitor for review by a radiologist. The radiologist is instructed to first review each case in the conventional manner. Then the radiologist

PMA P120004: FDA Summary of Safety and Effectiveness Data

{1}

uses Parascript® AccuDetect® 6.1.0 CAD to re-examine the case utilizing the information supplied by the CAD system before making a final assessment for the case.

The Parascript® AccuDetect® 6.1.0 system consists of proprietary software and general-purpose computing equipment. The device is delivered as an integrated system with specific installation configuration. The Parascript® AccuDetect® 6.1.0 CAD software is installed by trained technicians. Figure 1 illustrates the Parascript® AccuDetect® 6.1.0 System in context of hospital infrastructure:

Figure 1: Parascript® AccuDetect® 6.1.0 System in Context of Hospital Infrastructure

The Parascript® AccuDetect® 6.1.0 system is intended to work on a standalone computer and communicates with input and output devices through LAN via DICOM 3.0 protocol.

The Parascript® AccuDetect® 6.1.0 system receives images from the Acquisition Workstation (AWS), Picture Archiving and Communication System (PACS) or other enabled storage device via DICOM interface. The Parascript® AccuDetect® 6.1.0 CAD software processes digital images created by GE Senographe (Essential, DS, and Care) or Philips MicroDose Full Field Digital Mammography Systems (FFDMs).

The Parascript® AccuDetect® 6.1.0 results are sent in the form of a DICOM Mammography CAD Structured Report (SR) to a Review Workstation (RWS) and/or PACS.

For each mammogram, a radiologist is instructed to first review the images thoroughly before enabling display of the Parascript® AccuDetect® 6.1.0 results on the Review Workstation. The images and corresponding DICOM Mammography CAD Structured Reports can be stored on Review Workstation and/or in the PACS system.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{2}

The Parascript® AccuDetect® 6.1.0 CAD includes two main components:

1. DICOM Server
2. Recognizer

Figure 2: Parascript® AccuDetect® 6.1.0 logical components

DICOM Server is implemented as a Windows service that automatically starts on user's computer after rebooting the computer. Recognizer is a process that receives image data from a queue, executes recognition, creates DICOM Structured Reports and places them into the results queue.

The Parascript® AccuDetect® 6.1.0 CAD software produces two types of marks: soft tissue densities and calcifications. A soft tissue density mark consists of an area enclosed by an oval. A radiologist is instructed to carefully consider the area in and around a soft tissue mark for the possibility of a soft tissue density. A calcification mark consists of an area enclosed by a rectangle. A radiologist is instructed to carefully consider the area in and around a calcification mark for the possibility of a cluster of calcifications.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{3}

Border of a suspicious area

Figure 3: Soft Tissue density mark
Border of a suspicious area
Figure 4: Calcification mark

The Parascript® AccuDetect® 6.1.0 CAD software is able to detect calcifications and soft tissue densities including architectural distortions, focal asymmetries, and suspect opacities with a dense core or spiculated masses. Marks are placed at the region where the suspicious lesion is detected.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{4}

Figure 5. Lesion location

The directions for use specify that a radiologist will first read the digital mammogram case in the conventional, unaided manner. Next, the radiologist is instructed to reexamine the case utilizing the information supplied by the Parascript® AccuDetect® 6.1.0 system before making a final assessment for the case.

Below are examples of soft tissue density mark (Figure 6) and calcifications marks (Figure 7).

Figure 6. Biopsy-proven malignant soft tissue density

PMA P120004: FDA Summary of Safety and Effectiveness Data

{5}

Figure 7. Biopsy-proven malignant calcifications


The Parascript® AccuDetect® 6.1.0 can detect masses with a diameter between 5mm and 50mm. Parascript® AccuDetect® 6.1.0 CAD software can detect calcification clusters consisting of several calcifications with size between 0.1 mm and 0.8 mm in diameter within 2 cm² areas.

The Parascript® AccuDetect® 6.1.0 CAD software is installed on off-the-shelf computing equipment (see below) by trained technicians, using documented installation procedures and instructions. Users will be provided with an integrated user manual which includes detailed instructions.

Acceptable computing equipment for use with the Parascript® AccuDetect® 6.1.0 CAD must have the following minimum specifications: Microsoft Windows 7 Professional 64 bits, 2 GHz processor supporting SSE2 instruction set, 4 cores, RAM - 1 GB per core, 100 GB free space on hard drive.

VI. ALTERNATIVE PRACTICES AND PROCEDURES

The field of mammography contains many standard practices and procedures that are well defined under the Mammography Quality Standards Act (MQSA) for maximizing the accuracy of reading screening mammograms. Although not required by MQSA, some clinics use "double reading" to increase the accuracy of screening mammography.

There are also other systems that perform image analysis like that performed by the Parascript® AccuDetect® 6.1.0 CAD software. These systems are commonly referred to as "Mammography CAD" systems. Any other commercially available system approved for this intended use can be used as an alternative to the double reading of mammograms by two radiologists.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{6}

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 7

# VII. MARKETING HISTORY

The Parascript® AccuDetect® 6.1 has been marketed in France, Germany and Sweden in 2012. The previous versions were deployed in Italy, Netherlands, and Russia since 2010. The device has not been withdrawn from marketing for any reason related to its safety or effectiveness.

# VIII. POTENTIAL ADVERSE EFFECTS OF THE DEVICE ON HEALTH

There are no known direct safety risks caused by, or related to, the use of the device. Indirect inherent risks are that (a) the device may not identify some actionable areas (malignant lesions); and (b) the device may identify regions that are not actionable (nonmalignant lesions/false positive readings). These possibilities are explained in the Warnings section of the device labeling. Proper use of the information generated by the device is explained in the Directions for Use section of the device labeling.

If a radiologist determines that a false positive CAD mark indicates an area that is suspicious enough for follow-up, then the patient may be subjected to unnecessary anxiety and/or biopsy.

# IX. SUMMARY OF NON-CLINICAL STUDIES

Parascript utilizes the ISO 13485 standard mapped to 21 CFR 820 requirements as a way to control quality through product lifecycle process controls and reviews.

Non-clinical studies were conducted throughout the design and development of the Parascript® AccuDetect® 6.1.0 CAD software. These studies were designed to ensure that the Parascript® AccuDetect® 6.1.0 software meets its specifications and intended use.

Parascript performed verification and validation testing on the Parascript® AccuDetect® 6.1.0 CAD software.

Verification testing consisted of software unit testing, software integration testing, and software system testing. The purpose of the verification test was to assure that the software application satisfies the software requirements. It was found that two minor anomalies remain in the software application. These anomalies have no impact on the safety and effectiveness of the device and no impact on the operator usage. Verification testing was successfully completed.

Internal validation testing consisted of measuring device standalone performance including sensitivity, specificity and false positives rate (per image). Internal validation is used to determine whether a candidate CAD algorithm has the potential to perform as expected on an independent validation dataset. The results of internal testing using the in-house library of 3,489 cases are the following: overall sensitivity is 90.7%, case specificity is 48.6%, and false positive rate per image is 0.412. The performance of Parascript® AccuDetect® 6.1.0 CAD algorithm on the in-house dataset suggests it has the potential to perform satisfactorily on an independent clinical validation dataset.

{7}

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 8

# X. SUMMARY OF PRIMARY CLINICAL STUDIES

The primary objective of the clinical studies was to determine whether radiologists are more effective at reading digital mammograms when using the Parascript® AccuDetect® 6.1.0 Computer Aided Detection (CAD) software versus when not using CAD software. The purpose of the clinical performance assessment of Parascript® AccuDetect® 6.1.0 CAD was to demonstrate the clinical safety and effectiveness of the Parascript® AccuDetect® 6.1.0 CAD when used as a second reader in screening mammography.

Intrinsic Imaging, LLC served as the Clinical Research Organization for the reader study under the direction of the Primary Investigator, Dr. Amit Mehta.

The following retrospective studies were performed:

- A pivotal reader study “Comparison of Reader Performance for Screening Mammography Without and With Parascript® AccuDetect® 6.1.0 Computer Aided Detection” was conducted to compare the effectiveness of radiologists reading screening digital mammograms when using Parascript® AccuDetect® 6.1.0 CAD versus when not using CAD.
- The Parascript® AccuDetect® 6.1.0 CAD standalone study to measure behavior of the Parascript® AccuDetect® 6.1.0 CAD software.

# A. Study Readers and Cases

The Pivotal Study was conducted with 12 radiologists reading 240 cases. The group of radiologists included general and specialized radiologists. Six general radiologists read less than 3,000 cases per year and six specialized radiologists read more than 3,000 cases per year.

The cases for the study consisted from a randomly selected set of 240 screening FFDM cases acquired on Philips MDM (Microdose Mammography) L30 and GE Senographe Essential FFDM devices (120 cases with cancer, 108 normal cases, and 12 cases with actionable benign findings). The GE Senographe Essential data set contained the same number and types of cases as the Philips MDM L30 data set (each set contained 60 cancer cases, 54 normal cases, and 6 actionable benign cases).

The cases were collected from 3 sites in the United States and 2 sites in Europe (one in Belgium and one in Switzerland). All U.S. sites received approval to provide cases for this study by their respective Institutional Review Boards. The European sites were selected due to insufficient supply of retrospective images for Philips MicroDose FFDM in the United States. The European sites followed laws and regulations in their countries and received corresponding approvals for the data collection.

Cases used in the reader study were selected from the pool of accrued cases to conform to a representative sample of screening examinations.

{8}

- Positive cases were defined as ones for which there is a mammography visible lesion that was proven by biopsy to represent a cancer.
- Normal cases were defined as ones for which there is a negative index digital mammogram and a subsequent negative mammogram 320-450 days following the index exam (with no cancer diagnosed during this period);
- Recall cases were defined as ones for which further evaluation (e.g., additional mammographic views, breast US, etc.) was recommended during the interpretation of the screening mammograms.
- Actionable benign cases were defined as recall cases for which either subsequent diagnostic investigation or biopsy did not demonstrate cancer.

The inclusion and exclusion criteria for screening mammography cases were as follows:

**Inclusion Criteria:**

- Female.
- Any ethnic origin.
- Have a four-view screening digital mammogram.
- Have a negative index digital mammogram and a subsequent negative mammogram 320-450 days following the index exam (with no cancer diagnosed during this period); a finding shown by biopsy or follow-up imaging to be a benign abnormality; or, for cases with cancer, a lesion evident on the screening mammogram and a follow-up biopsy demonstrating breast cancer.
- Have provided consent (as required by IRB and local laws) for images and protected health information to be used for further research.
- Meet none of the exclusion criteria.

**Exclusion Criteria:**

- Subjects who presented with any contraindications to mammographic screening, including but not limited to:
- Significant existing breast trauma.
- Pregnancy.
- Lactating.
- Previous surgical biopsy.
- Previous breast cancer.
- Placement of an internal breast marker.
- Presence of a breast implant.
- Presence of a pacemaker.
- Inadequate technical quality mammography images, such as insufficient anatomical coverage.

Each site submitted de-identified digital mammograms, images (overlays) indicating the location of each known cancer, reference standard-based truth (biopsy for positive

PMA P120004: FDA Summary of Safety and Effectiveness Data

{9}

cases; follow up mammogram for normal cases and follow-up mammogram or biopsy with actionable benign cases), de-identified clinical reports (including radiology, surgical, and pathology reports), and study-specific case report forms. Tables 1 through 5 characterize the 240 cases used in the pivotal study.

Table 1: Age Distribution

| Age | Total |
| --- | --- |
| <= 44 | 36 |
| 45-54 | 78 |
| 55-64 | 58 |
| >= 65 | 67 |
| Unknown | 1 |

Table 2: Breast Density Distribution

| Breast Density Type | Cancer Cases | Normal or Benign Cases |
| --- | --- | --- |
| Almost Entirely Fat | 3 | 11 |
| Scattered Fibroglandular | 57 | 49 |
| Heterogeneously Dense | 50 | 54 |
| Extremely dense | 8 | 6 |
| Unknown | 2 | 0 |

Table 3: Lesions Size Distribution

| Size (mm) | Totals |
| --- | --- |
| 1 - 5 | 6 |
| 6 - 10 | 31 |
| 11 - 15 | 25 |
| 16 - 20 | 18 |
| >20 | 22 |
| Unknown | 18 |

Table 4: Pathology of Cancers

| Diagnosis | Totals |
| --- | --- |
| DCIS | 28 |
| DCIS+IDC | 8 |
| DCIS+ILC | 1 |
| FCC+IDCS | 1 |
| ICDL | 1 |

PMA P120004: FDA Summary of Safety and Effectiveness Data

{10}

| IDC | 69 |
| --- | --- |
| IDC+MetC+PC | 1 |
| IDC+ MF | 1 |
| ILC | 6 |
| IMC | 1 |
| LCIS | 1 |
| O | 1 |
| TC | 1 |

DCIS = Ductal Carcinoma In Situ
ICDL = Invasive Carcinoma with Ductal and Lobular Features
IDC = Invasive Ductal Carcinoma
ILC = Invasive Lobular Carcinoma
IMC = Invasive Mammary Carcinoma
LCIS = Lobular Carcinoma In Situ
MetC = Metastatic Carcinoma
MF = Multifocal
O = Other
PC = Papillary Carcinoma
TC = Tubular Carcinoma

Table 4: Distribution of Cases per Manufacturer

| Type | Number of Cases |
| --- | --- |
| GE Soft Tissue Densities | 40 |
| GE Calcifications | 27 |
| Philips Soft Tissue Densities | 38 |
| Philips Calcifications | 30 |
| Total Soft Tissue Densities | 78 |
| Total Calcifications | 57 |

# B. Study Execution

The clinical reader study was conducted at Intrinsic Imaging, LLC located in 8401 Datapoint, Suite 600, San Antonio, Texas 78229. Intrinsic Imaging is a clinical research organization and a medical imaging core lab providing expert image management and radiological review services to global pharmaceutical, biotechnology and medical device organizations. As a member of the South Texas Radiology Group (STRG) family of companies, Intrinsic Imaging was able to provide conditions of a typical clinical environment, including temperature, ambient light, light sources (less than 50 lux), level of comfort, type of furnishings, and ambient noise. All medical physics evaluations in the mammography room were performed prior to the study according to the prescriptions of state and national agencies.

12 Readers (certified with the American Board of Radiology, and qualified under the Mammography Quality Standard Act) performed sequential readings of 240 cases (120 with malignant lesions, 108 normal cases, and 12 cases with actionable benign findings). The readers read each case unassisted, recorded the results of that reading

PMA P120004: FDA Summary of Safety and Effectiveness Data

{11}

then turned on Parascript® AccuDetect® 6.1.0 CAD as a “second reader”. Once the findings of the unassisted read have been stored, they could not be altered.

For each case, reader performed the following actions in order:

1. Evaluate the case without CAD assistance.
2. Record the unassisted assessment for the case.
3. Turn on CAD marks and view the Parascript® AccuDetect® 6.1.0 marks for the case.
4. Record CAD assisted assessment for the case.

The unassisted and CAD assisted assessments included the following information:

- Readers noted their level of confidence that a malignancy is present on each finding using 1-100 probability of malignancy (POM) scale; they also noted the type of each finding (soft tissue density or calcification).
- Readers noted their overall level of confidence that a malignancy is present on the case.
- Readers assigned a BI-RADS screening score (0, 1, or 2) to each case.

All readers were trained on the workstation, use of CAD, hanging protocol, and electronic case report forms prior to beginning the reading sessions. In addition the readers were told that the CAD does not show any marks for a normal case more often compared to a cancer case and that on average the CAD shows more marks on cancer cases compared to normal cases.

Each case and each finding were classified as true negative, true positive, false negative, or false positive based on a comparison of the reading with the reference standard-based truth.

## C. Pivotal Study Statistical Methods

The statistical analysis estimated a smooth patient-based receiver operating characteristic (ROC) curve for each of the 12 study readers in each test condition (without CAD and with CAD) using the probability of malignancy (POM) ratings each reader provided for each case. The major efficacy criterion used in this study was the area under (AUC) the receiver operator characteristic (ROC) curve. The statistical analysis used the method of Dorfman, Berbaum, and Metz [1,2]. The ROC curves and AUC for each reader were estimated with two different models in version 2.32 build 3 of DBM-MRMC software: Wilcoxon-trapezoidal (non-parametric) and PROPROC (parametric). The results of the parametric model are reported below. The statistical analysis treated patients and readers as random effects.

The statistical analysis for secondary endpoints compared AUC between CAD assisted and unassisted lesion-based L-ROC curves (or, in another terminology, Figures of Merit for AFROC curves) which were based on the probability of malignancy (POM) ratings each reader provided for each finding. The analysis was performed with JAFROC software, version 4.1 (described at

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 12

{12}

http://www.devchakraborty.com), which is based on the Dorfman, Berbaum, and Metz method.

Also, recall-based specificity and recall-based sensitivities for soft tissue densities, calcifications and overall cancers were compared for unassisted and CAD assisted readings. Bootstrapping [3] and General Estimation Equations (GEE) [4] methods were used for calculation of 95% confidence intervals and statistical significance of improvement.

Subgroup analysis looked at results for 11 subgroups stratified by lesion type (soft tissue densities and calcifications), by manufacturer (GE and Philips), by reader experience (radiologists reading more than 3,000 cases per year and radiologists reading less than 3,000 cases per year), by breast density (categories 1-2 and categories 3-4), and by lesion size (&lt;=10mm, between 10mm and 20mm, and &gt;20mm).

## D. Safety and Effectiveness Results

### 1. Safety Results

The device does not enter in direct contact with the patient and does not modify the mammography acquisition protocol. There is no known direct safety or health risks caused by, or related to, the use of the device. Indirect risks are that the device may fail to identify some malignant lesions and may identify some nonmalignant areas (false positive readings). There were no adverse events reported in the clinical study.

### 2. Effectiveness Results

#### Pivotal Reader Study Results:

The primary goal of the Pivotal Study was to demonstrate that patient-based ROC curves from CAD assisted mammography are superior to patient-based ROC curves from unassisted mammography. The areas under unassisted and CAD assisted ROC curves, which provide an estimate of the probability that a reader rates malignant cases as more suspicious than non-malignant ones, were estimated using two different models: Wilcoxon-trapezoidal (non-parametric) and PROPROC (parametric, reported below).

In both cases, the average area under the patient-based ROC curves for radiologists using CAD increased with statistical significance comparing to the average area under the patient-based ROC curves for the same radiologists interpreting the same cases without CAD (Table 6). Figure 8 shows graphs of the unassisted and CAD assisted ROC curves averaged over the readers.

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 13

{13}

Table 5: Primary Results of Pivotal Study: The Increase in Patient-based ROC AUC

| Model | No CAD (CI) | CAD assisted (CI) | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| PROCROC | 0.914(0.885, 0.943) | 0.930(0.903, 0.957) | 0.016(0.006, 0.025) | 0.0010 |

Difference = CAD assisted - No CAD.
$\mathrm{CI} = 95\%$ Confidence Interval.

The results in the table above demonstrate that the primary goal of the study was met. The use of Parascript® AccuDetect® 6.1.0 led to a significant increase in effectiveness for the group of 12 radiologists reading digital mammograms. The non-parametric model produces similar results.

Figure 8. Unassisted and CAD assisted patient-based ROC curves averaged over the readers (parametric model)

# 3. Subgroup Analyses

The secondary analyses were performed for informational purposes. The analyses included the area under lesion-based L-ROC curves for overall cancers, for soft tissue densities, and for calcifications; it included the area under patient-based ROC curves for GE images and Philips images as well as for radiologists reading more than 3,000 cases per year and for radiologists reading less than 3,000 cases per year; it also included recall-based sensitivity (for overall cancers, for soft tissue densities, for calcifications, for GE images, for Philips images) and recall-based specificity (overall, GE images and Philips images).

PMA P120004: FDA Summary of Safety and Effectiveness Data

{14}

All confidence intervals were calculated without adjustment for multiplicity, thus the statistical significance of the difference of performance with and without CAD cannot be assessed for secondary endpoints.

Figure 9 shows graphs of the unassisted and CAD assisted L-ROC curves averaged over the readers.

Figure 9. Unassisted and CAD assisted lesion-based L-ROC curves averaged over the readers

The average area under the lesion-based L-ROC curves for radiologists using CAD increased comparing to the average area under the lesion-based L-ROC curves for the same radiologists interpreting the same cases without CAD. The increase was observed for both lesion type subgroups as well as for overall cancers (Table 7).

Table 7. The Increase in Lesion-based L-ROC AUC

| Group | No CAD (CI) | CAD assisted (CI) | Difference (CI) |
| --- | --- | --- | --- |
| Overall | 0.817(0.773, 0.861) | 0.838(0.798, 0.879) | 0.021(0.011, 0.031) |
| Soft Tissue Densities | 0.825(0.776, 0.874) | 0.848(0.803, 0.893) | 0.023(0.012, 0.035) |
| Calcifications | 0.897(0.858, 0.937) | 0.910(0.873, 0.947) | 0.013(0.003, 0.022) |

Difference = CAD assisted - No CAD.
CI = 95% Confidence Interval.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{15}

The average area under the patient-based ROC curves for radiologists using CAD increased comparing to the average area under the patient-based ROC curves for the same radiologists interpreting the same cases without CAD for two manufacturer subgroups (GE and Philips), and for two radiologists subgroups (radiologists reading more than 3,000 cases per year and radiologists reading less than 3,000 cases per year) (Table 8).

Table 8. The Increase in Patient-based ROC AUC for subgroups

| Subgroup | No CAD (CI) | CAD assisted (CI) | Difference (CI) |
| --- | --- | --- | --- |
| GE | 0.904(0.858, 0.951) | 0.932(0.896, 0.968) | 0.028(0.009, 0.047) |
| Philips | 0.892(0.849, 0.934) | 0.914(0.875, 0.952) | 0.022(0.008, 0.037) |
| Breast Specialists(>3,000 cases per year) | 0.905(0.873, 0.937) | 0.928(0.902, 0.954) | 0.023(0.007, 0.038) |
| GeneralRadiologists(<3,000 cases per year) | 0.887(0.834, 0.940) | 0.915(0.877, 0.952) | 0.028(0.007, 0.049) |

The average recall-based sensitivity increased from 0.919 without CAD to 0.934 with CAD (Table 9). This represents $1.5\%$ more cancers detected and $18.8\%$ of missed cancers detected. Also, sensitivity increased for both lesion type subgroups and for both manufacturer subgroups.

Table 9. The Increase in Recall-based Sensitivity

| Group | No CAD (CI) | CAD assisted (CI) | Difference (CI) |
| --- | --- | --- | --- |
| Overall | 0.919(0.878, 0.946) | 0.934(0.892, 0.960) | 0.015(0.003, 0.029) |
| Soft Tissue Densities | 0.854(0.785, 0.901) | 0.882(0.814, 0.927) | 0.027(0.011, 0.047) |
| Calcifications | 0.933(0.864, 0.962) | 0.939(0.870, 0.969) | 0.006(-0.009, 0.016) |
| GE | 0.931(0.884, 0.961) | 0.949(0.885, 0.978) | 0.018(0.003, 0.036) |
| Philips | 0.907(0.839, 0.946) | 0.919(0.851, 0.958) | 0.0125(-0.006, 0.037) |

Difference = CAD assisted - No CAD.
CI = 95% Confidence Interval.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{16}

The average recall-based specificity increased with from 0.655 without CAD to 0.703 with CAD (Table 10). This represents 4.9% less normal cases recalled. Recall-based specificity also increased for both manufacturer subgroups.

Table 10. The Increase in Recall-based Specificity

| Group | No CAD (CI) | CAD assisted (CI) | Difference (CI) |
| --- | --- | --- | --- |
| Overall | 0.655
(0.602, 0.705) | 0.703
(0.647, 0.753) | 0.049
(0.029, 0.067) |
| GE | 0.636
(0.558, 0.707) | 0.692
(0.608, 0.765) | 0.056
(0.028, 0.082) |
| Philips | 0.674
(0.603, 0.739) | 0.715
(0.638, 0.779) | 0.042
(0.017, 0.067) |

Difference = CAD assisted – No CAD.<br/>
CI = 95% Confidence Interval.

## 4. CAD Standalone Study

Sensitivity, specificity and false positive rate of CAD (overall and for different subgroups) were measured in the CAD Standalone Study. The data set consisted of 300 cases (150 with malignant lesions, 135 normal cases, and 15 cases with actionable benign findings). There were 140 GE cases and 160 Philips cases in the set. 10% of non-cancer cases were actionable benign cases for each FFDM system.

Sensitivity was measured as a number of positive cases where at least one true positive mark is present divided by the total number of positive cases; specificity was measured as a number of normal and actionable benign cases without marks divided by the total number of normal and actionable benign cases; false positive rate (per image) was measured as a total number of marks in normal and actionable benign cases divided by the total number of images in normal and actionable benign cases.

Table 11 shows CAD sensitivity, specificity and false positive rate on the 150 cancer and 150 normal and actionable benign cases used in the Standalone Study as well as separately for calcifications and soft tissue densities. It also shows CAD sensitivity by cancer lesion size and CAD sensitivity by pathology. Table 14, Table 15, and Table 16 show sensitivity, specificity and false positive rate for different breast density categories.

Table 11: Standalone Sensitivity, Specificity, False Positive Rate Per Image on normal and benign cases stratified by detector type and breast density

| Stratification | Type of Lesion | Sensitivity (CI) | Specificity (CI) | False Positive Rate (CI) |
| --- | --- | --- | --- | --- |
| Overall | All lesion types | 131/150 = 87.3%
(82.0-92.7) | 67/150 = 44.7%
(36.7-52.6) | 256/600 = 0.427
(0.346-0.508) |
| | Soft tissue Densities | 90/108 = 83.3%
(76.3-90.4) | 83/150=55.3%
(47.4-63.3) | 143/600 = 0.238
(0.195-0.281) |

PMA P120004: FDA Summary of Safety and Effectiveness Data

{17}

Table 12: Sensitivity to different lesion sizes

| Stratification | Stratum | All lesion types | Soft tissue densities | Calcifications |
| --- | --- | --- | --- | --- |
| Lesion size | ≤10 mm | 28/36 = 77.8% (64.2-91.4) | 18/27 = 66.7% (48.9-84.4) | 87.9% (71.8-100.0) |
| | >10 and ≤20 mm | 49/55 = 89.1% (80.9-97.3) | 41/47 = 87.2% (77.7-96.8) | 14/17 = 82.4% (64.2-100.0) |
| | >20 mm | 31/34 = 91.2% (81.6-100.0) | 21/23 = 91.3% (79.8-100.0) | 15/17 = 88.2% (72.9-100.0) |
| Unknown | 23/25 = 92.0% (81.4-100.0) | 10/11 = 90.9% (73.9-100.0) | 13/14 = 92.9% (79.4-100.0) | |

# XI. PANEL MEETING RECOMMENDATION AND FDA'S POST-PANEL ACTION

In accordance with the provisions of section 515(c)(2) of the act as amended by the Safe Medical Devices Act of 1990, this PMA was not referred to the Radiological Advisory Panel, an FDA advisory committee, for review and recommendation because the information in the PMA substantially duplicates information previously reviewed by this panel.

PMA P120004: FDA Summary of Safety and Effectiveness Data

{18}

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 19

# XII. CONCLUSIONS DRAWN FROM PRECLINICAL AND CLINICAL STUDIES

## A. Effectiveness Conclusions

In the pivotal study, use of the Parascript® AccuDetect® 6.1.0 CAD resulted in a statistically significant increase in effectiveness for radiologists reading digital mammograms.

## B. Safety Conclusions

The risks of the device are based on software and standalone testing as well as data collected in the clinical study conducted to support PMA approval as described above.

The device does not enter in direct contact with the patient and does not modify the mammography acquisition protocol. There is no known direct safety or health risks caused by, or related to, the use of the device. Indirect risks are that the device may fail to identify some malignant lesions and may identify some nonmalignant areas (false positive readings). There were no adverse events reported in the clinical study.

## C. Benefit-Risk Conclusions

The probable benefits of the device are also based on data collected in a clinical study conducted to support PMA approval as described above.

The addition of CAD as a second reader led to an increase of 1.6% (from 91.4% to 93%) of the reader AUC under the ROC curve. The device will benefit to a small number of patients whose cancer may be detected earlier and have better treatment outcome. If the patient is cured from cancer, the benefit is lifelong.

Additional factors were considered in determining probable risks and benefits for the Parascript® AccuDetect® 6.1.0 device. The MRMC design is adequate and commonly accepted to determine the increase in accuracy of radiologists when using the CAD.

In general when using a CAD algorithm, there is a chance that less experienced radiologists will be swayed in their final decision by the CAD, leading to additional work-up to work out the false positives to true positives or negatives. The additional workup includes diagnostic imaging and sometimes breast biopsy. The study conducted with Parascript® AccuDetect® 6.1.0 did not show a significant decrease in specificity. Given that breast cancer is a lethal disease and that survival rates improve with early detection, the benefit of early cancer detection outweighs the remote possibility of complications resulting from potential additional workup.

In conclusion, given the available information above, the data support that, for Parascript® AccuDetect® 6.1.0, the probable benefits outweigh the probable risks.

{19}

D. Overall Conclusions

The data in this application supports the reasonable assurance of safety and effectiveness of this device when used in accordance with the indications for use. The results of the pivotal study demonstrate that the performance of the radiologists in identifying cancerous cases has been improved with the use of the Parascript® AccuDetect® 6.1.0 CAD. In the pivotal study, both reader sensitivity and reader specificity increased. However the confidence intervals of these quantities have not been adjusted for multiplicity, therefore the statistical significance of the increase in sensitivity and specificity cannot be assessed.

XIII. CDRH DECISION

CDRH issued an approval order on August 22, 2013. The final conditions of approval are cited in the approval order.

The applicant’s manufacturing facility was inspected and found to be in compliance with the device Quality System (QS) regulation (21 CFR 820).

XIV. APPROVAL SPECIFICATIONS

Directions for use: See device labeling.

Hazards to Health from Use of the Device: See Indications, Contraindications, Warnings, Precautions, and Adverse Events in the device labeling.

Post-approval Requirements and Restrictions: See approval order.

XV. REFERENCES

1. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Investigative Radiology 27:723-731 (1992).
2. Obuchowski NA. Nonparametric analysis of clustered ROC curve data. Biometrics, 53:567-578 (1997).
3. Efron B., Tibshirani R. An Introduction to the Bootstrap. (1993).
4. Lipsitz, SH., Fitzmaurice, GM., Orav EJ., Laird NM. Performance of Generalized Estimating Equations in Practical Situations. Biometrics, 50, 270-278 (1994).

PMA P120004: FDA Summary of Safety and Effectiveness Data
Page 20