M-VU ALGORITHM ENGINE

P100005 · Icad, Inc. · MYN · Jan 23, 2012 · Radiology

Device Facts

• Record ID: P100005
• Device Name: M-VU ALGORITHM ENGINE
• Applicant: Icad, Inc.
• Product Code: MYN · Radiology
• Decision Date: Jan 23, 2012
• Decision: APRL
• Regulation: 21 CFR 892.2070
• Device Class: Class 2

Intended Use

The M-Vu Algorithm Engine is intended for use in screening mammography to identify areas consistent with breast cancer for radiologist review after completing an initial read.

Device Story

M-Vu Algorithm Engine is a CAD software application for screening mammography; processes digitized film images from Mammo Pro film digitizer. Automatically analyzes images to identify potential masses (5mm-5cm) and microcalcification clusters (min 3 microcalcifications, 0.2-0.6mm). Used in clinic/radiology lab; operated by radiologists. Requires M-Vu CAD Station for processing and M-Vu Viewer Station for display. Radiologist reviews films conventionally first, then re-examines CAD-marked regions. Output displayed as mass marks (solid red line) or calcification marks (dotted red line) on monitor or printout. Assists radiologist in identifying malignant lesions; improves sensitivity for cancer detection; potential benefit of earlier/more accurate diagnosis; risk of false positives leading to unnecessary follow-up/biopsy.

Clinical Evidence

Pivotal retrospective reader study (n=21 radiologists, 280 cases: 140 positive, 140 negative). Primary endpoints: JAFROC figure of merit and area under per-case ROC curve. Results: JAFROC FOM increased from 0.812 to 0.839 (p=0.001); AUC increased from 0.885 to 0.902 (p=0.013). Per-case sensitivity (recall) increased from 0.865 to 0.901 (p=0.002); specificity decreased from 0.649 to 0.623 (p<0.001). CAD standalone study showed 79.3% sensitivity and 28.6% specificity.

Technological Characteristics

Software-based CAD for digitized film mammography. Requires M-Vu CAD Station hardware. Connectivity via DICOM interface. Software development governed by IEC 62304:2006. Detects masses (5mm-5cm) and microcalcifications (0.2-0.6mm).

Indications for Use

Indicated for use in screening mammography to identify areas consistent with breast cancer in female patients at least 18 years of age. No contraindications.

Regulatory Classification

Identification

Medical image analyzers, including computer-assisted/aided detection (CADe) devices for mammography breast cancer, ultrasound breast lesions, radiograph lung nodules, and radiograph dental caries detection, is a prescription device that is intended to identify, mark, highlight, or in any other manner direct the clinicians' attention to portions of a radiology image that may reveal abnormalities during interpretation of patient radiology images by the clinicians. This device incorporates pattern recognition and data analysis capabilities and operates on previously acquired medical images. This device is not intended to replace the review by a qualified radiologist, and is not intended to be used for triage, or to recommend diagnosis.

Special Controls

*Classification.* Class II (special controls). The special controls for this device are:(1) Design verification and validation must include: (i) A detailed description of the image analysis algorithms including a description of the algorithm inputs and outputs, each major component or block, and algorithm limitations. (ii) A detailed description of pre-specified performance testing methods and dataset(s) used to assess whether the device will improve reader performance as intended and to characterize the standalone device performance. Performance testing includes one or more standalone tests, side-by-side comparisons, or a reader study, as applicable. (iii) Results from performance testing that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures ( *e.g.,* receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain a sufficient number of cases from important cohorts (*e.g.,* subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Appropriate software documentation ( *e.g.,* device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results; and cybersecurity).(2) Labeling must include the following: (i) A detailed description of the patient population for which the device is indicated for use. (ii) A detailed description of the intended reading protocol. (iii) A detailed description of the intended user and user training that addresses appropriate reading protocols for the device. (iv) A detailed description of the device inputs and outputs. (v) A detailed description of compatible imaging hardware and imaging protocols. (vi) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level ( *e.g.,* poor image quality or for certain subpopulations), as applicable.(vii) Device operating instructions. (viii) A detailed summary of the performance testing, including: test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.

Reference Devices

• M-Vu CAD Station (K061160)
• M-Vu Viewer Station (K060451)

Submission Summary (Full Text)

{0}

SUMMARY OF SAFETY AND EFFECTIVENESS DATA (SSED)

I. GENERAL INFORMATION

Device Generic Name: Image Analysis System
Device Trade Name: M-Vu Algorithm Engine
Applicant's Name and Address: VuCOMP, Inc.
2500 Dallas Parkway
Suite 500
Plano, Texas 75093
Date(s) of Panel Recommendation: None
Premarket Approval Application (PMA) Number: P100005
Date of FDA Notice of Approval: January 23, 2012
Expedited: Not Applicable

II. INDICATIONS FOR USE

The M-Vu Algorithm Engine is intended for use in screening mammography to identify areas consistent with breast cancer for radiologist review after completing an initial read.

III. CONTRAINDICATIONS

None

IV. WARNINGS AND PRECAUTIONS

The warnings and precautions can be found in the M-Vu Algorithm Engine labeling.

V. DEVICE DESCRIPTION

M-Vu Algorithm Engine is a Computer-Aided Detection (CAD) software device intended to aid radiologists reading mammograms. It is a proprietary software application designed to process digitized film images. The digital images are automatically analyzed to mark areas for review by a radiologist. Results are displayed on either a computer monitor or printout. The radiologist is instructed to first review each case in the conventional manner and then re-examine regions marked by the M-Vu system before making a final assessment for the case.

The M-Vu Algorithm Engine is used in combination with two specific components also produced by VuCOMP: the M-Vu CAD Station and the M-Vu Viewer Station. The M-Vu CAD Station, cleared under the Premarket Notification K061160 (June 9, 2006),

PMA P100005: FDA Summary of Safety and Effectiveness Data

{1}

provides the computing hardware for the M-Vu Algorithm Engine. The M-Vu Viewer Station, cleared under the Premarket Notification K060451 (March 22, 2006), provides a platform for reviewing the electronic output of the M-Vu Algorithm Engine.

The M-Vu Algorithm Engine requires the M-Vu CAD Station for operation. The Algorithm Engine software is installed at VuCOMP by trained technicians. Figure 1 illustrates the M-Vu System configuration.

Figure 1: M-Vu System Configuration

The M-Vu Algorithm Engine receives input images from the M-Vu CAD Station by way of a DICOM interface. The M-Vu Algorithm Engine will only process digitized film images created by the Mammo Pro film digitizer made by Array Corporation.

The M-Vu Algorithm Engine results are displayed for the radiologist on either a printout or the M-Vu Viewer Station. The M-Vu Viewer Station is typically positioned near a motorized film viewer or light box. The M-Vu Algorithm Engine results may be sorted in a desired order by using the M-Vu Viewer Station's barcode reader. Each case has a unique barcode that is printed at the top of the M-Vu CAD Station printed results page.

For each mammogram, a radiologist is instructed to first review the films thoroughly before enabling display of the mammogram's M-Vu Algorithm Engine results on the M-Vu Viewer Station.

The M-Vu Algorithm Engine produces two types of marks: mass marks and calcification marks. A mass mark consists of an area enclosed by a solid red line. A radiologist is instructed to carefully consider the area in a mass mark for the possibility of a mass. A calcification mark consists of an area enclosed by a dotted red line. A radiologist is instructed to carefully consider the area in a calcification mark for the possibility of a

PMA P100005: FDA Summary of Safety and Effectiveness Data

{2}

cluster of microcalcifications. Below, are examples of mass marks (Figure 2) and calcification marks (Figure 3).

Figure 2: Mass Mark

Figure 3: Calcification Mark

The M-Vu Algorithm Engine can detect masses with a diameter between 5 mm and 5 cm. The M-Vu Algorithm Engine can detect individual microcalcifications between 0.2 mm and 0.6 mm in diameter and can detect microcalcifications clusters consisting of at least 3 microcalcifications.

## VI. ALTERNATIVE PRACTICES AND PROCEDURES

There are alternatives to using a computer aided detection system for the detection of breast cancer. The current procedure for reviewing film mammograms involves a radiologist’s review of the films on a motorized film viewer or light box. Although not commonly performed, each mammogram may be read by more than one radiologist in order to increase the accuracy of screening mammography.

## VII. MARKETING HISTORY

The M-Vu Algorithm Engine has been sold in Japan since 2010.

## VIII. POTENTIAL ADVERSE EFFECTS OF THE DEVICE ON HEALTH

There are no known direct safety or health risks caused by, or related to, the use of the device. Indirect risks are that the device may fail to mark some malignant lesions and may mark some nonmalignant lesions (false positive readings). If a doctor determines that a false positive CAD mark indicates an area that is suspicious enough for follow-up, then the patient may be subjected to unnecessary concern and/or biopsy.

## IX. SUMMARY OF NON-CLINICAL STUDIES

VuCOMP utilizes the IEC 62304:2006 Medical device software – Software life-cycle processes standard to govern software development activities. Non-clinical studies were conducted throughout the design and development of the M-Vu Algorithm Engine. These studies were designed to ensure that the M-Vu Algorithm Engine meets its specifications and intended use.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{3}

PMA P100005: FDA Summary of Safety and Effectiveness Data
page 4

## A. Assessment of CAD Algorithm Performance

Quantitative assessment of the CAD algorithm was conducted using a VuCOMP in-house test library of 1,708 mammograms. The results demonstrate that the system is able to detect areas associated with microcalcifications and masses.

## B. Software/System Verification and Validation

VuCOMP performed verification and validation testing on the M-Vu Algorithm Engine software. Verification testing consisted of software unit testing, software integration testing, and software system testing. Internal validation testing consisted of measuring device standalone performance including sensitivity, specificity and false positives per image (FPPI) using an in-house test library of over 1,000 mammograms. Overall sensitivity was measured at 83.6%, case specificity at 36.9%, and FPPI of 0.399.

## C. System Repeatability

VuCOMP performed repeatability testing for the M-Vu Algorithm Engine. A set of mammograms were scanned multiple times on the same scanner to study the consistency of CAD marks related to the variability associated with the film digitization process. Test results demonstrated that the M-Vu Algorithm Engine performed to product specifications throughout the testing period. The M-Vu Algorithm Engine repeatability was measured at 93.5%.

## X. SUMMARY OF PRIMARY CLINICAL STUDIES

The primary objective of the clinical studies was to determine whether radiologists are more effective at reading screen-film mammograms when using the M-Vu Algorithm Engine versus when not using Computer-Aided Detection (CAD).

The University of North Carolina (UNC) served as the Clinical Research Organization for this study under the direction of the Primary Investigator, Etta D. Pisano, MD. The following retrospective studies were performed:

- A pivotal reader study to compare the effectiveness of radiologists reading screen-film mammograms when using the M-Vu Algorithm Engine versus when not using CAD.
- A CAD standalone study to measure behavior of the M-Vu Algorithm Engine separately from the radiologists.

## A. Study Readers and Cases

The Pivotal Study used 21 radiologists reading 280 cases. The radiologists were from a variety of academic, specialty, and community clinics located across the United States.

The cases were a randomly selected set of 140 positive cases and 140 negative cases drawn from 11 United States sites representing academic, specialty, and community clinics. Each site received approval to provide cases for this study by their respective

{4}

Institutional Review Boards. The cases were selected such that no more than 10% of the positive cases and no more than 10% of the negative cases came from any one site.

A positive case was defined as an exam having a biopsy-proven breast cancer found within 15 months following the exam date. A negative case was defined as an exam for which breast cancer had not been found within 15 months prior or 15 months after the exam date, and for which at least one associated subsequent negative exam had been taken at least 11 months after the exam date. The inclusion and exclusion criteria were as follows:

## Inclusion Criteria:

- Cases are screen-film mammograms from exams performed during the years 1998 through 2006.
- Patients who are female and at least 18 years of age, having had a mammography exam and relevant history for determining positive or negative status.
- Cases that include four views: LCC, RCC, LMLO, and RMLO (two for unilateral studies).

## Exclusion Criteria:

- Cases that are diagnostic (e.g., to explore palpable lesions or other symptoms) instead of screening.
- Cases that include any film copies.
- Cases with implants but no implant-displaced views.
- Cases not acquired from an MQSA certified facility.
- Cases without sufficient patient information to facilitate truthing, which includes basic demographic information (age, race, ethnicity) and appropriate radiology reports, biopsy reports, and pathology reports as necessary within two years after mammogram date.
- Cases of patients who were either pregnant or nursing at the time of imaging.
- Cases with film containing indelible marks, or markers in film intended to indicate prior biopsy sites (scars).
- Cases otherwise not meeting the inclusion criteria.

Each site submitted original screen-film mammograms, acetate overlays indicating the location of each known cancer, de-identified clinical reports (including radiology, surgical, and pathology reports), and study-specific case report forms. Tables 1 through 4 characterize the 280 cases used in the pivotal study.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{5}

Table 1: Age Distribution of Cases included in the Pivotal Study

| Age | Total |
| --- | --- |
| < 40 | 5 |
| 40-49 | 53 |
| 50-59 | 91 |
| 60-69 | 69 |
| 70-79 | 47 |
| >= 80 | 15 |

Table 2: Breast Density Distribution of Cases included in Pivotal Study

| Breast Density Type | Cancer Cases | Negative Cases |
| --- | --- | --- |
| Extremely Dense | 20 | 28 |
| Heterogeneously Dense | 43 | 32 |
| Scattered Fibroglandular | 55 | 57 |
| Mostly Fatty | 22 | 23 |

Table 3: Size Distribution of Cancers

| Size (mm) | Totals |
| --- | --- |
| <= 8 | 31 |
| 8 - 12.5 | 25 |
| 12.5 - 17 | 28 |
| > 17 | 27 |
| Unknown | 29 |

Table 4: Pathology of Cancers

| Diagnosis | Totals |
| --- | --- |
| IDC | 65 |
| ILC | 7 |
| DCIS | 32 |
| DCIS+LCIS | 1 |
| IDC+DCIS | 26 |
| IDC+DCIS+ILC | 1 |
| IDC+DCIS+ILC+LCIS | 2 |
| IDC+ILC | 1 |
| IDC+ILC+LCIS | 1 |
| IDC+LCIS | 1 |
| LCIS+ILC | 1 |
| TC | 1 |
| MC | 1 |

DCIS = Ductal Carcinoma In Situ
IDC = Invasive Ductal Carcinoma
ILC = Invasive Lobular Carcinoma
LCIS = Lobular Carcinoma In Situ
MC = Mucinous Carcinoma
TC = Tubular Carcinoma

PMA P100005: FDA Summary of Safety and Effectiveness Data

{6}

PMA P100005: FDA Summary of Safety and Effectiveness Data
page 7

## B. Study Execution

The clinical studies were conducted at the University of North Carolina (UNC) located in Chapel Hill, North Carolina. Care was taken to mimic the clinical environment of a radiology lab during the study. Environmental conditions similar to a typical clinical environment were established, including temperature, ambient light, light sources (less than 50 lux), level of comfort, level of furnishings, and ambient noise.

Cases were presented to the readers in random order. For each case, each reader performed the following actions in order:

1. Evaluate the case without seeing marks from the M-Vu Algorithm Engine
2. Record a “without CAD” assessment for the case
3. View the marks created by the M-Vu Algorithm Engine for the case
4. Record a “with CAD” assessment for the case

The “with CAD” and “without CAD” assessments included the following information:

- Whether the reader would recall the patient, and why (suspicious finding or technical problem)
- Screening BI-RADS (0, 1, 2, 3, 4a, 4b, 4c, 5)
- Forced BI-RADS (1, 2, 3, 4a, 4b, 4c, 5) if screening BI-RADS was “0”
- Lesion findings

The lesion findings included the following information for each individual lesion finding:

- Laterality (left or right)
- Type (Mass, Architectural Distortion, Asymmetry, or Calcification)
- BI-RADS (1, 2, 3, 4a, 4b, 4c, 5)
- Probability of Malignancy (0-100%)

## C. Pivotal Study Statistical Methods

The statistical analysis estimated a smooth receiver operating characteristic (ROC) curve $^{1,2}$ for each of the 21 study readers in each test condition (without CAD and with CAD) using the probability of malignancy (POM) ratings each reader provided. Each ROC curve was estimated using proper ROC models in DBM MRMC software $^{3}$. For each reader, the statistical analysis computed differences between the readings with CAD and without CAD in terms of area under the ROC curve as well as the uncertainty using 95% confidence intervals while taking into account correlations that arose because each reader interpreted the same cases in both conditions. The statistical analysis used the method of Dorfman, Berbaum, and Metz $^{4}$ with proper binomial models and random effects for readers in DBM MRMC software $^{3}$ to perform multi-reader, multi-case (MRMC) analysis and compare area under the ROC curve between conditions. The statistical analysis also used MRMC methods with fixed effect for reading condition and random effects for readers to analyze FROC

{7}

curves⁵, sensitivities (per-case and per-lesion), specificities (per-case), and false-positive marks per image⁶. Subgroup analyses looked at results for masses and for calcifications.

In all analysis using BI-RADS category, a “forced BI-RADS” value was used. This is a nonzero value (1, 2, 3, 4a, 4b, 4c, or 5) provided by the reader even if the reader would have normally used a value of zero in a screening context.

## D. Pivotal Study Primary Results

The primary aim of the Pivotal Study was to determine if radiologists reading screen-film mammograms were more effective at finding cancer when using the M-Vu Algorithm Engine versus when not using CAD. This aim was further divided into 1) effectiveness in finding malignant lesions, and 2) effectiveness in finding malignant cases.

The effectiveness of the radiologists in finding malignant lesions was analyzed with the JAFROC figure of merit⁷, which provides an estimate of the probability that a reader rates malignant lesions as more suspicious than non-malignant findings. The measured figure of merit for radiologists using CAD was significantly larger (p = 0.001) than the figure of merit for the same radiologists interpreting the same cases without CAD (Table 5).

The effectiveness of the radiologists in finding malignant cases was analyzed with the area under the per-case ROC curve, which provides an estimate of the probability that a reader rates malignant cases as more suspicious than non-malignant cases. The average area under the per-case ROC curve for radiologists using CAD was significantly larger (p = 0.013) than the average area under the ROC curve for the same radiologists interpreting the same cases without CAD (Table 5). Figure 4 shows graphs of the ROC curve for the without-CAD and with-CAD conditions.

Table 5: Primary Results of Pivotal Study – JAFROC Figure of Merit (FOM) and Area under the Per-Case ROC Curve (AUC)

| Analysis | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| FOM | 0.812 | 0.839 | 0.027 (0.012, 0.043) | 0.001 |
| AUC | 0.885 | 0.902 | 0.016 (0.004, 0.029) | 0.013 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

All primary aims of the study were met. This demonstrates that use of the M-Vu Algorithm Engine led to a significant increase in effectiveness for the group of 21 radiologists reading screen-film mammograms.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{8}

Figure 4: Radiologist ROC Curves

## E. Pivotal Study Secondary Results

Although the Pivotal Study was designed to only show statistical significance for the primary aims, additional secondary analysis was performed for informational purposes. This analysis included radiologist sensitivity, specificity, and area under the per-case ROC curve for two subgroups (masses and calcification clusters).

The average radiologist sensitivity (based on recall) increased significantly (p &lt; 0.002) from 0.865 without CAD to 0.901 with CAD (Table 6). This represents an increase of 4.2% more cancers detected and 26.7% of missed cancers detected. Radiologist sensitivity also increased significantly for calcification cases (p = 0.001) and mass cases (p = 0.016). The overall sensitivity increase was accompanied by a

PMA P100005: FDA Summary of Safety and Effectiveness Data

{9}

smaller, but still statistically significant (p &lt; 0.001) decrease in specificity (based on recall) from 0.649 to 0.623 (Table 9).

The average radiologist sensitivity (based on BI-RADS category 3 or higher) increased significantly (p = 0.004) from 0.851 without CAD to 0.885 with CAD (Table 7). Radiologist sensitivity also increased significantly for calcification cases (p = 0.003) and mass cases (p = 0.036). The overall sensitivity increase was accompanied by a smaller, but still statistically significant (p = 0.001) decrease in specificity (based on BI-RADS) from 0.684 to 0.658 (Table 9).

Table 6: Radiologist Per-Case Sensitivity Based on Recall

| Group | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| Overall | 0.865 | 0.901 | 0.036 (0.014, 0.058) | 0.002 |
| Calcification | 0.830 | 0.882 | 0.052 (0.021, 0.083) | 0.001 |
| Mass | 0.897 | 0.918 | 0.021 (0.004, 0.038) | 0.016 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

Table 7: Radiologist Per-Case Sensitivity Based on BI-RADS Category

| Group | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| Overall | 0.851 | 0.885 | 0.033 (0.011, 0.055) | 0.004 |
| Calcification | 0.817 | 0.866 | 0.049 (0.017, 0.080) | 0.003 |
| Mass | 0.885 | 0.904 | 0.018 (0.001, 0.035) | 0.036 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

The average radiologist per-lesion sensitivity (based on BI-RADS category 3 or higher) increased significantly for the Overall (p &lt; 0.001), Calcification (p &lt; 0.001), and Mass (p = 0.004) groups (Table 8).

Table 8: Radiologist Per-Lesion Sensitivity Based on BI-RADS Category

| Group | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| Overall | 0.792 | 0.834 | 0.043 (0.023, 0.063) | < 0.001 |
| Calcification | 0.731 | 0.797 | 0.067 (0.034, 0.100) | < 0.001 |
| Mass | 0.851 | 0.871 | 0.021 (0.007, 0.034) | 0.004 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{10}

Table 9: Radiologist Per-Case Specificity

| Basis | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| Recall | 0.649 | 0.623 | -0.026 (-0.039, -0.013) | < 0.001 |
| BI-RADS | 0.684 | 0.658 | -0.024 (-0.037, -0.011) | 0.001 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

Analysis of the average area under the radiologist per-case ROC curve was divided into calcification and mass subgroups. The calcification ROC analysis used the 69 malignant calcification cases along with all 140 negative cases. The mass ROC analysis used the 86 malignant mass cases along with all 140 negative cases. All reader findings were used regardless of finding type. Consequently, to show improvement in the calcification ROC curve, the radiologist improvement in finding malignant calcifications must individually outweigh any specificity degradation due to both calcification false positives and mass false positives. Similarly, improvement in finding malignant masses must individually outweigh any specificity degradation due to both calcification false positives and mass false positives. This may result in a conservative estimate of improvement in radiologist performance for each subgroup. Table 10 shows that the area under the curve increased for both calcifications and masses. The increase was statistically significant for calcifications (p = 0.007), but not for masses (p = 0.27). Figures 5 and 6 show graphs of the ROC curves for the subgroups.

Table 10: Subgroup Analysis of Area under the Per-Case ROC Curve Based on Probability of Malignancy where specificity is calculated using false positives of all types (both calcification and mass)

| Group | Without CAD | With CAD | Difference (CI) | P-value |
| --- | --- | --- | --- | --- |
| Calcification | 0.867 | 0.891 | 0.024 (0.007, 0.042) | 0.007 |
| Mass | 0.910 | 0.914 | 0.004 (-0.004, 0.012) | 0.27 |

Difference = with CAD - without CAD.
CI = 95% Confidence Interval.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{11}


Figure 5: Radiologist Calcification ROC Curves Based on Probability of Malignancy where specificity is calculated using false positives of all types (both calcification and mass)

PMA P100005: FDA Summary of Safety and Effectiveness Data
page 12

{12}


Figure 6: Radiologist Mass ROC Curves Based on Probability of Malignancy where specificity is calculated using false positives of all types (both calcification and mass)

PMA P100005: FDA Summary of Safety and Effectiveness Data
page 13

{13}

# F. CAD Standalone Study

CAD sensitivity was measured as the proportion of cancer cases that were true positives. A case true positive occurred if a case had at least one malignant region found by CAD. A malignant region was considered found if a CAD mark centroid was inside the region or if the region centroid was inside a CAD mark. CAD specificity was measured as the proportion of negative cases that were true negatives. A case true negative occurred if a negative case had no CAD marks on it. Table 11 shows the CAD sensitivity on the 140 cancer cases used in the Pivotal Study. Table 12 shows the CAD sensitivity by lesion size. Table 13 shows the CAD sensitivity by pathology result. Table 14 shows the CAD sensitivity by breast density type. Table 15 shows the CAD specificity for the 140 negative cases used in the Pivotal Study.

Table 11: CAD Sensitivity

| | Cases | Sensitivity | Confidence Interval |
| --- | --- | --- | --- |
| Overall | 140 | 79.3% | (72.6%, 86.0%) |
| Calcification | 69 | 79.7% | (70.2%, 89.2%) |
| Mass | 86 | 81.4% | (73.2%, 89.6%) |

Table 12: CAD Sensitivity by Lesion Size with Confidence Intervals in Parenthesis

| Lesion Size (mm) | Calcification Sensitivity | Mass Sensitivity | Overall Sensitivity |
| --- | --- | --- | --- |
| <= 8 | 7/13=53.8%
(26.7%, 80.9%) | 15/20=75.0%
(56.0%, 94.0%) | 21/31=67.7%
(51.3%, 84.2%) |
| > 8 and <= 12.5 | 5/6=83.3%
(53.5%, 100%) | 16/21=76.2%
(58.0%, 94.4%) | 19/25=76.0%
(59.3%, 92.7%) |
| > 12.5 and <= 17 | 12/12=100%
(100%, 100%) | 19/21=90.5%
(77.9%, 100%) | 26/28=92.9%
(83.3%, 100%) |
| > 17 | 14/16=87.5%
(71.3%, 100%) | 13/15=86.7%
(69.5%, 100%) | 23/27=85.2%
(71.8%, 98.6%) |
| No Measurement | 17/22=77.3%
(59.8%, 94.8%) | 7/9=77.8%
(50.6%, 100%) | 22/29=75.9%
(60.3%, 91.4%) |

PMA P100005: FDA Summary of Safety and Effectiveness Data

{14}

Table 13: CAD Sensitivity by Pathology with Confidence Intervals in Parenthesis

| Pathology | Calcification Sensitivity | Mass Sensitivity | Overall Sensitivity |
| --- | --- | --- | --- |
| IDC | 32/39=82.1%
(70.0%, 94.1%) | 59/69=85.5%
(77.2%, 93.8%) | 80/97=82.5%
(74.9%, 90.0%) |
| DCIS | 34/44=77.3%
(64.9%, 89.7%) | 16/24=66.7%
(47.8%, 85.5%) | 45/62=72.6%
(61.5%, 83.7%) |
| Other | 3/4=75.0%
(32.6%, 100%) | 11/14=78.6%
(57.1%, 100%) | 13/17=76.5%
(56.3%, 96.6%) |

Table 14: CAD Sensitivity by Breast Density Type with Confidence Intervals in Parenthesis

| Breast Density | Calcification Sensitivity | Mass Sensitivity | Overall Sensitivity |
| --- | --- | --- | --- |
| Extremely Dense | 8/11 = 72.7%
(46.4%, 99.0%) | 7/9 = 77.8%
(50.6%, 100%) | 15/20 = 75.0%
(56.0%, 94.0%) |
| Heterogeneously Dense | 18/22 = 81.8%
(65.7%, 97.9%) | 23/25 = 92.0%
(81.4%, 100%) | 37/43 = 86.0%
(75.7%, 96.4%) |
| Scattered Fibroglandular | 21/28 = 75.0%
(59.0%, 91.0%) | 27/35 = 77.1%
(63.2%, 91.1%) | 41/55 = 74.5%
(63.0%, 86.1%) |
| Fatty | 8/8 = 100%
(100%, 100%) | 13/17 = 76.5%
(56.3%, 96.6%) | 18/22 = 81.8%
(65.7%, 97.9%) |

Table 15: CAD Specificity

| | Cases | Specificity | Confidence Interval |
| --- | --- | --- | --- |
| All Cases | 140 | 28.6% | (21.1%, 36.1%) |
| Extremely Dense | 28 | 50.0% | (31.5%, 68.5%) |
| Heterogeneously Dense | 32 | 28.1% | (12.5%, 43.7%) |
| Scattered Fibroglandular | 57 | 22.8% | (11.9%, 33.7%) |
| Fatty | 23 | 17.4% | (1.9%, 32.9%) |

PMA P100005: FDA Summary of Safety and Effectiveness Data

{15}

Table 16 shows the average CAD false positives per image (FPPI) for the 140 negative cases used in the Pivotal Study.

Table 16: CAD FPPI with Confidence Intervals in Parenthesis

| | Total
FPPI | Calcification
FPPI | Mass
FPPI |
| --- | --- | --- | --- |
| All Cases | 0.418
(0.346, 0.490) | 0.088
(0.042, 0.133) | 0.330
(0.275, 0.385) |
| Extremely Dense | 0.205
(0.092, 0.318) | 0.063
(0.000, 0.148) | 0.143
(0.084, 0.202) |
| Heterogeneously Dense | 0.438
(0.253, 0.622) | 0.133
(0.000, 0.290) | 0.305
(0.207, 0.402) |
| Scattered Fibroglandular | 0.443
(0.336, 0.550) | 0.079
(0.030, 0.128) | 0.364
(0.270, 0.458) |
| Fatty | 0.587
(0.419, 0.755) | 0.076
(0.004, 0.148) | 0.511
(0.362, 0.660) |

Due to the randomness of the film digitization process, no two images created by a film digitizer are ever exactly the same. This causes some variation in the CAD results.

The repeatability of cancer detection was measured by randomly selecting 27 cancer cases, scanning them multiple times on multiple digitizers, running each case’s scans through CAD, and then analyzing how often outcomes (either true positive case or false negative case) were repeated. Three different MammoPro film digitizers (Array Corporation USA, Hampton, NH) were used. Each case was scanned 10 times (3 times on one digitizer, 3 times on another, and 4 times on the third). Repeatability was measured as the proportion of the number of outcomes that were in the majority. Mathematically, this is expressed by $\max(\mathrm{TP}, \mathrm{FN}) / (\mathrm{TP} + \mathrm{FN})$, where TP is the number of true positive case outcomes and FN is the number of false negative case outcomes. Table 17 shows the CAD repeatability over the 27 cancer cases and 3 film digitizers.

Table 17: CAD Repeatability with Confidence Intervals in Parenthesis

| Overall | Scanner A | Scanner B | Scanner C |
| --- | --- | --- | --- |
| 0.922 (0.866, 0.979) | 0.963 (0.923, 1.000) | 0.963 (0.923, 1.000) | 0.944 (0.897, 0.992) |

The variability of false marks was evaluated by randomly selecting 23 negative cases, scanning them multiple times on multiple digitizers, running each case’s scans through CAD, and then measuring the False Positives per Image (FPPI). The same three digitizers were again used in the same way as above to scan each case 10 times. The CAD FPPI mean was measured by first calculating the mean FPPI for each case

PMA P100005: FDA Summary of Safety and Effectiveness Data

{16}

over all the scans of that case and then calculating the mean and standard error across all the case means. This process was repeated for the 27 cancer case scans used in the cancer case repeatability analysis above. The CAD FPPI mean over the 23 negative cases and 27 cancer cases is shown in Table 18.

Table 18: CAD FPPI Mean for Repeatability Cases with the Associated Standard Error in Parenthesis

| | Overall | Scanner A | Scanner B | Scanner C |
| --- | --- | --- | --- | --- |
| Negative cases | 0.339 | 0.344 | 0.369 | 0.313 |
| | (0.060) | (0.063) | (0.067) | (0.057) |
| Cancer cases | 0.396 | 0.426 | 0.380 | 0.387 |
| | (0.056) | (0.061) | (0.052) | (0.059) |

## XI. PANEL MEETING RECOMMENDATION AND FDA'S POST-PANEL ACTION

In accordance with the provisions of section 515(c)(2) of the act as amended by the Safe Medical Devices Act of 1990, this PMA was not referred to the Radiological Advisory Panel, an FDA advisory committee, for review and recommendation because the information in the PMA substantially duplicates information previously reviewed by this panel.

## XII. CONCLUSIONS DRAWN FROM PRECLINICAL AND CLINICAL STUDIES

### A. Safety Conclusions

There are no known direct safety or health risks caused by, or related to, the use of the device. Indirect risks are that the device may fail to mark some malignant lesions and may mark some nonmalignant areas (false positive readings). There were no adverse events reported in the clinical study.

### B. Effectiveness Conclusions

In the pivotal study, use of the M-Vu Algorithm Engine resulted in a statistically significant increase in effectiveness for radiologists reading screen-film mammograms.

### C. Overall Conclusions

The data in this application support the reasonable assurance of safety and effectiveness of this device when used in accordance with the indications for use. The results of the pivotal study demonstrate that the performance of the radiologists in identifying cancerous cases improved with the use of the M-Vu CAD Algorithm Engine. The improvement in finding malignant lesions and identifying malignant cases outweighed the associated reduction in specificity.

PMA P100005: FDA Summary of Safety and Effectiveness Data

{17}

# XIII. CDRH DECISION

CDRH issued an approval order on January 23, 2012. The final conditions of approval are cited in the approval order.

The applicant’s manufacturing facility was inspected and found to be in compliance with the device Quality System (QS) regulation (21 CFR 820).

# XIV. APPROVAL SPECIFICATIONS

Directions for use: See device labeling.

Hazards to Health from Use of the Device: See Indications, Contraindications, Warnings, Precautions, and Adverse Events in the device labeling.

Post-approval Requirements and Restrictions: See approval order.

# XV. REFERENCES

1. Pesce LL, Metz CE. Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. Academic Radiology 14(7):814-29 (2007).

2. Metz CE, Pan X. “Proper” binormal ROC curves: Theory and maximum-likelihood estimation. Journal of Mathematical Psychology 43:1–33 (1999).

3. DBM MRMC software: http://krl.bsd.uchicago.edu/KRL_ROC/software_index.htm

4. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Investigative Radiology 27:723-731 (1992).

5. Chakraborty DP, Berbaum KS. Observer studies involving detection and localization: modeling, analysis, and validation. Medical Physics 31(8):2313-30 (2004).

6. Obuchowski NA, Zepp RC. Simple steps for improving multiple-reader studies in radiology. AJR American Journal of Roentgenology 166(3):517-21 (1996)

7. JAFROC_V3d_BETA software and choice of figure of merit: http://www.devchakraborty.com/index.php

PMA P100005: FDA Summary of Safety and Effectiveness Data
page 18