What is the FDA product code for Genius Digital Diagnostics System with the Genius Cervical AI algorithm?

QYV. It is classified under 21 CFR 864.3900 as a Class 2 device in the Pathology panel.

What is the regulatory pathway for Genius Digital Diagnostics System with the Genius Cervical AI algorithm?

De Novo classification (submission type: Direct).

Who can help prepare an FDA De Novo submission like Genius Digital Diagnostics System with the Genius Cervical AI algorithm?

Innolitics — a medical-device software consultancy — provides regulatory and engineering support for FDA De Novo submissions. See https://innolitics.com/services/regulatory/ or contact https://innolitics.com/contact.

Genius Digital Diagnostics System with the Genius Cervical AI algorithm

Q: Who is the manufacturer of Genius Digital Diagnostics System with the Genius Cervical AI algorithm?

Hologic, Inc. filed the De Novo classification request for Genius Digital Diagnostics System with the Genius Cervical AI algorithm (DEN210035).

Q: Does Genius Digital Diagnostics System with the Genius Cervical AI algorithm use AI or machine learning?

Yes. Genius Digital Diagnostics System with the Genius Cervical AI algorithm (DEN210035) is flagged as an AI/ML-enabled medical device in the FDA's AI/ML device list.

Q: Can an AI/ML device like Genius Digital Diagnostics System with the Genius Cervical AI algorithm get a guaranteed FDA De Novo clearance?

Yes. Innolitics offers a guaranteed FDA De Novo classification engagement for AI/ML Software as a Medical Device — combining engineering and regulatory execution under one agreement with a guaranteed clearance outcome. Eligibility: the device must be an AI/ML Software as a Medical Device (SaMD), and its performance must meet the criteria Innolitics determines during an initial regulatory strategy engagement to be necessary for FDA clearance. See https://innolitics.com/services/end-to-end-samd/ or contact https://innolitics.com/contact.

DEN210035 · Hologic, Inc. · QYV · Jan 31, 2024 · Pathology

Device Facts

Record ID	DEN210035
Device Name	Genius Digital Diagnostics System with the Genius Cervical AI algorithm
Applicant	Hologic, Inc.
Product Code	QYV · Pathology
Decision Date	Jan 31, 2024
Decision	DENG
Submission Type	Direct
Regulation	21 CFR 864.3900
Device Class	Class 2
Attributes	AI/ML

Intended Use

The Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended for the creation and viewing of digital images of scanned ThinPrep® Pap Test glass slides. Objects of interest selected by the Genius™ Cervical AI algorithm from the scanned digital image are presented in a gallery format next to the image of the whole cell spot on the Genius™ Review Station for review and interpretation. The Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended to aid in cervical cancer screening for the presence of atypical cells, cervical neoplasia, including its precursor lesions (Low Grade Squamous Intraepithelial Lesions, High Grade Squamous Intraepithelial Lesions) and carcinoma, as well as all other cytological categories as defined by The Bethesda System for Reporting Cervical Cytology. After digital review with the Genius Cervical AI algorithm, if there is uncertainty in the diagnosis, then direct examination of the glass slide by light microscopy should be performed. Digital images from the Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm should be interpreted by qualified cytologists in conjunction with the patient's screening history, other risk factors, and professional guidelines which guide patient management.

Device Story

System scans ThinPrep Pap test slides to generate whole slide images; Genius Cervical AI algorithm (CNN-based) analyzes images to identify and select objects of interest (OOIs). OOIs presented in gallery format on Genius Review Station for cytologist/pathologist review. Used in clinical laboratories; operators are cytologists/pathologists. System includes Digital Imager, Image Management Server (IMS), and Review Station. Reviewer evaluates gallery and whole slide image to determine diagnostic category per Bethesda System. If uncertain, reviewer performs direct light microscopy of glass slide. Output aids in cervical cancer screening; helps identify abnormal cells/lesions; potentially improves sensitivity for detecting cervical neoplasia compared to manual review.

Clinical Evidence

Multi-center clinical study (1,994 slides) compared Genius Digital Diagnostics System (with AI) to manual glass slide review and ThinPrep Imaging System (TIS). Primary endpoints: sensitivity and specificity at ASCUS+, LSIL+, ASC-H+, HSIL+, and Cancer thresholds. Genius AI review showed statistically significant higher sensitivity for LSIL+ (89.1% vs 84.7%), ASC-H+ (87.8% vs 79.6%), and HSIL+ (81.5% vs 74.0%) compared to manual review. Specificity was slightly lower for Genius AI compared to manual review. Study confirmed non-inferiority/superiority in sensitivity for detecting cervical neoplasia.

Technological Characteristics

System includes digital slide scanner (imager), image management server, and review station software. Features Z-focal plane acquisition (Z-stacking/merging) to capture 3D cellular depth. Includes image processing for composition, stitching, and display. Connectivity via network architecture for image management. Software algorithm uses machine learning to detect and present objects of interest.

Indications for Use

Indicated for cervical cancer screening in women undergoing Pap testing. Aids in detection of atypical cells, cervical neoplasia (LSIL, HSIL), and carcinoma. For use by qualified cytologists and pathologists. Contraindicated if glass slide examination is required due to diagnostic uncertainty.

Regulatory Classification

Identification

The digital cervical cytology slide imaging system with artificial intelligence (AI) algorithm is a prescription in vitro diagnostic device consisting of an automated digital slide creation, viewing, and management system intended to aid in the review of digital images of slides prepared from Pap test specimens and conventional Pap smears by selecting and presenting areas of interest to facilitate interpretation by the reader. The AI software algorithm uses machine learning techniques to provide information to the user about the presence, location, or characteristics of areas of the image with clinical implications. Information from this device is intended to assist the user in rendering a cytology diagnosis.

Related Devices

P020002 — THINPREP (R) IMAGING SYSTEM · Hologic, Inc. · Jun 6, 2003
K964689 — TRACCELL 2000 SLIDE MAPPING SYSTEM · Accumed Intl., Inc. · Aug 18, 1997
K982786 — TRACCELL 2000 SLIDE MAPPING SYSTEM · Accumed Intl., Inc. · Oct 16, 1998
K040591 — DUET SYSTEM · Bioview , Ltd. · Sep 16, 2004

Submission Summary (Full Text)

{0}------------------------------------------------ Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food & Drug Administration (FDA). The logo consists of two parts: a symbol on the left and the text "FDA U.S. FOOD & DRUG ADMINISTRATION" on the right. The symbol is a stylized representation of a staff entwined with a serpent, a common symbol associated with medicine and healthcare. The text is written in a clear, sans-serif font, with "FDA" in a bold, blue color and the rest of the text in a smaller, lighter blue. # EVALUATION OF AUTOMATIC CLASS III DESIGNATION FOR # Genius TM Digital Diagnostics System with the Genius ™ Cervical Al Algorithm # DECISION SUMMARY #### I Background Information: - A De Novo Number DEN210035 - B Applicant Hologic Inc. ## C Proprietary and Established Names Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm ## D Regulatory Information | Product Code(s) | Classification | Regulation Section | Panel | |-----------------|---------------------------------------------------------------------------------------------|-----------------------|-----------| | QYV | Digital cervical cytology slide imaging system with artificial intelligence algorithm | 21 CFR 864.3900 | Pathology | #### Submission/Device Overview: II ### A Purpose for Submission: De Novo request for evaluation of automatic class III designation for the Hologic's Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm. ## B Measurand: Not applicable ## C Type of Test: Digital cytology whole slide imaging system with AI algorithm to assist in review of digital images of ThinPrep Pap test slides. {1}------------------------------------------------ #### III Indications for Use: ### A Intended Use(s): See Indications for Use below. #### B Indication(s) for Use: The Genius TM Digital Diagnostics System with the Genius™ Cervical AI algorithm includes the Genius TM Digital Imager, Genius™ Image Management Server (IMS), the Genius™ Review Station, and the Genius™ Cervical AI algorithm. The Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended for the creation and viewing of digital images of scanned ThinPrep® Pap Test glass slides. Objects of interest selected by the Genius™ Cervical AI algorithm from the scanned digital image are presented in a gallery format next to the image of the whole cell spot on the Genius™ Review Station for review and interpretation. The Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended to aid in cervical cancer screening for the presence of atypical cells, cervical neoplasia, including its precursor lesions (Low Grade Squamous Intraepithelial Lesions, High Grade Squamous Intraepithelial Lesions) and carcinoma, as well as all other cytological categories as defined by The Bethesda System for Reporting Cervical Cytology1. After digital review with the Genius Cervical AI algorithm, if there is uncertainty in the diagnosis, then direct examination of the glass slide by light microscopy should be performed. Digital images from the Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm should be interpreted by qualified cytologists in conjunction with the patient's screening history, other risk factors, and professional guidelines which guide patient management. #### C Special Conditions for Use Statement(s): For in vitro diagnostic (IVD) use only For prescription use only #### D Special Instrument Requirements: - . Genius Digital Imager - . Genius Image Management Server (IMS) - . Genius Review Station - . Genius Cervical AI algorithm #### IV Device/System Characteristics: ### A. Device Description: 1 Nayar R, Wilbur DC. (eds), The Bethesda System for Reporting Cervical Cytology: Definitions, Criteria, and Explanatory Notes. 3rd ed. Cham, Switzerland: Springer: 2015 {2}------------------------------------------------ The Genius Digital Diagnostics System includes, Genius Digital Image. Management Server, Genius Review Station(s), and Genius Cervical AI algorithm, as shown in Figure 1 below. Image /page/2/Picture/1 description: The image shows three pieces of medical equipment. The first piece of equipment is a Genius machine by Hologic. The second piece of equipment is a black server tower. The third piece of equipment is a computer monitor displaying medical images. ## Figure 1. The Genius Digital Diagnostics System ### Genius Digital Imager (Digital Imager) The Genius Digital Imager digitally scans ThinPrep® Pap test slides that have been prepared using the ThinPrep 2000 system. ThinPrep 5000 processor or ThinPrep Genesis processor. stained with the ThinPrep stain (Papanicolaou-based stain) and cover slipped. The Digital Imager is operated using the user interface which is a menu-driven, graphical display touch screen. The Digital Imager consists of the following: - . Digital Imager processor which images the slides, one at a time. - Digital Imager computer which captures the images and controls the electromechanical . components of the system. - . Image Management Server which stores the slide ID and pertinent image data. The Digital Imager requires a connection to the Image Management Server. ### Genius Image Management Server (IMS) The Genius IMS subsystem is responsible for data management and includes a database, an image repository, and an imager web service, which is a HTTPS server for interfacing with the Digital Imager. The IMS hosts web applications for the review station. Information stored on the IMS's database subsystem is tracked by unique accession IDs, which are read from the microscope slide labels when they are scanned on the Digital Imager. The Digital Imager, IMS, and Review Stations are connected by a local area network. The IMS stores whole slide images in the image repository, (a physical structure of the files on a local storage disk), along with associated information from the review of the slides in a SQL database to facilitate the display of images for cytologist (CTs) and pathologists (PTs) to review. In addition to the network connection between the Digital Imager. IMS and Review Station. another network connection is required for accessing an optional archive storage system. The minimum computer requirements for the IMS are listed in the Table 1. {3}------------------------------------------------ | | Dual Intel Xeon Silver 4214 2.20 GHz processor | |------------------|----------------------------------------------------------------| | | 64 GB memory | | | 240 GB SSD for OS (boot) | | | RAID 10 Array configuration | | Server Hardware | 30 TB configured storage capacity | | | Two 10 GB Ethernet ports 3 USB 2.0 (or faster) ports | | | Display interface (HDMI or Display Port) | | | Dual, hot plug, redundant power supply (1+1), 750 W or greater | | Operating System | Windows 64-bit minimum; Windows Server 2016 recommended | | Display | A minimum resolution of 1366 by 768 pixels | ## Table 1. Genius Image Management Server (IMS) Minimum Requirements ## Genius Review Station The Genius Review Station includes a dedicated computer ruming the IMS client software and the Barco MDPC-8127 display which is a high-resolution display that has been validated for diagnostic review of objects of interest (OOIs) or whole cell spot images. The Review Station is used by CTs and PTs to screen the ThinPrep® Pap Test slides that have been imaged using the Digital Imager. The minimum specifications for the dedicated computer and specifications of the high-resolution display are listed in Tables 2 and 3. Table 2. Review Station Computer Minimum Requirements | Hardware | • x86 processor, Intel® Core i7 2.4Ghz (4C, 8T) | |------------------|-----------------------------------------------------------| | | • 16 GB DDR4 Memory or greater | | | • 256 GB Drive or greater | | | • 1Gb or faster network connection | | | • Barco MXRT display control card | | | • PCIe Gen3 x 16 slot for the display control card | | | • A keyboard and a mouse | | Operating System | Windows 10, 64-bit | | Browser | Google Chrome, Version 85 (1.3.35.451) or greater | | Environmental | • Operating temperature range: 16-32°C | | | • Non- Operating temperature range: -28°C to 50°C | | | • Operating humidity range: 20-80% RH, noncondensing | | | • Non- operating humidity range: 15-95% RH, noncondensing | {4}------------------------------------------------ | Manufacturer | Barco | |-------------------------------|-----------------------------------------------------------------------------------------------| | Model number | MDPC-8127 | | Hologic part number | CMP-01669 | | Screen technology | IPS LED | | Active screen size (diagonal) | 684 mm (27") | | Active screen size (H x V) | 569 x 335 mm (22.4 x 13.2") | | Aspect ratio (H:V) | 16:9 | | Resolution | 8 MP (3840 x 2160 pixels @ 120 Hz) | | Pixel pitch | 0.155 mm | | Color imaging | Yes | | Power rating | 100-240 Vac, 50/60 Hz, 3.6-1.6 A | | Power consumption | 75 W (nominal) @ calibrated luminance of 450 cd/m < 0.5 W (hibernate) < 0.5 W (standby) | | Operating temperature | 0 °C to 35 °C (20 °C to 30 °C within specs) | ### Genius Cervical AI algorithm The Genius Cervical AI algorithm is a software algorithm which is pre-loaded on the Genius Digital Imager. The Genius Cervical AI algorithm uses Convolutional Neural Network (CNN) technology to analyze ThinPrep® Pap Test whole slide images scanned using the Genius Digital Imager and selects OOIs to be displayed for review by a CT or PT. The Genius Cervical AI algorithm software is separate with its own software subsystem. The user selects the appropriate specimen type [i.e., GYN (Pap test slide)] to be scanned before initiating the scanning process. The analysis of the image is performed automatically by the Genius Cervical AI algorithm after the scanned digital image is generated by the Genius Digital Imager. A typical input image is a high-resolution color scan of a ThinPrep cell spot, containing over 6x10° pixels, from 5,000 to 100,000 cells. The output of the algorithm is a set of OOIs. organized into rows and designed to provide information for the reviewer to determine the diagnostic category for that particular image (case) per The Bethesda System for Reporting Cervical Cytology. There are 5 rows of OOIs and 6 primary OOIs per row. There are an additional 6 secondary OOIs available per row that the user can review, as needed. The OOIs are grouped into rows of particular diagnostic categories. Priority is given to abnormal cells when populating the first four rows, followed by cell types needed for assessing specimen adequacy (Endocervical cells and Squamous Metaplasia). The final row shows infectious organisms when present. The algorithm also provides a total count of squamous cells on the slide. This count may be used by the reviewer in assessing specimen adequacy. The cells and cell clusters presented in a gallery format on the Genius Review {5}------------------------------------------------ Station, as shown in Figure 2 below, allows the reviewer to view all OOIs presented by the algorithm. The CT or PT can digitally mark OOIs for subsequent review. Image /page/5/Figure/1 description: The image shows a screenshot of a medical analysis software interface. On the left side of the screen, there is a grid of small images, likely representing individual cells or regions of interest. The text at the top indicates a patient ID "21062029999", the type of test "Gyn", and the approximate cell count "58700". On the right side, there is a larger image, possibly a full slide view, with a smaller inset image for context or magnification. Figure 2. Example Review Screen with OOI Gallery # B. Principle of Operation The Genius Digital Diagnostics System is designed for scanning and generating digital images of ThinPrep® Pap test slides. The process is displayed in Figure 3. These digitized images can then be reviewed and interpreted by CTs and PTs for the purposes of reporting Pap test results per The Bethesda System for Reporting Cervical Cytology. Stained ThinPrev® Pap-test slides are loaded into slide carriers, which are placed into the Digital Imager. The operator uses a touch screen on the Digital Imager to interact with the instrument via a graphic, menu-driven interface. The slide ID reader scans the case's accession ID and locates the position of the cell spot. The Digital Imager contains slide handling robotics and digital imaging components to scan the entire ThinPrep cell spot to create an in-focus whole slide image. For ThinPrep® Pap test slides, the Genius Cervical AI algorithm identifies OOIs found on the digital image of the slide. The objects classified as most clinically relevant are presented in a gallery to the CT and PT for review. The slide image data, the slide ID, are transmitted to the Slide is returned to its slide carrier. The IMS functions as the central data manager for the Genius Digital Diagnostics System. As slides are imaged by the Digital Imager and reviewed by the CT or PT at the Review Station, the IMS stores, retrieves, and transmits information based on the case ID. The Review Station is a dedicated computer running the Review Station software application, with a display validated for diagnostic review of OOls and/or whole cell spot images. The Review Station is connected to a keyboard and a mouse. When a valid case accession ID has been identified at the Review Station, the server sends the images for that case ID to the user. The CT or PT is presented with a gallery of images of OOIs for that slide for review. When any image is being reviewed, the CT or PT has the option to digitally mark objects of interest and include the marks in the case review. The reviewer can move and zoom into any area of interest to evaluate details of the cell spot using a mouse or arrow keys for examination. The reviewer is responsible for {6}------------------------------------------------ ensuring the validity of the interpretation of the digital images created by the Genius Digital Diagnostics System. Image /page/6/Figure/1 description: This image shows the process of cervical specimen analysis using ThinPrep technology. The process starts with a request form and a ThinPrep vial containing the cervical specimen collected from the patient. The sample is then processed on a ThinPrep 5000 processor, a ThinPrep Genesis processor, or the ThinPrep 2000 system using ThinPrep Imaging System slides. The digital imager scans the cell spot and sends the images to the digital imager computer, which is connected to the Image Management Server, and abnormal cases are reviewed by a pathologist. ## Figure 3. Genius Digital Diagnostics System: Laboratory Workflow for Cervical Cancer Screening (Lab Workflow for ThinPrep® Pap Test Cases) ### C. Interpretation of Results The Genius Cervical AI algorithm presents a gallery of clinically relevant objects. After reviewing the entire gallery and the digital images, the CT and/or PT provide an interpretation and diagnosis of the image per The Bethesda System for Reporting Cervical Cytology. It is the responsibility of qualified CTs and PTs to employ appropriate procedures and safeguards to assure the validity of the interpretation of images obtained using this system. The Genius Digital Diagnostics System should be used in conjunction with the standard of care {7}------------------------------------------------ evaluation of the slide image, including professional screening and management guidelines to guide patient care. Users must always review the entire gallery prior to rendering an interpretation to minimize interpretive errors. After review of the digital images with the Genius Cervical AI algorithm, if there is uncertainty in the diagnosis, then direct examination of the glass slide by light microscopy is performed. # D. Software The Genius™ Digital Diagnostics System with the Genius™ Cervical AI software was identified to have a moderate level of concern as defined in FDA's Guidance for the Content of Premarket Submission for Software Contained in Medical Devices (May 11, 2005). Software Verification and Validation (V&V) testing was performed to demonstrate that the software conforms to user needs, intended uses, and that design outputs meet design requirements. The verification plan included test coverage for hardware specifications, software specifications and error diagnostics specification. - Software Description: Hologic Inc. provided a general description of the features in the i. software documentation and in the device description. The description of the software is consistent with the device functionality described in the device description. - ii. Device Hazard Analysis: Hologic Inc. provided separate analyses of the device and cybersecurity concerns. The content of the hazard analysis is sufficient and assesses preand post-mitigation risks. The device hazard analysis includes: - Identification of the hazard g - Cause of the hazard (hazardous situation) . - . Probability of the hazard - . Severity of the hazard - . Method of control or mitigation - . Corrective measures taken, including an explanation of the aspects of the device design/requirements, that eliminate, reduce, or warn of a hazardous event verification of the control implementation, which is traceable through the enumerated traceability matrix. - iii. Software Requirement Specifications (SRS): The SRS includes user, engineering, algorithmic, cybersecurity, and various other types of requirements that give a full description of the functionality of the device. The SRS is consistent with the device description and software description. - iv. Architecture Design Chart: Hologic Inc. provided the software overview and included flow diagrams representative of process flow for various features of the Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm. - Software Design Specification (SDS): The SDS is traceable to the SRS and demonstrates v. how individual requirements are implemented in the software design and includes appropriate linkages to predefined verification testing. - Traceability Analysis/Matrix: Hologic Inc. provided traceability between all documents vi. including the SRS, SDS, and subsequent verification. Hazard mitigations are traceable throughout all documents. - vii. Software Development Environment: Hologic Inc. outlined the software development environment and the processes/procedures used for medical device software development. The content is consistent with expected quality system norms. {8}------------------------------------------------ - viii. Verification and Validation Testing: The validation and system level verifications procedures are based upon the requirements with clearly defined test procedures and pass/fail criteria. All tests passed. Unit level test procedures, actual, and expected results are included for all design specifications. - Revision Level History: System Software Version (v) 1.1.0 was released prior to its use in ix. all the performance studies, including analytical (standalone and precision) and clinical reader study. The Genius Cervical AI algorithm v1.0.16.0 will remain locked for use with the authorized device and will not be continually trained and improved with each cohort analyzed in clinical practice, after marketing authorization. - Unresolved Anomalies: There are no unresolved anomalies. x. - xi. Cybersecurity: The cybersecurity documentation is consistent with the recommendations for information that should be included in premarket submissions outlined in the FDA guidance document "Content of Premarket Submissions for Management of Cybersecurity in Medical Devices: Guidance for Industry and Food and Drug Administration Staff" (issued October 2, 2014). Information related to cybersecurity reviewed included: - Hazard analysis related to cybersecurity risks, . - . Traceability documentation linking cybersecurity controls to risks considered. - . Summary plan for validating software updates and patches throughout the lifecycle of the medical device. - . Summary describing controls in place to ensure that the medical device will maintain its integrity, and - . Device instructions for use and product specifications related to recommended cybersecurity controls appropriate for the intended use of the device. - xii. Remote Use: Hologic provided information about the device environment and risks such as connection method and requirements, optimal connection speed, network requirements, minimum network specifications, image quality, etc., with respect to remote use. A network speed value of at least 100 MBps is recommended for optimal speed in loading images and case data. Users must contact network administrator if the speed test value is below this threshold. Users who choose to remotely connect the Genius Review Station to the Genius Image Management Server are responsible for the maintenance of their network's cybersecurity. #### V Standards/Guidance Documents Referenced: | Standard Number | Title | |-------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | EN ISO 14971:2019 | Medical Devices – Application of risk management to medical devices | | EN 61010-1:3rd edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 1: General Requirements | | IEC 61010-1:3rd edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 1: General Requirements | | CSA C22.2 #61010-1-12:3rd edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 1: General Requirements | | Standard Number | Title | | EN 61010-2-101:2nd Edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 2-101: Particular requirements for IVD medical equipment | | IEC 61010-2-101:2nd Edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 2-101: Particular requirements for IVD medical equipment | | CSA C22.2 # 61010-2-101:2nd edition | Safety requirements for electrical equipment for measurement, control and laboratory use – Part 2-101: Particular requirements for IVD medical equipment | | IEC 60601-1-2:4th Edition | Medical electrical equipment – Part 1-2: General requirements for basic safety and essential performance — Collateral Standard: Electromagnetic disturbances —Requirements and tests | | IEC TR 62366-2 | Medical Devices-Part 2: Guidance on the application of usability engineering to medical devices | | EN 60601-1-2:4th edition | Medical electrical equipment – Part 1-2: General requirements for basic safety and essential performance — Collateral Standard: Electromagnetic disturbances —Requirements and tests | | EN 61326-2-6:2nd edition | Electrical equipment for measurement, control and laboratory use– EMC requirements-Part 2-6 Particular requirements-IVD medical equipment | | IEC 61326-2-6:2nd Edition | Electrical equipment for measurement, control and laboratory use– EMC requirements-Part 2-6 Particular requirements-IVD medical equipment | | IEC 61326-1:2nd edition | Electrical equipment for measurement, control and laboratory use – EMC requirements – Part 1: General requirements | | EN 61326-1:2nd edition | Electrical equipment for measurement, control and laboratory use – EMC requirements – Part 1: General requirements | | EN ISO 13485:2016 | Medical Devices – Quality management systems-requirements for regulatory purposes | | EN 13612:2002 | Performance evaluation of IVD medical devices | | ASTM D4169-16:2016 | Standard Practice for Performance Testing of Shipping Containers and Systems | | EN ISO 18113-1:2011 | In vitro diagnostic medical devices – Information supplied by the manufacturer (labeling) Part 1: Terms, definitions and general requirements | | EN ISO 18113-3:2011 | In vitro diagnostic medical devices – Information supplied by the manufacturer (labeling) Part 3: In vitro instruments for professional use | | EN 62304:2015 | Medical Device Software – Software life cycle processes | | Standard Number | Title | | EN 62366-1:2015 | Medical Devices - Application of usability engineering to medical devices | | ISO 15223-1:2016 | Medical devices- symbols to be used with medical device labels, labelling and information to be supplied-Part 1 General requirements | | FDA Guidance: 2016 | Technical Performance Assessment of Digital Pathology Whole Slide Imaging Devices | | FDA Guidance: May 2005 | Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices | | FDA Guidance September 2019 | Guidance for Off-the-Shelf Software Use in Medical Devices | | FDA Guidance: February 2016 | Applying Human Factors and Usability Engineering to Medical Devices | {9}------------------------------------------------ {10}------------------------------------------------ #### VI Performance Characteristics: ## A. Technical Performance Assessment Multiple studies were performed to verify that the Genius Digital Diagnostic System with the Genius Cervical AI algorithm met all product specification and performance requirements for safe and effective use of the device. Detailed descriptions of the device components and bench testing results were provided for the following: - . Slide Feeder: Information was provided on the slide feed mechanism, slide configuration, class of automation, user interaction, and failure mode and effects analysis. - . Light Source: Information and specifications on the light source (bulb type, manufacturer and model, wattage, spectral power distribution, expected lifetime, and output adjustment control) and the condenser (illumination format, manufacturer and model, numeric aperture, and working distance) were provided. The expected variation of intensity and spectra was evaluated for three periods: - . Duration of a single scan - . Duration of a single workday - . Lifetime of the device - Information on capability of tracking intensity and spectral degradation with . lifetime was provided. Spectral distribution of the light incident on the slide was measured. - . Imaging Optics: Information and specifications on the optical elements, including optical schematic, microscope objective, auxiliary lens, and magnification, were provided. The imaging optical system was validated with the following tests: - Relative irradiance . - Distortion . - . Lateral chromatic aberrations - Longitudinal chromatic aberrations . {11}------------------------------------------------ - . Z-focal plane acquisition method: Information on the volumetric imaging method was provided. - . Mechanical Scanner Movement: Information and specifications on the imaging robotics were provided. The stage position and camera trigger signal are synchronized by utilizing position monitoring and close-loop control. - . Digital Imaging Sensor: Information and specifications on the digital imaging sensor, including sensor type, sensor manufacturer, number of pixels, dimension of pixel, configuration of color filter array, spectral transmission of color filter mask, relative response versus wavelength, linearity, spatial uniformity, dark current level, read noise, readout rate, and digital output format, were provided. - . Image Processing and Composition: The whole slide image creation was validated with the following tests: - Exposure control . - . White balance - . Color correction - Sub-sampling and pixel offset correction . - Flat-field correction . - Pixel defect correction . - . Scanning method - . Stitching and merging - . Scanning speed - . Number of planes - Image Files Format: Information and specifications on the full-resolution TIFF file and . pyramid TIFF file were provided. - . Image Review Manipulation Software: Information and specifications on continuous panning, continuous zooming, tracking of visited objects, and digital marking were provided. - . Computer Environment: Information and specifications on the computer hardware, operating system, graphics card and driver, color management settings, color profile, and display interfaces were provided. - . Display: Information and specifications on the display device (Barco MDPC-8127, CMP-01669) were provided, including the technology characteristics, physical size of the viewable area and aspect ratio, backlight type and properties, frame rate and refresh rate, pixel array, pitch, pixel aperture ratio and subpixel matrix scheme, subpixel driving for greyscale, display interface, ambient light adaptation, color calibration tools, and frequency and nature of quality-control tests. Technical performance assessments at a system level were performed for the following: - . Coverage of Z-depth to represent the 3D aspects of the cvtological sample: The volumetric imaging method was validated with bench tests using a Ronchi pattern and compared with a conventional microscope. The acquired Z-depth was ≥ 24 microns. - . Color Reproducibility: Accuracy and precision of the colors displayed on the Genius Review Station Computer were validated with the following tests: - Reproducibility of Imager calibration across multiple Sierra slides . - . Reproducibility of Imager calibration across multiple Imagers {12}------------------------------------------------ - . Reproducibility of calibration across multiple displays - Overall system accuracy, displayed color compared to the intended color . The colors displayed on the Genius Review Stations for scans taken using the Genius Digital Imager meets the acceptance criteria specified. - Structural Similarity Index Measurement (SSIM) Study: Test data to assess the . reproducibility of image capture across multiple imagers, multiple runs on a single imager, and over the calibration cycles of the imager was provided. A set of 18 Pap test slides were imaged using the Genius Digital Diagnostics System with the Genius Cervical AI algorithm, as listed in Table 4. | Category Name | Quantity | |---------------|----------| | NILM | 3 | | ASCUS / AGUS | 3 | | LSIL | 3 | | ASC-H | 3 | | HSIL | 3 | | CANCER | 3 | Table 4. Bethesda Diagnostic Category Slide Distribution NILM: negative for intraepithelial lesion or malignancy; ASCUS: atypical squamous cells of undetermined significance; AGC: atypical glandular cells; LSIL: low grade squamous intraepithelial lesion: ASC-H; atypical squamous cells - cannot exclude HSIL; HSIL: High grade squamous intraepithelial lesion. Three separate Genius Digital Imagers were used for imaging slides. Between-imager reproducibility was evaluated by imaging slides once on 3 separate Genius Digital Imagers. Within-imager reproducibility was evaluated by imaging the slides 3 times on a single Genius Digital Imager. Imaging Stability was evaluated by imaging the slides 3 times on a single system with an instrument auto-calibration performed between each run. The study evaluation was based on assessing the structural similarity index metric (SSIM) which combines measurements of luminance, contrast, and structure at the pixel level. The acceptance criteria of SSIM above 0.9 was set. Study results showed all SSIM were greater than the 0.9. Performance met the defined criteria. - . Spatial Resolution: The spatial resolution of the optics in the Digital Imager was validated with the following tests: - Edge-based spatial frequency response data including horizontal and . vertical targets - . Edge-based spatial frequency response data including 45-degree target - . Sinusoidal spatial frequency response data including Siemens Star target The spatial resolution of the optics in the digital cytology imager meets the prespecified acceptance criteria. - Focusing Testing: The focus quality of whole slide images was validated with the . following tests: - Tallying the fraction of slides where there are quality control (QC) focus . errors - . Manual review of slides for out of focus region identification - . Estimate sensitivity of the OC focus error check {13}------------------------------------------------ The focus quality of whole slide images produced by the Digital Diagnostics system meets the acceptance criteria specified. - Whole Cell-Spot Coverage: A set of 223 slides were imaged, and then analyzed to . determine whether the entire cell spot was included within the image. An in-house software tool was created to analyze the swath scan pattern and the presence of cells in each swath for a human review. The Genius Digital Imager scanned the entire cell spot of the slide and the cell spot coverage was determined to be acceptable based on visual inspection. - . Stitching Error: A set of slides was imaged and the swath data before and after stitching were captured. The stitched image was compared with an unstitched image of the same material using the multi-scale structural similarity metric (MS-SSIM). The stitching of image swaths in the Genius Digital Imager met the pre-specified acceptance criteria. - . Turnaround Time: A stopwatch was used to determine the time required to complete the following operations: - View the whole slide image and the OOI gallery . - . Display initial images when a new slide review is selected - . Pan to the left on the whole slide image - . Zoom in 40x on the whole slide image - . Zoom out 10x on the whole slide image - . Pan up in full screen and view the whole slide image - . Select an OOI from the gallery and view the whole slide image once the complete image is zoomed in - . Select an OOI from the more/like row in the gallery and view the whole slide image once the complete image is zoomed in. The turnaround time used in the Genius Digital Imager was determined to be acceptable. #### B. Analytical Performance: #### 1. Objects of Interest (OOI) Reproducibility A single site study was conducted to demonstrate that the Genius Cervical AI algorithm accurately and reproducibly selects OOIs. An OOI is a cell or cluster of cells on a glass slide scanned by the Genius Diagnostics System that most likely contains clinically relevant information for diagnostic purposes. In the study, thirty-seven ThinPrep Pap test slides were enrolled which were selected from slides used in the Genius Digital Diagnostics System with the Genius Cervical AI algorithm clinical study. The selected slides covered the full range of abnormal diagnostic categories as defined in The Bethesda System for Reporting Cervical Cytology. These slides were prepared on the ThinPrep 2000 system, ThinPrep 5000 processor, and ThinPrep Genesis processor. The slides were imaged three times on three different Genius Digital Imagers. Three CTs independently reviewed the nine runs of each case on the Genius Digital Diagnostics System, blinded to the reference diagnosis for the case. A minimum of a {14}------------------------------------------------ two-week washout period was used between the review of the images from each run. During each review on the Genius Digital Diagnostics System with Genius Cervical AI algorithm, the CT recorded the observation in every tile in the gallery for the case on the Genius Review Station. The study compared OOIs selected by the Genius Cervical AI algorithm to the reference diagnosis which was the adjudicated diagnosis for the slide that was determined during the clinical study (See clinical study section below for details of the method used to determine the reference diagnosis for each slide). The accuracy and reproducibility of the Genius Digital Diagnostics System with Genius Cervical AI algorithm was measured by comparison to the adjudicated reference diagnoses. The study evaluated the performance of the Genius Cervical AI algorithm in presenting images suitable for diagnosing Pap test cases. The study also measured reproducibility of the Genius Digital Diagnostics System, as a whole. A summary of the performance data is shown in Table 5 below. | Reference Dx | # Slides | # of Evaluations | Proportion of Abnormal OOIs | Median # of Abnormal OOIs | Range of # of Abnormal OOIs (Min;Max) | Proportion Category+ OOIs | Median # Category+ OOIs | Range of # of Category+ OOIs (Min; Max) | |--------------|----------|------------------|-----------------------------|---------------------------|---------------------------------------|---------------------------|-------------------------|-----------------------------------------| | UNSAT | 2 | 54 | 31% | 0 | 0; 5 | | | | | NILM | 5 | 135 | 16% | 0 | 0; 4 | | | | | ASCUS | 5 | 135 | 100% | 6 | 2; 17 | 100% | 6 | 2; 17 | | LSIL | 5 | 135 | 100% | 10 | 3; 23 | 96% | 5 | 0; 23 | | ASC-H | 5 | 135 | 100% | 13 | 4; 22 | 100% | 11 | 3; 19 | | AGC | 5 | 135 | 100% | 12 | 3; 24 | 100% | 12 | 3; 24 | | HSIL | 5 | 135 | 100% | 18 | 12; 25 | 100% | 9 | 2; 21 | | CANCER | 5 | 135 | 100% | 14 | 5; 20 | 92% | 6 | 0; 14 | | All Abnormal | 30 | 810 | 100% | 13 | 3; 25 | 98% | 8 | 0; 24 | Table 5. OOI Summary by Reference Category (all CTs) NILM: negative for intraepithelial lesion or malignancy: ASCUS: atypical squamous cells of undetermined significance; AGC: atypical glandular cells; LSIL: low grade squamous intraepithelial lesion; ASC-H: atypical squamous cells - cannot exclude HSIL; HSIL: High grade squamous intraepithelial lesion. - = # of evaluations = (total valid runs) * (# of CTs for the given diagnosis subset of slides) - · Proportion abnormal = the fraction of evaluations for which at least one abnormal OOI was observed - · Median # abnormal = the median number of abnormal OOIs in the evaluations - · Proportion category+ = the fraction of evaluations for which at least one OOI that is equal or greater than the reference diagnosis observed. - · Median # category + = the median number of OOIs that are category+ in the evaluations Note: For the reference cancer diagnosis slide reviews, while 100% had OOIs marked by the CTs as ASCUS+, 92% had OOIs marked as cancer. #### Agreement Rates by Threshold {15}------------------------------------------------ Table 6 below shows the positive agreement rate of the OOIs at various abnormal thresholds. For example, there were 20 LSIL+ slides (combined LSIL, ASC-H, HSIL, and CANCER), evaluated by 3 CTs over 9 imaging runs for a total of 540 evaluations. Of those, 530 had LSIL OOIs or higher for an agreement rate of 530/540 = 98%. | Threshold | # of Evaluations | Agreement Rate | |------------------------------------------------------------|------------------|----------------| | ASCUS+ (includes ASCUS, LSIL, ASC-H, AGC, HSIL, Cancer) | 810 | 100% | | LSIL+ (includes LSIL, ASC-H, HSIL, Cancer) | 540 | 98% | | ASC-H+ (includes ASC-H, HSIL, Cancer) | 405 | 99% | | HSIL+ (includes HSIL, Cancer) | 270 | 99% | | CANCER | 135 | 92% | Table 6. Agreement rates by Reference Threshold ## OOI Reproducibility Table 7 below shows the between-instrument and within-instrument agreement rates for the presence of Category+ OOIs. ## Table 7. OOI Reproducibility | | # of Pairs | % Agreement | |--------------------|------------|-------------| | Between-instrument | 999 | 96% | | Within-instrument | 999 | 99% | ## 2. Cell Count Study: A study was conducted to evaluate the performance of the cell count metric produced by the Genius Cervical AI algorithm compared to a manual cell count. A total of 50 ThinPrep Pap test slides including at least 8 slides with counts near the clinically important threshold of 5000 cells, were enrolled in the study (described in Table 8 below). These slides were prepared on a ThinPrep 5000 processor, stained and cover slipped. The same slides were imaged on three Genius Digital Imagers three separate times. To obtain the manual cell count for the slides in the study, a CT viewed the whole slide image presented on the Genius Review Station, counted the cells presented in a portion of the cell spot image, and estimated the total number of cells based on the portion, similar to the normal process for counting cells on slides viewed on a microscope. The cell counts derived on each Digital Imager by the algorithm in the Genius Digital Diagnostics System were compared to the manual cell count estimate. ### Table 8: Diagnostic category of slides used in the Cell Count Study | Category | # of Slides | |----------|-------------| | AGUS | 4 | | ASC-H | 3 | | HSIL | 12 | | LSIL | 5 | | NILM | 23 | {16}------------------------------------------------ | CANCER | 3 | |-------------|----| | Grand Total | 50 | Within-imager reproducibility was evaluated by imaging the slides 3 times on a single Genius Digital Imager and comparing cell count results. Between-imager reproducibility was evaluated by imaging slides once on 3 separate Genius Digital Imagers and comparing cell count results. Cell count accuracy was evaluated by comparing the Genius Cervical A1 derived cell counts to the cytologist manual count estimate for each slide. The reproducibility was compared across three runs on one instrument (between-instrument) and across one run on three instruments (within-instrument) and the mean cell count and %CV were calculated. Data showed that the within-imager precision %CV was 0.6% and between-imager %CV was 2.7%. Figure 4 compares the cell counts between the Genius Cervical AI algorithm and a manual cell count method for each specimen. The appropriate linear regression analysis was performed, and slope was 1.06 with 95% CI: (1.01; 1.11) and the intercept of 213 with 95% CI: (28: 398). The relative systematic difference between the cell count determined by the CT during the review of glass slides to the Genius Cervical AI algorithm derived cell counts at 5,000 cells was 10% with 95% CI: (4%; 17%). The results of the cell count study were acceptable. Scatter plot of agreement is shown in Figure 4. Image /page/16/Figure/4 description: The image is a scatter plot titled "Cell Counts". The x-axis is labeled "Manual" and ranges from 0 to 8x10^4. The y-axis is labeled "DC individual runs" and ranges from 0 to 10x10^4. The scatter plot shows a positive correlation between the manual cell counts and the DC individual runs. Figure 4. Scatter Plot of Agreement between Interpretations Using Genius Cervical AI algorithm Review versus Manual Glass Slide Review {17}------------------------------------------------ #### C. Clinical Studies: ## Genius Digital Diagnostics System with the Genius Cervical AI algorithm Review (Genius Cervical AI Clinical Study) Compared to Manual Microscope Review of Glass Slides A multi-center Genius Cervical AI Algorithm Clinical Study was performed within the United States. The objective of the study was to show that routine screening of ThinPrep Pap test slide images using the Genius Digital Diagnostics System with the Genius Cervical AI algorithm was comparable to the approved method of screening using glass slides with a light microscope, which is the current standard of care. The study included 1994 slides and 4 clinical sites (laboratories). Slides were prepared from residual material after the clinical sites signed out the case, from women who were screened for cervical cancer using the ThinPrep Pap test. Samples that were enrolled were processed on the ThinPrep® 2000 system, the ThinPrep® 5000 processor or the ThinPrep® Genesis™ processor. At each of 4 clinical sites, 3 independent teams consisting of 1 CT and 1 PT at each (site CT/PT teams) reviewed all cases at their site. All cases at the corresponding site were reviewed independently by the three teams at that particular site and, therefore, the number of reviews at the site were 3 times the number of slides at the site. Site CT/PT teams screened cases in 3 review phases as follows: manual review of glass slides with a light microscope without the assistance of the ThinPrep Imaging System (TIS) (Manual review), review of glass slides with the ThinPrep Imaging System (TIS review) and review of digital images with the Genius Digital Diagnostics System with Genius Cervical AI algorithm (Genius Cervical AI review/Digital Review), in that order. Cases with an ASCUS, AGC, LSIL, ASC-H, HSIL, Cancer or unsatisfactory for evaluation (UNSAT) result by the CT were also reviewed by the PT. A minimum 14-dav washout period occurred between each review phase. The cases were randomized prior to each review phase. Cytological diagnoses and specimen adequacy were determined in accordance with the Bethesda System criteria. An adjudicated diagnosis was used as a "gold standard" ("reference" or "ground truth"). Cases were screened by an adjudication panel, composed of 3 adjudication CT/PT teams, consisting of 1 CT and 1 PT each (adjudication CT/PT team). Slides were reviewed independently by the three teams. All cases, regardless of result. were reviewed by CTs and PTs. For each case, results from each adjudication CT/PT team were used to obtain a consensus result, defined as the result for which there was majority agreement (by at least two of the three adjudication CT/PT teams). If a consensus result was not obtained initially, these cases underwent review by the three adjudication PTs simultaneously using a multi-headed microscope (multi-head review). The reference result was based on either the consensus result (if met initially) or the multi-head review result (if consensus was not obtained initially). Cytological diagnoses and specimen adequacy were determined in accordance with the Bethesda System criteria: NILM, ASCUS, AGC, LSIL, ASC-H, HSIL, Cancer and UNSAT. {18}------------------------------------------------ ### Laboratory and Patient Characteristics The cytology laboratories participating in the study were comprised of four (4) sites. All sites selected had extensive experience in the processing and evaluation of gynecologic ThinPrep Pap test slides and were trained in the use of the Genius Digital Diagnostics System with Genius Cervical AI algorithm. There were 1995 slides that were eligible for the study. Of these, 1994 slides were included in the study and one (1) was excluded from the study because the slide failed the quality audit due to a scratched coverslip, an exclusion criterion. The total number of reviews was 5,982 (3 x 1994 slides). Thirty-four (34) cases (102 reviews) had adjudication results of UNSAT and the remaining 1960 cases (5,880 reviews) were Satisfactory (SAT) for evaluation and had reference adjudication diagnoses. Table 9 provides characteristics of the participating clinical sites. Table 10 describes the patient populations with SAT slides, at each of the study sites. #### Table 9. Site Characteristics | Site | 1 | 2 | 3 | 4 | |---------------------------------|--------|---------|---------|-------| | ThinPrep Pap Tests Per Year | 48,000 | 239,750 | 329,500 | 4,500 | | Number of Cytologists in Study | 3 | 3 | 3 | 3 | | Number of Pathologists in Study | 3 | 3 | 3 | 3 | | Site Number | Total number | Median Age (yrs) | # Hysterectomy (% of enrolled) | # Postmenopausal (% of enrolled) | |-------------|--------------|------------------|--------------------------------|----------------------------------| | 1 | 488 | 33.0 | 18 (3.7) | 37 (7.6) | | 2 | 494 | 36.0 | 6 (1.2) | 24 (4.9) | | 3 | 490 | 35.0 | 22 (4.5) | 43 (8.8) | | 4 | 488 | 37.0 | 6 (1.2) | 41 (8.4) | | Overall | 1960 | 35.0 | 52 (2.6) | 141 (7.4) | #### Table 10. Site Demographics #### Eligibility Criteria Cases were eligible to be included in the study if they met the following criteria: ThinPrep slides of known diagnoses generated from residual cytological specimens (within 6 weeks from date of collection). The approximate number of slides from the Bethesda System diagnostic categories that were enrolled in the study are as follows: - · NILM: 1060 cases - · ASCUS: 225 cases - · AGC: 20 cases - · LSIL: 225 cases - · ASC-H: 225 cases - · HSIL: 225 cases - · Cancers: 20 cases (squamous and/or adenocarcinoma) - · UNSAT 20 cases {19}------------------------------------------------ Cases were excluded from the study if any slide was deemed not adequate due to broken slide, dilute specimen or slide is otherwise unreadable). ## Objective of the Clinical Study The primary objectives of this study included comparing the sensitivity and specificity when diagnosing cases imaged and reviewed on the Genius Digital Diagnostics System with Genius Cervical AI algorithm with the sensitivity and specificity of Manual review and also with TIS review. An adjudicated diagnosis was used as a gold standard ("reference" or "ground truth"). The comparison of sensitivities and specificities was performed at the following thresholds (described in Table 11 below): ASCUS+, LSIL+, ASC-H+. HSIL+, Cancer. | Threshold | Negative | Positive | |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|---------------------------------------| | ASCUS+ | NILM | ASCUS, AGC, LSIL, ASC-H, HSIL, Cancer | | LSIL+ | NILM, ASCUS, AGC | LSIL, ASC-H, HSIL, Cancer | | ASC-H+ | NILM, ASCUS, AGC, LSIL | ASC-H, HSIL, Cancer | | HSIL+ | NILM, ASCUS, AGC, LSIL, ASC-H | HSIL, Cancer | | Cancer | NILM, ASCUS, AGC, LSIL, ASC-H, HSIL | Cancer | | NILM: negative for intraepithelial lesion or malignancy; ASCUS: atypical squamous cells of undetermined significance; AGC: atypical glandular cells; LSIL: low grade squamous intraepithelial lesion; ASC-H: atypical squamous cells - cannot exclude HSIL; HSIL: High grade squamous intraepithelial lesion | | | #### Table 11. Bethesda System Diagnostic Category Partitions Sensitivity and specificity of each review type (Genius Cervical AI algorithm review, Manual review, and TIS review) were calculated on all cases with a satisfactory reference result at the ASCUS+, LSIL+, ASC-H+, HSIL+ and Cancer diagnostic thresholds. Of these cases, UNSAT Genius Cervical AI, Manual, or TIS review results were considered positive at each diagnostic threshold. Sensitivity was calculated separately on all cases with an UNSAT reference result, where sensitivity was defined as the proportion of Genius Cervical AI. Manual, or TIS review results of UNSAT or ASCUS+. Specificity was also calculated, where specificity was defined as the proportion of satisfactory Genius Cervical AI, Manual, or TIS review results on all cases with a satisfactory reference result. Differences in sensitivities and differences in specificities were calculated along with two-sided 95% confidence intervals (95% CI). {20}------------------------------------------------ C.1.1 Performance of Genius Cervical AI Review and Manual Glass Slide Review | Diagnostic Threshold | Genius (95% CI) | Sensitivity % Manual (95% CI) | Difference (Genius – Manual) (95% CI)¹ | Specificity % Genius (95% CI) | Manual (95% CI) | Difference (Genius – Manual) (95% CI)¹ | |-------------------------|-------------------------------------|-------------------------------------|-------------------------------------------------|-------------------------------------|-------------------------------------|-------------------------------------------------| | ASCUS+ | 91.7 [1950/2127] (90.1, 93.3) | 90.1 [1917/2127] (88.7, 91.8) | 1.6 [33/2127] (-0.1, 3.2) | 91.0 [3414/3753] (89.7, 92.1) | 92.2 [3461/3753] (91.1, 93.2) | -1.3 [-47/3753] (-2.3, -0.2) | | LSIL+ | 89.1 [1467/1647] (87.2, 91.0) | 84.7 [1395/1647] (82.3, 86.8) | 4.4 [72/1647] (2.1, 6.7) | 91.7 [3883/4233] (90.5, 92.9) | 94.1 [3984/4233] (93.1, 95.0) | -2.4 [-101/4233] (-3.5, -1.4) | | ASC-H+ | 87.8 [938/1068] (84.8, 90.2) | 79.6 [850/1068] (76.3, 82.5) | 8.2 [88/1068] (4.8, 11.6) | 94.2 [4531/4812] (93.2, 95.1) | 97.0 [4669/4812] (96.4, 97.7) | -2.9 [-138/4812] (-3.8, -1.9) | | HSIL+ | 81.5 [699/858] (78.5, 84.4) | 74.0 [635/858] (70.1, 77.5) | 7.5 [64/858] (4.0, 11.4) | 94.8 [4763/5022] (94.0, 95.6) | 97.2 [4882/5022] (96.6, 97.8) | -2.4 [-119/5022] (-3.0, -1.7) | Table 12. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Compared to Adjudicated Diagnosis The sensitivity of the Genius Cervical AI was statistically significantly higher for LSIL+, ASC-H+ and HSIL+. Increase in sensitivity was 4.4%, 8.2% and 7.5% for LSIL+, ASC-H+ and HSIL+, respectively. There were statistically significant decreases in specificity for ASCUS+, LSIL+, ASC-H+, and HSIL+ diagnostic thresholds. The decrease in specificity was 1.3%, 2.4%, 2.9% and 2.4% for ASCUS+, LSIL+, ASC-H+, and HSIL+, respectively (described in Table 12 above). # C.1.2 Genius Cervical AI Algorithm Review vs. Manual Glass Slide Review Stratified by Site ## ASCUS+ Sensitivity is a percent of "reference" ASCUS+ cases classified in Genius Cervical AI reviews or in Manual reviews as ASCUS+ or UNSAT, and specificity is a percent of "reference" NILM cases classified in either review as NILM (described in Table 13 below). {21}------------------------------------------------ | Sites | Number of Cases | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|--------------------|-------------------------------------|-------------------------------------|-----------------------------------|-------------------------------------|------------------------------------|------------------------------------| | | | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 488 | 93.4 [538/576] (90.0. 96.1) | 87.8 [506/576] (83.9, 91.3) | 5.6 [32/576] (1.7, 8.7) | 91.7 [814/888] (88.6, 94.1) | 95.6 [849/888] (93.6, 97.3) | -3.9 [-35/888] (-6.3, -1.7) | | Site 2 | 494 | 87.7 [479/546] (83.6. 90.9) | 93.2 [509/546] (90.0. 95.8) | -5.5 [-30/546] (-9.0, -2.0) | 93.3 [873/936] (91.2, 95.2) | 90.9 [851/936] (88.4, 93.5 | 2.4 [22/936] (0.3, 4.7) | | Site 3 | 490 | 92.2 [506/549] (88.9, 95.0) | 88.7 [487/549] (85.4. 92.0) | 3.5 [19/549] (0.4.6.1) | 92.6 [853/921] (90.1, 94.9) | 92.0 [847/921] (89.9. 93.8) | 0.7 [6/921] (-1.9, 2.8) | | Site 4 | 488 | 93.6 [427/456] (90.8, 96.1) | 91.0 [415/456] (87.3, 94.7) | 2.6 [12/546] (-0.6, 5.8) | 86.7 [874/1008] (83.9, 89.4) | 90.7 [914/1008] (88.1. 93.0) | -4.0 [-40/1008] (-6.2, -1.6) | | Total | 1960 | 91.7 [1950/2127] (90.1, 93.3) | 90.1 [1917/2127] (88.7, 91.8) | 1.6 [33/2127] (-0.1, 3.2) | 91.0 [3414/3753] (89.7, 92.1) | 92.2 [3461/3753] (91.1.93.2) | -1.3 [-47/3753] (-2.3, -0.2) | Table 13. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Stratified by Site at ASCUS+ # LSIL+ Sensitivity is a percent of "reference" LSIL+ cases classified in Genius Cervical AI algorithm reviews or in Manual reviews as LSIL+ or UNSAT, and specificity is a percent of "reference" (NILM or ASCUS or AGC) cases classified in either review as NILM or ASCUS or AGC, (described in Table 14 below). Table 14. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Stratified by Site at LSIL+ | Sites | Number of Cases | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|--------------------|-------------------------------------|-------------------------------------|------------------------------------|-------------------------------------|-------------------------------------|-------------------------------------| | | | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 488 | 88.5 [401/453] (84.2, 92.2) | 83.7 [379/453] (78.9, 87.8) | 4.9 [22/453] (0.5, 9.5) | 91.0 [920/1011] (88.2, 93.8) | 94.3 [953/1011] (92.3, 96.4) | -3.3 [-33/1011] (-5.6, -1.1) | | Site 2 | 494 | 85.9 [348/405] (81.0, 89.8) | 93.1 [377/405] (89.7, 96.2) | -7.2 [-29/405] (-11.1, -3.3) | 92.9 [1000/1077] (90.8, 94.8) | 92.3 [994/1077] (89.8, 94.5) | 0.6 [6/1077] (-1.5, 2.7) | | Site 3 | 490 | 89.7 [390/435] (86.2, 93.0) | 72.6 [316/435] (66.9, 77.6) | 17.0 [74/435] (12.2, 22.3) | 92.4 [956/1035] (89.9, 94.5) | 97.1 [1005/1035] (95.9, 98.3) | -4.7 [-49/1035] (-7.1, -2.9) | | Site 4 | 488 | 92.7 [328/354] (89.5, 95.1) | 91.2 [323/354] (87.2, 94.6) | 1.4 [5/354] (-2.7, 5.9) | 90.7 [1007/1110] (88.4, 92.9) | 93.0 [1032/1110] (90.8, 94.9) | -2.3 [-25/1110] (-4.1,0.1) | | Total | 1960 | 89.1 [1467/1647] (87.2, 91.0) | 84.7 [1395/1647] (82.3, 86.8) | 4.4 [72/1647] (2.1, 6.7) | 91.7 [3883/4233] (90.5, 92.9) | 94.1 [3984/4233] (93.1, 95.0) | -2.4 [-101/4233] (-3.5, -1.4) | # ASC-H+ Sensitivity is a percent of "reference" ASC-H+ cases classified in Genius reviews or in Manual reviews as ASC-H+ or UNSAT, and specificity is a percent of "reference" (NILM or ASCUS or {22}------------------------------------------------ AGC or LSIL) cases classified in either review as NILM or ASCUS or AGC or LSIL, (described in Table 15 below). | Sites | Number | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|----------|------------------------------------|------------------------------------|------------------------------------|-------------------------------------|-------------------------------------|-------------------------------------| | | of Cases | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 488 | 85.7 [257/300] (80.0, 90.4) | 80.0 [240/300] (74.1, 85.3) | 5.7 [17/300] (0.0, 11.8) | 92.4 [1075/1164] (89.7, 94.6) | 96.1 [1119/1164] (94.5, 97.7) | -3.8 [-44/1164] (-5.6, -2.0) | | Site 2 | 494 | 83.3 [230/276] (77.3, 88.7) | 90.9 [251/276] (86.1, 95.4) | -7.6 [-21/276] (-13.4, -2.7) | 96.5 [1164/1206] (94.9. 97.9) | 96.0 [1158/1206] (94.5, 97.5) | 0.5 [6/1206] (-1.0, 2.1) | | Site 3 | 490 | 92.3 [241/261] (87.8, 95.9) | 69.7 [182/261] (62.6, 77.2) | 22.6 [59/261] (15.6, 28.9) | 94.5 [1143/1209] (92.5. 96.4) | 98.5 [1191/1209] (97.7, 99.2) | -4.0 [-48/1209] (-5.7, -2.3) | | Site 4 | 488 | 90.9 [210/231] (87.0. 94.4) | 76.6 [177/231] (68.8, 84.0) | 14.3 [33/231] (6.3, 22.8) | 93.2 [1149/1233] (91.2, 95.1) | 97.4 [1201/1233] (96.3. 98.5) | -4.2 [-52/1233] (-6.2, -2.4) | | Total | 1960 | 87.8 [938/1068] (84.8, 90.2) | 79.6 [850/1068] (76.3, 82.5) | 8.2 [88/1068] (4.8, 11.6) | 94.2 [4531/4812] (93.2, 95.1) | 97.0 [4669/4812] (96.4, 97.7) | -2.9 [-138/4812] (-3.8, -1.9) | Table 15. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Stratified by Site at ASC-H+ # HSIL+ Sensitivity is a percent of "reference" HSIL+ cases classified in Genius reviews or in Manual reviews as HSIL+ or UNSAT, and specificity is a percent of "reference" (NILM or ASCUS or AGC or LSIL or ASC-H) cases classified in either review as NILM or ASCUS or AGC or LSIL or ASC-H, (described in Table 16 below). | Table 16. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and | | |-------------------------------------------------------------------------------------------|--| | Manual Glass Slide Review (Manual) Stratified by Site at HSIL+ | | | Sites | Number of Cases | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|-----------------|-----------------------------------|-----------------------------------|-------------------------------------|-------------------------------------|-------------------------------------|-------------------------------------| | | | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 488 | 79.4 [193/243] (72.4, 86.3) | 74.5 [181/243] (68.4, 81.0) | 4.9 [12/243] (-2.4, 12.3) | 93.5 [1142/1221] (91.1, 95.4) | 95.7 [1169/1221] (94.0, 97.2) | -2.2 [-27/1221] (-3.9, -0.9) | | Site 2 | 494 | 77.5 [179/231] (70.3, 84.6) | 87.4 [202/231] (80.3, 93.3) | -10.0 [-23/231] (-17.0, -4.1) | 96.8 [1211/1251] (95.5, 97.9) | 96.8 [1211/1251] (95.4, 98.0) | 0.0 [0/1251] (-1.1, 1.0) | | Site 3 | 490 | 83.8 [171/204] (77.8, 89.5) | 54.4 [111/204] (45.7, 62.9) | 29.4 [60/204] (22.4, 37.5) | 95.6 [1210/1266] (94.0, 97.0) | 99.4 [1259/1266] (98.9, 99.8) | -3.9 [-49/1266] (-5.3, -2.5) | | Site 4 | 488 | 86.7 [156/180] (82.1, 91.3) | 78.3 [141/180] (70.7, 86.8) | 8.3 [15/180] (0.0, 15.7) | 93.5 [1200/1284] (91.8, 95.1) | 96.8 [1243/1284] (95.5, 98.0) | -3.3 [-43/1284] (-4.9, -1.7) | | Total | 1960 | 81.5 [699/858] (78.5, 84.4) | 74.0 [635/858] (70.1, 77.5) | 7.5 [64/858] (4.0, 11.4) | 94.8 [4763/5022] (94.0, 95.6) | 97.2 [4882/5022] (96.6, 97.8) | -2.4 [-119/5022] (-3.0, -1.7) | # Cancer Sensitivity is a percent of "reference" Cancer cases classified in Genius Cervical AI algorithm reviews or in Manual reviews as Cancer or UNSAT, and specificity is a percent of "reference" {23}------------------------------------------------ (NILM or ASCUS or AGC or LSIL or ASC-H or HSIL) cases classified in either review as NILM or ASCUS or AGC or LSIL or ASC-H or HSIL, (described in Table 17 below). | Sites | Number of Cases | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|--------------------|----------------------------------|----------------------------------|----------------------------------|-------------------------------------|-------------------------------------|----------------------------------| | | | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 488 | 66.7 [14/21] (25.0, 100.0) | 76.2 [16/21] (50.0, 100.0) | -9.5 [-2/21] (-33.3, 11.1) | 98.3 [1418/1443] (97.0, 99.2) | 98.6 [1423/1443] (97.7, 99.3) | -0.3 [-5/1443] (-1.1, 0.3) | | Site 2 | 494 | 66.7 [14/21] (20.8, 100.0) | 85.7 [18/21] (63.0, 100.0) | -19.0 [-4/21] (-44.4, 0.0) | 98.6 [1440/1461] (97.8, 99.3) | 97.7 [1428/1461] (96.5, 98.8) | 0.8 [12/1461] (0.1, 1.6) | | Site 3 | 490 | 60.6 [20/33] (33.3, 84.6) | 39.4 [13/33] (16.7, 66.7) | 21.2 [7/33] (3.7, 40.0) | 98.9 [1421/1437] (98.2, 99.5) | 99.4 [1429/1437] (98.8, 99.9) | -0.6 [-8/1437] (-1.3, 0.1) | | Site 4 | 488 | 76.2 [16/21] (44.4, 100.0) | 81.0 [17/21] (55.6, 100.0) | -4.8 [-1/21] (-22.2, 13.3) | 98.4 [1420/1443] (97.6, 99.1) | 98.4 [1420/1443] (97.6, 99.2) | 0.0 [0/1443] (-0.8, 0.8) | | Total | 1960 | 66.7 [64/96] (51.7, 80.6) | 66.7 [64/96] (54.3, 79.0) | 0.0 [0/96] (-9.8, 11.1) | 98.5 [5699/5784] (98.0, 98.9) | 98.5 [5700/5784] (98.1, 98.9) | -0.0 [-1/5784] (-0.4, 0.4) | Table 17. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Stratified by Site at Cancer # UNSAT Sensitivity is a percent of "reference" UNSAT cases classified in Genius reviews or in Manual reviews as UNSAT or ASCUS+, and specificity is a percent of "reference" Satisfactory (SAT) slides classified in either review as SAT, (described in Table 18 below). | Table 18. Sensitivity and Specificity of Genius Cervical AI Algorithm Review (Genius) and | | |-------------------------------------------------------------------------------------------|--| | Manual Glass Slide Review (Manual) Stratified by Site at UNSAT | | | Sites | Number of Cases | Sensitivity (95%CI) | | | Specificity (95%CI) | | | |--------|--------------------|----------------------------------|----------------------------------|----------------------------------|-------------------------------------|-------------------------------------|----------------------------------| | | | Genius | Manual | Difference | Genius | Manual | Difference | | Site 1 | 503 | 86.7 [39/45] (71.1, 100) | 51.1 [23/45] (26.7, 73.3) | 35.6 [16/45] (11.1, 57.8) | 99.6 [1458/1464] (98.9, 100) | 99.9 [1463/1464] (99.8, 100) | -0.3 [-5/1464] (-1.0, 0.1) | | Site 2 | 500 | 77.8 [14/18] (55.6, 94.4) | 77.8 [14/18] (55.6, 100) | 0.0 [0/18] (-16.7, 16.7) | 99.6 [1476/1482] (99.1, 100) | 99.7 [1478/1482] (99.3, 100) | -0.1 [-2/1482] (-0.5, 0.1) | | Site 3 | 495 | 80.0 [12/15] (40.0, 100) | 53.3 [8/15] (26.7, 66.7) | 26.7 [-4/15] (13.3, 33.3) | 99.7 [1465/1470] (99.2, 100) | 99.9 [1468/1470] (99.7, 100) | -0.2 [-3/1470] (-0.6, 0.1) | | Site 4 | 496 | 70.8 [17/24] (37.5, 95.8) | 75.0 [18/24] (50.0, 95.8) | -4.2 [-1/24] (-29.2, 25.0) | 100 [1464/1464] (100, 100) | 99.3 [1454/1464] (98.8, 99.8) | 0.7 [10/1464] (0.2, 1.2) | | Total | 1994 | 80.4 [82/102] (67.6, 91.2) | 61.8 [63/102] (50.0, 72.5) | 18.6 [19/102] (5.9, 31.4) | 99.7 [5863/5880] (99.5, 99.9) | 99.7 [5863/5880] (99.5, 99.9) | 0.0 [0/5880] (-0.2, 0.2) | {24}------------------------------------------------ ## C.1.3 Tables of performance of each Bethesda Category Table 19 through Table 26 summarize results from Genius Cervical AI algorithm-assisted review and Manual review for each of the major descriptive diagnosis classifications of the Bethesda System as determined by the adjudication diagnosis: NILM, ASCUS, AGC, LSIL, ASC-H, HSIL, Cancer, and the diagnostic category UNSAT. | | Manual | | | | | | | | | | |--------|--------|-------|------|-------|-----|------|-------|------|--------|-------| | | | UNSAT | NILM | ASCUS | AGC | LSIL | ASC-H | HSIL | Cancer | Total | | | UNSAT | 3 | 10 | 1 | 0 | 0 | 0 | 0 | 0 | 14 | | | NILM | 10 | 3250 | 113 | 12 | 8 | 19 | 2 | 0 | 3414 | | | ASCUS | 0 | 122 | 43 | 0 | 7 | 4 | 1 | 0 | 177 | | | AGC | 1 | 19 | 1 | 0 | 0 | 2 | 2 | 0 | 25 | | Genius | LSIL | 0 | 16 | 22 | 0 | 4 | 0 | 0 | 0 | 42 | | | ASC-H | 1 | 30 | 10 | 0 | 1 | 5 | 1 | 1 | 49 | | | HSIL | 1 | 10 | 6 | 0 | 3 | 2 | 5 | 0 | 27 | | | Cancer | 0 | 4 | 0 | 1 | 0 | 0 | 0 | 0 | 5 | | | Total | 16 | 3461 | 196 | 13 | 23 | 32 | 11 | 1 | 3753 | Table 19. Genius Cervical Al Algorithm Review (Genius) and Manual Glass Slide Review (Manual) Results for All Diagnostic Categories in Slides with Adjudicated Diagnoses of Among the 3753 reviews determined by the adjudication…

Genius Digital Diagnostics System with the Genius Cervical AI algorithm

Device Facts

Intended Use

Device Story

Clinical Evidence

Technological Characteristics

Indications for Use

Regulatory Classification

Identification

Related Devices

Submission Summary (Full Text)

Panel 1

Section Matches

Product Code Matches

Special Control Matches

My Alerts

Genius Digital Diagnostics System with the Genius Cervical AI algorithm

Device Facts

Intended Use

Device Story

Clinical Evidence

Technological Characteristics

Indications for Use

Regulatory Classification

Identification

Related Devices

Submission Summary (Full Text)

Panel 1

Section Matches

Product Code Matches

Special Control Matches

My Alerts

Create Alert

Search Filters

Welcome

Check your email

Welcome!

Genius Digital Diagnostics System with the Genius Cervical AI algorithm