Who is the manufacturer of Contour ProtégéAI+?

Mim Software, Inc. filed the 510(k) submission for Contour ProtégéAI+ (K250035).

What is the FDA product code for Contour ProtégéAI+?

QKB. It is classified under 21 CFR 892.2050 as a Class 2 device in the Radiology panel.

What is the regulatory pathway for Contour ProtégéAI+?

510(k) clearance (submission type: Special).

Does Contour ProtégéAI+ use AI or machine learning?

Yes. Contour ProtégéAI+ (K250035) is flagged as an AI/ML-enabled medical device in the FDA's AI/ML device list.

Who can help prepare an FDA 510(k) submission like Contour ProtégéAI+?

Innolitics — a medical-device software consultancy — provides regulatory and engineering support for FDA 510(k) submissions. See https://innolitics.com/services/510ks/ or contact https://innolitics.com/contact.

Can an AI/ML device like Contour ProtégéAI+ get a guaranteed FDA 510(k) clearance?

Yes. Innolitics offers a guaranteed FDA 510(k) clearance engagement for AI/ML Software as a Medical Device — combining engineering and regulatory execution under one agreement with a guaranteed clearance outcome. Eligibility: the device must be an AI/ML Software as a Medical Device (SaMD), and its performance must meet the criteria Innolitics determines during an initial regulatory strategy engagement to be necessary for FDA clearance. See https://innolitics.com/services/end-to-end-samd/ or contact https://innolitics.com/contact.

Contour ProtégéAI+

K250035 · Mim Software, Inc. · QKB · Feb 3, 2025 · Radiology

Device Facts

Record ID	K250035
Device Name	Contour ProtégéAI+
Applicant	Mim Software, Inc.
Product Code	QKB · Radiology
Decision Date	Feb 3, 2025
Decision	SESE
Submission Type	Special
Regulation	21 CFR 892.2050
Device Class	Class 2
Attributes	AI/ML, Software as a Medical Device

Intended Use

Contour ProtégéAl is an accessory to MIM software used for the contouring of anatomical structures in imaging data using machine-learning-based algorithms automatically. Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI. Contour ProtégéAl is not intended to detect or contour lesions.

Device Story

Contour ProtégéAI+ is a software accessory for MIM software that automatically segments anatomical structures on CT and MR images. It utilizes machine-learning neural network models to generate contours, which are then reviewed and edited by clinicians using visualization software. The device is used in clinical settings (e.g., radiation oncology) to assist in quantitative analysis, adaptive therapy, and treatment planning. It operates on Windows, Mac, or Linux workstations, or via the MIMcloud service. By automating the time-consuming manual contouring process, the device aims to improve efficiency in radiation therapy workflows. Clinicians use the output to inform treatment planning and patient management. The device is not intended for lesion detection.

Clinical Evidence

Bench testing only. Evaluated 770 images from 32 institutions. Performance measured against MIM Maestro reference device using Dice coefficient, Mean Distance to Agreement (MDA), and cumulative Added Path Length (APL). Results demonstrated non-inferiority (p=0.05) for most structures. User evaluation on a three-point scale confirmed time-saving functionality. Training data included diverse scanner vendors (GE, Siemens, Philips, Toshiba) and patient demographics (53.4% female, 31.3% male, 15.3% unknown; 50% over age 60).

Technological Characteristics

Software-based medical image processing system. Utilizes machine-learning neural network models for automated segmentation. Operates on Windows, Mac, and Linux platforms; supports local or cloud-based (MIMcloud) deployment. Compatible with ACR/NEMA DICOM 3.0 standard images. No physical materials or energy sources.

Indications for Use

Indicated for trained medical professionals to assist in automated processing of digital CT and MR images (ACR/NEMA DICOM 3.0). Supports creation of contours via machine-learning for quantitative analysis, adaptive therapy, transfer to radiation therapy planning systems, and archiving. Enables segmentation of anatomical structures across CT locations and prostate, seminal vesicles, and urethra in T2-weighted MR images.

Regulatory Classification

Identification

A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.

Special Controls

*Classification.* Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).

Predicate Devices

Contour ProtégéAl (K231765)

Reference Devices

MIM 4.1 (SEASTAR) [i.e., MIM Maestro] (K071964)

Related Devices

K231765 — Contour ProtégéAI · Mim Software, Inc. · Nov 8, 2023
K213976 — Contour ProtégéAI · Mim Software, Inc. · Feb 3, 2022
K210632 — Contour ProtegeAI · Mim Software, Inc. · Oct 20, 2021
K193252 — Contour ProtegeAI · Mim Software, Inc. · Jul 2, 2020
K223774 — Contour ProtégéAI · Mim Software, Inc. · Apr 6, 2023

Submission Summary (Full Text)

{0}------------------------------------------------ February 3, 2025 Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: on the left, there is a seal of the Department of Health & Human Services - USA, which features an abstract image of an eagle. To the right of the seal, there is the text "FDA U.S. FOOD & DRUG ADMINISTRATION" in blue color. The word "FDA" is in a square box. MIM Software Inc. Sydney Lindner Regulatory Affairs Engineer 25800 Science Park Drive, Suite 180 Cleveland, Ohio 44122 Re: K250035 Trade/Device Name: Contour ProtégéAI+ Regulation Number: 21 CFR 892.2050 Regulation Name: Medical Image Management And Processing System Regulatory Class: Class II Product Code: QKB, Dated: January 7, 2025 Received: January 7, 2025 Dear Sydney Lindner: We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading. If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register. Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" {1}------------------------------------------------ (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download). Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181). Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050. All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems. For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory {2}------------------------------------------------ assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100). Sincerely, Locon Weidner Lora D. Weidner, Ph.D. Assistant Director Radiation Therapy Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health Enclosure {3}------------------------------------------------ ## Indications for Use Submission Number (if known) K250035 Device Name Contour ProtégéAl+ Indications for Use (Describe) Trained medical professionals use Contour ProtégéAI as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications: · Creation of contours using machine-learning algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management. · Segmenting anatomical structures across a variety of CT anatomical locations. · And segmenting the prostate, the seminal vesicles, and the urethra within T2-weighted MR images. Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI. Type of Use (Select one or both, as applicable) > Prescription Use (Part 21 CFR 801 Subpart D) Over-The-Counter Use (21 CFR 801 Subpart C) #### CONTINUE ON A SEPARATE PAGE IF NEEDED. This section applies only to requirements of the Paperwork Reduction Act of 1995. #### *DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW!* The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to: > Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov "An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number. {4}------------------------------------------------ Image /page/4/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a simple color palette. #### 510(k) Summary (The following information is in conformance with 21 CFR 807.92) ## Submitter K250035 MIM Software Inc. 25800 Science Park Drive - Suite 180 Cleveland, OH 44122 | Phone: Fax: | 216-455-0600 216-455-0601 | |------------------------|------------------------------| | Contact Person: | Sydney Lindner | | Date Summary Prepared: | January 7, 2025 | ## Device Name Trade Name: Common Name: Regulation Number / Product Code: Classification Name: Contour ProtégéAl+ Medical Imaging Software 21 CFR 892.2050 Product Code QKB System, Imaging Processing, Radiological ## Predicate and Reference Devices | Predicate: | K231765 | Contour ProtégéAl | MIM Software Inc. | |------------|---------|------------------------------------------|-------------------| | Reference: | K071964 | MIM 4.1 (SEASTAR) [i.e., MIM Maestro] | MIMvista Corp. | {5}------------------------------------------------ Image /page/5/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, to the left of the text "mim SOFTWARE." The text is in a sans-serif font, with "mim" in a larger size than "SOFTWARE." ## Intended Use Contour ProtégéAl is an accessory to MIM software used for the contouring of anatomical structures in imaging data using machine-learning-based algorithms automatically. Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI. Contour ProtégéAl is not intended to detect or contour lesions. #### Indications for Use Trained medical professionals use Contour ProtégéAl as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications: - Creation of contours using machine-learning algorithms for applications ● including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management. - · Segmenting anatomical structures across a variety of CT anatomical locations. - And segmenting the prostate, the seminal vesicles, and the urethra within ● T2-weighted MR images. Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAl. ## Device Description Contour ProtégéAl+ is an accessory to MIM software that automatically creates contours on medical images through the use of machine-learning algorithms. It is designed for use in the processing of medical images and operates on Windows, Mac, and Linux computer systems. Contour ProtégéAl+ is deployed on a remote server using the MIMcloud service for data management and transfer; or locally on the workstation or server running MIM software. {6}------------------------------------------------ Image /page/6/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle cut out of the red square where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. Indications for use have not been modified. The intended use is the same as the last 510(k) clearance for Contour ProtégéAl (K231765). In the upcoming 1.4.0 release, one new neural network model is being added and three existing neural network models are being updated by retraining with additional or updated contours which prompted this premarket submission of the subject device. | ITEM | Subject Device: Contour ProtégéAl+ (K250035) | Predicate Device: Contour ProtégéAl (K231765) | Substantial Equivalence Discussion | |---------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Clearance Date | TBD | November 8, 2023 | N/A | | Operating Platform | Server-based application supporting: • Linux-based OS • Local deployment on Windows or Mac | Server-based application supporting: • Linux-based OS • Local deployment on Windows or Mac | No change | | Modalities | CT and MR | CT and MR | No change | | Atlas-based Segmentation | No | No | No change | | Automatically contour imaging data using machine- learning | Yes | Yes | No change | | Cloud-based Deployment | Yes | Yes | No change | | Local Deployment or Installation | Yes | Yes | No change | | Neural | (1.0.0 models) | (1.0.0 models) | The subject device contains | | | Subject Device: | Predicate Device: | | | ITEM | Contour ProtégéAl+ (K250035) | Contour ProtégéAl (K231765) | Substantial Equivalence Discussion | | Network Models Included | • Head and Neck CT • Prostate CT • Thorax CT • Liver CT • Prostate MR (1.1.0 model) • Prostate MR (2.0.0 models) • Head and Neck CT • Prostate CT • Thorax CT • Abdomen CT • Lungs and Liver CT (3.1.0 models)* • Head and Neck CT • Prostate CT • Thorax CT • Abdomen CT • Lungs and Liver CT • MRT Additional Structures CT (4.0.0 models) • Head and Neck CT | • Head and Neck CT • Prostate CT • Thorax CT • Liver CT • Prostate MR (1.1.0 model) • Prostate MR (2.0.0 models) • Head and Neck CT • Prostate CT • Thorax CT • Abdomen CT • Lungs and Liver CT (3.1.0 models)* • Head and Neck CT • Prostate CT • Thorax CT • Abdomen CT • Lungs and Liver CT • MRT Additional Structures CT (4.0.0 models) • Head and Neck CT | one new neural network model and three updated neural network models compared to the predicate. These 4 models were all trained on the same architecture as the predicate without any changes. These 4 models cover all the existing structures for contouring and additional, new structures, which are denoted in bold font. These 4 models were all tested according to the same procedures and acceptance criteria as the predicate. No unexpected results were observed. The changes do not raise new questions for safety and effectiveness. | | | • Thorax CT • Abdomen CT • Pelvis CT • SurePlan MRT CT | • Thorax CT • Abdomen CT • Pelvis CT • SurePlan MRT CT | | | | (4.1.0 models) • Head and Neck CT • Thorax CT • Whole Body - Physiological • Uptake Organs CT | (4.1.0 models) • Head and Neck CT • Thorax CT • Whole Body - Physiological • Uptake Organs CT | | | | (4.2.0 models) | | | | | Subject Device: | Predicate Device: | | | ITEM | Contour ProtégéAl+ (K250035) | Contour ProtégéAl (K231765) | Substantial Equivalence Discussion | | | • Thorax CT - LN_IMN_L - LN_IMN_R - LN_Sclav_L - LN_Sclav_R - LN_Ax_L1_L - LN_Ax_L1_R - LN_Ax_L2_L - LN_Ax_L2_R - LN_Ax_L3_L - LN_Ax_L3_R - BrachialPlex_L*** - BrachialPlex_R*** - Breast_L*** - Breast_R*** - Breast_L_RTOG** - Breast_R_RTOG** - Bronchus - Carina - Cricoid - Esophagus - Glnd_Thyroid - GreatVes*** - Heart*** - Humerus_Head_L - Humerus_Head_R - Kidney_L - Kidney_R - Larynx*** - Liver - Lung_L - Lung_R - Musc_Constrict - Pancreas - SpinalCord - Stomach*** - Trachea - Ribs** - Chestwall_L** - Chestwall_R** - A_Asc_Aorta** - A_LAD** • Abdomen CT - Bladder - Bowel*** | | | | | Subject Device: | Predicate Device: | Substantial Equivalence | | ITEM | Contour ProtégéAl+ (K250035) | Contour ProtégéAl (K231765) | Discussion | | | - Bowel Bag*** - Cauda Equina - Left Kidney - Right Kidney - Liver - Spinal Cord - Stomach*** - Female Pelvis CT** - Bowel Bag** - Bowel** - LN Pelvics** - Bladder** - Uterocervix** - Colon Sigmoid** - Cauda Equina** - Sacral Plexus** - Sacrum** - Rectum** - Femur Head L** - Femur Head R** - SurePlan MRT CT - Bone*** - Glnd_Lacrimal_L - Glnd_Lacrimal_R - Glnd_Submand_L - Glnd_Submand_R - Glnd_Thyroid - Kidney_L (Kidney_L w/o Renal Pelvis) - Kidney_R (Kidney_R w/o Renal Pelvis) - Liver - Lung_L - Lung_R - Parotid_L - Parotid_R - Spleen | | | #### Table 1 - Comparison to Predicate Device. {7}------------------------------------------------ Image /page/7/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the word "SOFTWARE" in a smaller, sans-serif font. {8}------------------------------------------------ Image /page/8/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, and the colors are eye-catching. {9}------------------------------------------------ Image /page/9/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle in the red square. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the text "SOFTWARE" in a smaller, sans-serif font. * The 3.1.0 models share the same training images and architecture as the 3.0.0 models. Some errors and style inconsistencies in the training segmentations were corrected before re-training, resulting in the 3.1.0 models. ** Indicates new contour/model. *** Indicates updated contour. {10}------------------------------------------------ Image /page/10/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a focus on simplicity and clarity. ## Testing and Performance Data Changes within this submission compared to the predicate device, Contour ProtégéAl (K231765), include one new 4.2.0 neural network model and three updated 4.2.0 neural network models with additional or updated contours. Non-inferiority testing was used to compare the Contour ProtégéAl+ subject device to Atlases created from the MIM Maestro (K071964) reference device. Verification and validation tests were performed for each of the core features of Contour ProtégéAl+: - Contour creation time savings ● - Clinical quality contour generation ● The testing methods include both internal verification by MIM Software's own qualified testers and external validation by trained medical professionals. The Contour ProtégéAl+ subject device was evaluated on the test subjects from a pool of 770 images gathered from 32 institutions. The CT images for this training set were obtained from clinical treatment plans for patients prescribed external beam or molecular radiotherapy, but the original segmentations were not used. Instead, the images were re-segmented by consultants (physicians and dosimetrists) specifically for this purpose, outside of clinical practice. Detailed instructions derived from relevant published contouring guidelines were prepared for the dosimetrists. The initial seqmentations were then reviewed and corrected by radiation oncologists against the same standards and guidelines. Qualified staff at MIM Software (MD or licensed dosimetrists) then performed a final review and correction. All segmenters and reviewers were instructed to spend additional time to ensure the highest quality training data. In particular, the consultants were asked to contour all specified OAR structures on all images according to referenced standards, whether or not they were proximal to the treatment field. All patients were imaged on an indexed couch in the treatment position ("simulation CT"). Series that were no-axial, had slices thinner than 0.5mm, or had non-Fan Beam or MV acquisitions were excluded. The verification data used for testing is from a set of institutions that are totally disjoint from the datasets used to train each model in the Contour ProtégéAl+ device. The MIM Maestro (K071964) reference device was configured with Atlases built from the same training data used to train the models. We tested Contour ProtégéAl+ against the reference device, and the goal of this testing is to show that the subject device is equivalent or superior to the reference device. The performance of both segmentation devices was measured by calculating both the Dice score and MDA of the novel segmentations with the original ground-truth contours. User beta testing was also used to evaluate the performance of Contour ProtégéAl+ in the context of time savings {11}------------------------------------------------ Image /page/11/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, on the left side. To the right of the squares is the text "mim" in a stylized, lowercase font, with the word "SOFTWARE" underneath in a smaller font. compared to contouring from scratch. This user evaluation was made on a three-point scale for each contour, with one indicating negligible time savings, two indicating moderate, and three indicating significant time savings. Our acceptance criteria combine the statistical tests and the user evaluation - only structures that pass two or more of the following three tests could be included in the final models: - Statistical non-inferiority of the Dice score compared with the reference predicate. - Statistical non-inferiority of the MDA score compared with the reference predicate. - Average user evaluation of 2 or higher, when measured on a three-point scale. ● Further clinical validation was also conducted in-house to qualitatively evaluate contours compared to detailed criteria based on established clinical guidelines. In addition, each model as a whole was also evaluated. In order to be included in the released product, the cumulative Added Path (APL) of the contours in the model was evaluated relative to the ground-truth. Cumulative APL has been found to correlate well with the total time spent in editing and correcting auto-segmented contours (Vaassen et al, 2020)1. Each model was required to have statistically non-inferior cumulative APL compared to the reference predicate. Finally, the localization accuracy of each structure in all models was measured. We did not impose a passing criterion on localization accuracy; the results however are included in this summary document as well as in our User Guide and White Paper to allow the user to better understand the performance of the device. Across the testing data images, 53.4% were female, 31.3% were male, and 15.3% were unknown for sex. The manufacturer was GE for 43.2%, Siemens for 21.7%, Phillips for 15.5%, Toshiba for 0.4%, and unknown or other scanner vendors for 19.2%. 0.3% were between the ages of 0-20, 4.7% were between the aqes of 20-40, 20.9% were between 40-60, and 50.0% were over the age of 60. 24.1% were unknown for age. The mean and standard deviation Dice coefficients and MDA scores, along with the lower 95th percentile confidence bound, were calculated for both the proposed Contour ProtégéAl device and the MIM Maestro atlas segmentation reference device for each structure of each neural network model. Contour ProtégéAl+ results were equivalent or had better performance than the MIM Maestro atlas segmentation reference device. Equivalence is defined such that the lower 95th percentile confidence bound of the Contour ProtégéAl+ segmentation is greater than 0.1 Dice lower than the mean MIM atlas seqmentation reference device performance. Results over the validation set compared to the reference device are presented below. {12}------------------------------------------------ Image /page/12/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a lowercase, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, with a clean design. | 4.2.0 CT Model | Structure | Dice MIM Atlas | Dice Contour ProtégéAl | MDA MIM Atlas | MDA Contour ProtégéAl | External Evaluation Score | |-------------------|---------------------|-------------------|------------------------------|---------------------|-----------------------------|---------------------------------| | Thorax | BrachialPlex_L | 0.30 ± 0.14 | 0.41 ± 0.15 (0.28) * | 3.01 ± 1.19 | 2.89 ± 1.07 (3.94) * | 2.43 | | | BrachialPlex_R | 0.31 ± 0.12 | 0.39 ± 0.15 (0.26) * | 3.08 ± 1.54 | 3.11 ± 1.38 (4.55) * | 2.43 | | | Breast_L | 0.74 ± 0.11 | 0.79 ± 0.07 (0.73) * | 5.73 ± 3.10 | 5.00 ± 2.53 (6.98) * | 2.38 | | | Breast_R | 0.76 ± 0.12 | 0.80 ± 0.11 (0.71) * | 5.03 ± 2.72 | 4.49 ± 2.02 (6.24) * | 2.57 | | | Breast_L_RTO_G | 0.74 ± 0.11 | 0.77 ± 0.11 (0.69) * | 5.73 ± 3.10 | 5.47 ± 3.72 (7.87) | 2.5 | | | Breast_R_RTO_G | 0.76 ± 0.12 | 0.77 ± 0.15 (0.67) * | 5.03 ± 2.72 | 5.10 ± 3.17 (7.24) | 2.5 | | | Bronchus | 0.57 ± 0.19 | 0.62 ± 0.14 (0.50) ** | 2.69 ± 2.22 | 1.86 ± 0.90 (3.19) * | 2.63 | | | Carina | 0.37 ± 0.18 | 0.50 ± 0.12 (0.41) * | 2.67 ± 2.65 | 1.93 ± 0.83 (3.07) * | 2.43 | | | Cricoid | 0.02 ± 0.04 | 0.06 ± 0.05 (0.02) * | 4.77 ± 1.53 | 5.01 ± 1.40 (6.27) * | 2.86 | | | Esophagus | 0.47 ± 0.17 | 0.69 ± 0.16 (0.63) * | 2.73 ± 2.19 | 1.08 ± 1.53 (1.71) * | 2.5 | | | Glnd_Thyroid | 0.46 ± 0.18 | 0.66 ± 0.18 (0.53) * | 2.84 ± 1.90 | 1.68 ± 1.44 (2.86) * | 2.86 | | | GreatVes | 0.64 ± 0.17 | 0.70 ± 0.16 (0.53) ** | 4.59 ± 1.99 | 3.44 ± 1.99 (5.46) * | 2.63 | | | Heart | 0.88 ± 0.08 | 0.90 ± 0.07 (0.87) * | 3.05 ± 2.03 | 2.49 ± 1.84 (3.08) * | 2.63 | | | Humerus_Head_L | 0.60 ± 0.24 | 0.62 ± 0.24 (0.42) ** | 0.38 ± 0.23 | 0.33 ± 0.22 (0.52) * | 3 | | | Humerus_Head_R | 0.57 ± 0.22 | 0.60 ± 0.22 (0.41) ** | 0.78 ± 2.00 | 0.33 ± 0.35 (1.60) * | 3 | | 4.2.0 CT Model | Structure | Dice MIM Atlas | Dice Contour ProtégéAl | MDA MIM Atlas | MDA Contour ProtégéAl | External Evaluation Score | | | Kidney_L | 0.73 ± 0.19 | 0.90 ± 0.09 (0.83) * | 3.75 ± 2.66 | 1.32 ± 1.13 (2.27) * | 2.75 | | | Kidney_R | 0.73 ± 0.19 | 0.89 ± 0.09 (0.83) * | 3.97 ± 2.60 | 1.41 ± 1.06 (2.34) * | 2.75 | | | Larynx | 0.47 ± 0.19 | 0.59 ± 0.14 (0.51) * | 3.42 ± 1.32 | 2.87 ± 1.34 (3.51) * | 2.63 | | | Liver | 0.84 ± 0.12 | 0.90 ± 0.13 (0.85) * | 5.06 ± 4.18 | 3.56 ± 9.12 (6.39) * | 2.71 | | | Lung_L | 0.95 ± 0.02 | 0.96 ± 0.02 (0.96) * | 1.12 ± 0.46 | 0.79 ± 0.40 (0.94) * | 2.75 | | | Lung_R | 0.95 ± 0.03 | 0.97 ± 0.03 (0.96) * | 1.36 ± 0.67 | 0.85 ± 0.48 (1.04) * | 2.75 | | | Musc_Constrict | 0.40 ± 0.17 | 0.50 ± 0.17 (0.39) * | 2.00 ± 1.79 | 1.67 ± 1.55 (2.71) * | 3 | | | Pancreas | 0.17 ± 0.17 | 0.47 ± 0.21 (0.36) * | 16.43 ± 16.42 | 6.80 ± 8.89 (14.66) ** | 2.17 | | | SpinalCord | 0.66 ± 0.16 | 0.64 ± 0.17 (0.59) * | 1.29 ± 0.91 | 1.28 ± 0.72 (1.50) * | 2.5 | | | Stomach | 0.46 ± 0.22 | 0.73 ± 0.21 (0.64) * | 12.56 ± 13.48 | 6.89 ± 20.57 (14.36) ** | 2.13 | | | Trachea | 0.68 ± 0.15 | 0.74 ± 0.17 (0.66) * | 1.43 ± 0.65 | 1.12 ± 0.64 (1.46) * | 2.63 | | | A LAD | 0.07 ± 0.09 | 0.32 ± 0.12 (0.22) * | 8.77 ± 10.69 | 3.43 ± 4.46 (11.41) ** | 2.67 | | | A_Aorta_Asc | 0.72 ± 0.16 | 0.83 ± 0.17 (0.68) ** | 2.90 ± 2.16 | 1.11 ± 0.60 (2.64) * | 2.14 | | | Rib | 0.25 ± 0.09 | 0.28 ± 0.11 (0.21) * | 35.64 ± 12.65 | 39.04 ± 14.62 (49.68) ** | 2.63 | | | Chestwall_L | 0.40 ± 0.16 | 0.39 ± 0.17 (0.19) ** | 4.35 ± 1.12 | 4.44 ± 1.24 (5.87) * | 2.43 | | | Chestwall_R | 0.45 ± 0.18 | 0.42 ± 0.18 (0.22) ** | 4.66 ± 1.65 | 4.69 ± 1.56 (6.51) * | 2.43 | | | LN_Ax_L1_L | 0.59 ± 0.09 | 0.63 ± 0.10 (0.75) * | 2.76 ± 0.89 | 2.25 ± 0.80 (3.32) * | 2.67 | | 4.2.0 CT Model | Structure | Dice MIM Atlas | Dice Contour ProtégéAl | MDA MIM Atlas | MDA Contour ProtégéAl | External Evaluation Score | | | LN_Ax_L1_R | 0.52 ± 0.12 | 0.58 ± 0.13 (0.69) * | 3.46 ± 1.04 | 2.90 ± 1.52 (4.09) * | 2.67 | | | LN_Ax_L2_L | 0.57 ± 0.16 | 0.64 ± 0.13 (0.77) ** | 2.57 ± 1.25 | 1.93 ± 0.88 (2.88) * | 2.67 | | | LN_Ax_L2_R | 0.53 ± 0.20 | 0.60 ± 0.18 (0.73) ** | 2.91 ± 1.29 | 2.44 ± 1.26 (3.31) * | 2.67 | | | LN_Ax_L3_L | 0.55 ± 0.17 | 0.62 ± 0.19 (0.76) ** | 2.21 ± 1.11 | 1.90 ± 1.26 (2.87) * | 2.67 | | | LN_Ax_L3_R | 0.46 ± 0.16 | 0.52 ± 0.17 (0.65) * | 3.30 ± 1.95 | 2.74 ± 1.73 (4.24) * | 2.67 | | | LN_IMN_L | 0.17 ± 0.11 | 0.41 ± 0.17 (0.59) * | 4.75 ± 3.26 | 1.82 ± 0.85 (5.05) ** | 3 | | | LN_IMN_R | 0.23 ± 0.16 | 0.48 ± 0.20 (0.63) * | 3.36 ± 1.95 | 1.69 ± 1.38 (3.09) * | 3 | | | LN_Sclav_L | 0.58 ± 0.15 | 0.66 ± 0.13 (0.79) * | 2.84 ± 1.94 | 2.49 ± 1.70 (4.18) * | 2 | | | LN_Sclav_R | 0.48 ± 0.09 | 0.55 ± 0.09 (0.64) * | 2.97 ± 0.75 | 2.67 ± 0.90 (3.56) * | 2.33 | | Abdomen | Bladder | 0.72 ± 0.23 | 0.92 ± 0.16 (0.81) * | 3.97 ± 3.00 | 0.78 ± 0.73 (1.93) * | 2.6 | | | Bowel | 0.34 ± 0.14 | 0.52 ± 0.19 (0.37) * | 9.26 ± 3.49 | 5.05 ± 3.17 (8.13) ** | 2.5 | | | BowelBag | 0.30 ± 0.09 | 0.36 ± 0.11 (0.29) * | 14.13 ± 3.62 | 10.08 ± 3.36 (12.50) * | 2.89 | | | CaudaEquina | 0.62 ± 0.15 | 0.69 ± 0.13 (0.59) * | 1.17 ± 0.51 | 0.95 ± 0.53 (1.33) * | 2.6 | | | Kidney_L | 0.74 ± 0.17 | 0.94 ± 0.03 (0.85) * | 3.86 ± 2.48 | 0.82 ± 0.41 (2.12) * | 2.9 | | | Kidney_R | 0.75 ± 0.18 | 0.92 ± 0.07 (0.83) * | 3.97 ± 3.35 | 0.96 ± 0.62 (2.74) * | 2.9 | | | Liver | 0.84 ± 0.12 | 0.93 ± 0.08 (0.86) * | 5.23 ± 3.65 | 2.01 ± 2.14 (3.85) * | 2.8 | | | SpinalCord | 0.60 ± 0.16 | 0.65 ± 0.14 (0.56) * | 1.13 ± 0.60 | 0.83 ± 0.32 (1.10) * | 2.9 | | 4.2.0 CT Model | Structure | Dice MIM Atlas | Dice Contour ProtégéAl | MDA MIM Atlas | MDA Contour ProtégéAl | External Evaluation Score | | | Stomach | 0.49 ± 0.21 | 0.82 ± 0.11 (0.72) * | 11.56 ± 14.68 | 2.68 ± 2.02 (9.37) ** | 2.5 | | Female Pelvis | Bladder | 0.61 ± 0.18 | 0.91 ± 0.06 (0.83) * | 5.53 ± 3.76 | 1.04 ± 0.94 (2.66) * | 2.75 | | | Bag_Bowel | 0.45 ± 0.17 | 0.52 ± 0.20 (0.36) ** | 8.89 ± 2.79 | 6.89 ± 2.25 (9.06) * | 2.63 | | | Bowel | 0.34 ± 0.10 | 0.55 ± 0.19 (0.42) * | 8.81 ± 3.05 | 4.84 ± 3.03 (7.44) * | 2.75 | | | Colon_Sigmoid | 0.05 ± 0.05 | 0.47 ± 0.24 (0.36) * | 20.55 ± 12.23 | 14.72 ± 15.14 (23.56) ** | 2.5 | | | Femur_Head_ L | 0.86 ± 0.07 | 0.91 ± 0.08 (0.84) * | 1.43 ± 0.78 | 0.95 ± 1.03 (1.75) * | 2.75 | | | Femur_Head_ R | 0.87 ± 0.05 | 0.92 ± 0.02 (0.89) * | 1.22 ± 0.59 | 0.69 ± 0.28 (1.09) * | 2.75 | | | UteroCervix | 0.16 ± 0.16 | 0.65 ± 0.27 (0.47) * | 10.65 ± 5.73 | 4.39 ± 13.21 (13.49) ** | 2 | | | LN_Pelvics | 0.64 ± 0.05 | 0.74 ± 0.06 (0.69) * | 4.37 ± 1.11 | 4.12 ± 1.52 (5.28) * | 2.75 | | | Rectum | 0.38 ± 0.16 | 0.75 ± 0.12 (0.67) * | 6.30 ± 3.30 | 1.52 ± 1.66 (3.04) * | 2.63 | | | SacralPlex | 0.02 ± 0.01 | 0.03 ± 0.01 (0.02) * | 13.03 ± 1.73 | 12.81 ± 1.86 (14.38) * | 2.5 | | | Sacrum | 0.84 ± 0.02 | 0.89 ± 0.01 (0.87) * | 1.58 ± 0.32 | 1.09 ± 0.17 (1.31) * | 3 | | | CaudaEquina | 0.65 ± 0.11 | 0.66 ± 0.11 (0.58) * | 1.19 ± 0.58 | 0.94 ± 0.59 (1.39) * | 2.57 | | SurePlan MRT | Bone | 0.76 ± 0.08 | 0.83 ± 0.05 (0.71) * | 4.77 ± 1.97 | 4.57 ± 3.35 (9.50) ** | 3 | | | Glnd_Lacrimal _L | 0.23 ± 0.17 | 0.30 ± 0.21 (0.16) * | 1.99 ± 0.86 | 1.26 ± 0.57 (1.96) * | 2.67 | | | Glnd_Lacrimal _R | 0.23 ± 0.16 | 0.36 ± 0.23 (0.22) * | 1.53 ± 0.80 | 1.18 ± 0.87 (1.97) * | 2.67 | | | Glnd_Subman | 0.58 ± 0.12 | 0.67 ± 0.29 | 2.19 ± | 1.00 ± 0.35 | 3 | | 4.2.0 CT Model | Structure | Dice MIM Atlas | Dice Contour ProtégéAl | MDA MIM Atlas | MDA Contour ProtégéAl | External Evaluation Score | | | d_L | | (0.52) * | 0.76 | (1.45) * | | | | GInd_Subman d_R | 0.56 ± 0.16 | 0.66 ± 0.31 (0.48) * | 2.41 ± 1.14 | 0.97 ± 0.33 (1.68) * | 3 | | | Glnd_Thyroid | 0.47 ± 0.19 | 0.75 ± 0.12 (0.63) ** | 2.98 ± 1.95 | 1.29 ± 1.20 (2.46) * | 2.67 | | | Kidney_L | 0.72 ± 0.18 | 0.91 ± 0.04 (0.82) * | 4.08 ± 2.52 | 1.56 ± 0.61 (2.89) * | 3 | | | Kidney_R | 0.76 ± 0.17 | 0.91 ± 0.03 (0.82) * | 3.87 ± 3.14 | 1.45 ± 0.63 (3.12) * | 3 | | | Liver | 0.85 ± 0.12 | 0.93 ± 0.07 (0.88) * | 4.82 ± 3.51 | 1.79 ± 1.50 (3.25) * | 2.67 | | | Lung_L | 0.94 ± 0.03 | 0.96 ± 0.04 (0.93) * | 1.42 ± 0.51 | 0.90 ± 0.49 (1.32) * | 3 | | | Lung_R | 0.94 ± 0.04 | 0.96 ± 0.05 (0.92) * | 1.66 ± 0.75 | 1.07 ± 0.86 (1.75) * | 3 | | | Parotid L | 0.71 ± 0.09 | 0.81 ± 0.05 (0.78) * | 2.17 ± 0.77 | 1.42 ± 0.40 (1.73) * | 3 | | | Parotid_R | 0.71 ± 0.09 | 0.82 ± 0.05 (0.78) * | 2.17 ± 0.73 | 1.34 ± 0.48 (1.67) * | 3 | | | Spleen | 0.72 ± 0.10 | 0.95 ± 0.02 (0.87) * | 4.38 ± 1.87 | 0.62 ± 0.45 (2.08) * | 2.67 | # Table 2 – Dice, MDA, and external evaluation results. {13}------------------------------------------------ Image /page/13/Picture/0 description: The image shows the MIM Software logo. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a stylized font, with the word "SOFTWARE" below it. {14}------------------------------------------------ Image /page/14/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a stylized font, with the word "SOFTWARE" underneath in smaller letters. The logo is simple and modern, with a focus on the company's name. {15}------------------------------------------------ Image /page/15/Picture/0 description: The image shows the logo for MIM Software. The logo consists of a gray square overlapping a red square with a white circle in the middle. To the right of the squares is the text "mim" in a sans-serif font, with the word "SOFTWARE" below it in a smaller font. {16}------------------------------------------------ Image /page/16/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a lowercase, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, with a clean design. Mean ± Std MDA mm (upper 95th percentile confidence bound based on normal distribution in parentheses) * Equivalence demonstrated at p=0.05 significance level between Contour ProtégéAI and MIM Atlas (green) ** Equivalence was NOT demonstrated because the minimum sample size was not met for this contour (yellow) Cells highlighted in orange indicate that equivalence was NOT demonstrated. Additionally, preliminary user evaluation conducted as part of testing demonstrated that Contour ProtégéAl+ yields comparable time-saving functionality when creating contours as other commercially available automatic segmentation products. {17}------------------------------------------------ Image /page/17/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle in the intersection. To the right of the squares is the text "mim" in a sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, and the colors are eye-catching. ## Table 3 – 4.2.0 CT Models cumulative APL. | 4.2.0 CT Models | MIM Atlas | Contour ProtégéAl | |-----------------|-----------------|----------------------------| | Thorax | 220.33 ± 232.42 | 181.44 ± 219.43 (200.79) * | | Abdomen | 354.53 ± 386.05 | 238.05 ± 309.40 (312.07) * | | Female Pelvis | 433.24 ± 392.19 | 314.13 ± 349.77 (383.86) * | | SurePlan MRT | 216.52 ± 207.87 | 133.01 ± 160.23 (167.60) * | Mean ± Std APL mm (upper 95th percentile confidence bound based on normal distribution in parentheses). * Equivalence demonstrated at p=0.05 significance level between Contour ProtégéAl and MIM Atlas. | 4…