Who is the manufacturer of AutoContour (RADAC V5)?

Radformation, Inc. filed the 510(k) submission for AutoContour (RADAC V5) (K260509).

What is the FDA product code for AutoContour (RADAC V5)?

QKB. It is classified under 21 CFR 892.2050 as a Class 2 device in the Radiology panel.

What is the regulatory pathway for AutoContour (RADAC V5)?

510(k) clearance (submission type: Special).

Does AutoContour (RADAC V5) use AI or machine learning?

Yes. AutoContour (RADAC V5) (K260509) is flagged as an AI/ML-enabled medical device in the FDA's AI/ML device list.

Who can help prepare an FDA 510(k) submission like AutoContour (RADAC V5)?

Innolitics — a medical-device software consultancy — provides regulatory and engineering support for FDA 510(k) submissions. See https://innolitics.com/services/510ks/ or contact https://innolitics.com/contact.

Can an AI/ML device like AutoContour (RADAC V5) get a guaranteed FDA 510(k) clearance?

Yes. Innolitics offers a guaranteed FDA 510(k) clearance engagement for AI/ML Software as a Medical Device — combining engineering and regulatory execution under one agreement with a guaranteed clearance outcome. Eligibility: the device must be an AI/ML Software as a Medical Device (SaMD), and its performance must meet the criteria Innolitics determines during an initial regulatory strategy engagement to be necessary for FDA clearance. See https://innolitics.com/services/end-to-end-samd/ or contact https://innolitics.com/contact.

AutoContour (RADAC V5)

Q: What are the predicate devices for AutoContour (RADAC V5)?

AutoContour Model RADAC V4 (K242729).

K260509 · Radformation, Inc. · QKB · Mar 19, 2026 · Radiology

Device Facts

Record ID	K260509
Device Name	AutoContour (RADAC V5)
Applicant	Radformation, Inc.
Product Code	QKB · Radiology
Decision Date	Mar 19, 2026
Decision	SESE
Submission Type	Special
Regulation	21 CFR 892.2050
Device Class	Class 2
Attributes	AI/ML, Software as a Medical Device

Intended Use

AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning.

Device Story

Software-based system for radiation therapy treatment planning; inputs DICOM-compliant CT or MR images; utilizes machine learning-based contouring models to automatically segment anatomical structures (head/neck, thorax, abdomen, pelvis); allows user review and manual modification of generated contours; outputs DICOM-compliant structure sets for import into treatment planning systems. Comprises a .NET Windows client, a local Windows agent service for monitoring network storage, and a cloud-based contouring service. Operated by trained radiation oncology personnel (physicists, dosimetrists, therapists) in clinical settings. Automates contouring tasks to improve efficiency and consistency in treatment planning; output affects clinical decision-making by providing initial structure definitions for radiation dose calculation.

Clinical Evidence

Bench testing only. Performance validated using independent datasets sequestered from training data. Metrics included Mean Dice Similarity Coefficient (DSC) and Hausdorff Distance. CT models (n=459 training, 49 testing) and MR models (n=243-493 training, 24-133 testing) evaluated against manual ground truth (NRG/RTOG/ESTRO guidelines). CT structure models achieved mean DSC of 0.91 (large), 0.86 (medium), and 0.75 (small). Qualitative clinical appropriateness graded by experts (mean score 4.37/5 for CT, 4.3/5 for MR).

Technological Characteristics

Software-only application; no energy delivered to patients. Operates on Windows OS (client/agent) and Linux (cloud server). Uses machine learning (CNN-based) for automated segmentation. Connectivity via TCP/IP. DICOM-compliant input/output. No hardware components.

Indications for Use

Indicated for adult patients (22 years and older) requiring radiation therapy treatment planning, specifically to assist radiation treatment planners in contouring and reviewing structures within medical images.

Regulatory Classification

Identification

A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.

Special Controls

*Classification.* Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).

Predicate Devices

AutoContour Model RADAC V4 (K242729)

Reference Devices

Limbus Contour (K241837)

Related Devices

K200323 — AutoContour · Radformation, Inc. · Oct 30, 2020
K230685 — AutoContour Model RADAC V3 · Radformation, Inc. · Apr 14, 2023
K212915 — MVision AI Segmentation · MVision AI · May 3, 2022
K242729 — AutoContour (Model RADAC V4) · Radformation, Inc. · Dec 9, 2024
K173420 — Radiomics App v1.0 · Microsoft Corp. · Dec 27, 2017

Submission Summary (Full Text)

{0} FDA U.S. FOOD & DRUG ADMINISTRATION March 19, 2026 Radformation, Inc. Jennifer Wampler Senior Regulatory Affairs Specialist 261 Madison Ave. 9th Floor New York, New York 10016 Re: K260509 Trade/Device Name: AutoContour Model RADAC V5 Regulation Number: 21 CFR 892.2050 Regulation Name: Medical Image Management And Processing System Regulatory Class: Class II Product Code: QKB Dated: February 13, 2026 Received: February 17, 2026 Dear Jennifer Wampler: We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading. If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register. U.S. Food & Drug Administration 10903 New Hampshire Avenue Silver Spring, MD 20993 www.fda.gov {1} K260509 - Jennifer Wampler Page 2 Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download). Your device is also subject to, among other requirements, the Quality Management System Regulation (QMSR) (21 CFR Part 820), which includes, but is not limited to, ISO 13485 clause 7.3 (Design controls), ISO 13485 clause 8.3 (Nonconforming product), ISO 13485 clause 8.5.2 (Corrective action), and ISO 13485 clause 8.5.3 (Preventative action). Please note that regardless of whether a change requires premarket review, the QMSR requires device manufacturers to review and approve changes to device design and production (ISO 13485 clause 7.3 and ISO 13485 clause 7.5) and document changes and approvals in the Medical Device File (ISO 13485 clause 4.2.3). Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the Quality Management System Regulation (QMSR) (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050. All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/unique-device-identification-system-udi-system. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medical-device-reporting-mdr-how-report-medical-device-problems. For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory- {2} K260509 - Jennifer Wampler Page 3 assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100). Sincerely, ![img-0.jpeg](img-0.jpeg) Lora D. Weidner, Ph.D. Assistant Director Radiation Therapy Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health Enclosure {3} | Indications for Use | | | | --- | --- | --- | | Please type in the marketing application/submission number, if it is known. This textbox will be left blank for original applications/submissions. | K260509 | ? | | Please provide the device trade name(s). | | ? | | AutoContour Model RADAC V5 | | | | Please provide your Indications for Use below. | | ? | | AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning. | | | | Please select the types of uses (select one or both, as applicable). | ☑ Prescription Use (21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C) | ? | | Please select the age group(s) for which the device(s) is to be used. | ☐ Neonates/Newborns (Birth to < 29 days old) ☐ Infants (29 days old to < 2 years old) ☐ Children (2 years old to < 12 years old) ☐ Adolescents (12 years old to < 22 years old) ☑ Adults (22 years old and greater) | ? | {4} RAD formation K260509 # 510(k) Summary - AutoContour Model RADAC V5 This 510(k) Summary has been created per the requirements of the Safe Medical Device Act (SMDA) of 1990, and the content is provided in conformance with 21 CFR Part 807.92. # 1. Submitter's Information | Table 1: Submitter's Information | | | --- | --- | | Submitter's Name: | Kevin Robinson | | Company: | Radformation, Inc. | | Address: | 261 Madison Avenue, 9th Floor New York, NY 10016 | | Contact Person: | Kevin Robinson VP of Regulatory Affairs, Radformation | | Phone: | 585-500-6996 | | Fax: | — | | Email: | regulatory@radformation.com | | Date of Summary Preparation | 2/13/2026 | # 2. Device Information | Table 2 : Device Information | | | --- | --- | | Trade Name: | AutoContour Model RADAC V5 | | Common Name: | AutoContour, AutoContouring, AutoContour Agent, AutoContour Cloud Server | | Classification Name: | Class II | | Classification: | Medical image management and processing system | | Regulation Number: | 892.2050 | | Product Code: | QKB | | Classification Panel: | Radiology | AutoContour Model RADAC V5 Traditional 510(k) - Summary {5} AutoContour Model RADAC V5 Traditional 510(k) # 3. Predicate Device Information ## Predicate Device AutoContour Model RADAC V5 (Subject Device) makes use of its prior submissions - AutoContour Model RADAC V4 (K242729) as the Predicate Device. The Indications for Use, patient population, functionality and technical components of this Predicate Device remain unchanged in AutoContour Model RADAC V5. This submission is intended to build on the functionality and technological components of the 510(k) cleared AutoContour Model RADAC V4. ## Reference Device AutoContour Model RADAC V5 also makes use of Limbus Contour (K241837) as a reference device for the new structure models from Limbus Contour that have been integrated into this version. Limbus Contour is a Radformation product. # 4. Device Description As with AutoContour Model RADAC V4, the AutoContour Model RADAC V5 device is software that uses DICOM-compliant image data (CT or MR) as input to: (1) automatically contour various structures of interest for radiation therapy treatment planning using machine learning based contouring. The deep-learning-based structure models are trained using imaging datasets consisting of anatomical organs of the head and neck, thorax, abdomen, and pelvis for adult male and female patients, (2) allow the user to review and modify the resulting contours, and (3) generate DICOM-compliant structure set data that can be imported into a radiation therapy treatment planning system. AutoContour Model RADAC V5 consists of 3 main components: 1. A .NET client application designed to run on the Windows Operating System, allowing the user to load image and structure sets for upload to the cloud-based server for automatic contouring, perform registration with other image sets, as well as review, edit, and export the structure set. 2. A local "agent" service designed to run on the Windows Operating System that is configured by the user to monitor a network storage location for new CT and MR datasets that are to be automatically contoured. 3. A cloud-based automatic contouring service that produces initial contours based on image sets sent by the user from the .NET client application. # 5. Indications for Use AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning. {6} # 6. Technological Characteristics AutoContour RADAC V5 (Subject Device) makes use of its prior submission - AutoContour RADCC V2 (K242729), a Predicate Device. The Indications for Use, patient population, functionality, and technical components of this Predicate Device remain unchanged in AutoContour RADAC V5. The main UI outputs are equivalent to the Predicate Device as well, allowing the user to properly visualize and analyze the calculations. This submission is intended to build on the functionality and technological components of the 510(k) cleared AutoContour RADAC V4. | Table 11: Substantial Equivalence AutoContour Model RADAC V5 vs. AutoContour Model RADAC V4 (K242729) | | | | | --- | --- | --- | --- | | Characteristic | Subject Device: AutoContour Model RADAC V5 | Predicate Device: AutoContour Model RADAC V4 (K242729) | Reference Device Limbus Contour (K241837) Used for Verification | | AutoContour vs. PredicateDevices: Technological Characteristics | | | | | Indications for Use | AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning | AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning | Limbus Contour is a software-only medical device intended for use by trained radiation oncologists, dosimetrists, and physicists to derive optimal contours for input to radiation treatment planning. | | Target Population | Any patient type for whom relevant modality scan data is available. | Any patient type for whom relevant modality scan data is available. | Any patient type for whom relevant modality scan data is available. | | Energy Used and/or Delivered | None – software-only application. The software application does not deliver or depend on energy delivered to or from patients | None – software-only application. The software application does not deliver or depend on energy delivered to or from patients | None – software-only application. The software application does not deliver or depend on energy delivered to or from patients | | Intended users | Trained radiation oncology personnel | Trained radiation oncology personnel | Trained radiation oncology personnel | | Design: Data Visualization/Graphical User Interface | Contains both an automated processing component and Data Visualization / Graphical User Interface | Contains both an automated processing component and Data Visualization / Graphical User Interface | Contains an automated processing component. | | Design: View manipulation and Volume rendering | Window and level, pan, zoom, cross-hairs, slice navigation, fused views. | Window and level, pan, zoom, cross-hairs, slice navigation, fused views. | None | AutoContour Model RADAC V5 Traditional 510(k) {7} | Design: Image registration | Manual and Automatic Rigid registration. Automatic Deformable Registration | Manual and Automatic Rigid registration. Automatic Deformable Registration | None | | --- | --- | --- | --- | | AutoContour vs. Predicate and Reference: Model Comparison | | | | | Regions and Volumes of interest (ROI) | CT or MR input for contouring of anatomical regions: Head and Neck, Thorax, Abdomen and Pelvis. Machine learning based contouring of 420 CT-based and 62 MR-based models and manual ROI manipulation CT Models: • A_Aorta • A_Aorta_Asc • A_Aorta_Dsc • A_Brachiocephls • A_Carotid_L • A_Carotid_R • A_Celiac* • A_Circumflex_L • A_Coronary_2d_R • A_Coronary_L • A_Coronary_R • A_LAD • A_Mesenteric_S* • A_Pulmonary* • A_Subclavian_L • A_Subclavian_R • Atrium_L* • Atrium_R* • AV_Node • Barrigel™ • BileDuct_Common • Bladder* • Bladder_CBCT* • Bladder_F* • Body* • Body+Mask* • Bone_Hyoid* • Bone_Ilium* • Bone_Ilium_L* • Bone_Ilium_R* • Bone_Ischium_L* • Bone_Ischium_R* • Bone_Mandible* • Bone_Pelvic* • Bone_Pterygoid_L • Bone_Pterygoid_R | CT or MR input for contouring of anatomical regions: Head and Neck, Thorax, Abdomen and Pelvis. Machine learning based contouring of 260 CT-based and 35 MR-based models and manual ROI manipulation CT Models: • A_Aorta • A_Aorta_Asc • A_Aorta_Dsc • A_Brachiocephls • A_Carotid_L • A_Carotid_R • A_Coronary • A_LAD • A_Pulmonary • A_Subclavian_L • A_Subclavian_R • Atrium_L • Atrium_R • Bladder • Bladder_F • Bone_Hyoid • Bone_Ilium_L • Bone_Ilium_R • Bone_Mandible • Bone_Pelvic • Bone_Skull • Bone_Sternum • Bone_Teeth • Bowel • Bowel_Bag • Bowel_Large • Bowel_Small • BrachialPlex_L • BrachialPlex_R • Brain • Brainstem • Breast_L • Breast_R • Breast_Prone • Bronchus • BuccalMucosa | CT or MR input for contouring of anatomical regions: Head and Neck, Thorax, Abdomen and Pelvis. CT Models • A_Aorta • A_Aorta_Base • A_Aorta_I • A_Aorta_I • A_Celiac • A_LAD • A_Mesenteric_S • A_Pulmonary • Atrium_L • Atrium_R • Bag_Bowel • Bag_Bowel_Extend • Bag_Bowel_Full • Bag_Bowel_S • Bladder • Bladder_CBCT • Bladder_HDR • Body • Body+Mask • Bone_Hyoid • Bone_Hyoid • Bone_Ilium_L • Bone_Ilium • Bone_Ilium_R • Bone_Ischium_L • Bone_Ischium_R • Bone_Mandible • Bone_Pelvic • BoneMarrow_Pelvic • Bowel • Bowel_Bag • Bowel_Bag_Extend • Bowel_Bag_Full • Bowel_Bag_Superior • Bowel_Extend • Bowel_Full • Bowel_HDR • Bowel_S • Bowel_Superior • BrachialPlex_L • BrachialPlex_R • BrachialPlexs • Brain • Brainstem • Breast_Implant_L • Breast_Implant_R • Breast_L | AutoContour Model RADAC V5 Traditional 510(k) {8} AutoContour Model RADAC V5 Traditional 510(k) | | • Bone_PubicSymphys* • Bone_Sacrum* • Bone_Skull • Bone_Sternum* • Bone_Teeth • Bowel • Bowel_Bag* • Bowel_Bag_F • Bowel_F • Bowel_Large • Bowel_Large_F • Bowel_Small • Bowel_Small_F • BrachialPlex_L • BrachialPlex_R • BrachialPlexs • Brain • Brainstem • Breast_Implant_L* • Breast_Implant_R* • Breast_L • Breast_Prone • Breast_R • Breast_RTOG_L* • Breast_RTOG_R* • Breast_Wire_L* • Breast_Wire_R* • Breasts_RTOG* • Bronchus • BuccalMucosa • Canal_Anal* • Canal_Anal_F* • Cardiac_Avoidance_Area • Carina • CaudaEquina • Cavity_Oral* • Cavity_Oral_Ext • Cerebellum* • Chestwall_2cm_L* • Chestwall_2cm_R* • Chestwall_Anat* • Chestwall_Anat_L* • Chestwall_Anat_R* • Chestwall_ESTRO_L • Chestwall_ESTRO_R • Chestwall_L • Chestwall_OAR • Chestwall_R • Chestwall_R_C • Chestwall_RC_R • Clavicle_L* • Clavicle_R* • Cochlea_L • Cochlea_R • Colon_Sigmoid | • Carina • CaudaEquina • Cavity_Oral • Cavity_Oral_Ext • Chestwall_L • Chestwall_OAR • Chestwall_R • Chestwall_RC_L • Chestwall_RC_R • Clavicle_L • Clavicle_R • Cochlea_L • Cochlea_R • Colon_Sigmoid • Cornea_L • Cornea_R • Dental_Artifact • Duodenum • Ear_Internal_L • Ear_Internal_R • Esophagus • External • Eye_L • Eye_R • Femur_Head_L • Femur_Head_L_CBCT • Femur_Head_R • Femur_Head_R_CBCT • Femur_Heads • Gallbladder • Glnd_Lacrimal_L • Glnd_Lacrimal_R • Glnd_Submand_L • Glnd_Submand_R • Glnd_Thyroid • GreatVes • Heart • Heart+A_Pulm • Hippocampus_L • Hippocampus_R • Humerus_L • Humerus_R • InternalAuditoryCanal_L • InternalAuditoryCanal_R • Kidney_L • Kidney_R • Kidneys • Larynx • Lens_L • Lens_R • Lips • Liver • LN_Ax_L1_L • LN_Ax_L1_R • LN_Ax_L2_L • LN_Ax_L2_R • LN_Ax_L3_L • LN_Ax_L3_R • LN_Ax_Sclav_L • LN_Ax_Sclav_R • LN_IMN_L | | --- | --- | --- | {9} AutoContour Model RADAC V5 Traditional 510(k) 6 | • Colon_Sigmoid_F • Cornea_2d_L* • Cornea_2d_R* • Cornea_L • Cornea_R • CorpusCallosum • Cribriform • Dental_Artifact • Duodenum • Ear_Internal_L • Ear_Internal_R • Esophagus • External • Eye_L • Eye_R • Eyes* • Falx • Femur_Head_L • Femur_Head_R • Femur_L • Femur_L_CBCT* • Femur_R • Femur_R_CBCT* • Femur_RTOG_L* • Femur_RTOG_R* • Femurs_RTOG* • Foley_Balloon • GallBladder • Genitals • GInd_Lacrimal_L* • GInd_Lacrimal_R* • GInd_Submand_L • GInd_Submand_R • GInd_Thyroid* • GreatVessels* • HDR_Bladder • HDR_Bowel • HDR_Canal_Anal* • HDR_Colon_Sigmoid* • HDR_Cylinder* • HDR_Rec-Canal_MR • HDR_Rectum • HDR_Ring* • HDR_Urethra* • Heart* • Heart_Prone • Heart+A_Pulm* • Hippocampus_L • Hippocampus_R • Humerus_L • Humerus_R • Iliac_Int_L • Iliac_Int_R • Iliac_L • Iliac_R | • Kidney_Outer_L • Kidney_Outer_R • Larynx • Larynx_Glottic • Larynx_NRG • Larynx_SG • Lens_L • Lens_R • Lips • Liver • LN_Ax_L • LN_Ax_L1_ESTRO_L • LN_Ax_L1_ESTRO_R • LN_Ax_L1_L • LN_Ax_L1_R • LN_Ax_L2_ESTRO_L • LN_Ax_L2_ESTRO_R • LN_Ax_L2_L • LN_Ax_L2_L3_L • LN_Ax_L2_L3_R • LN_Ax_L2_R • LN_Ax_L3_ESTRO_L • LN_Ax_L3_ESTRO_R • LN_Ax_L3_L • LN_Ax_L3_R • LN_Ax_R • LN_IMN_L • LN_IMN_R • LN_IMN_RC_L • LN_IMN_RC_R • LN_Inguinofem_L • LN_Inguinofem_R • LN_InPec_ESTRO_L • LN_InPec_ESTRO_R • LN_Neck_IA • LN_Neck_IB_L • LN_Neck_IB_R • LN_Neck_IB_V_L • LN_Neck_IB_V_R • LN_Neck_II_L • LN_Neck_II_R • LN_Neck_II-IV_L • LN_Neck_II-V_R • LN_Neck_II-V_L • LN_Neck_II-V_R • LN_Neck_IV_L • LN_Neck_V_R • LN_Neck_VIA | • LN_IMN_R • LN_Neck_234_L • LN_Neck_234_R • LN_Neck_2347AB_L • LN_Neck_2347AB_R • LN_Neck_IA • LN_Neck_IA6 • LN_Neck_IB_L • LN_Neck_IB_R • LN_Neck_II_L • LN_Neck_II_R • LN_Neck_III_L • LN_Neck_III_R • LN_Neck_IV_L • LN_Neck_IV_R • LN_Neck_L • LN_Neck_R • LN_Neck_V_L • LN_Neck_V_R • LN_Neck_VI • LN_Neck_VIIAB_L • LN_Neck_VIIAB_R • LN_Pelvics_CBCT • LN_Pelvis • LN_Sclav_L • LN_Sclav_R • Lung_L • Lung_R • Lungs • Mesorectum • Musc_Constrict • Musc_PecMinor_L • Musc_PecMinor_R • Musc_Sclmast_L • Musc_Sclmast_R • OpticChiasm • OpticNrv_L • OpticNrv_R • Optics • Pancreas • Parotid_L • Parotid_R • PelvisVessels • PenileBulb • Pericardium • Pericardium+A_Pulm • Pituitary • Prostate • Prostate_CBCT • Prostate+SeminalVes • ProstateBed • PubicSymphys • Rectum • Rectum_CBCT • Rectum_HDR • Retina_L • Retina_R • Ribs • Ribs Sacrum • Ribs_L • Ribs_R • Sacrum • SeminalVes | | --- | --- | --- | {10} AutoContour Model RADAC V5 Traditional 510(k) 7 | • InternalAuditoryCanal_L* • InternalAuditoryCanal_R* • Kidney_L • Kidney_Outer_L* • Kidney_Outer_R* • Kidney_R* • Kidneys • Kidneys_Outer • Larynx • Larynx_Glottic • Larynx_NRG • Larynx_SG • Lens_L • Lens_R • Lips* • Liver • LN_Ax_L • LN_Ax_L1_2d_L* • LN_Ax_L1_2d_R* • LN_Ax_L1_ESTRO_2d_L • LN_Ax_L1_ESTRO_2d_R • LN_Ax_L1_ESTRO_L • LN_Ax_L1_ESTRO_R • LN_Ax_L1_L • LN_Ax_L1_R • LN_Ax_L2_2d_L* • LN_Ax_L2_2d_R* • LN_Ax_L2_ESTRO_2d_L* • LN_Ax_L2_ESTRO_2d_R* • LN_Ax_L2_ESTRO_L • LN_Ax_L2_ESTRO_R • LN_Ax_L2_InPec_ESTRO_2d_L* • LN_Ax_L2_InPec_ESTRO_2d_R* • LN_Ax_L2_L • LN_Ax_L2_L3_L • LN_Ax_L2_L3_R • LN_Ax_L2_R • LN_Ax_L3_2d_L* • LN_Ax_L3_2d_R* • LN_Ax_L3_ESTRO_2d_L* • LN_Ax_L3_ESTRO_2d_R* • LN_Ax_L3_ESTRO_L • LN_Ax_L3_ESTRO_R • LN_Ax_L3_L • LN_Ax_L3_R • LN_Ax_R • LN_Ax_Sclav_2d_L* • LN_Ax_Sclav_2d_R* • LN_IMN_2d_L* • LN_IMN_2d_R* • LN_IMN_ESTRO_2d_L* | • LN_Neck_VIIA_L • LN_Neck_VIIA_R • LN_Neck_VIIB_L • LN_Neck_VIIB_R • LN_Paraortic • LN_Pelvics_F • LN_Pelvics • LN_Pelvic_NRG • LN_Post_Neck_L • LN_Post_Neck_R • LN_Presacral • LN_Sclav_ESTRO_L • LN_Sclav_ESTRO_R • LN_Sclav_L • LN_Sclav_R • LN_Sclav_RADCOMP_L • LN_Sclav_RADCOMP_R • Lobe_Temporal_L • Lobe_Temporal_R • Lung_L • Lung_R • Macula_L • Macula_R • Marrow_Ilium_L • Marrow_Ilium_R • Musc_Constrict • Musc_Iliopsoas_L • Musc_Iliopsoas_R • Myocardium • Nipple_L • Nipple_Prone • Nipple_R • OpticChiasm • OpticNrv_L • OpticNrv_R • Pancreas • Parotid_L • Parotid_R • PenileBulb • Pericardium • Pharynx • Pituitary • Prostate • ProstateBed • Rectum • Rectum_F • Retina_L • Retina_R • Rib • Rib01_L • Rib01_R • Rib02_L • Rib02_R | • SeminalVes_CBCT • Skin • SpinalCanal • SpinalCord • Spleen • Sternum • Stomach • Trachea • Urethra_HDR • Uterus_Cervix • Uterus+Cervix • V_Venacava_I • V_Venacava_S • Vagina • VB_C1 • VB_C2 • VB_C3 • VB_C4 • VB_C5 • VB_C6 • VB_C7 • VB_L1 • VB_L2 • VB_L3 • VB_L4 • VB_L5 • VB_T01 • VB_T02 • VB_T03 • VB_T04 • VB_T05 • VB_T06 • VB_T07 • VB_T08 • VB_T09 • VB_T10 • VB_T11 • VB_T12 • VBs • Ventricle_L • Ventricle_R MR Models: • Bladder • Bladder_HDR • Bowel_HDR • Brainstem • Canal_Anal_HDR • Colon_Sigmoid_HDR • Cornea_L • Cornea_R • Eye_L • Eye_R • Femur_Head_L • Femur_Head_R • Hippocampus_L • Hippocampus_R • Optics • PenileBulb • Prostate • PubicSymphys | | --- | --- | --- | {11} AutoContour Model RADAC V5 Traditional 510(k) 8 | • LN_IMN_ESTRO_2d_R* • LN_IMN_Expand_2d_L* • LN_IMN_Expand_2d_R* • LN_IMN_Expand_ESTRO_2d_L* • LN_IMN_Expand_ESTRO_2d_R* • LN_IMN_L • LN_IMN_R • LN_IMN_RC_L • LN_IMN_RC_R • LN_Inguinofem_L* • LN_Inguinofem_R* • LN_InPec_ESTRO_2d_L* • LN_InPec_ESTRO_2d_R* • LN_InPec_ESTRO_L • LN_InPec_ESTRO_R • LN_Mesorectum* • LN_Neck_2d_L • LN_Neck_2d_R • LN_Neck_CerVII_2d_L • LN_Neck_CerVII_2d_R • LN_Neck_IA • LN_Neck_IA_2d* • LN_Neck_IA_VI_2d* • LN_Neck_IB_2d_L* • LN_Neck_IB_2d_R* • LN_Neck_IB_L • LN_Neck_IB_R • LN_Neck_IB-V_L • LN_Neck_IB-V_R • LN_Neck_II_2d_L* • LN_Neck_II_2d_R* • LN_Neck_II_L • LN_Neck_II_R • LN_Neck_III_2d_L* • LN_Neck_III_2d_R* • LN_Neck_III_L • LN_Neck_III_R • LN_Neck_II-IV_2d_L* • LN_Neck_II-IV_2d_R* • LN_Neck_II-IV_L • LN_Neck_II-IV_R • LN_Neck_IV_2d_L* • LN_Neck_V_2d_R* • LN_Neck_V_L • LN_Neck_V_R | • Rib03_L • Rib03_R • Rib04_L • Rib04_R • Rib05_L • Rib05_R • Rib06_L • Rib06_R • Rib07_L • Rib07_R • Rib08_L • Rib08_R • Rib09_L • Rib09_R • Rib10_L • Rib10_R • Rib11_L • Rib11_R • Rib12_L • Rib12_R • Rib_L • Rib_R • SacralPlex_L • SacralPlex_R • SeminalVes • SpinalCanal • SpinalCord • Spleen • Stomach • Trachea • UteroCervix • V_Brachioceph_L • V_Brachioceph_R • V_Jugular_L • V_Jugular_R • V_Venacava_I • V_Venacava_S • VB • VB_C1 • VB_C2 • VB_C3 • VB_C4 • VB_C5 • VB_C6 • VB_C7 • VB_L1 • VB_L2 • VB_L3 • VB_L4 • VB_L5 • VB_T01 • VB_T02 • VB_T03 • VB_T04 • VB_T05 | • Rectum • Rectum_HDR • Retina_L • Retina_R • Sacrum • SeminalVes • Urethra_HDR | | --- | --- | --- | {12} AutoContour Model RADAC V5 Traditional 510(k) 9 | • LN_Neck_VI_2d* • LN_Neck_VIA • LN_Neck_VII_2d_L* • LN_Neck_VII_2d_R* • LN_Neck_VIIA_2d_L* • LN_Neck_VIIA_2d_R* • LN_Neck_VIIA_L • LN_Neck_VIIA_R • LN_Neck_VIIB_2d_L* • LN_Neck_VIIB_2d_R* • LN_Neck_VIIB_L • LN_Neck_VIIB_R • LN_Paraaortic • LN_Pelvics • LN_Pelvics_CBCT* • LN_Pelvics_F • LN_Pelvics_NRG* • LN_Post_Neck_L • LN_Post_Neck_R • LN_Presacral • LN_Sclav_2d_L* • LN_Sclav_2d_R* • LN_Sclav_ESTRO_2d_L* • LN_Sclav_ESTRO_2d_R* • LN_Sclav_ESTRO_L • LN_Sclav_ESTRO_R • LN_Sclav_L • LN_Sclav_R • LN_Sclav_RC_L • LN_Sclav_RC_R • Lobe_Temporal_L* • Lobe_Temporal_R* • Lung_L • Lung_R • Lungs • Macula_L • Macula_R • Marrow_Ilium_L • Marrow_Ilium_R • Marrow_Pelvis* • Medulla • Midbrain • Musc_Constrict_2d* • Musc_Constrict • Musc_Iliopsoas_L • Musc_Iliopsoas_R • Musc_PecMinor_L* • Musc_PecMinor_R* • Musc_Sclmast_L* • Musc_Sclmast_R* • Myocardium • Nipple_L • Nipple_Prone • Nipple_R • OpticChiasm* | • VB_T06 • VB_T07 • VB_T08 • VB_T09 • VB_T10 • VB_T11 • VB_T12 • Ventricle_L • Ventricle_R | | --- | --- | | MR Models: • A_Pud_Int_L • A_Pud_Int_R • Bladder • Bladder_Trigone • Brain • Brainstem • Cerebellum • Colon_Sigmoid • External_Pelvis • Eye_L • Eye_R • Femur_L • Femur_R • Glnd_Prostate • Hippocampus_L • Hippocampus_R • Hypo_True • Hypothalamus • Lens_L • Lens_R • NVB_L • NVB_R • OpticChiasm • OpticNrv_L • OpticNrv_R • OpticTract_L • OpticTract_R • PenileBulb • Pituitary • Prostate • Rectal_Spacer • Rectum • SeminalVes • SpinalCord_Cerv • Urethra | | {13} AutoContour Model RADAC V5 Traditional 510(k) 10 | | • OpticNrv_L* • OpticNrv_R* • Optics* • Pancreas* • Parametrium • Parotid_GInd_L • Parotid_GInd_R • Parotid_L • Parotid_R • PenileBulb • Pericardium • Pericardium_Inf* • Pericardium_Inf+A_Pulm* • Pharynx • Pituitary* • Pituitary_3d • Pons • Prostate* • Prostate_CBCT* • Prostate+SeminalVes* • Prostate+SV-Spacergel* • ProstateBed • ProstateFiducials* • Prostate-Gel_Fiducials* • Prostate-Spacergel_Vue* • Rectum • Rectum_CBCT* • Rectum_F • Rectum-Spacergel_Vue* • Retina_2d_L* • Retina_2d_R* • Retina_L • Retina_R • Rib* • Rib_L* • Rib_R* • Rib01_L • Rib01_R • Rib02_L • Rib02_R • Rib03_L • Rib03_R • Rib04_L • Rib04_R • Rib05_L • Rib05_R • Rib06_L • Rib06_R • Rib07_L • Rib07_R • Rib08_L • Rib08_R • Rib09_L • Rib09_R • Rib10_L | | | | --- | --- | --- | --- | {14} AutoContour Model RADAC V5 Traditional 510(k) 11 | | • Rib10_R • Rib11_L • Rib11_R • Rib12_L • Rib12_R • SacralPlex_L • SacralPlex_R • SeminalVes* • SeminalVes_CBCT* • Sinuses • Skin* • SpaceOAR™Vue* • SpinalCanal • SpinalCord* • Spleen • Stomach • Tentorium • Trachea* • UteroCervix • V_Brachioceph_L • V_Brachioceph_R • V_Jugular_L • V_Jugular_R • V_Pulmonary • V_Venacava_I* • V_Venacava_I_UKSABR • V_Venacava_S • V_Venacava_S_UKSABR • Vagina* • Valve_Aortic • Valve_Mitral • Valve_Pulmonic • Valve_Tricuspid • VB • VB_C1 • VB_C2 • VB_C3 • VB_C4 • VB_C5 • VB_C6 • VB_C7 • VB_L1 • VB_L2 • VB_L3 • VB_L4 • VB_L5 • VB_T01 • VB_T02 • VB_T03 • VB_T04 • VB_T05 • VB_T06 • VB_T07 • VB_T08 • VB_T09 | | | | --- | --- | --- | --- | {15} AutoContour Model RADAC V5 Traditional 510(k) 12 | | • VB_T10 • VB_T11 • VB_T12 • Ventricle_Brain • Ventricle_L* • Ventricle_R* • Vessels_Pelvis* MR Models: • A_Pud_Int_L • A_Pud_Int_R • Amygdala_L • Amygdala_R • Bladder* • Bladder_Trigone • Bladder_TRUFI • Bone_PubicSymphys* • Bone_Sacrum* • Brain • Brainstem • Cerebellum • Colon_Sigmoid • Cornea_L* • Cornea_R* • CorpusCallosum • External_Pelvis • Eye_L* • Eye_R* • Falx • Femur_L* • Femur_R* • Glnd_Prostate • HDR_Bladder* • HDR_Bowel* • HDR_Canal_Anal* • HDR_Colon_Sigmoid* • HDR_Rec-Canal* • HDR_Rectum* • HDR_Urethra* • Hippocampus_L* • Hippocampus_R* • Hypo_True • Hypothalamus • Lens_L • Lens_R • Medulla • Midbrain • NVB_L • NVB_R • OpticChiasm • OpticNrv_L • OpticNrv_R • Optics* • OpticTract_L | | | | --- | --- | --- | --- | {16} AutoContour Model RADAC V5 Traditional 510(k) | | • OpticTract_R • PenileBulb* • PenileBulb_TRUFI • Pituitary • Pons • Prostate* • Rectal_Spacer • Rectum* • Retina_L* • Retina_R* • SeminalVes* • Sinuses • SpinalCord_Cerv • Tentorium • Thalamus • Urethra • Ventricle_Brain | | | | --- | --- | --- | --- | | Design: Region/volume of interest measurements and size measurements | None – not applicable | None – not applicable | None – not applicable | | Design: Region/Volume Quantification | None – not applicable | None – not applicable | None – not applicable | | Design: Supported modalities | CT or MR input for contouring or registration/fusion. PET/CT input for registration/fusion only. DICOM RTSTRUCT and REGISTRATION for input | CT or MR input for contouring or registration/fusion. PET/CT input for registration/fusion only. DICOM RTSTRUCT and REGISTRATION for input | CT or MR input for contouring. | | Design: Reporting and data routing | No built-in reporting, supports exporting DICOM RTSTRUCT, REGISTRATION and DOSE files for output. | No built-in reporting, supports exporting DICOM RTSTRUCT, REGISTRATION and DOSE files for output. | No built-in reporting, supports exporting DICOM RTSTRUCT files for output | | Compatibility with the environment and other devices | Compatible with data from any DICOM compliant scanners for the applicable modalities. Agent Uploader component compatible with Microsoft Windows. Cloud-based automatic contouring service compatible with Linux. | Compatible with data from any DICOM compliant scanners for the applicable modalities. Agent Uploader component compatible with Microsoft Windows. Cloud-based automatic contouring service compatible with Linux. | No Limitation on scanner model, DICOM 3.0 compliance required. | 13 {17} AutoContour Model RADAC V5 Traditional 510(k) 14 | | Web application server-based application compatible with Linux. | Web application server-based application compatible with Linux. | | | --- | --- | --- | --- | | Communication s/ Networking | TCP/IP | TCP/IP | N/A | | Computer platform & Operating System | Windows Operating System | Windows Operating System | Operating System Windows 10 / Windows Server 2016 and Above | # 7. Performance Verification testing confirmed that RADAC V5 features and functionality met predefined acceptance criteria consistent with the predicate device. RADAC models were divided into 4 categories and tested using protocols consistent with the category, while applying consistent acceptance criteria across each. Models were classified as 1) New Models that did not exist in the predicate or reference device, 2) Models that did not exist in the predicate, but were unchanged from the reference device, 3) Models that previously existed in the predicate or reference device but were changed in RADAC V5, and 4) Models that previously existed in the predicate device (AC V4) and are unchanged. | Model Verification Categories and Acceptance Criteria *, **, ***, and *** refer to specified acceptance criteria. | | | | | --- | --- | --- | --- | | New Models - Not previously in AC or LC | Models unchanged from LC 1.8 | Updated Models Regardless of source (AC or LC) | Unchanged Models From AC 2.6 | | New Structure Model Validation: • Training DSC Evaluation* • External Dataset DSC Evaluation* • Internal Likert Qualitative Review** • External Likert Qualitative Review** | Existing Structure Model Tests: • DSC Comparison between Limbus v1.8 and AutoContour v2.7.*** • Hausdorff Distance between Limbus v1.8 and Autocontour v2.7.*** | Existing Structure Model Tests: • DSC Comparison between Limbus v1.8 and AutoContour v2.7 or between AutoContour v2.6 and AutoContour v2.7.*** • Hausdorff Distance between Limbus v1.8 and AutoContour v2.7 or between AutoContour v2.6 and AutoContour v2.7.*** | Existing Structure Model Tests: • DSC Comparison between AutoContour v2.6 and AutoContour v2.7.*** • Hausdorff Distance between AutoContour v2.6 and AutoContour v2.7.*** | | | | New Structure Model Validation: • Training DSC Evaluation* • External Dataset DSC Evaluation* • Internal Likert Qualitative Review** • External Likert Qualitative Review** | | {18} # Model Acceptance testing utilized the following predefined acceptance criteria. * Training DSC and External Dataset DSC Evaluations: Mean DSC threshold for passing Large Structures: 0.80, Medium Structures: 0.65, and Small Structures: 0.50 ** Internal and External Likert Qualitative Review: Each structure model was determined to pass if the average grade exceeded 3 across all external image sets reviewed. *** Existing Structure Model DSC Evaluation: The DSC threshold for passing Large Structures: >0.99, Medium Structures >0.98, and Small Structures: >0.95. \*\*\* Existing Structure Model Hausdorff Distance Evaluation: Hausdorff Distance threshold for passing structures is $\leq 3\mathrm{mm}$ # Performance Data Testing was performed according to Radformation's AutoContour Validation Test Protocol and Report, which demonstrates that AutoContour Model RADAC V5 performs as intended per its indications for use. Further tests were performed on independent datasets from those included in training and validation sets in order to validate the generalizability of the machine learning model. # Description of Changes to Test Protocol Changes to the test protocol include an update to the existing structure model tests to include Hausdorff Distance as an additional metric for regression testing of existing model output changes, and the inclusion of a comparison to Limbus v1.8. There were no changes to the new and updated structure model testing protocol between AutoContour RADAC V4 and RADAC V5. # Testing Summary Mean Dice Similarity Coefficient (DSC) was used to validate the accuracy of structure model outputs when tested on image data sequestered from the original training data population. The test datasets were independent from those used for training and consisted of approximately $10\%$ of the number of training image sets used as input for the model. For CT structure models there were an average of 459 training and 49 testing image sets. CT training images were gathered from several institutions, in several different countries. Ground truthing of each test data set were generated manually using consensus (NRG/RTOG/ESTRO) guidelines as appropriate by six clinically experienced experts consisting of 2 radiation therapy physicists, 1 radiation dosimetrist, and 3 radiation therapists with specialized training in radiation therapy contouring. Structure models were categorized into three size categories as DSC metrics can be sensitive to structure volume. A structure would pass initial validation if the mean DSC exceeded 0.8 for Large volume structures (eg. Bladder, Spleen) 0.65 for Medium volume structures (eg. Gallbladder, Duodenum) and 0.5 for Small structures (eg. Cornea, Retina). For CT Structure models large, medium and small structures resulted in a mean DSC of $0.91 + / - 0.14$ , $0.86 + / - 0.13$ , and $0.75 + / - 0.20$ respectively. A full summary of the CT structure DSC is available below: Table 4: CT Training Data Results for AutoContour Model RADAC V5 AutoContour Model RADAC V5 Traditional 510(k) {19} | CT Structure | Size | Pass Criteria | # of Training Sets | # of Testing Sets | DSC (Avg) | DSC Std Dev | Lower Bound 95% Confidence Interval | | --- | --- | --- | --- | --- | --- | --- | --- | | A_Aorta** (Update) | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | A_Aorta_Asc** (Update) | Medium | 0.65 | 240 | 60 | 0.92 | 0.02 | 0.91 | | A_Carotid_L** (Update) | Medium | 0.65 | 328 | 83 | 0.79 | 0.13 | 0.58 | | A_Carotid_R** (Update) | Medium | 0.65 | 328 | 83 | 0.79 | 0.13 | 0.58 | | A_Celiac | Small | 0.50 | 435 | 44 | 0.87 | 0.24 | 0.47 | | A_Circumflex_L | Small | 0.50 | 415 | 45 | 0.52 | 0.36 | -0.08 | | A_Coronary_2d_R | Small | 0.50 | 415 | 45 | 0.83 | 0.26 | 0.41 | | A_Coronary_L | Small | 0.50 | 415 | 45 | 0.50 | 0.36 | -0.11 | | A_Coronary_R** (Update) | Small | 0.50 | 408 | 103 | 0.56 | 0.09 | 0.41 | | A_Mesenteric_S | Small | 0.50 | 428 | 43 | 0.79 | 0.28 | 0.33 | | A_Pulmonary** (Update) | Medium | 0.65 | 1338 | 50 | 0.91 | 0.18 | 0.60 | | Atrium_L** (Update) | Medium | 0.65 | 398 | 45 | 0.82 | 0.19 | 0.51 | | Atrium_R** (Update) | Medium | 0.65 | 398 | 45 | 0.82 | 0.20 | 0.50 | | AV_Node | Medium | 0.65 | 398 | 45 | 0.87 | 0.20 | 0.53 | | Barrigel | Medium | 0.65 | 111 | 28 | 0.77 | 0.06 | 0.66 | | BileDuct_Common | Small | 0.50 | 643 | 162 | 0.57 | 0.20 | 0.24 | | Bladder** (Update) | Large | 0.80 | 1105 | 50 | 0.93 | 0.18 | 0.63 | | Body+Mask* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | Bone_Ischium_L | Large | 0.80 | 521 | 50 | 0.92 | 0.15 | 0.68 | | Bone_Ischium_R | Large | 0.80 | 35 | 4 | 0.92 | 0.15 | 0.68 | | Bone_Mandible** (Update) | Medium | 0.65 | 234 | 25 | 0.88 | 0.16 | 0.62 | | Bone_Pelvic** (Update) | Large | 0.80 | 234 | 25 | 0.94 | 0.12 | 0.74 | | Bone_Pterygoid_L | Small | 0.50 | 308 | 36 | 0.73 | 0.29 | 0.26 | | Bone_Pterygoid_R | Small | 0.50 | 308 | 36 | 0.73 | 0.29 | 0.26 | | Bone_Skull** (Update) | Large | 0.80 | 80 | 20 | 0.92 | 0.01 | 0.90 | | Bone_Teeth** (Update) | Medium | 0.65 | 340 | 76 | 0.88 | 0.02 | 0.84 | | Bowel** (Update) | Medium | 0.65 | 221 | 13 | 0.78 | 0.03 | 0.72 | | Bowel_Bag** (Update) | Large | 0.80 | 454 | 48 | 0.95 | 0.05 | 0.87 | | Bowel_Bag_F* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | Bowel_F* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Bowel_Large** (Update) | Medium | 0.65 | 805 | 52 | 0.89 | 0.17 | 0.61 | | Bowel_Large_F* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Bowel_Small** (Update) | Medium | 0.65 | 705 | 45 | 0.93 | 0.05 | 0.85 | AutoContour Model RADAC V5 Traditional 510(k) {20} AutoContour Model RADAC V5 Traditional 510(k) | Bowel_Small_F* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | --- | --- | --- | --- | --- | --- | --- | --- | | BrachialPlex_L** (Update) | Medium | 0.65 | 527 | 132 | 0.73 | 0.08 | 0.61 | | BrachialPlex_R** (Update) | Medium | 0.65 | 527 | 132 | 0.73 | 0.08 | 0.61 | | BrachialPlexs* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Brain** (Update) | Large | 0.80 | 1033 | 50 | 0.95 | 0.14 | 0.71 | | Brainstem** (Update) | Medium | 0.65 | 614 | 50 | 0.90 | 0.14 | 0.67 | | Breast_RTOG_L | Large | 0.80 | 660 | 50 | 0.88 | 0.25 | 0.46 | | Breast_RTOG_R | Large | 0.80 | 660 | 50 | 0.88 | 0.25 | 0.46 | | Breasts_RTOG* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | BuccalMucosa** (Update) | Medium | 0.65 | 392 | 98 | 0.70 | 0.05 | 0.61 | | Canal_Anal* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Canal_Anal_F* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | InternalAuditoryCanal_L | Small | 0.50 | 253 | 27 | 0.76 | 0.34 | 0.19 | | CaudaEquina** (Update) | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Cavity_Oral_Ext** (Update) | Medium | 0.65 | 392 | 98 | 0.94 | 0.02 | 0.90 | | Chestwall_2cm_L | Large | 0.80 | 223 | 25 | 0.92 | 0.11 | 0.74 | | Chestwall_2cm_R | Large | 0.80 | 223 | 25 | 0.92 | 0.11 | 0.74 | | Chestwall_ESTRO_L | Large | 0.80 | 223 | 20 | 0.87 | 0.21 | 0.52 | | Chestwall_ESTRO_R | Large | 0.80 | 223 | 20 | 0.87 | 0.21 | 0.52 | | Cochlea_L** (Update) | Small | 0.50 | 106 | 26 | 0.65 | 0.10 | 0.49 | | Cochlea_R** (Update) | Small | 0.50 | 106 | 26 | 0.65 | 0.10 | 0.49 | | Cornea_2d_L | Small | 0.50 | 729 | 50 | 0.90 | 0.17 | 0.62 | | Cornea_2d_R | Small | 0.50 | 729 | 50 | 0.90 | 0.17 | 0.62 | | CorpusCallosum | Medium | 0.65 | 58 | 15 | 0.76 | 0.07 | 0.64 | | Cribriform | Medium | 0.65 | 361 | 41 | 0.74 | 0.30 | 0.24 | | Dental_Artifact** (Update) | Medium | 0.65 | 342 | 86 | 0.76 | 0.10 | 0.60 | | Ear_Internal_L** (Update) | Small | 0.50 | 580 | 146 | 0.64 | 0.21 | 0.29 | | Ear_Internal_R** (Update) | Small | 0.50 | 580 | 146 | 0.64 | 0.21 | 0.29 | | Esophagus** (Update) | Medium | 0.65 | 1116 | 279 | 0.76 | 0.13 | 0.55 | | Eyes* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Falx | Medium | 0.65 | 177 | 48 | 0.76 | 0.06 | 0.66 | | HDR_Canal_Anal* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | GreatVessels* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | HDR_Colon_Sigmoid | Medium | 0.65 | 494 | 48 | 0.83 | 0.22 | 0.48 | | Heart** (Update) | Large | 0.80 | 1000 | 50 | 0.81 | 0.31 | 0.30 | | Heart+A_Pulm | Large | 0.80 | 33 | 4 | 0.96 | 0.11 | 0.78 | {21} AutoContour Model RADAC V5 Traditional 510(k) | Hippocampus_L** (Update) | Medium | 0.65 | 226 | 57 | 0.67 | 0.10 | 0.51 | | --- | --- | --- | --- | --- | --- | --- | --- | | Hippocampus_R** (Update) | Medium | 0.65 | 226 | 57 | 0.67 | 0.10 | 0.51 | | Iliac_Int_L** (Update) | Medium | 0.65 | 160 | 40 | 0.66 | 0.28 | 0.19 | | Iliac_Int_R** (Update) | Medium | 0.65 | 160 | 40 | 0.66 | 0.28 | 0.19 | | Iliac_L** (Update) | Medium | 0.65 | 162 | 82 | 0.83 | 0.07 | 0.71 | | Iliac_R** (Update) | Medium | 0.65 | 162 | 82 | 0.83 | 0.07 | 0.71 | | Kidneys* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | Larynx** (Update) | Medium | 0.65 | 172 | 43 | 0.85 | 0.05 | 0.77 | | LN_Ax_L1_ESTRO_2d_L | Medium | 0.65 | 395 | 44 | 0.88 | 0.21 | 0.53 | | LN_Ax_L1_ESTRO_2d_R | Medium | 0.65 | 395 | 44 | 0.88 | 0.21 | 0.53 | | LN_Ax_L2_ESTRO_2d_L | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Ax_L2_ESTRO_2d_R | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Ax_L3_ESTRO_2d_L | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Ax_L3_ESTRO_2d_R | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_InPec_ESTRO_2d_L | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_InPec_ESTRO_2d_R | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Ax_L2_InPec_ESTRO_2d_L | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Ax_L2_InPec_ESTRO_2d_R | Medium | 0.65 | 404 | 45 | 0.89 | 0.21 | 0.54 | | LN_Mesorectum | Large | 0.80 | 302 | 33 | 0.86 | 0.27 | 0.42 | | LN_Neck_2d_L | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_2d_R | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_CerVII_2d_L | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_CerVII_2d_R | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_IB_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_IB_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_II_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_II_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_II-IV_2d_L | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_II-IV_2d_R | Large | 0.80 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_III_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_III_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_IV_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_IV_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_VII_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_VII_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_VIIA_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | {22} AutoContour Model RADAC V5 Traditional 510(k) | LN_Neck_VIIA_2d_R | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | --- | --- | --- | --- | --- | --- | --- | --- | | LN_Neck_VIIB_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Neck_VIIB_2d_L | Medium | 0.65 | 672 | 50 | 0.85 | 0.18 | 0.55 | | LN_Pelvics_NRG** (Update) | Large | 0.80 | 921 | 50 | 0.89 | 0.16 | 0.63 | | LN_Presacral** (Update) | Medium | 0.65 | 191 | 48 | 0.78 | 0.16 | 0.52 | | Lobe_Temporal_L** (Update) | Large | 0.80 | 211 | 23 | 0.81 | 0.27 | 0.37 | | Lobe_Temporal_R** (Update) | Large | 0.80 | 211 | 23 | 0.81 | 0.27 | 0.37 | | Lungs* | Large | 0.80 | N/A | N/A | N/A | N/A | N/A | | Marrow_Pelvis | Medium | 0.65 | 35 | 4 | 0.94 | 0.12 | 0.74 | | Medulla | Medium | 0.65 | 614 | 50 | 0.90 | 0.14 | 0.67 | | Midbrain | Medium | 0.65 | 614 | 50 | 0.906 | 0.14 | 0.67 | | Musc_Iliopsoas_L** (Update) | Large | 0.80 | 1082 | 65 | 0.94 | 0.08 | 0.81 | | Musc_Iliopsoas_R** (Update) | Large | 0.80 | 1082 | 65 | 0.94 | 0.14 | 0.71 | | Musc_PecMinor_L | Medium | 0.65 | 398 | 42 | 0.83 | 0.22 | 0.48 | | Myocardium** (Update) | Medium | 0.65 | 1082 | 65 | 0.90 | 0.05 | 0.82 | | Pancreas** (Update) | Medium | 0.65 | 765 | 50 | 0.82 | 0.15 | 0.57 | | Parametrium | Medium | 0.65 | 1095 | 50 | 0.85 | 0.19 | 0.54 | | Parotid_L** (Update) | Medium | 0.65 | 787 | 197 | 0.88 | 0.051 | 0.80 | | Parotid_R** (Update) | Medium | 0.65 | 787 | 197 | 0.88 | 0.051 | 0.80 | | Pericardium** (Update) | Large | 0.80 | 160 | 41 | 0.94 | 0.02 | 0.91 | | Pericardium_Inf | Large | 0.80 | 734 | 50 | 0.94 | 0.165 | 0.68 | | Pericardium_Inf+A_Pulm | Large | 0.80 | 1000 | 50 | 0.81 | 0.31 | 0.30 | | Pharynx** (Update) | Medium | 0.65 | 57 | 15 | 0.90 | 0.02 | 0.87 | | Pituitary** (Update) | Small | 0.50 | 295 | 33 | 0.83 | 0.27 | 0.40 | | Pituitary_3d | Small | 0.50 | 373 | 41 | 0.68 | 0.09 | 0.53 | | Pons | Medium | 0.65 | 614 | 50 | 0.90 | 0.14 | 0.67 | | ProstateBed** (Update) | Medium | 0.65 | 133 | 34 | 0.87 | 0.04 | 0.79 | | ProstateFiducials | Small | 0.50 | 888 | 50 | 0.88 | 0.20 | 0.55 | | Rectum-Spacergel_VUE* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Retina_2d_L | Small | 0.50 | 729 | 50 | 0.90 | 0.17 | 0.62 | | Retina_2d_R | Small | 0.50 | 729 | 50 | 0.90 | 0.17 | 0.62 | | Rib01_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib01_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib02_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib02_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib03_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | {23} AutoContour Model RADAC V5 Traditional 510(k) 20 | Rib03_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | --- | --- | --- | --- | --- | --- | --- | --- | | Rib04_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib04_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib05_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib05_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib06_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib06_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib07_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib07_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib08_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib08_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib09_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib09_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib10_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib10_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib11_L** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib11_R** (Update) | Medium | 0.65 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib12_L** (Update) | Small | 0.50 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib12_R** (Update) | Small | 0.50 | 148 | 37 | 0.90 | 0.09 | 0.76 | | Rib** (Update) | Large | 0.80 | 84 | 11 | 0.91 | 0.10 | 0.74 | | Rib_R** (Update) | Large | 0.80 | 84 | 11 | 0.91 | 0.10 | 0.74 | | SacralPlex_L** (Update) | Medium | 0.65 | 326 | 83 | 0.75 | 0.06 | 0.66 | | SacralPlex_R** (Update) | Medium | 0.65 | 326 | 83 | 0.75 | 0.06 | 0.66 | | Sinuses | Medium | 0.65 | 177 | 48 | 0.78 | 0.07 | 0.67 | | SpinalCanal** (Update) | Large | 0.80 | 87 | 22 | 0.90 | 0.02 | 0.87 | | SpinalCord** (Update) | Medium | 0.65 | 1000 | 49 | 0.91 | 0.07 | 0.79 | | Spleen** (Update) | Large | 0.80 | 765 | 70 | 0.94 | 0.14 | 0.71 | | Tentorium | Medium | 0.65 | 177 | 48 | 0.70 | 0.10 | 0.55 | | Trachea** (Update) | Medium | 0.65 | 865 | 50 | 0.90 | 0.17 | 0.63 | | UteroCervix** (Update) | Medium | 0.65 | 594 | 50 | 0.92 | 0.20 | 0.61 | | V_Jugular_L** (Update) | Medium | 0.65 | 264 | 68 | 0.78 | 0.18 | 0.48 | | V_Jugular_R** (Update) | Medium | 0.65 | 264 | 68 | 0.78 | 0.18 | 0.48 | | V_Pulmonary | Medium | 0.65 | 398 | 45 | 0.68 | 0.21 | 0.34 | | V_Venacava_I** (Update) | Medium | 0.65 | 1000 | 50 | 0.88 | 0.16 | 0.62 | | V_Venacava_I_UKSABR | Medium | 0.65 | 627 | 49 | 0.91 | 0.11 | 0.72 | | V_Venacava_S** (Update) | Medium | 0.65 | 168 | 26 | 0.83 | 0.08 | 0.69 | {24} AutoContour Model RADAC V5 Traditional 510(k) | V_Venacava_S_UKSABR | Medium | 0.65 | 619 | 48 | 0.89 | 0.16 | 0.63 | | --- | --- | --- | --- | --- | --- | --- | --- | | Vagina | Medium | 0.65 | 446 | 50 | 0.85 | 0.21 | 0.51 | | Valve_Aortic | Medium | 0.65 | 398 | 45 | 0.88 | 0.20 | 0.57 | | Valve_Mitral | Medium | 0.65 | 398 | 45 | 0.85 | 0.21 | 0.50 | | Valve_Pulmonic | Medium | 0.65 | 398 | 45 | 0.91 | 0.17 | 0.63 | | Valve_Tricuspid | Medium | 0.65 | 398 | 45 | 0.88 | 0.21 | 0.54 | | VB** (Update) | Large | 0.80 | 1051 | 64 | 0.99 | 0.01 | 0.97 | | VB_C1** (Update) | Medium | 0.65 | 196 | 14 | 0.90 | 0.25 | 0.49 | | VB_C2** (Update) | Medium | 0.65 | 202 | 13 | 0.99 | 0.01 | 0.97 | | VB_C3** (Update) | Medium | 0.65 | 214 | 14 | 0.97 | 0.03 | 0.92 | | VB_C4** (Update) | Medium | 0.65 | 234 | 15 | 0.90 | 0.24 | 0.51 | | VB_C5** (Update) | Medium | 0.65 | 328 | 20 | 0.89 | 0.21 | 0.54 | | VB_C6** (Update) | Medium | 0.65 | 520 | 33 | 0.87 | 0.24 | 0.48 | | VB_C7** (Update) | Medium | 0.65 | 651 | 36 | 0.98 | 0.01 | 0.96 | | VB_L1** (Update) | Large | 0.80 | 731 | 49 | 0.97 | 0.09 | 0.82 | | VB_L2** (Update) | Large | 0.80 | 650 | 44 | 0.99 | 0.03 | 0.94 | | VB_L3** (Update) | Large | 0.80 | 582 | 44 | 0.99 | 0.03 | 0.94 | | VB_L4** (Update) | Large | 0.80 | 569 | 44 | 0.99 | 0.01 | 0.97 | | VB_L5** (Update) | Large | 0.80 | 550 | 43 | 0.99 | 0.02 | 0.96 | | VB_T01** (Update) | Medium | 0.65 | 663 | 37 | 0.98 | 0.01 | 0.96 | | VB_T02** (Update) | Medium | 0.65 | 682 | 38 | 0.96 | 0.12 | 0.76 | | VB_T03** (Update) | Medium | 0.65 | 700 | 38 | 0.96 | 0.14 | 0.73 | | VB_T04** (Update) | Medium | 0.65 | 695 | 39 | 0.95 | 0.16 | 0.69 | | VB_T05** (Update) | Medium | 0.65 | 687 | 39 | 0.97 | 0.08 | 0.84 | | VB_T06** (Update) | Medium | 0.65 | 682 | 36 | 0.97 | 0.06 | 0.87 | | VB_T07** (Update) | Medium | 0.65 | 686 | 39 | 0.94 | 0.16 | 0.68 | | VB_T08** (Update) | Medium | 0.65 | 729 | 45 | 0.96 | 0.14 | 0.73 | | VB_T09** (Update) | Medium | 0.65 | 778 | 49 | 0.98 | 0.04 | 0.91 | | VB_T10** (Update) | Medium | 0.65 | 803 | 49 | 0.97 | 0.09 | 0.82 | | VB_T11** (Update) | Medium | 0.65 | 814 | 48 | 0.98 | 0.07 | 0.86 | | VB_T12** (Update) | Medium | 0.65 | 795 | 50 | 0.97 | 0.13 | 0.76 | | Ventricle_Brain | Medium | 0.65 | 177 | 48 | 0.84 | 0.05 | 0.75 | | Ventricle_L** (Update) | Medium | 0.65 | 398 | 45 | 0.72 | 0.27 | 0.28 | | Ventricle_R** (Update) | Medium | 0.65 | 398 | 45 | 0.84 | 0.23 | 0.45 | | Vessels_Pelvis | Medium | 0.65 | 921 | 50 | 0.892 | 0.16 | 0.63 | {25} $^{*}\mathrm{N / A}$ : Structures are generated based on a post-processing/boolean operation from previously released structure models (Eye, Rib) rather than generated from a CNN model. Quantitative and Qualitative testing for these structures is still performed in the following sections in order to validate appropriate contour generation and clinical acceptability. ** A_Aorta, A_Aorta_Asc, A_Carotid_L, A_Carotid_R, A_Coronary_R, A_Pulmonary, Atrium_L, Atrium_R, Bladder, Bone_Skull, Bone_Teeth, Bone_Mandible, Bone_Pelvic, Bowel, Bowel_Bag, Bowel_Large, Bowel_Small, BrachialPlex_L, BrachialPlex_R, Brain, Brainstem, BuccalMucosa, CaudaEquina, Cavity_Oral_Ext, Cochlea_L, Cochlea_R, Dental_Artifact, Ear_Internal_L, Ear_Internal_R, Esophagus, Heart, Hippocampus_L, Hippocampus_R, Iliac_Int_L, Iliac_Int_R, Iliac_L, Iliac_R, Kidneys, Larynx, LN_Pelvics_NRG, LN_Presacral, Lobe_Temporal_L, Lobe_Temporal_R, Musc_liopsoas_L, Musc_liopsoas_R, Myocardium, Parotid_L, Parotid_R, Pancreas, Pericardium, Pharynx, Pituitary, ProstateBed, Rib01_L, Rib01_R, Rib02_L, Rin02_R, Rib03_L, Rib03_R, Rib04_L, Rib04_R, Rib05_L, Rib05_R, Rib06_L, Rib06_R, Rib07_L, Rib07_R, Rib08_L, Rib08_R, Rib09_L, Rib09_R, Rib10_L, Rib10_R, Rib11_L, Rib11_R, Rib12_L, Rib12_R, Rib, Rlb_R, SacralPlex_L, SacralPlex_R, SpinalCanal, SpinalCord, Spleen, Trachea, UteroCervix, V_Jugular_L, V_Jugular_R, V_Venacava_I, V_Venacava_S, VB_VB_C1, VB_C2, VB_C3, VB_C4, VB_C5, VB_C6, VB_C7, VB_L1, VB_L2, VB_L3, VB_L4, VB_L5, VB_T01, VB_T02, VB_T03, VB_T04, VB_T05, VB_T06, VB_T07, VB_T08, VB_T09, VB_T10, VB_T11, VB_T12, Ventricle_L, and Ventricle_R models were previously released/tested, but updates to the training datasets and contour output necessitated additional release testing for these models. Additional external clinical testing was performed in order to validate the accuracy of the models on image sets acquired that were unique to the training datasets. Both AutoContour and manually added ground truth contours following the same structure guidelines used for structure model training were added to the image sets. | Table 5: CT External Clinical Dataset References | | | | --- | --- | --- | | Model Group | Data Source ID | Data Citation | | CT Pelvis | TCIA - Pelvic-Ref | Afua A. Yorke, Gary C. McDonald, David Solis Jr., Thomas Guerrero. (2019) Pelvic Reference Data. The Cancer Imaging Archive. DOI: 10.7937/TCIA.2019.woskg5oo | | CT Head and Neck | TCIA - Head-Neck-PET-CT | Martin Vallières, Emily Kay-Rivest, Léo Jean Perrin, Xavier Liem, Christophe Furstoss, Nader Khaouam, Phuc Félix Nguyen-Tan, Chang-Shu Wang, Khalil Sultanem. (2017). Data from Head-Neck-PET-CT. The Cancer Imaging Archive. doi: 10.7937/K9/TCIA.2017.8oje5q00 | | CT Abdomen | TCIA - Pancreas-CT-CB | Hong, J., Reyngold, M., Crane, C., Cuaron, J., Hajj, C., Mann, J., Zinovoy, M., Yorke, E., LoCastro, E., Apte, A. P., & Mageras, G. (2021). Breath-hold CT and cone-beam CT images with expert manual organ-at-risk segmentations from radiation treatments of locally advanced pancreatic cancer [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.ESHQ-4D90 | | CT Thorax: | TCIA - NSCLC | Aerts, H. J. W. L., Wee, L., Rios Velazquez, E., Leijenaar, R. T. H., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., Hoebers, F., Rietbergen, M. M., Leemans, C. R., Dekker, A., Quackenbush, J., Gillies, R. J., Lambin, P. (2019). Data From | | | | the Cancer Imaging Archive. http://www.cancerimagingarchive.org/ | | CT Head and Neck | TCIA - Head-Neck-PET-CT | Hassan, M. M., & Khaouam, M. (2019). The Cancer Imaging Archive. http://www.cancerimagingarchive.org/ | | CT Abdomen | TCIA - Pancreas-CT-CB | Hassan, M. M., & Khaouam, M. (2019). The Cancer Imaging Archive. http://www.cancerimagingarchive.org/ | | CT Thorax: | TCIA - NSCLC | Hassan, M. M., & Khaouam, M. (2019). The Cancer Imaging Archive. http://www.cancerimagingarchive.org/ | AutoContour Model RADAC V5 Traditional 510(k) {26} DSC values were calculated between ground truth contour data and AutoContour structures and rated on the same DSC passing criteria used for the Training DSC validation. All structures passed the minimum DSC criteria for small, medium and large structures with a mean DSC of $0.73 + / - 0.11$ , $0.84 + / - 0.09$ , and $0.93 + / - 0.05$ respectively: Additionally, the qualitative clinical appropriateness of AutoContour structures generated on these scans was graded by clinical experts. Autocontour structures were graded on a scale from 1 to 5 where 5 refers to contour requiring no additional edits, and 1 refers to a score in which full manual re-contour of the structure would be required. An average score $>= 3$ was used to determine whether a structure model would ultimately be beneficial clinically. An average rating of 4.37 was found across all CT structure models demonstrating that only minor edits would be required in order to make the structure models acceptable for clinical use. Table 6: CT External Reviewer Results for AutoContour Model RADAC V5 | CT Structure | Size | Pass Criteria | # Testings Sets | Average DSC | Average DSC Std. Dev | Lower Bound 95% Confidence Interval | External Reviewer Average Rating (1-5) | | --- | --- | --- | --- | --- | --- | --- | --- | | A_Aorta (Update) | Large | 0.80 | 20 | 0.94 | 0.02 | 0.91 | 4.80 | | A_Aorta_Asc (Update) | Medium | 0.65 | 20 | 0.91 | 0.03 | 0.86 | 4.40 | | A_Carotid_L (Update) | Medium | 0.65 | 37 | 0.79 | 0.06 | 0.68 | 4.30 | | A_Carotid_R (Update) | Medium | 0.65 | 37 | 0.78 | 0.05 | 0.69 | 4.20 | | A_Celiac | Small | 0.50 | 24 | 0.78 | 0.22 | 0.41 | 4.40 | | A_Circumflex_L | Small | 0.50 | 40 | 0.56 | 0.16 | 0.30 | 4.25 | | A_Circumflex_R | Small | 0.50 | 40 | 0.56 | 0.16 | 0.30 | 4.25 | AutoContour Model RADAC V5 Traditional 510(k) {27} AutoContour Model RADAC V5 Traditional 510(k) | A_Coronary_2d_R | Small | 0.50 | 39 | 0.65 | 0.21 | 0.31 | 3.60 | | --- | --- | --- | --- | --- | --- | --- | --- | | A_Coronary_L | Small | 0.50 | 18 | 0.69 | 0.22 | 0.33 | 4.35 | | A_Coronary_R (Update) | Small | 0.50 | 40 | 0.55 | 0.13 | 0.33 | 4.25 | | A_Mesenteric_S | Small | 0.50 | 25 | 0.78 | 0.13 | 0.57 | 4.50 | | A_Pulmonary (Update) | Medium | 0.65 | 20 | 0.89 | 0.02 | 0.85 | 4.50 | | Atrium_L (Update) | Medium | 0.65 | 20 | 0.87 | 0.02 | 0.85 | 4.50 | | Atrium_R (Update) | Medium | 0.65 | 20 | 0.85 | 0.06 | 0.75 | 4.50 | | AV_Node | Medium | 0.65 | 40 | 0.87 | 0.12 | 0.67 | 4.45 | | Barrigel | Medium | 0.65 | 14 | 0.79 | 0.03 | 0.74 | 4.30 | | BileDuct_Common | Small | 0.50 | 25 | 0.66 | 0.11 | 0.47 | 4.55 | | Bladder (Update) | Large | 0.80 | 20 | 0.96 | 0.02 | 0.94 | 4.60 | | Body+Mask | Large | 0.80 | 22 | 0.99 | 0.01 | 0.98 | 4.50 | | Bone_Ischium_L | Large | 0.80 | 41 | 0.98 | 0.02 | 0.94 | 4.20 | | Bone_Ischium_R | Large | 0.80 | 41 | 0.98 | 0.03 | 0.94 | 4.20 | | Bone_Mandible (Update) | Medium | 0.65 | 22 | 0.88 | 0.02 | 0.84 | 4.40 | | Bone_Pelvic (Update) | Large | 0.80 | 41 | 0.91 | 0.01 | 0.89 | 4.25 | | Bone_Pterygoid_L | Small | 0.50 | 22 | 0.81 | 0.03 | 0.77 | 4.20 | | Bone_Pterygoid_R | Small | 0.50 | 22 | 0.81 | 0.03 | 0.76 | 4.30 | | Bone_Skull (Update) | Large | 0.80 | 22 | 0.94 | 0.01 | 0.92 | 4 | | Bone_Teeth (Update) | Medium | 0.65 | 9 | 0.87 | 0.02 | 0.84 | 4.20 | | Bowel (Update) | Medium | 0.65 | 46 | 0.87 | 0.03 | 0.82 | 3.50 | | Bowel_Bag (Update) | Large | 0.80 | 46 | 0.91 | 0.05 | 0.83 | 4.40 | | Bowel_Bag_F | Large | 0.80 | 20 | 0.89 | 0.05 | 0.81 | 4.40 | | Bowel_F | Medium | 0.65 | 20 | 0.91 | 0.04 | 0.84 | 4.40 | | Bowel_Large (Update) | Medium | 0.65 | 46 | 0.84 | 0.08 | 0.71 | 4.20 | | Bowel_Large_F | Medium | 0.65 | 20 | 0.79 | 0.20 | 0.47 | 4.40 | | Bowel_Small (Update) | Medium | 0.65 | 46 | 0.83 | 0.06 | 0.73 | 4.10 | | Bowel_Small_F | Medium | 0.65 | 20 | 0.87 | 0.10 | 0.70 | 4.30 | | BrachialPlex_L (Update) | Medium | 0.65 | 22 | 0.91 | 0.03 | 0.87 | 4.50 | | BrachialPlex_R (Update) | Medium | 0.65 | 22 | 0.90 | 0.03 | 0.86 | 4.40 | | BrachialPlexs | Medium | 0.65 | 22 | 0.91 | 0.03 | 0.87 | 4.40 | | Brain (Update) | Large | 0.80 | 22 | 0.97 | 0.00 | 0.97 | 4.70 | | Brainstem (Update) | Medium | 0.65 | 22 | 0.87 | 0.02 | 0.84 | 4.60 | | Breast_RTOG_L | Large | 0.80 | 20 | 0.99 | 0.01 | 0.97 | 5 | | Breast_RTOG_R | Large | 0.80 | 20 | 0.99 | 0.02 | 0.95 | 4.90 | | Breasts_RTOG | Large | 0.80 | 20 | 0.99 | 0.02 | 0.96 | 4.95 | {28} AutoContour Model RADAC V5 Traditional 510(k) | BuccalMucosa (Update) | Medium | 0.65 | 22 | 0.66 | 0.09 | 0.51 | 4.40 | | --- | --- | --- | --- | --- | --- | --- | --- | | Canal_Anal | Medium | 0.65 | 20 | 0.72 | 0.17 | 0.44 | 4.70 | | Canal_Anal_F | Medium | 0.65 | 20 | 0.75 | 0.21 | 0.40 | 4.30 | | InternalAuditoryCanal_L | Small | 0.50 | 20 | 0.88 | 0.10 | 0.72 | 4.40 | | CaudaEquina (Update) | Medium | 0.65 | 41 | 0.80 | 0.08 | 0.66 | 4.40 | | Cavity_Oral_Ext (Update) | Medium | 0.65 | 22 | 0.95 | 0.01 | 0.94 | 4.10 | | Chestwall_2cm_L | Large | 0.80 | 20 | 0.99 | 0.002 | 0.99 | 4.80 | | Chestwall_2cm_R | Large | 0.80 | 20 | 0.99 | 0.003 | 0.99 | 5 | | Chestwall_ESTRO_L | Large | 0.80 | 18 | 0.90 | 0.04 | 0.83 | 4.20 | | Chestwall_ESTRO_R | Large | 0.80 | 18 | 0.89 | 0.03 | 0.84 | 4.20 | | Cochlea_L (Update) | Small | 0.50 | 22 | 0.70 | 0.12 | 0.51 | 4.50 | | Cochlea_R (Update) | Small | 0.50 | 22 | 0.80 | 0.13 | 0.59 | 4.50 | | Cornea_2d_L | Small | 0.50 | 22 | 0.80 | 0.05 | 0.73 | 4.40 | | Cornea_2d_R | Small | 0.50 | 22 | 0.78 | 0.06 | 0.69 | 4.40 | | CorpusCallosum | Medium | 0.65 | 22 | 0.75 | 0.05 | 0.67 | 4.20 | | Cribriform | Medium | 0.65 | 22 | 0.74 | 0.11 | 0.55 | 4.10 | | Dental_Artifact (Update) | Medium | 0.65 | 20 | 0.89 | 0.10 | 0.73 | 4.10 | | Ear_Internal_L (Update) | Small | 0.50 | 22 | 0.72 | 0.09 | 0.58 | 4 | | Ear_Internal_R (Update) | Small | 0.50 | 22 | 0.73 | 0.08 | 0.60 | 4.10 | | Esophagus (Update) | Medium | 0.65 | 40 | 0.83 | 0.04 | 0.78 | 4.75 | | Eyes | Medium | 0.65 | 21 | 0.97 | 0.01 | 0.96 | 4.50 | | Falx | Medium | 0.65 | 22 | 0.79 | 0.04 | 0.73 | 4.40 | | GreatVessels | Medium | 0.65 | 20 | 0.92 | 0.02 | 0.89 | 4.30 | | HDR_Canal_Anal | Large | 0.80 | 20 | 0.79 | 0.10 | 0.62 | 4.30 | | HDR_Colon_Sigmoid | Medium | 0.65 | 20 | 0.70 | 0.20 | 0.38 | 4.10 | | Heart (Update) | Large | 0.80 | 20 | 0.97 | 0.04 | 0.91 | 5 | | Heart+A_Pulm | Large | 0.80 | 20 | 0.98 | 0.01 | 0.96 | 4.50 | | Hippocampus_L (Update) | Medium | 0.65 | 22 | 0.77 | 0.11 | 0.59 | 4.50 | | Hippocampus_R (Update) | Medium | 0.65 | 22 | 0.81 | 0.06 | 0.71 | 4.50 | | Iliac_Int_L (Update) | Medium | 0.65 | 41 | 0.80 | 0.05 | 0.72 | 4.45 | | Iliac_Int_R (Update) | Medium | 0.65 | 41 | 0.80 | 0.06 | 0.69 | 4.45 | | Iliac_L (Update) | Medium | 0.65 | 41 | 0.88 | 0.06 | 0.79 | 4.50 | | Iliac_R (Update) | Medium | 0.65 | 41 | 0.84 | 0.06 | 0.74 | 4.50 | | Kidneys | Large | 0.80 | 25 | 0.99 | 0.03 | 0.94 | 4.80 | | Larynx (Update) | Medium | 0.65 | 22 | 0.88 | 0.10 | 0.72 | 4.40 | | LN_Ax_L1_ESTRO_2d_L | Medium | 0.65 | 20 | 0.95 | 0.05 | 0.87 | 4.50 | {29} AutoContour Model RADAC V5 Traditional 510(k) | LN_Ax_L1_ESTRO_2d_R | Medium | 0.65 | 20 | 0.94 | 0.05 | 0.87 | 4.50 | | --- | --- | --- | --- | --- | --- | --- | --- | | LN_Ax_L2_ESTRO_2d_L | Medium | 0.65 | 20 | 0.92 | 0.04 | 0.85 | 4.50 | | LN_Ax_L2_ESTRO_2d_R | Medium | 0.65 | 20 | 0.93 | 0.04 | 0.86 | 4.60 | | LN_Ax_L3_ESTRO_2d_L | Medium | 0.65 | 18 | 0.69 | 0.13 | 0.48 | 4.50 | | LN_Ax_L3_ESTRO_2d_R | Medium | 0.65 | 18 | 0.67 | 0.14 | 0.44 | 4.50 | | LN_InPec_ESTRO_2d_L | Medium | 0.65 | 20 | 0.90 | 0.07 | 0.78 | 4.60 | | LN_InPec_ESTRO_2d_R | Medium | 0.65 | 19 | 0.902 | 0.09 | 0.76 | 4.50 | | LN_Ax_L2_InPec_ESTRO_2d_L | Medium | 0.65 | 20 | 0.93 | 0.04 | 0.86 | 4.50 | | LN_Ax_L2_InPec_ESTRO_2d_R | Medium | 0.65 | 20 | 0.93 | 0.05 | 0.86 | 4.50 | | LN_Mesorectum | Large | 0.80 | 41 | 0.93 | 0.03 | 0.88 | 4.50 | | LN_Neck_2d_L | Large | 0.80 | 22 | 0.90 | 0.02 | 0.86 | 4.50 | | LN_Neck_2d_R | Large | 0.80 | 22 | 0.90 | 0.03 | 0.86 | 4.50 | | LN_Neck_CerVII_2d_L | Large | 0.80 | 22 | 0.94 | 0.04 | 0.87 | 4.20 | | LN_Neck_CerVII_2d_R | Large | 0.80 | 22 | 0.94 | 0.03 | 0.89 | 4.20 | | LN_Neck_IB_2d_L | Medium | 0.65 | 22 | 0.79 | 0.05 | 0.71 | 4.30 | | LN_Neck_IB_2d_R | Medium | 0.65 | 22 | 0.82 | 0.03 | 0.76 | 4.50 | | LN_Neck_II_2d_L | Medium | 0.65 | 21 | 0.75 | 0.04 | 0.68 | 4.50 | | LN_Neck_II_2d_R | Medium | 0.65 | 21 | 0.77 | 0.05 | 0.68 | 4.50 | | LN_Neck_II-IV_2d_L | Large | 0.80 | 22 | 0.94 | 0.05 | 0.87 | 4.40 | | LN_Neck_II-IV_2d_R | Large | 0.80 | 22 | 0.94 | 0.04 | 0.89 | 4.40 | | LN_Neck_III_2L | Medium | 0.65 | 22 | 0.74 | 0.06 | 0.65 | 4.40 | | LN_Neck_III_2R | Medium | 0.65 | 22 | 0.72 | 0.06 | 0.61 | 4.40 | | LN_Neck_IV_2L | Medium | 0.65 | 22 | 0.94 | 0.04 | 0.87 | 4.30 | | LN_Neck_IV_2R | Medium | 0.65 | 22 | 0.93 | 0.04 | 0.86 | 4.30 | | LN_Neck_VII_2d_L | Medium | 0.65 | 22 | 0.75 | 0.10 | 0.58 | 4 | | LN_Neck_VII_2d_R | Medium | 0.65 | 22 | 0.72 | 0.09 | 0.58 | 4 | | LN_Neck_VIIA_2d_L | Medium | 0.65 | 22 | 0.85 | 0.07 | 0.74 | 4.20 | | LN_Neck_VIIA_2d_R | Medium | 0.65 | 22 | 0.82 | 0.10 | 0.66 | 4.20 | | LN_Neck_VIIB_2d_L | Medium | 0.65 | 22 | 0.69 | 0.13 | 0.48 | 4 | | LN_Neck_VIIB_2d_R | Medium | 0.65 | 22 | 0.68 | 0.12 | 0.49 | 4 | | LN_Pelvics_NRG (Update) | Large | 0.80 | 41 | 0.82 | 0.03 | 0.77 | 4.50 | | LN_Presacral (Update) | Medium | 0.65 | 41 | 0.81 | 0.07 | 0.70 | 4.45 | | Lobe_Temporal_L (Update) | Large | 0.80 | 22 | 0.95 | 0.07 | 0.84 | 4.50 | | Lobe_Temporal_R (Update) | Large | 0.80 | 22 | 0.95 | 0.08 | 0.83 | 4.50 | {30} AutoContour Model RADAC V5 Traditional 510(k) | Lungs | Large | 0.80 | 20 | 0.98 | 0.01 | 0.97 | 5 | | --- | --- | --- | --- | --- | --- | --- | --- | | Marrow_Pelvis | Medium | 0.65 | 21 | 0.94 | 0.01 | 0.93 | 4.45 | | Medulla | Medium | 0.65 | 22 | 0.76 | 0.06 | 0.67 | 4.50 | | Midbrain | Medium | 0.65 | 22 | 0.75 | 0.13 | 0.55 | 4.50 | | Musc_Iliopsoas_L (Update) | Large | 0.80 | 41 | 0.92 | 0.02 | 0.88 | 4.90 | | Musc_Iliopsoas_R (Update) | Large | 0.80 | 41 | 0.92 | 0.02 | 0.88 | 4.90 | | Musc_PecMinor_L | Medium | 0.65 | 18 | 0.89 | 0.04 | 0.82 | 4.50 | | Myocardium (Update) | Medium | 0.65 | 20 | 0.91 | 0.02 | 0.87 | 4.40 | | Pancreas (Update) | Medium | 0.65 | 25 | 0.73 | 0.13 | 0.52 | 4.20 | | Parametrium | Medium | 0.65 | 19 | 0.77 | 0.11 | 0.59 | 4.70 | | Parotid_L (Update) | Medium | 0.65 | 22 | 0.90 | 0.03 | 0.86 | 4.20 | | Parotid_R (Update) | Medium | 0.65 | 22 | 0.91 | 0.02 | 0.87 | 4.10 | | Pericardium (Update) | Large | 0.80 | 20 | 0.98 | 0.02 | 0.95 | 4.30 | | Pericardium_Inf | Large | 0.80 | 20 | 0.98 | 0.01 | 0.96 | 4.40 | | Pericardium_Inf+A_Pulm | Large | 0.80 | 20 | 0.98 | 0.01 | 0.97 | 4.30 | | Pharynx (Update) | Medium | 0.65 | 22 | 0.84 | 0.04 | 0.77 | 4.30 | | Pituitary (Update) | Small | 0.50 | 18 | 0.60 | 0.29 | 0.13 | 3.10 | | Pituitary_3d | Small | 0.50 | 20 | 0.63 | 0.10 | 0.47 | 4.50 | | Pons | Medium | 0.65 | 22 | 0.85 | 0.03 | 0.80 | 4.50 | | ProstateBed (Update) | Medium | 0.65 | 20 | 0.89 | 0.05 | 0.80 | 4.40 | | ProstateFiducials | Small | 0.50 | 10 | 0.52 | 0.09 | 0.37 | 3.80 | | Rectum-Spacergel_VUE | Medium | 0.65 | 5 | 0.95 | 0.02 | 0.92 | 3.90 | | Retina_2d_L | Small | 0.50 | 22 | 0.88 | 0.03 | 0.83 | 4.40 | | Retina_2d_R | Small | 0.50 | 22 | 0.88 | 0.03 | 0.83 | 4.40 | | Rib01_L (Update) | Medium | 0.65 | 40 | 0.84 | 0.08 | 0.72 | 4.20 | | Rib01_R (Update) | Medium | 0.65 | 40 | 0.85 | 0.08 | 0.71 | 4.15 | | Rib02_L (Update) | Medium | 0.65 | 40 | 0.84 | 0.08 | 0.71 | 4.15 | | Rib02_R (Update) | Medium | 0.65 | 39 | 0.82 | 0.12 | 0.63 | 4.30 | | Rib03_L (Update) | Medium | 0.65 | 40 | 0.84 | 0.09 | 0.69 | 4.15 | | Rib03_R (Update) | Medium | 0.65 | 39 | 0.85 | 0.11 | 0.67 | 4.20 | | Rib04_L (Update) | Medium | 0.65 | 38 | 0.85 | 0.06 | 0.76 | 4.25 | | Rib04_R (Update) | Medium | 0.65 | 39 | 0.82 | 0.12 | 0.61 | 4.30 | | Rib05_L (Update) | Medium | 0.65 | 37 | 0.85 | 0.09 | 0.70 | 4.20 | | Rib05_R (Update) | Medium | 0.65 | 33 | 0.86 | 0.09 | 0.70 | 3.95 | | Rib06_L (Update) | Medium | 0.65 | 40 | 0.85 | 0.11 | 0.68 | 4.30 | | Rib06_R (Update) | Medium | 0.65 | 40 | 0.82 | 0.15 | 0.57 | 4.35 | {31} AutoContour Model RADAC V5 Traditional 510(k) | Rib07_L (Update) | Medium | 0.65 | 40 | 0.86 | 0.07 | 0.75 | 4.40 | | --- | --- | --- | --- | --- | --- | --- | --- | | Rib07_R (Update) | Medium | 0.65 | 40 | 0.84 | 0.13 | 0.62 | 4.35 | | Rib08_L (Update) | Medium | 0.65 | 40 | 0.87 | 0.02 | 0.83 | 4.35 | | Rib08_R (Update) | Medium | 0.65 | 40 | 0.83 | 0.15 | 0.59 | 4.20 | | Rib09_L (Update) | Medium | 0.65 | 40 | 0.87 | 0.03 | 0.82 | 4.30 | | Rib09_R (Update) | Medium | 0.65 | 40 | 0.81 | 0.20 | 0.48 | 4.20 | | Rib10_L (Update) | Medium | 0.65 | 40 | 0.85 | 0.11 | 0.67 | 4.35 | | Rib10_R (Update) | Medium | 0.65 | 40 | 0.77 | 0.26 | 0.35 | 4.40 | | Rib11_L (Update) | Medium | 0.65 | 40 | 0.83 | 0.11 | 0.65 | 4.40 | | Rib11_R (Update) | Medium | 0.65 | 40 | 0.76 | 0.27 | 0.32 | 4.40 | | Rib12_L (Update) | Small | 0.50 | 30 | 0.79 | 0.12 | 0.59 | 4.35 | | Rib12_R (Update) | Small | 0.50 | 27 | 0.81 | 0.06 | 0.71 | 4.30 | | Rib (Update) | Large | 0.80 | 20 | 0.81 | 0.02 | 0.79 | 4.50 | | Rib_R (Update) | Large | 0.80 | 20 | 0.81 | 0.02 | 0.79 | 4.60 | | SacralPlex_L (Update) | Medium | 0.65 | 41 | 0.67 | 0.04 | 0.60 | 4.45 | | SacralPlex_R (Update) | Medium | 0.65 | 41 | 0.69 | 0.05 | 0.61 | 4.45 | | Sinuses | Medium | 0.65 | 22 | 0.71 | 0.04 | 0.63 | 4.10 | | SpinalCanal (Update) | Large | 0.80 | 25 | 0.93 | 0.05 | 0.84 | 4.50 | | SpinalCord (Update) | Medium | 0.65 | 45 | 0.89 | 0.04 | 0.82 | 4.50 | | Spleen (Update) | Large | 0.80 | 24 | 0.97 | 0.10 | 0.96 | 4.50 | | Tentorium | Medium | 0.65 | 22 | 0.61 | 0.07 | 0.48 | 4.30 | | Trachea (Update) | Medium | 0.65 | 22 | 0.84 | 0.04 | 0.78 | 4.40 | | UteroCervix (Update) | Medium | 0.65 | 12 | 0.85 | 0.06 | 0.76 | 4.50 | | V_Jugular_L (update) | Medium | 0.65 | 37 | 0.76 | 0.13 | 0.55 | 3.90 | | V_Jugular_R (update) | Medium | 0.65 | 36 | 0.81 | 0.06 | 0.71 | 4.10 | | V_Pulmonary | Medium | 0.65 | 40 | 0.89 | 0.09 | 0.74 | 4.35 | | V_Venacava_I (Update) | Medium | 0.65 | 11 | 0.80 | 0.09 | 0.65 | 4.65 | | V_Venacava_I_UKSABR | Medium | 0.65 | 18 | 0.97 | 0.06 | 0.87 | 4.50 | | V_Venacava_S (Update) | Medium | 0.65 | 20 | 0.88 | 0.02 | 0.85 | 4.50 | | V_Venacava_S_UKSABR | Medium | 0.65 | 18 | 0.95 | 0.08 | 0.83 | 4.40 | | Vagina | Medium | 0.65 | 20 | 0.89 | 0.05 | 0.81 | 4.50 | | Valve_Aortic | Medium | 0.65 | 36 | 0.82 | 0.18 | 0.52 | 4.45 | | Valve_Mitral | Medium | 0.65 | 40 | 0.91 | 0.10 | 0.75 | 4.45 | | Valve_Pulmonic | Medium | 0.65 | 39 | 0.81 | 0.18 | 0.52 | 4.35 | | Valve_Tricuspid | Medium | 0.65 | 40 | 0.91 | 0.09 | 0.77 | 4.45 | | VB (Update) | Large | 0.80 | 62 | 0.94 | 0.02 | 0.91 | 4.125 | {32} | VB_C1 (Update) | Medium | 0.65 | 22 | 0.89 | 0.02 | 0.86 | 4.10 | | --- | --- | --- | --- | --- | --- | --- | --- | | VB_C2 (Update) | Medium | 0.65 | 22 | 0.91 | 0.01 | 0.89 | 4.30 | | VB_C3 (Update) | Medium | 0.65 | 22 | 0.88 | 0.02 | 0.85 | 4.30 | | VB_C4 (Update) | Medium | 0.65 | 22 | 0.88 | 0.02 | 0.85 | 4.40 | | VB_C5 (Update) | Medium | 0.65 | 22 | 0.87 | 0.02 | 0.84 | 4.30 | | VB_C6 (Update) | Medium | 0.65 | 22 | 0.88 | 0.02 | 0.85 | 4 | | VB_C7 (Update) | Medium | 0.65 | 22 | 0.90 | 0.01 | 0.88 | 4.30 | | VB_L1 (Update) | Large | 0.80 | 19 | 0.89 | 0.21 | 0.54 | 4.40 | | VB_L2 (Update) | Large | 0.80 | 19 | 0.88 | 0.21 | 0.53 | 4.50 | | VB_L3 (Update) | Large | 0.80 | 20 | 0.90 | 0.19 | 0.58 | 4.50 | | VB_L4 (Update) | Large | 0.80 | 20 | 0.90 | 0.21 | 0.56 | 4.50 | | VB_L5 (Update) | Large | 0.80 | 20 | 0.90 | 0.19 | 0.58 | 4.30 | | VB_T01 (Update) | Medium | 0.65 | 20 | 0.95 | 0.13 | 0.73 | 4.50 | | VB_T02 (Update) | Medium | 0.65 | 20 | 0.96 | 0.07 | 0.85 | 4.50 | | VB_T03 (Update) | Medium | 0.65 | 20 | 0.93 | 0.18 | 0.63 | 4.50 | | VB_T04 (Update) | Medium | 0.65 | 20 | 0.92 | 0.21 | 0.57 | 4.30 | | VB_T05 (Update) | Medium | 0.65 | 20 | 0.91 | 0.21 | 0.57 | 4.40 | | VB_T06 (Update) | Medium | 0.65 | 20 | 0.87 | 0.23 | 0.49 | 4.40 | | VB_T07 (Update) | Medium | 0.65 | 20 | 0.82 | 0.32 | 0.28 | 4.10 | | VB_T08 (Update) | Medium | 0.65 | 20 | 0.82 | 0.34 | 0.26 | 4.40 | | VB_T09 (Update) | Medium | 0.65 | 20 | 0.82 | 0.34 | 0.26 | 4.20 | | VB_T10 (Update) | Medium | 0.65 | 20 | 0.82 | 0.34 | 0.26 | 4.10 | | VB_T11 (Update) | Medium | 0.65 | 20 | 0.81 | 0.34 | 0.24 | 4 | | VB_T12 (Update) | Medium | 0.65 | 17 | 0.89 | 0.23 | 0.52 | 4.40 | | Ventricle_Brain | Medium | 0.65 | 22 | 0.85 | 0.06 | 0.76 | 4.30 | | Ventricle_L (Update) | Medium | 0.65 | 20 | 0.93 | 0.10 | 0.92 | 4.50 | | Ventricle_R (Update) | Medium | 0.65 | 20 | 0.84 | 0.03 | 0.78 | 4.50 | | Vessels_Pelvis | Medium | 0.65 | 41 | 0.93 | 0.05 | 0.85 | 4 | The MR training data set used for initial testing of the Brain models (Amygdala_L/R, CorpusCallosum, Cornea_L/R, Falx, Retina_L/R, Tentorium, Sinuses, Thalamus and Ventricle_Brain) had an average of 259 training image sets and 58 testing image sets and were acquired from several different institutions in several countries. The MR training data used for initial testing of the MR Pelvis models (Rectal_Spacer) had an average of 243 training image sets and 50 testing image sets and were taken from 2 open source datasets, and several institutions within several countries, such as the United States, AutoContour Model RADAC V5 Traditional 510(k) {33} Canada, and Spain. Datasets used for testing were removed from the training dataset pool before model training began, and used exclusively for testing. Ground truthing of each test data set was generated manually using consensus (NRG/RTOG) guidelines as appropriate by six clinically experienced experts consisting of 2 radiation therapy physicists, 1 radiation dosimetrist, and 3 radiation therapists with specialized training in radiation therapy contouring. For MR Structure models, a mean training DSC of $0.82 + / - 0.12$ for medium models, and $0.72 + / - 0.09$ for small models. | Table 8: MR Training Data Results for AutoContour Model RADAC V5 | | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | | MR Models | Size | Pass Criteria | # of Training Sets | # of Testing Sets | DSC (Avg) | DSC Std Dev (Avg) | Lower Bound 95% Confidence Interval | | A_Pud_Int_L** (Update) | Small | 0.50 | 221 | 56 | 0.69 | 0.08 | 0.56 | | A_Pud_Int_R** (Update) | Small | 0.50 | 221 | 56 | 0.69 | 0.08 | 0.56 | | Amygdala_L | Small | 0.50 | 493 | 133 | 0.73 | 0.07 | 0.61 | | Amygdala_R | Small | 0.50 | 493 | 133 | 0.73 | 0.07 | 0.61 | | Bladder_Trigone** (Update) | Medium | 0.65 | 256 | 65 | 0.72 | 0.05 | 0.64 | | Brainstem** (Update) | Medium | 0.65 | 422 | 50 | 0.88 | 0.16 | 0.62 | | CorpusCallosum | Medium | 0.65 | 56 | 15 | 0.72 | 0.25 | 0.31 | | Falx | Medium | 0.65 | 178 | 45 | 0.83 | 0.03 | 0.78 | | HDR_Bowel | Medium | 0.65 | 307 | 37 | 0.83 | 0.19 | 0.51 | | HDR_Colon_Sigmoid | Medium | 0.65 | 240 | 24 | 0.81 | 0.21 | 0.46 | | Lens_L** (Update) | Small | 0.50 | 208 | 52 | 0.81 | 0.09 | 0.66 | | Lens_R** (Update) | Small | 0.50 | 208 | 52 | 0.81 | 0.09 | 0.66 | | Medulla* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Midbrain* | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | NVB_L** (Update) | Small | 0.50 | 167 | 43 | 0.61 | 0.08 | 0.48 | | NVB_R** (Update) | Small | 0.50 | 167 | 43 | 0.61 | 0.08 | 0.48 | | OpticNrv_L** (Update) | Small | 0.50 | 164 | 32 | 0.64 | 0.08 | 0.51 | | OpticNrv_R** (Update) | Small | 0.50 | 164 | 32 | 0.64 | 0.08 | 0.51 | | OpticTract_L** (Update) | Small | 0.50 | 304 | 76 | 0.72 | 0.08 | 0.59 | | OpticTract_R** (Update) | Small | 0.50 | 304 | 76 | 0.72 | 0.08 | 0.59 | | PenileBulb_TRUFI | Small | 0.50 | 228 | 58 | 0.79 | 0.12 | 0.59 | | Pons * | Medium | 0.65 | N/A | N/A | N/A | N/A | N/A | | Rectal_Spacer** (Update) | Small | 0.50 | 233 | 26 | 0.82 | 0.24 | 0.43 | | Sinuses | Medium | 0.65 | 178 | 45 | 0.83 | 0.04 | 0.762 | AutoContour Model RADAC V5 Traditional 510(k) {34} | Tentorium | Small | 0.50 | 180 | 46 | 0.77 | 0.05 | 0.69 | | --- | --- | --- | --- | --- | --- | --- | --- | | Thalamus | Medium | 0.65 | 361 | 40 | 0.83 | 0.13 | 0.62 | | Urethra** (Update) | Small | 0.50 | 390 | 100 | 0.68 | 0.09 | 0.53 | | Ventricle_Brain | Medium | 0.65 | 182 | 47 | 0.90 | 0.04 | 0.8342 | *These structures were generated from post-processing operations from previously released or re-tested models, Brainstem, rather than from a CNN model. Qualitative and Quantitative analysis of these structures is still performed. **A_Pud_Int_L, A_Pud_Int_R, Bladder_Trigone, Brainstem, Lens_L, Lens_R, NVB_L, NVB_R, OpticNrv_L, OpticNrv_R, OpticTract_L, OpticTract_R, Rectal_Spacer, and Urethra models were previously released/tested, but updates to the training datasets and contour output necessitated additional release testing for these models. Additional external clinical testing was performed in order to validate the accuracy of the models on image sets acquired that were unique to the training datasets. | Table 9: MR External Clinical Dataset References | | | | --- | --- | --- | | Model Group | Data Source ID | Data Citation | | MR Brain | MR - Renown | N/A | | MR Pelvis | Gold Atlas Pelvis | Nyholm, Tufve, Stina Svensson, Sebastian Andersson, Joakim Jonsson, Maja Sohlin, Christian Gustafsson, Elisabeth Kjellén, et al. 2018. “MR and CT Data with Multi Observer Delineations of Organs in the Pelvic Area - Part of the Gold Atlas Project.” Medical Physics 12 (10): 3218-21. doi:10.1002/mp.12748. | | MR Pelvis_2 | SynthRad | Thummerer A, van der Bijl E, Galapon Jr A, Verhoeff JJ, Langendijk JA, Both S, van den Berg CAT, Maspero M. 2023. SynthRAD2023 Grand Challenge dataset: Generating synthetic CT for radiotherapy. Medical Physics, 50(7), 4664-4674. https://doi.org/10.1002/mp.16529 | | MRLinac Pelvis | MR Linac | N/A- Testing data was shared by 2 institutions utilizing MR Linacs for image acquisitions. | | MR Female HDR Brachy | Female HDR MR Pelvis | N/A- Testing data was shared by 1 institution in Canada | | MR Pelvis Barrigel | Barrigel | N/A- Testing data was shared by several institutions in Australia. | For the Brain models, datasets acquired via data-use agreement from a clinical partner were acquired containing 20 MR T1 Ax post (BRAVO) image scans acquired with a GE MR750w scanner. Images had an average slice thickness of $1.6\mathrm{mm}$ , In-plane resolution between 0.94 mm, and acquisition parameters of TR=5.98ms, TE=96.8s. Data for testing of the MR Pelvis structure models were acquired from 2 publicly available datasets, which contained images of patients with prostate or rectal cancer, as well as 1 dataset shared from 2 institutions utilizing an MR Linac. Various scanner models and acquisition settings were used. Data for testing of the MR Pelvis HDR structure models were acquired from 1 institution using two different slice thicknesses, 1mm and 4mm, and two different in-plane resolutions, 1mm and .72mm. DSC values were calculated between ground truth contour data and AutoContour structures and rated on the same DSC passing criteria as was used for the training DSC validation. All structures, but one passed the minimum DSC criteria for small, and medium structures with a Auto…