Tutorial: Build a Contouring Library Using ProKnow

Overview

The ProKnow software is an excellent tool to help an organization design a standardized contouring accuracy library published using our online, customizable Quality System for Anatomy Contouring.

A contouring accuracy library can be used to measure the accuracy and quantify the variation of anatomy contouring (1) for a population of users and/or (2) to test and validate auto-segmentation methods.

Your library can be deployed in one of two ways: (1) as a private library that is available only to your organization's members and collaborators or (2) as a public library that is available to the international user base for research and/or in conjunction with workshops or scientific meetings.

This article will help you get started by describing the basic steps of how to design a contouring library using the ProKnow software as your data repository, editing tool, and asset staging guide.

Initial Set Up

There are some necessary preparatory steps that you will need to do only once before you begin building your contouring library.

1. VERIFY YOUR LOG IN

   Navigate your supported browser to your assigned ProKnow domain and log in. See Tutorial: Logging In for the First Time and Beyond for help.

2. CREATE, OR BE ASSIGNED, PROKNOW WORKSPACE(S) FOR YOUR CONTOURING ATLASES

   If you are a ProKnow customer with a secure, private domain that you use for other clinical and research purposes, then you may wish to create one or more dedicated workspaces (e.g., one per body site) that will house the image and contour data that you may ultimately deploy in your contouring accuracy library. Dedicated workspaces allow you to better organize data and also control who has permissions to access, upload, review, and edit patient data in the workspace(s).

   If you are not a ProKnow customer and you have been given access to a ProKnow-hosted domain, then at least one dedicated workspace will already be created for you.

3. ASSIGN EXPERTS

   You will designate specific experts to create and/or review the contours that you will be putting forth as references, you will need to assign appropriate experts and make sure they also have access to the dedicated anatomy atlas workspace(s).

Plan Your Library

1. PRIORITIZE BODY SITES

   It is a good idea to organize your contouring library into body sites and/or contouring protocols. Think about which body sites you eventually wish to cover, then prioritize them to determine the order in which you will build them.

2. PLAN IMPORTANT STRUCTURES PER BODY SITE

   Develop a list of important anatomical structures per body site or treatment protocol category. You may want to start with the highest-priority organs-at-risk (OARs) and target volumes for which assessing contouring accuracy is most critical.

3. ASSIGN EXPERTS

   You will designate specific experts to create and/or review the contours that you will be putting forth as references, you will need to assign appropriate experts and make sure they also have access to the dedicated anatomy atlas workspace(s).

4. SELECT SUITABLE PATIENT IMAGE SETS

   You will want to start collecting representative and appropriate image sets to cover the body sites and structures you have prioritized. Image sets can be computed tomography (CT) or magnetic resonance (MR). For the contouring exercises, the images must be axial and not oblique.

   Note About Non-Axial Image Series

   If your source images are oblique, then you can resample them to axial planes and then render those as a new image series to use with the contouring library, which degrade image quality unless they original oblique series has high resolution (i.e., small) slice spacing.

   After you collect each image set (and, potentially, RT structure set per image set), it is a good idea to anonymize all DICOM files to remove sensitive information and to assign generic and descriptive patient names.

5. CREATE AND/OR ASSEMBLE SUPPLEMENTARY EDUCATIONAL MATERIALS, PER STRUCTURE TYPE

   The contouring accuracy library allows for anatomical structure-specific educational materials (documents, movies, etc.) to be accessed for each exercise in the library. Some materials can be accessed at any time from the Background tab while other materials can be published on the Learn tab and available only after the user has made one attempt at contouring the full structure.

   These supplementary educational materials are not required, but they certainly enhance the usefulness of your contouring library, especially if it used for educational or skills assessment purposes.

Create Reference (Expert) Contours

The following steps describe how to build a library of reference/expert contours.

1. SIGN IN

   Navigate your supported browser to your assigned ProKnow domain and log in. If you don't know how to sign in, see the Tutorial: Logging In for the First Time and Beyond for help.

2. SELECT THE WORKSPACE DESIGNATED AS THE ANATOMY ATLAS

   From the workspace drop-down list in the upper left, select the applicable workspace where you store (or aim to store) the patient data standards used for your contouring library.

3. CREATE NEW (OR SELECT EXISTING) PATIENT

   On the left dashboard, click Patients to see the patient list. If the patient is already created, you can simply select it and proceed with remaining steps. If this is to be a new dataset, then create a new patient dataset using the labels (ID, name, etc.) that you wish to use for display and organization in ProKnow.

4. UPLOAD PATIENT IMAGES

   After creating the patient, you can upload the axial patient images directly to this patient using the Actions --> Upload options (upload by selecting files or by selecting a directory and having it upload all data contained, recursively). The images must be an axial series of DICOM CT or MR images. Oblique MR planes must be resampled to axial planes to use in contouring.

   Anonymization Reminder

   You will control if these patient images will ultimately be available via download to those with access to your contouring library, or if contouring will be online via ProKnow only. Regardless, it's is good practice to fully anonymize the DICOM files to remove any sensitive information. Edit, but do not remove, required tags (e.g., patient ID and name). Non-essential tags (referring physician, institution, etc.) you can clear.

5. UPLOAD OR CREATE STANDARD (EXPERT) CONTOURS

   If you have already contoured all necessary targets and critical structures using another software system (e.g., your TPS), then you can upload the DICOM RT structure set directly to this patient.

   As an alternative, you can create a new structure set for this patient and use the integrated contouring tools to define all targets and critical structures.

6. REVIEW AND EDIT TO FINALIZE ANATOMY CONTOURS

   Remember that your experts are building standard anatomical contours against which all your contouring library users will be compared and accuracy measured, so be sure to review all the contours for anatomical accuracy.

   Have your expert or team of experts sign in and review all the applicable structures and their contours, making edits using the ProKnow software if applicable.

   For more information on how to edit labels, colors, and/or contours and how to save the contours as a new version of the structure set, visit the Editing Structures article.

7. EDIT EACH STRUCTURE NAME TO BE HOW YOU WANT IT DISPLAYED IN THE LIBRARY

   For each structure that will be added to your contouring accuracy library, edit the name to be how you want it displayed in your contouring accuracy library.

   That list will be sort-able alphabetically, so you may want to make use of easily-to-read prefixes for modality (e.g., "[CT] ") or structure (e.g., "Heart ") or a combination (e.g., "[CT] Heart "), then have the suffix be something like the instance number or reference image set. A good example of an easily-sorted structure instance might be, "[CT] Heart, Imageset 01."

8. WRITE BRIEF INSTRUCTIONS, PER STRUCTURE, USING THE "NOTES" TOOL

   Use the patient notes tool to specify concise instructions for each structure that requires special guidance or detail. These instructions will appear for the user prior to their contouring (in addition to any supplementary education materials, as described below).

   For example, for a longitudinal structure (e.g., spinal cord), you may wish to tell the user the superior and inferior limits (by slice position, in mm) on which to contour, unless knowing those limits is part of the assessment.

   Define one note per structure. Write the instructions exactly as you would like them to appear to the users.

9. UPLOAD SUPPLEMENTARY EDUCATIONAL MATERIALS (BODY SITE- or STRUCTURE-SPECIFIC)

   As described earlier (in the "Plan Your Library" section), you can provide direct access to educational materials for the particular structure. You would probably use these structure-specific instructions for each instance of the structure, so that they are not imageset-specific but rather applying to that structure, in general.

   To do this, you should create a new patient called "Educational Materials" in your workspace. Then, you can upload your educational document using the Documents tool for that patient.

   PDF documents are ideal. Convert your original source documents (DOC, PPT, etc.) to PDF before uploading.

   If you have videos, you can host them on Vimeo and provide the Vimeo ID which will allow the video to be embedded in your ProKnow contouring accuracy library. Movies are a great way to do "contouring walkthroughs" that explain how the experts came up with their contours.

Request Publication of the Structures in Your Contouring Accuracy Library

Contact ProKnow by emailing support@proknowsystems.com when you are ready to port the image sets and anatomical structures into the applicable ProKnow online contouring accuracy library.

You will need to provide a list of the anonymized patient IDs and names (as displayed in ProKnow) for the applicable patient datasets, as well as the structure names per dataset, that you wish to add to the library.

Make sure you have (1) the structure-specific instructions entered as Notes and (2) the general educational materials for the structure category uploaded as PDF in the separate "Educational Materials" patient's Documents tab.

If you are using a ProKnow-hosted sub-domain, ProKnow staff will have access to the data they need. If you are a ProKnow customer using your private domain, then you will need to give temporary access (to the dedicated workspace only) to a ProKnow staff member so that he/she can retrieve the data and port it to your contouring accuracy library.