IN THIS ARTICLE
- Who Should Participate
- Getting Started with a Plan Study
- General Planning Instructions, FAQs and Online Help
- Plan Quality Scoring
- Statistical Analysis & Performance vs. Population
Goals Common to Every Plan Study
- Quantify the relative capabilities of different planners and delivery modalities to create high quality treatment plans, where the measure of quality is 100% objective and transparent with real-time scoring and analysis.
- Study the variability across all submitted plans and look for statistical significance and correlations.
- Derive, record, and distribute the “best practices” (i.e. useful tips and successful techniques), sharing with the community via presentations, along with written and recorded interviews with high performers.
- Form the basis for a feature scientific presentation at an associated scientific meeting and/or as the basis of a peer-reviewed publication.
Goals Unique to Each Individual Plan Study
Individual plan studies will often have specific and unique goals beyond the common goals. These specific goals will be spelled out in the study-specific instructions on the "Instructions" tab for that plan study.
Who Should Participate
Any and all people in the worldwide radiation oncology community are encouraged to participate in public plan studies. Participants can use any planning system and delivery method available to them and suitable for the particular patient dataset and prescription. In some instances, participation may target a certain modality, but most plan studies are open to all modalities. Most participants are professionals who do radiation treatment planning as part of their job, but we encourage students (e.g., dosimetry students, medical physics residents, radiation oncology residents) to participate as well.
Are you concerned about your performance? Have no fear, because all results are secure and anonymous (for both individuals and institutions), with the exception of "high performers" who may be contacted at the close of the study and will be given the option to be recognized by name and interviewed about their planning methods.
Getting Started with a Plan Study
- SIGN IN to ProKnow (www.proknowsystems.com). If you do not yet have a ProKnow account, then create one (it's easy, and it's free).
- Once you are signed in, SELECT THE "PLAN STUDIES" main menu item.
- In the organization selector drop-down (upper-right), SELECT THE ORGANIZATION hosting the plan study.
- For public plan studies, the organization will either be "ProKnow" or else will be specific to the meeting or organization (e.g., TROG Cancer Research, Radiosurgery Society, or others). All public plan studies (and their respective organizations) will be accessible to all ProKnow users.
- For private plan studies, you will have to be part of the specific organization to see that organization as an option in the drop-down menu in the upper right. Select the organization to see the plan studies for your organization. (Note: If you do not see the organization, then it means you have not yet been invited, or have not yet accepted the invitation, to that organization.)
- SELECT THE STUDY ID from the library of available studies. This will bring up the specific study's workspace.
- READ THE INSTRUCTIONS thoroughly.
- DOWNLOAD the required data, which will be a zip file containing the necessary CT images and RT Structure Set.
- UNZIP the data to get all the DICOM source files.
- IMPORT the images and structures into your TPS. (If this is your first time, please consult our TPS-specific instructions.)
- CREATE YOUR PLAN and CALCULATE THE DOSE using the planning system and modality of choice. Please refer to the Important Instructions on Specific Topics section for more details.
- EXPORT your plan and dose as DICOM RT Plan and DICOM RT Dose, respectively.
- UPLOAD your RT Plan and RT Dose to ProKnow.
- GENERATE YOUR PLAN SCORE, which will be calculated in real time by the ProKnow site. Comprehensive results will be displayed, both as a summary table and as per-metric drill-down analysis of each metric result.
- CONTINUE TO EDIT your plan (if you wish) in order to optimize your score. There is no limit to the number of trials and attempts.
- When you are ready (and/or at the deadline), SUBMIT your results to the study. Note: You can remove and/or replace a submission as long as the ProKnow data collection phase is open.
- If you see a section on the "Your Submission" page called ADDITIONAL DATA, please take the time to answer the questions. These will capture important data and feedback that cannot be derived from the plan and dose files.
- Once the study statistics are "released" by ProKnow, you can INTERACTIVELY STUDY YOUR RESULTS VS. THE POPULATION. Even after the study data collection phase is closed, you can continue to upload different plans and score them to see how they compare to your original submission and vs. the population.
- Over time, ProKnow will post various, useful materials on the LEARN tab. These materials might include official presentations of study results, resulting publications, and/or written and recorded interviews with some of the study's high performers providing tips and techniques to their success.
General Planning Instructions, FAQs and Online Help
Important Instructions on Specific Topics
Searchable Database and Online Help
We have a searchable database of questions that you can access here: http://support.proknowsystems.com/
Enter a search term or topic and see a list of applicable articles. Or, browse the category of topics and click the topic link to see a subset of articles.
If you have questions that are not addressed in the documents or searchable topics specified above, you can submit your questions by emailing firstname.lastname@example.org.
Plan Quality Scoring
Online and Real-Time
ProKnow plan studies most often use a scoring method and plan metrics that are far more advanced, comprehensive, and challenging than typical pass/fail goals based on published tolerances. A set of metrics will be extracted and scored with a weighted score function that defines minimal requirements up to the ideal result. For a general description of metric types in this and other studies, please visit this support page:
The plans will be scored based on target coverage and organ-at-risk sparing defined by a collection of specific metrics, each with its own score function that encapsulates both the (1) priority (i.e., weight) and (2) objectives (i.e., minimal to ideal performance) of each metric.
Advanced Scoring Mechanism
ProKnow plan studies most often use a scoring method and plan metrics that are far more advanced, comprehensive, and challenging than typical pass/fail goals based on published tolerances. A set of metrics will be extracted and scored with a weighted score function that defines minimal requirements up to the ideal result. For a general description of metric types in this and other studies, please visit our Metric Library.
Please see the "Plan Metric Summary" pages in the study-specific instructions for details about metrics and their objectives.
Statistical Analysis & Performance vs. Population
Each and every participant will be able to return to their ProKnow portal at any time to interactively study their individual results vs. the population, not only for the plan composite score, but also for any individual metric component.
This "Population Results" feature will be enabled (i.e. turned on) after one of two conditions is met:
- After the study data collection phase is over and ProKnow has "released" the results, or
- If an organization's study manager decides to dynamically display the population results during data collection