Plan Scoring Overview
Each ProKnow Plan Study is configured with a scoring method, and there are several options (see: Plan Scoring Details, below). All methods share the traits of: 1) identifying a list of all the pertinent metrics to be extracted, scored against goals, and studied; 2) setting a unique and objective score function or set of rules/objectives per-metric to reward success; and 3) creating a composite plan score over all metrics.
A metric can be described as "something that's important" and is chosen from a vast library of options ranging from simple min/max/mean, to DVH points (dose-at-volume, volume-at-dose, etc.), to calculated metrics (conformation number, conformality index, homogeneity index, etc.), and the list goes on.
After you extract each metric, you can then ask the question, "How do we measure performance for this metric?" and that is in the per-metric score function, i.e. how each metric earns some kind of objective score or performance level. This per-metric scoring options are explained in the next section, called "Plan Scoring Details," but for any methodology, after all metrics' scores are calculated, they are added up to yield the overall plan score. Therefore, all available scoring methods produce a summary/composite score over all metrics.
Plan Scoring Details
Currently a plan study organizer can select one of three scoring options:
- PlanIQ Method. This method allows each metric to have a numeric score function customized for each metric. The score function has no restrictions other than it must be a true function (i.e. no metric result, or x-value, can have more than one score, or y-value, assigned). The functions can be as simple as lines from a score of zero at the minimum required result to a score of max at the ideal result, or they can be as complex as multi-sloped lines, curves, and even assign negative scores over regions if you wish to create penalty regions. This method is allowed on ProKnow via license agreement with the PlanIQ product.
- Performance Bin Method. This method allows you to set up any number of performance bins for each metric, where each bin represents a range of metric results and the bin value is assigned both a description (e.g. "Acceptable") and a score level. This method is nice if you have a standard system of goal objectives defined by something like: Unacceptable, Marginal, Acceptable, Good, and Ideal. The composite score will be a tally of all the scores for the bins achieved.
- Simple Pass/Fail Method. Each metric has a single threshold that determines the pass/fail status of the result. The composite plan score is basically a tally of the number of passing metrics vs. the total number of metrics, e.g. 13 metrics passed out of 14.
- Metrics will depend on ROI names matching those specified in the plan quality scoring algorithm. This is guaranteed for ProKnow plan studies because the original DICOM RT Structure Set provided with each study is what is used to generate DVHs and metrics, so if you edit or add contours during your planning, or even if you change names, it will not cause any problems because you will not upload your edited structures.
- Any metric row that has the label *NLS means that the score function uses non-linear scoring, so the score is not a simple line with a zero for the minimal requirement and max score for the ideal result. Please consult the plan study instructional PDF to see graphs of each metric's score function.
Nelms BE, Robinson G, Markham J, Velasco K, Boyd S, Narayan S, Wheeler J, Sobczak M. Variation in external beam treatment plan quality: An inter-institutional study of planners and planning systems. Practical Radiation Oncology 2012 Oct;2(4):296-305.