Performance assessment of an algorithm-based decision-making platform for patient prioritization during cancer care.

Abstract

e13639 Background: More than 75% of patients receiving cancer treatment suffer Adverse Events (AE), and more than 45% develops at least one severe AE. Remote patient monitoring tools have demonstrated that proactively and timely addressing symptoms optimize patient care, reducing unplanned hospital visits and improving patient survival and quality of life (QoL). Step Oncology is a federated health infrastructure powered by Artificial Intelligence to support evidence-based decision-making in-patient care. The objective of this study was to validate the algorithm that Step Oncology® for patients features to identify patients at risk using captured patient’s real-time real-world data to support informed decisions in patient-centric care. Methods: Patients diagnosed with cancer were asked to use the patient application of Step Oncology to report real-time treatment-related symptoms and QoL questionnaires. Patient Reported Outcomes Measurements (PROMs) are real-time categorized by the algorithm in a 5-level color scale to indicate priority, from green (less severe) to dark red (most severe), according to CTCAE standard. Patients were prioritized based on their risk, allowing healthcare teams to proactively take care of patients in greatest need. To validate algorithm performance, three expert clinicians were asked to assign a risk level to the information reported by patients without having access to algorithm classifications. Accuracy, True Negative Rate (Specificity) and True Positive Rate (Sensitivity) were calculated from a total of 1254 severity assessments. Results: Out of 1254 severity classifications, 1152 were an exact match between the algorithm and the expert classifications, accomplishing a 91,87% (95% CI: 90,27% - 93,30%) accuracy. Considering every possible categorization has the same weight on the algorithm performance, specificity was 97,42% (95% CI: 96,68% - 97,90%) and sensitivity was 88,60% (95% CI: 84,46% - 91,44%). Out of 102 nonequivalent classifications, in 49% the algorithm categorized severity on a conservative approach compared to expert classification. Main non-equivalence reason was differences between the algorithm and clinicians in the time span considered for severity calculation. Conclusions: Safety and performance of Step Oncology for patients has been clinically validated with outstanding results against clinical expert evaluation. Its patient early detection of complications and categorization abilities have been demonstrated to be highly accurate. Step Oncology use in routine clinical practice has been proven to be effective in routine care patient monitoring and prioritization during and after treatment.