Exploring DORIS: A cancer-focused AI knowledge platform for diagnosis and treatment information.

Abstract

9035 Background: Clinicians in oncology have high demands on their time due to lack of sufficient staffing, leading to the need to work additional hours outside of practice, “pajama time”, and high levels of stress. Key approaches to improving the situation for clinicians include: (a) increasing staffing and resources, (b) improved teamwork, (c) utilization of ambient AI for notetaking, and (d) the use of AI assistants for medical knowledge access and interpretation. We’re focused on (d). Regarding the use of AI assistants, our informal survey of oncology fellows and attendees has revealed that over 90% of them are using a general AI chat tool to help them find key medical information. The use of a general AI chat tool is problematic since there is no medical review or curation, leading to errors and gaps in the information provided. Methods: Our team has built an alternative to general AI chat, called DORIS (Dynamic Oncology Reference Information System). The DORIS for breast cancer prototype was built in alignment with ASCO’s Guiding Principals for AI, and targets easy access to information around 5 pillars of medical knowledge needed for cancer care: molecular biomarkers, diagnostic testing, treatment pathways, drug-drug comparison, and clinical trial search. The system uses a Large Language Model (LLM) to facilitate access to a carefully curated set of medical documents covering the 5 pillars, including treatment pathways (with digitization of flow diagram logic), FDA, NIH and other publicly accessible datasets. To be accurate, the curation of DORIS’ knowledgebase needs to be kept up-to-date, while removing out-of-date information (something general AI platforms don’t do). To further increase the ease of use, DORIS includes different approaches to query for information while also using its embedded intelligence to suggest follow up questions. Results: Medical review of DORIS is ongoing. Early errors were identified related to omissions from curation. Other errors occurred due to bugs within the LLM, e.g. the responses suddenly shifted to Spanish. The reference materials have been expanded to reduce omissions and code implemented to double check responses from the LLM. We are on target for onboarding over 250 users for the early access program. Conclusions: Early user feedback has been positive with regards to relevance and utility. We will report on the first 6 months of use of this prototype platform, providing multiple metrics on how it is being used, and an analysis of user feedback on the value of the tool.