Clinical and technological validation of locally-installed, privacy-compliant, and hematology-tailored ambient AI platform for clinical report transcription assistance and clinical trial patient management

Abstract

Abstract Background Effective communication between clinicians and patients is vital in hematology for personalized treatment. However, traditional practice is time-consuming and prioritizes clinical data over patient-reported outcomes (PROs), limiting direct patient interaction. Existing speech-to-text AI-based solutions promise to solve these issues but still present significant barriers: these solutions are often costly due to reliance on third-party models, lack of specialization on specific medical domains, are not easy to install on-premise and integrated with hospital systems, raising privacy and compliance issues. To overcome these limitations, we have developed a hematology-tailored platform called Ambient AI. This work presents results of an ongoing prospective study, currently under validation with real patients. The aim is to assess the platform's ability to support medical documentation and optimize patient enrollment and management in clinical trials within a real-world hematology setting. Methods Ambient AI records, transcribes in real-time clinician-patient conversations and extracts clinical information while filtering out non-essential details. It leverages four key modules: real-time conversation transcription, an AI-powered system for generating medical reports, a clinical data extraction module for research dataset and a dedicated component for optimizing patient management in clinical trials. The platform generates structured medical reports for physician review, modification, and approval. Once validated, reports are saved, and intermediate data is deleted to ensure privacy and compliance. The platform integrates local AI models using speech-to-text technology and open-source large language models (LLM) fine-tuned for hematology. The validation framework assesses: 1) technological performance, using metrics as Jaccard Similarity (JS) for text accuracy and Word Mover Distance (WMD) for contextual understanding, and an LLM as a judge for expert assessment; 2) clinical fidelity and physician satisfaction; and 3) patient experience and engagement measured through surveys. Results A preliminary analysis of 100 simulated reports, using an open-source Gemma3 12B quantized model, showed that AI-assisted transcription improves documentation efficiency by reducing manual data entry. AI-generated reports had high clinical relevance, requiring minimal physician edits. Strong transcription accuracy (JS 0.85) and effective contextual interpretation (WMD 0.81) were observed. However, some medical terminology or sentences were not properly captured. To address this, the model was fine-tuned with domain knowledge, and a sample of 30 reports from the initial 100 was used for performance assessment. Results showed improved metrics (JS 0.88, WMD 0.73). For a final assessment, a team of five physicians was presented with transcribed text and two reports, and tasked with selecting the more appropriate. The fine-tuned model's report was preferred on average in 28 out of 30 instances (93.3%). The Ambient AI platform validation is currently undergoing at Humanitas Research Hospital, Italy, involving 1,000 patients. Early results indicate increased physician satisfaction, improved workflow efficiency, and enhanced patient engagement. Moreover, we developed an AI tool leveraging an LLM for automatic data extraction from medical records, enabling structured dataset generation for research. To optimize clinical trial management we implemented a module that: 1) automatically scans medical records to identify eligible patients for ongoing clinical trials; 2) reports and grades adverse events during trial visits according to international guidelines; 3) suggests potential drug modification schedules based on study protocols; and 4) generates automated patient visit reports for Clinical Research Organizations (CROs), reducing the need for in-person monitoring visits. Conclusion The Ambient AI platform aims to improve hematology workflows by enhancing communication, streamlining documentation, and integrating PROs into clinical practice. This fosters better doctor-patient interaction, a key aspect of patient-centered care, providing support and empowering individuals. Preliminary findings show its effectiveness in accurate data collection, with ongoing validation. The technology may also facilitate patient selection and management in clinical trials, supporting precision medicine.