Sunburst plot: A tool for interpretation and prediction of inter-reader variability relationships in BICR studies.

Abstract

e13690 Background: Blinded independent central review (BICR) is recommended by the USFDA to minimize evaluation bias and improve imaging assessment consistency. The "Double read with adjudication" model is preferred for oncology trials, ensuring uniform interpretation of imaging results. Disagreement rates in image evaluations range from 20% to 40%, regardless of medical specialty or technology. Reader performance can be influenced by factors such as experience, fatigue, workload, stress, biases and random measurement variability. Methods: There was double read for 482 subjects by five board-certified radiologists using RECIST 1.1 criteria. A total of 1,282 follow-up visits, with 2,564 assessments, were analyzed. The data was visualized using a "sunburst plot" created with Python and data visualization libraries. This tool can be configured for automated monitoring of reader performance, addressing a key challenge in current BICR setups. The sunburst plot begins at the center or core and expands outward, forming distinct layers: Core/Center Layer: Represents total number of follow-up reads performed in the study, the foundation for subsequent layers. First/Inner Layer: Shows the number of follow-up reads by each reader. Second/Middle Layer: Indicates the number of follow-up reads shared with any particular reader. Third/Outer Layer: Depicts the outcomes of adjudications, detailing whether they were won, lost, or remained unadjudicated between the two readers. This visualization effectively illustrates all inter-reader relationships and adjudication outcomes in the study. Results: Table 1 provides an overview of the follow-up reads for Reader 2. Reader 2 conducted a total of 640 follow-up reads, as shown in Layer 1. In Layer 2, Reader 2 shared 263 reads with Reader 4, of which 237 (90.11%) were in agreement and did not require adjudication. With Reader 3, Reader 2 shared 56 reads, with 50 (89.23%) remaining unadjudicated. Reader 2 also shared 124 reads with Reader 1, where 94 (75.8%) were unadjudicated, 19 (15.32%) were won, and 11 (8.87%) were lost. Lastly, Reader 2 shared 197 reads with Reader 5, with 141 (71.57%) not requiring adjudication and amongst those adjudicated, winning 44 (22.33%) and losing 12 (6.09%) of them. Conclusions: The sunburst plot provides an innovative, hierarchical visualization, offering fresh insights over traditional methods. Combined with robust statistical analysis, it enhances BICR study assessments and informs better decision-making. Here's a table summarizing the follow-up reads for Reader 2. Reader Pair Total Reads No ADJ Won Lost R4 263 237 (90%) 8 (3%) 18 (7%) R3 56 50 (89%) 0 (0%) 6 (11%) R1 124 94 (76%) 19 (15%) 11 (9%) R5 197 141 (72%) 44 (22%) 12 (6%) This table shows Reader 2's adjudicated and unadjudicated reads. A sunburst plot consolidates all reader interactions, simplifying the interpretation of complex data patterns and enhancing decision-making.