Discordance, accuracy and reproducibility study of pathologists’ diagnosis of melanoma and melanocytic tumors
Abstract
Abstract Accurate melanoma diagnosis is crucial for patient outcomes and reliability of AI diagnostic tools. We assess interrater variability among eight expert pathologists reviewing histopathological images and clinical metadata of 792 melanoma-suspicious lesions prospectively collected at eight German hospitals. Moreover, we provide access to the largest panel-validated dataset featuring dermoscopic and histopathological images with metadata. Complete agreement is achieved in 53.5% of cases (424/792), and a majority vote ( ≥ five pathologists) in 90.9% (720/792). Considerable discordance is observed for non-invasive melanomas (complete agreement in only 10/73 cases). The expert panel disagrees with the local pathologists’ and dermatologists’ diagnoses in 14.9% and 33.5% of cases, respectively. This variability highlights the diagnostic challenges of early-stage melanomas and the need to reconsider how ground truth is established in routine care and AI research. Including at least two pathologists or virtual panels may contribute to more consistent diagnostic results.
Article Details
Authors (47)
Sarah Haggenmüller
Christoph Wies
Julia Abels
Jana T. Winterstein
Lukas Heinlein
Carina Nogueira Garcia
Jochen S. Utikal
Sebastian A. Wohlfeil
Friedegund Meier
Sarah Hobelsberger
Frank F. Gellrich
Mildred Sergon
Axel Hauschild
Department of Dermatology, University Hospital, Kiel, Germany
Lars E. French
Lucie Heinzerling
Justin G. Schlager
Kamran Ghoreschi
Max Schlaak
Franz J. Hilke
Gabriela Poch
Sören Korsing
Cosimo Sarfert
Carola Berking
Markus V. Heppt
Michael Erdmann
Department of Dermatology, Uniklinikum Erlangen, Friedrich Alexander University
Sebastian Haferkamp
Konstantin Drexler
Dirk Schadendorf
Wiebke Sondermann
Matthias Goebeler
Bastian Schilling
Jakob Nikolas Kather
Stefan Fröhling
Mar Llamas-Velasco
Luis C. Requena
Gerardo Ferrara
Maite Fernandez-Figueras
Sylvie Fraitag
Laboratoire d’Anatomie Pathologique, Hôpital Necker-Enfants Malades, Assistance Publique Hôpitaux de Paris
Cornelia S. L. Müller
Hans Starz
Heinz Kutzner
Raymond Barnhill
Richard Carr
Kenneth S. Resnik
Stephan Alexander Braun
Tim Holland-Letz
Titus J. Brinker