AI immuno-profiling: Predicting T and B cells directly from H&E-stained slides.

Abstract

e13592 Background: Tumor-infiltrating immune cells, especially T (CD3+) and B (CD20+) lymphocytes, are important prognostic and predictive biomarkers in oncology. However, traditional immunohistochemistry (IHC) and immunofluorescence (IF) methods are labor-intensive. By leveraging a robust dataset linking hematoxylin and eosin (H&E) images with the corresponding IF-based ground truth (GT) masks, we aimed to develop and evaluate an efficient semantic segmentation model capable of identifying CD3+ and CD20+ immune cells directly from H&E-stained slides. Methods: We constructed an H&E-stained whole-slide image (WSI) dataset spanning various cancer types, each paired with IF-derived GT masks. These masks were generated by staining the same WSIs with H&E and then de-staining and re-staining them with IF, allowing precise alignment of the target cells in both H&E and IF images. Consequently, we obtained separate H&E–IF image pairs for the CD3+ and CD20+ datasets, each derived from different WSIs. We extracted patches of 984 × 984 pixels, yielding 42,122 patches for CD3+ and 43,901 for CD20+, which were split into training, validation, and test sets. First, we trained two independent models—one to predict CD3+ cells and another to predict CD20+ cells—from the H&E images. We then applied these CD3+ and CD20+ models to each other’s H&E training and validation sets to create combined GT masks containing both CD3+ and CD20+ labels. For instance, when applying the CD20+ model to the CD3+ dataset, regions already labeled as CD3+ remained unchanged, while any newly predicted cells that did not overlap with those CD3+ labels were assigned CD20+ labels. Finally, using these combined GT masks, we developed a distillation model capable of predicting both CD3+ and CD20+ cells directly from H&E-stained images. We used the original IF-derived independent CD3+ and CD20+ datasets to evaluate the model’s predictive performance for each marker. We then calculated the Dice and correlation coefficients for the number of positive cells between the predicted and GT masks. Results: We used 4,212 images for the CD3+ test set and 4,390 patch images for the CD20+ test set. The Dice coefficients for CD3+ and CD20+ cells were 0.520 and 0.592, respectively. The correlation coefficients between the predicted and GT masks were 0.742 for CD3+ cells and 0.946 for CD20+ cells. Conclusions: Our results demonstrate that deep learning-based semantic segmentation models can predict the distribution of CD3+ and CD20+ immune cells directly from H&E-stained images, achieving high concordance with the IF-derived GT. By distilling two separate models into a single network, we streamlined the process to enable simultaneous detection of both cell types. We further speculate that the model not only captures cellular morphology but also surrounding features specific to T and B cells, thereby enhancing classification accuracy.