LIMPACAT: Multi-omics attention transformer for immune prediction in liver cancer using whole-slide imaging

Abstract

Characterizing the tumor immune microenvironment from histopathological images offers opportunities for ex vivo immune profiling and prognostic assessment. However, the TCGA-LIHC dataset lacks direct immune cell composition data. Therefore, this study aims to introduce Liver Immune Microenvironment Prediction and Classification Attention Transformer (LIMPACAT), a deep learning framework that leverages whole-slide images (WSIs) to predict immune cell levels relevant to hepatocellular carcinoma (HCC) prognosis. Immune cell compositions were inferred using a deconvolution approach, with bulk RNA-seq profiles simulated from liver-specific single-cell RNA sequencing data and processed with multiple normalization methods. These inferred compositions served as supervision signals to train a multiple instance learning model with an attention transformer. LIMPACAT exhibited ~80% accuracy in classifying immune cell levels from HCC WSIs, showing strong concordance between model prediction and deconvolution-derived estimates. These findings suggest that WSIs can serve as a proxy for immune profiling, facilitating pathology-based tumor microenvironment assessment and supporting personalized therapeutic strategies.