BioFuse: an embedding fusion framework for biomedical foundation models

Abstract

The biomedical field has witnessed a surge in pre-trained foundation models excelling in specific subdomains such as radiology and histopathology. While integrating these models promises a more comprehensive understanding of biomedical data, it poses challenges in model compatibility and feature fusion. We present BioFuse, a novel open-source framework designed to generate optimised biomedical embeddings. BioFuse utilises a pool of 9 state-of-the-art (SOTA) foundation models to create task-specific embeddings. It employs grid search to automatically identify the optimal combination of models, fusing their embeddings through vector concatenation. On the MedMNIST+ benchmark, using XGBoost as the downstream classifier, BioFuse outperforms several existing methods, achieving SOTA AUC in 5/12 datasets, while maintaining near-SOTA performance across most remaining datasets. Remarkably, our experiments reveal unexpected cross-modal capabilities, with histopathology and radiology models showing strong performance when applied to other imaging modalities. BioFuse features a high-level API for immediate deployment and an extensible architecture to incorporate future models and fusion techniques. We anticipate BioFuse will not only enhance the utility of foundation models in biomedicine but also open new avenues for uncovering cross-modal relationships in biomedical data.