From spots to cells: Cell segmentation in spatial transcriptomics with BOMS

Abstract

Imaging-based Spatial Transcriptomics methods enable the study of gene expression and regulation in complex tissues at subcellular resolution. However, inaccurate cell segmentation procedures lead to misassignment of mRNAs to individual cells which can introduce errors in downstream analysis. Current methods estimate cell boundaries using auxiliary DAPI/Poly(A) stains. These stains can be difficult to segment, thus requiring manual tuning of the method, and not all mRNA molecules may be assigned to the correct cells. We describe a new method, based on mean shift, that segments the cells based on the spatial locations and the gene labels of the mRNA spots without requiring any auxiliary images. We evaluate the performance of BOMS across various publicly available datasets and demonstrate that it achieves comparable results to the best existing method while being simple to implement and significantly faster in execution. Open-source code is available at https://github.com/sciai-lab/boms .