TY - JOUR

T1 - Cell segmentation in imaging-based spatial transcriptomics

AU - Petukhov, Viktor

AU - Xu, Rosalind J.

AU - Soldatov, Ruslan A.

AU - Cadinu, Paolo

AU - Khodosevich, Konstantin

AU - Moffitt, Jeffrey R.

AU - Kharchenko, Peter V.

N1 - Publisher Copyright: © 2021, The Author(s), under exclusive licence to Springer Nature America, Inc.

PY - 2022

Y1 - 2022

N2 - Single-molecule spatial transcriptomics protocols based on in situ sequencing or multiplexed RNA fluorescent hybridization can reveal detailed tissue organization. However, distinguishing the boundaries of individual cells in such data is challenging and can hamper downstream analysis. Current methods generally approximate cells positions using nuclei stains. We describe a segmentation method, Baysor, that optimizes two-dimensional (2D) or three-dimensional (3D) cell boundaries considering joint likelihood of transcriptional composition and cell morphology. While Baysor can take into account segmentation based on co-stains, it can also perform segmentation based on the detected transcripts alone. To evaluate performance, we extend multiplexed error-robust fluorescence in situ hybridization (MERFISH) to incorporate immunostaining of cell boundaries. Using this and other benchmarks, we show that Baysor segmentation can, in some cases, nearly double the number of cells compared to existing tools while reducing segmentation artifacts. We demonstrate that Baysor performs well on data acquired using five different protocols, making it a useful general tool for analysis of imaging-based spatial transcriptomics.

AB - Single-molecule spatial transcriptomics protocols based on in situ sequencing or multiplexed RNA fluorescent hybridization can reveal detailed tissue organization. However, distinguishing the boundaries of individual cells in such data is challenging and can hamper downstream analysis. Current methods generally approximate cells positions using nuclei stains. We describe a segmentation method, Baysor, that optimizes two-dimensional (2D) or three-dimensional (3D) cell boundaries considering joint likelihood of transcriptional composition and cell morphology. While Baysor can take into account segmentation based on co-stains, it can also perform segmentation based on the detected transcripts alone. To evaluate performance, we extend multiplexed error-robust fluorescence in situ hybridization (MERFISH) to incorporate immunostaining of cell boundaries. Using this and other benchmarks, we show that Baysor segmentation can, in some cases, nearly double the number of cells compared to existing tools while reducing segmentation artifacts. We demonstrate that Baysor performs well on data acquired using five different protocols, making it a useful general tool for analysis of imaging-based spatial transcriptomics.

U2 - 10.1038/s41587-021-01044-w

DO - 10.1038/s41587-021-01044-w

M3 - Journal article

C2 - 34650268

AN - SCOPUS:85117193174

VL - 40

SP - 345

EP - 354

JO - Nature Biotechnology

JF - Nature Biotechnology

SN - 1087-0156

ER -