多重免疫荧光（mIF）染色，例如 CODEX 和 MIBI，在疾病诊断、生物研究和药物开发等各个领域具有重要的临床价值。然而，这些技术往往受到高时间和成本要求的阻碍。在这里，我们提出了一种基于多模态注意力的虚拟 mIF 染色（MAS）系统，该系统利用深度学习模型从双模态非抗体中提取潜在的抗体相关特征-染色荧光成像，特别是自发荧光 (AF) 和 DAPI 成像。 MAS 系统利用自注意力和多注意力学习机制，同时生成胃癌中多种生存相关生物标志物的 mIF 预测。来自 94 名胃癌患者的 180 张病理切片的实验结果证明了 MAS 系统在胃癌中的效率和一致的性能。癌症和非癌症胃组织。此外，我们还展示了七个胃癌相关生物标志物（包括 CD3、CD20、FOXP3、PD1、CD8、CD163 和 PD-L1）的虚拟 mIF 图像的预后准确性，这与标准 mIF 染色获得的结果相当。 MAS系统快速生成可靠的多重染色，大大降低mIF成本并改善临床工作流程。斯坦福2022年HAI种子资助；美国国立卫生研究院 1R01CA256890。版权所有 © 2024 作者。由 Elsevier B.V. 出版。保留所有权利。

Multiplexed immunofluorescence (mIF) staining, such as CODEX and MIBI, holds significant clinical value for various fields, such as disease diagnosis, biological research, and drug development. However, these techniques are often hindered by high time and cost requirements.Here we present a Multimodal-Attention-based virtual mIF Staining (MAS) system that utilises a deep learning model to extract potential antibody-related features from dual-modal non-antibody-stained fluorescence imaging, specifically autofluorescence (AF) and DAPI imaging. The MAS system simultaneously generates predictions of mIF with multiple survival-associated biomarkers in gastric cancer using self- and multi-attention learning mechanisms.Experimental results with 180 pathological slides from 94 patients with gastric cancer demonstrate the efficiency and consistent performance of the MAS system in both cancer and noncancer gastric tissues. Furthermore, we showcase the prognostic accuracy of the virtual mIF images of seven gastric cancer related biomarkers, including CD3, CD20, FOXP3, PD1, CD8, CD163, and PD-L1, which is comparable to those obtained from the standard mIF staining.The MAS system rapidly generates reliable multiplexed staining, greatly reducing the cost of mIF and improving clinical workflow.Stanford 2022 HAI Seed Grant; National Institutes of Health 1R01CA256890.Copyright © 2024 The Author(s). Published by Elsevier B.V. All rights reserved.