研究动态
Articles below are published ahead of final publication in an issue. Please cite articles in the following format: authors, (year), title, journal, DOI.

多模态共同注意融合网络与在线数据增强用于癌症亚型分类。

Multimodal Co-attention Fusion Network with Online Data Augmentation for Cancer Subtype Classification.

发表日期:2024 May 27
作者: Saisai Ding, Juncheng Li, Jun Wang, Shihui Ying, Jun Shi
来源: IEEE TRANSACTIONS ON MEDICAL IMAGING

摘要:

在计算病理学中准确诊断癌症亚型以实现个性化癌症治疗是一项重要任务。最近的研究表明,多模态数据(例如全幻灯片图像(WSI)和多组学数据)的结合可以实现更准确的诊断。然而,由于多模态数据之间的异质性,以及多模态患者数据不足导致的性能下降,稳健的癌症诊断仍然具有挑战性。在这项工作中,我们提出了一种新颖的多模式共同注意融合网络(MCFN)和在线数据增强(ODA),用于癌症亚型分类。具体来说,提出了一种多模态相互引导共同注意(MMC)模块来有效地执行密集的多模态交互。它使多模态数据能够在集成过程中相互指导和校准,以减轻模态间和模内的异质性。随后,开发了自归一化网络(SNN)-Mixer,以允许不同组学数据之间的信息通信,并缓解多组学数据中的高维小样本问题。最重要的是,为了弥补模型训练的多模态样本不足的问题,我们在 MCFN 中提出了 ODA 模块。 ODA模块利用多模态知识来指导WSI的数据增强,并在模型训练期间最大化数据多样性。在公共 TCGA 数据集上进行了大量实验。实验结果表明,所提出的 MCFN 优于所有比较算法,表明其有效性。
It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.