MBFusion：用于癌症诊断和预后的多模式平衡融合和多任务学习
MBFusion: Multi-modal balanced fusion and multi-task learning for cancer diagnosis and prognosis
影响因子:6.30000
分区:医学2区 / 数学与计算生物学1区 生物学2区 计算机：跨学科应用2区 工程：生物医学2区
发表日期:2024 Oct
作者:
Ziye Zhang, Wendong Yin, Shijin Wang, Xiaorou Zheng, Shoubin Dong
摘要
病理图像和分子法是预测诊断和预后的重要信息。两种异质模态数据包含互补信息，两种模态的有效融合可以更好地揭示癌症的复杂机制。但是，由于表示不同的表示方法，不同任务中不同模态的表达强度差异很大，因此许多多模式融合不会取得最佳结果。在本文中，提出了MBFusion，以实现多个任务，例如通过多模式平衡融合来预测诊断和预后。 MBFusion框架使用两种专门构建的图形卷积网络来提取分子幻象数据的特征，并使用Resnet来提取病理图像数据的特征，并通过使用注意力和聚类来保留重要的深度特征，从而有效地提高了两种特征的表现，从而使其表达能力平衡和可比性。然后，通过交叉意见变压器融合了这两个模态数据的功能，并使用融合功能来通过使用多任务学习来学习癌症亚型分类和生存分析的两项任务。在本文中，在两个公共癌症数据集上比较了MBFusion和其他最先进的方法，而MBFusion通过三种评估指标显示了高达10.1％的改善。在消融实验中，MBFusion探讨了每个模态数据和每个框架模块对性能的贡献。此外，详细解释了MBFusion的解释性，以显示应用的价值。
Abstract
Pathological images and molecular omics are important information for predicting diagnosis and prognosis. The two kinds of heterogeneous modal data contain complementary information, and the effective fusion of the two modals can better reveal the complex mechanisms of cancer. However, due to the different representation learning methods, the expression strength of different modals in different tasks varies greatly, so that many multimodal fusions do not achieve the best results. In this paper, MBFusion is proposed, to achieve multiple tasks such as prediction of diagnosis and prognosis through multi-modal balanced fusion. The MBFusion framework uses two kinds of specially constructed graph convolutional network to extract the features of molecular omics data, and uses ResNet to extract the features of pathological image data and retain important deep features by using attention and clustering, which effectively improves both kinds of the features representation, making their expressive ability balanced and comparable. The features of these two modal data are then fused through cross-attention Transformer, and the fused features are used to learn both tasks of cancer subtype classification and survival analysis by using multi-task learning. In this paper, MBFusion and other state of the art methods are compared on two public cancer datasets, and MBFusion shows an improvement of up to 10.1% by three kinds of evaluation metrics. In the ablation experiment, MBFusion explores the contribution of each modal data and each framework module to the performance. Furthermore, the interpretability of MBFusion is explained in detail to show the value of application.