单独使用单一模式数据通常无法捕捉患者之间复杂的异质性，包括抗 HER2 治疗耐药性的变异性以及 HER2 阳性胃癌 (GC) 联合治疗方案的结果。许多研究尚未充分考虑这种方式缺陷。此外，人工智能在预测治疗反应方面的应用，特别是在GC等复杂疾病中，仍处于起步阶段。因此，本研究旨在采用综合分析方法准确预测 HER2 阳性 GC 患者对抗 HER2 治疗或抗 HER2 联合免疫治疗的治疗反应。我们收集了 429 名患者的多模式数据，包括放射学、病理学和临床信息：310 名患者接受抗 HER2 治疗，119 名患者接受抗 HER2 和抗 PD-1/PD-L1 抑制剂联合治疗免疫疗法。我们引入了一种深度学习模型，称为多模态模型 (MuMo)，它集成这些数据以做出精确的治疗反应预测。 MuMo 抗 HER2 治疗的曲线下面积得分为 0.821，联合免疫治疗的曲线下面积得分为 0.914。此外，被 MuMo 分类为低风险的患者表现出显着延长的无进展生存期和总生存期（对数秩检验，P<0.05）。这些发现不仅强调了多模式数据分析在增强 HER2 阳性胃癌治疗评估和个性化医疗方面的重要性，而且还强调了我们模型的潜力和临床价值。© 2024。作者。

The sole use of single modality data often fails to capture the complex heterogeneity among patients, including the variability in resistance to anti-HER2 therapy and outcomes of combined treatment regimens, for the treatment of HER2-positive gastric cancer (GC). This modality deficit has not been fully considered in many studies. Furthermore, the application of artificial intelligence in predicting the treatment response, particularly in complex diseases such as GC, is still in its infancy. Therefore, this study aimed to use a comprehensive analytic approach to accurately predict treatment responses to anti-HER2 therapy or anti-HER2 combined immunotherapy in patients with HER2-positive GC. We collected multi-modal data, comprising radiology, pathology, and clinical information from a cohort of 429 patients: 310 treated with anti-HER2 therapy and 119 treated with a combination of anti-HER2 and anti-PD-1/PD-L1 inhibitors immunotherapy. We introduced a deep learning model, called the Multi-Modal model (MuMo), that integrates these data to make precise treatment response predictions. MuMo achieved an area under the curve score of 0.821 for anti-HER2 therapy and 0.914 for combined immunotherapy. Moreover, patients classified as low-risk by MuMo exhibited significantly prolonged progression-free survival and overall survival (log-rank test, P < 0.05). These findings not only highlight the significance of multi-modal data analysis in enhancing treatment evaluation and personalized medicine for HER2-positive gastric cancer, but also the potential and clinical value of our model.© 2024. The Author(s).