准确的组织病理学亚型预测对于癌症诊断和肿瘤微环境分析具有临床意义。然而，实现准确的组织病理学亚型预测是一项具有挑战性的任务，因为（1）组织病理学图像的实例级区分，（2）组织病理学图像在形状和染色质纹理方面的类间和类内差异较小，以及（3 ）不同图像上的异构特征分布。在本文中，我们将亚型预测制定为细粒度表示学习，并提出了一种新颖的多实例选择变压器（MIST）框架，有效地实现了准确的组织病理学亚型预测。所提出的 MIST 设计了一种有效的选择性自注意力机制，具有多实例学习（MIL）和视觉变换器（ViT），以自适应识别信息实例以进行细粒度表示。创新的是，MIST 根据每个实例与实例和包的交互，赋予每个实例对包表示的不同贡献。具体来说，具有选择性多头自注意力（S-MSA）的 SiT 模块经过精心设计，可以通过对实例到实例的交互进行建模来识别代表性实例。相反，提出了具有信息瓶颈的 MIFD 模块，通过对与所选实例的实例到袋子的交互进行建模来学习组织病理学图像的判别性细粒度表示。对五个临床基准的大量实验表明，MIST 实现了准确的组织病理学亚型预测，并获得了最先进的性能，准确度为 0.936。 MIST 在处理细粒度医学图像分析（例如临床应用中的组织病理学亚型预测）方面显示出巨大潜力。版权所有 © 2024 Elsevier B.V。保留所有权利。

Accurate histopathological subtype prediction is clinically significant for cancer diagnosis and tumor microenvironment analysis. However, achieving accurate histopathological subtype prediction is a challenging task due to (1) instance-level discrimination of histopathological images, (2) low inter-class and large intra-class variances among histopathological images in their shape and chromatin texture, and (3) heterogeneous feature distribution over different images. In this paper, we formulate subtype prediction as fine-grained representation learning and propose a novel multi-instance selective transformer (MIST) framework, effectively achieving accurate histopathological subtype prediction. The proposed MIST designs an effective selective self-attention mechanism with multi-instance learning (MIL) and vision transformer (ViT) to adaptive identify informative instances for fine-grained representation. Innovatively, the MIST entrusts each instance with different contributions to the bag representation based on its interactions with instances and bags. Specifically, a SiT module with selective multi-head self-attention (S-MSA) is well-designed to identify the representative instances by modeling the instance-to-instance interactions. On the contrary, a MIFD module with the information bottleneck is proposed to learn the discriminative fine-grained representation for histopathological images by modeling instance-to-bag interactions with the selected instances. Substantial experiments on five clinical benchmarks demonstrate that the MIST achieves accurate histopathological subtype prediction and obtains state-of-the-art performance with an accuracy of 0.936. The MIST shows great potential to handle fine-grained medical image analysis, such as histopathological subtype prediction in clinical applications.Copyright © 2024 Elsevier B.V. All rights reserved.