PathoDuet:H 病理幻灯片分析的基础模型
PathoDuet: Foundation models for pathological slide analysis of H&E and IHC stains.
发表日期:2024 Jul 31
作者:
Shengyi Hua, Fang Yan, Tianle Shen, Lei Ma, Xiaofan Zhang
来源:
MEDICAL IMAGE ANALYSIS
摘要:
大量数字化组织病理学数据显示了通过自我监督学习方法开发病理基础模型的广阔前景。使用这些方法预训练的基础模型可以为下游任务奠定良好的基础。然而,自然图像和组织病理学图像之间的差距阻碍了现有方法的直接应用。在这项工作中,我们提出了 PathoDuet,一系列组织病理学图像的预训练模型,以及组织病理学中新的自我监督学习框架。该框架的特点是新引入的借口标记和后来的任务提升器,以明确利用图像之间的某些关系,例如多个放大倍数和多个污点。在此基础上,设计了跨尺度定位和跨染色转移两个借口任务,在苏木精和曙红(H
Large amounts of digitized histopathological data display a promising future for developing pathological foundation models via self-supervised learning methods. Foundation models pretrained with these methods serve as a good basis for downstream tasks. However, the gap between natural and histopathological images hinders the direct application of existing methods. In this work, we present PathoDuet, a series of pretrained models on histopathological images, and a new self-supervised learning framework in histopathology. The framework is featured by a newly-introduced pretext token and later task raisers to explicitly utilize certain relations between images, like multiple magnifications and multiple stains. Based on this, two pretext tasks, cross-scale positioning and cross-stain transferring, are designed to pretrain the model on Hematoxylin and Eosin (H&E) images and transfer the model to immunohistochemistry (IHC) images, respectively. To validate the efficacy of our models, we evaluate the performance over a wide variety of downstream tasks, including patch-level colorectal cancer subtyping and whole slide image (WSI)-level classification in H&E field, together with expression level prediction of IHC marker, tumor identification and slide-level qualitative analysis in IHC field. The experimental results show the superiority of our models over most tasks and the efficacy of proposed pretext tasks. The codes and models are available at https://github.com/openmedlab/PathoDuet.Copyright © 2024 Elsevier B.V. All rights reserved.