Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology

作者：Wenhao Tang, Fengtao Zhou, Sheng Huang, Xiang Zhu, Yi Zhang, Бо Лю · 年份：2024 · DOI：10.1109/cvpr52733.2024.01078 · 被引用次数：85 · 研究领域：AI in cancer detection、Digital Imaging for Blood Diseases、Radiomics and Machine Learning in Medical Imaging

Multiple instance learning (MIL) is the most widely used framework in computational pathology, encompassing sub-typing, diagnosis, prognosis, and more. However, the ex-isting MIL paradigm typically requires an offline instance feature extractor, such as a pre-trained ResNet or a foun-dation model. This approach lacks the capability for feature fine-tuning within the specific downstream tasks, limiting its adaptability and performance. To address this issue, we propose a Re-embedded Regional Transformer (R2T) for re-embedding the instance features online, which captures fine-grained local features and establishes connections across different regions. Unlike existing works that focus on pre-training powerful feature extractor or designing sophisticated instance aggregator, R2T is tailored to re-embed instance features online. It serves as a portable module that can seamlessly integrate into mainstream MIL models. Extensive experimental results on common computational pathology tasks validate that: 1) feature re-embedding improves the performance of MIL models based on ResNet-50 features to the level of foundation model features, and further enhances the performance of foundation model features; 2) the R2T can introduce more signifi-cant performance improvements to various MIL models; 3) R2T-MIL, as an R2T-enhanced AB-MIL, outperforms other latest methods by a large margin. The code is available at: https://github.com/DearCaat/RRT-MIL.