Research Accomplishments in Pathology AI

One major accomplishment was mapping tumor-infiltrating lymphocytes (TILs) in H&E-stained whole-slide images from The Cancer Genome Atlas (TCGA) to study the tumor–immune microenvironment. In a Cell Reports 2018 study, Saltz et al. processed 5,202 digitized slides across 13 cancer types to create TIL maps using a deep convolutional neural network that “computationally stained” regions containing lymphocytes. The network classified image patches to detect lymphocyte-rich areas, effectively highlighting immune cell infiltrates within tumors. These TIL maps were validated by showing strong agreement with pathologist annotations and with established molecular assays for T-cell density.

Crucially, the spatial patterns of TILs extracted from the slides were correlated with molecular and clinical data. Affinity propagation clustering revealed distinct local TIL spatial structures associated with patient overall survival. The study found that TIL density and spatial arrangement varied significantly across tumor types, immune subtypes, and tumor molecular subtypes. Certain TIL spatial patterns were enriched for specific T-cell subpopulations (from gene expression data) and linked to particular tumor genomic aberrations. In summary, this work demonstrated that routine pathology slides contain rich immunological information: by applying deep learning to map lymphocyte infiltration, one can connect tissue morphology with immune genomics and patient outcomes. This computational pathology approach to immune profiling provides a foundation for objective TIL quantification in diagnostics and for guiding immunotherapy decisions.

Another significant contribution is a patch-based CNN framework for classifying gigapixel whole-slide images (WSIs) into cancer subtypes. Hou et al. (CVPR 2016) recognized the impracticality of training a single CNN on entire WSIs due to their enormous size (often exceeding 10^9 pixels). Instead, they proposed breaking the WSI into smaller patches and training a patch-level classifier, then intelligently aggregating patch predictions to classify the whole slide. The core challenge was to combine patch outputs while accounting for the fact that not all patches are informative (e.g. many patches may be mostly background or benign tissue).

Their solution was a two-stage model: first a CNN was trained on image patches (e.g. 256×256 regions) to predict patch-level tumor subtype labels; second, a decision fusion model (e.g. logistic regression on the distribution of patch predictions) was trained to produce the slide-level diagnosis. Moreover, they introduced an Expectation-Maximization (EM) algorithm to iteratively identify and up-weight discriminative patches while down-weighting non-informative ones. This EM-based attention mechanism exploited spatial relationships between patches to refine which regions truly characterize the cancer subtype. They applied this approach to classify subtypes of glioma and non–small cell lung carcinoma, achieving accuracy on par with inter-pathologist agreement. Notably, in controlled experiments with smaller images, the patch-based CNN outperformed a conventional image-level CNN, confirming the benefit of preserving high-resolution detail.

Text here

Text here

Text here

Text here

Text here

1. J. Saltz et al., “Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images,” Cell Reports, vol. 23, no. 1, pp. 181–193.e7, Apr. 2018. DOI: 10.1016/j.celrep.2018.03.086
2. L. Hou et al., “Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification,” in Proc. CVPR, 2016, pp. 2424–2433. DOI: 10.1109/CVPR.2016.267
3. S. Yellapragada et al., “ZoomLDM: Latent Diffusion Model for Multi-Scale Image Generation,” in Proc. CVPR, 2025 (to appear). arXiv:2411.16969 [cs.CV] (Nov. 2024)
4. S. Kapse et al., “Attention De-sparsification Matters: Inducing Diversity in Digital Pathology Representation Learning,” Med. Image Anal., vol. 93, 103070, 2024. DOI: 10.1016/j.media.2023.103070
5. S. Kapse et al., “SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel Histopathology,” in Proc. CVPR, 2024, pp. 11226–11237. DOI: 10.1109/CVPR.2024.01145
6. S. Chakraborty et al., “Decoding the Visual Attention of Pathologists to Reveal Their Level of Expertise,” in Proc. MICCAI, LNCS 13433, 2022, pp. 90–100. DOI: 10.1007/978-3-031-16440-8_9
7. L. Hou et al., “Robust Histopathology Image Analysis: To Label or to Synthesize?,” in Proc. CVPR, 2019, pp. 8525–8534. DOI: 10.1109/CVPR.2019.00873
8. S. Abousamra et al., “Topology-Guided Multi-Class Cell Context Generation for Digital Pathology,” in Proc. CVPR, 2023, pp. 3323–3333. DOI: 10.1109/CVPR46375.2023.00330
9. X. Xu et al., “ViT-DAE: Transformer-Driven Diffusion Autoencoder for Histopathology Image Analysis,” in Proc. MICCAI Workshops (DGM4MICCAI), LNCS 14533, 2023, pp. 66–76. DOI: 10.1007/978-3-031-53767-7_7
10. P. Howlader et al., “Beyond Pixels: Semi-Supervised Semantic Segmentation with a Multi-Scale Patch-Based Multi-Label Classifier,” in Proc. ECCV, 2024 (to appear). [This work explores patch-level MIL to improve WSI segmentation].
11. J. Yun et al., “Uncertainty Estimation for Tumor Prediction with Unlabeled Data,” in Proc. CVPR Workshops, 2024, pp. 6946–6954. DOI: 10.1109/CVPRW.2024.00711
12. G. Zelinsky et al., “Predicting pathologist attention during cancer-image readings,” Journal of Vision, vol. 25, no. 9, p. 2736, 2025. DOI: 10.1167/jov.25.9.2736
13. A. Aji et al., “Hadoop-GIS: A High Performance Spatial Data Warehousing System over MapReduce,” Proc. VLDB Endow., vol. 6, no. 11, pp. 1009–1020, 2013. DOI: 10.14778/2536222.2536227