Abstract: The COVID-19 pandemic has emphasized the need for large-scale collaborations between the clinical and scientific communities to tackle global healthcare challenges. However, regulatory constraints around data sharing and patient privacy might hinder access to data representing diverse and clinically relevant patient populations. Federated learning (FL) technology allows us to work around such restrictions while respecting patient privacy. This talk will show how FL was used to predict clinical outcomes in patients with COVID-19 while allowing collaborators to retain governance over their data. Furthermore, I will introduce several recent advances in FL, including quantifying potential data leakage, automated machine learning (AutoML), federated neural architecture search (NAS), and personalization that can allow us to build more accurate and robust AI models for medical imaging, large-language-modeling, and beyond.

Abstract: There are more than 7,000 rare diseases, collectively affecting 300-400 million people worldwide, yet each disease has a very low prevalence, involving no more than 50 per 100,000 individuals. Due to their low prevalence, most front-line clinicians lack disease experience, resulting in numerous specialty referrals and expensive clinical workups for patients across multiple years and institutions. Such challenges make rare disease diagnosis extremely difficult; approximately 70% of individuals seeking a diagnosis and up to 50% of the suspected Mendelian conditions remain undiagnosed. These diagnostic odysseys can lead to redundant testing or unnecessary medical procedures, inappropriate or delayed disease management, and irreversible disease progression if the time window for intervention is missed. We introduce SHEPHERD, a deep learning approach for multi-faceted rare disease diagnosis. SHEPHERD is guided by existing knowledge of diseases, phenotypes, and genes to learn novel connections between a patient’s clinico-genetic information and phenotype and gene relationships. We train SHEPHERD exclusively on simulated patients and independently evaluate on an external patient cohort representing 299 diseases in the Undiagnosed Diseases Network. SHEPHERD excels at several diagnostic facets: performing causal gene discovery, retrieving “patients-like-me” with the same gene or disease, and providing interpretable characterizations of novel disease presentations. SHEPHERD demonstrates the potential of artificial intelligence to accelerate the diagnosis of rare disease patients and has implications for using deep learning on medical datasets with very few labels. This is joint work with Emily Alsentzer, Michelle M. Li, Shilpa N. Kobren, Zak Kohane, and the Undiagnosed Diseases Network.

1. Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging
2. SEPAL: Spatial Gene Expression Prediction from Local Graphs
3. Do CNNs need some Attention? Boosted Organ Segmentation with Attention Maps from Self-Supervised ViTs
4. RRc-UNet 3D for lung tumor segmentation from CT scans of Non-Small Cell Lung Cancer patients
5. Topo-CXR: Chest X-ray TB and Pneumonia Screening with Topological Machine Learning
6. Contrastive Image Synthesis and Self-supervised Feature Adaptation for Cross-Modality Biomedical Image segmentation
7. Cross-grained Contrastive Representation for Unsupervised Lesion Segmentation in Medical Image
8. Semi-supervised Quality Evaluation of Colonoscopy Procedures
9. Ensuring a connected structure for Retinal Vessels Deep-Learning Segmentation
10. CLIPath: Fine-tune CLIP with Visual Feature Fusion for Pathology Image Analysis Towards Minimizing Data Collection Efforts
11. Implicit Neural Representation in Medical Imaging: A Comparative Survey
12. HyperCoil-Recon: A Hypernetwork-based Adaptive Coil Configuration Task Switching Network for MRI Reconstruction
13. Self-Supervised Anomaly Detection from Anomalous Training Data via Iterative Latent Token Masking
14. Robust AMD Stage Grading with Exclusively OCTA Modality Leveraging 3D Volume
15. Segmentation-based Assessment of Tumor-Vessel Involvement for Surgical Resectability Prediction of Pancreatic Ductal Adenocarcinoma
16. Sharing is Caring: Concurrent Interactive Segmentation and Model Training using a Joint Model
17. Robust MSFM Learning Network for Classification and Weakly Supervised Localization
18. Standardized Image-Based Polysomnography Database and Deep Learning Algorithm for Sleep Stage Classification
19. BlindHarmony: “Blind” Harmonization for multi-site MR Image processing via Flow model
20. SP3D: Learning Keypoint Detection and Description in 3D Medical Images
21. DISGAN: Wavelet-informed discriminator guides GAN to MRI image Super-resolution with noise cleaning
22. Studying the Impact of Augmentations on Medical Confidence Calibration
23. Multimodal Contrastive Learning and Tabular Attention for Automated Alzheimer's Disease Prediction
24. Weakly Semi-supervised Detector-based Video Classification with Temporal Constraints for Lung Ultrasound
25. Order-ViT: Order Learning Vision Transformer for Cancer Classification in Pathology Images
26. MIND THE CLOT: AUTOMATED LVO DETECTION ON CTA USING DEEP LEARNING
27. A Comparative Study of Vision Transformer Encoders and Few-shot Learning for Medical Image Classification
28. RheumaVIT: transformer-based model for Automated Scoring of Hand Joints in Rheumatoid Arthritis
29. AW-Net: A Novel Fully Connected Attention-based Medical Image Segmentation Model
30. Geodesic Regression Characterizes 3D Shape Changes in the Female Brain During Menstruation
31. Computational Evaluation of the Combination of Semi-Supervised and Active Learning for Histopathology Image Segmentation with Missing Annotations
32. Towards Robust Natural-looking Mammography Lesion Synthesis On Ipsilateral Dual-views Breast Cancer Analysis
33. End-to-End Deep Learning for Reconstructing Segmented 3D CT Image from Multi-Energy X-ray Projections
34. SlAction: Non-intrusive, Lightweight Obstructive Sleep Apnea Detection using Infrared Video
35. Combating Coronary Calcium Scoring Bias for Non-gated CT by Semantic Learning on Gated CT
36. Comprehensive Multimodal Segmentation in Medical Imaging: Combining YOLOv8 with SAM and HQ-SAM Models
37. Semantic Parsing of Colonoscopy Videos with Multi-Label Temporal Networks
38. Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation
39. Causality-Driven One-Shot Learning for Prostate Cancer Grading from MRI
40. ShaRPy:Shape Reconstruction and Hand Pose Estimation From RGB-D with Uncertainty
41. An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports
42. Fusion Approaches to Predict Post-stroke Aphasia Recovery from Multimodal Neuroimaging Data
43. Automated Biomarker Detection in Diagnostics of Colorectal Cancer
44. Self-supervised Semantic Segmentation: Consistency over Transformation
45. ALFA: Leveraging All Levels of Feature Abstraction for Enhancing the Generalization of Histopathology Image Classification Across Unseen Hospitals
46. Pathology-Based Ischemic Stroke Etiology Classification via Clot Composition Guided Multiple Instance Learning
47. Enhancing Medical Image Segmentation: Optimizing Cross-Entropy Weights and Post-Processing with Autoencoders
48. Looking Beyond What You See: An empirical analysis on subgroup intersectional fairness for multi-label chest x-ray classification using social determinants of racial health inequities
49. Double pre-training for detecting fetal heart diseases with Deep Learning: A Chilean case
50. Using Large Text To Image Models with Structured Prompts for Skin Disease Identification: A Case Study
51. CheXFusion: Effective Fusion of Multi-View Features using Transformers for Long-Tailed Chest X-Ray Classification
52. Robust Asymmetric Loss for Multi-Label Long-Tailed Learning
53. How Can We Tame the Long-Tail of Chest X-ray Datasets?
54. Effect of Stage Training for Long-Tailed Multi-Label Image Classification
55. Bag of Tricks for Long-Tailed Multi-Label Classification on Chest X-Rays
56. Advanced Augmentation and Ensemble Approaches for Classifying Long-Tailed Multi-Label Chest X-Rays
57. An Optimized Ensemble-based Framework for Multi-Label Long-Tailed Classification on Chest X-rays
58. Multi-Label Long-Tailed Classification on Chest X-Rays by upgrading ML-GCN
59. Chest X-Ray Feature Pyramid Sum Model with Diseased Area Data Augmentation Method

Abstract: There are numerous efforts on technical development and translation of AI/ML in the medical imaging domain. In addition to some of the classic challenges such as small datasets, limited annotations, imbalanced classes, how to gain and enhance trust from the users and practitioners of medical AI/ML is an emerging topic, and key for successful applications of AI to patient care. In this talk, the speaker will elaborate important pillars in developing trustworthy medical AI tools, how to marry medical intelligence and AI to enhance trust from clinicians, and showcase a range of applications of AI/ML in medical imaging.

1. Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging
2. SEPAL: Spatial Gene Expression Prediction from Local Graphs
3. Do CNNs need some Attention? Boosted Organ Segmentation with Attention Maps from Self-Supervised ViTs
4. RRc-UNet 3D for lung tumor segmentation from CT scans of Non-Small Cell Lung Cancer patients
5. Topo-CXR: Chest X-ray TB and Pneumonia Screening with Topological Machine Learning
6. Contrastive Image Synthesis and Self-supervised Feature Adaptation for Cross-Modality Biomedical Image segmentation
7. Cross-grained Contrastive Representation for Unsupervised Lesion Segmentation in Medical Image
8. Semi-supervised Quality Evaluation of Colonoscopy Procedures
9. Ensuring a connected structure for Retinal Vessels Deep-Learning Segmentation
10. CLIPath: Fine-tune CLIP with Visual Feature Fusion for Pathology Image Analysis Towards Minimizing Data Collection Efforts
11. Implicit Neural Representation in Medical Imaging: A Comparative Survey
12. HyperCoil-Recon: A Hypernetwork-based Adaptive Coil Configuration Task Switching Network for MRI Reconstruction
13. Self-Supervised Anomaly Detection from Anomalous Training Data via Iterative Latent Token Masking
14. Robust AMD Stage Grading with Exclusively OCTA Modality Leveraging 3D Volume
15. Segmentation-based Assessment of Tumor-Vessel Involvement for Surgical Resectability Prediction of Pancreatic Ductal Adenocarcinoma
16. Sharing is Caring: Concurrent Interactive Segmentation and Model Training using a Joint Model
17. Robust MSFM Learning Network for Classification and Weakly Supervised Localization
18. Standardized Image-Based Polysomnography Database and Deep Learning Algorithm for Sleep Stage Classification
19. BlindHarmony: “Blind” Harmonization for multi-site MR Image processing via Flow model
20. SP3D: Learning Keypoint Detection and Description in 3D Medical Images
21. DISGAN: Wavelet-informed discriminator guides GAN to MRI image Super-resolution with noise cleaning
22. Studying the Impact of Augmentations on Medical Confidence Calibration
23. Multimodal Contrastive Learning and Tabular Attention for Automated Alzheimer's Disease Prediction
24. Weakly Semi-supervised Detector-based Video Classification with Temporal Constraints for Lung Ultrasound
25. Order-ViT: Order Learning Vision Transformer for Cancer Classification in Pathology Images
26. MIND THE CLOT: AUTOMATED LVO DETECTION ON CTA USING DEEP LEARNING
27. A Comparative Study of Vision Transformer Encoders and Few-shot Learning for Medical Image Classification
28. RheumaVIT: transformer-based model for Automated Scoring of Hand Joints in Rheumatoid Arthritis
29. AW-Net: A Novel Fully Connected Attention-based Medical Image Segmentation Model
30. Geodesic Regression Characterizes 3D Shape Changes in the Female Brain During Menstruation
31. Computational Evaluation of the Combination of Semi-Supervised and Active Learning for Histopathology Image Segmentation with Missing Annotations
32. Towards Robust Natural-looking Mammography Lesion Synthesis On Ipsilateral Dual-views Breast Cancer Analysis
33. End-to-End Deep Learning for Reconstructing Segmented 3D CT Image from Multi-Energy X-ray Projections
34. SlAction: Non-intrusive, Lightweight Obstructive Sleep Apnea Detection using Infrared Video
35. Combating Coronary Calcium Scoring Bias for Non-gated CT by Semantic Learning on Gated CT
36. Comprehensive Multimodal Segmentation in Medical Imaging: Combining YOLOv8 with SAM and HQ-SAM Models
37. Semantic Parsing of Colonoscopy Videos with Multi-Label Temporal Networks
38. Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation
39. Causality-Driven One-Shot Learning for Prostate Cancer Grading from MRI
40. ShaRPy:Shape Reconstruction and Hand Pose Estimation From RGB-D with Uncertainty
41. An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports
42. Fusion Approaches to Predict Post-stroke Aphasia Recovery from Multimodal Neuroimaging Data
43. Automated Biomarker Detection in Diagnostics of Colorectal Cancer
44. Self-supervised Semantic Segmentation: Consistency over Transformation
45. ALFA: Leveraging All Levels of Feature Abstraction for Enhancing the Generalization of Histopathology Image Classification Across Unseen Hospitals
46. Pathology-Based Ischemic Stroke Etiology Classification via Clot Composition Guided Multiple Instance Learning
47. Enhancing Medical Image Segmentation: Optimizing Cross-Entropy Weights and Post-Processing with Autoencoders
48. Looking Beyond What You See: An empirical analysis on subgroup intersectional fairness for multi-label chest x-ray classification using social determinants of racial health inequities
49. Double pre-training for detecting fetal heart diseases with Deep Learning: A Chilean case
50. Using Large Text To Image Models with Structured Prompts for Skin Disease Identification: A Case Study
51. CheXFusion: Effective Fusion of Multi-View Features using Transformers for Long-Tailed Chest X-Ray Classification
52. Robust Asymmetric Loss for Multi-Label Long-Tailed Learning
53. How Can We Tame the Long-Tail of Chest X-ray Datasets?
54. Effect of Stage Training for Long-Tailed Multi-Label Image Classification
55. Bag of Tricks for Long-Tailed Multi-Label Classification on Chest X-Rays
56. Advanced Augmentation and Ensemble Approaches for Classifying Long-Tailed Multi-Label Chest X-Rays
57. An Optimized Ensemble-based Framework for Multi-Label Long-Tailed Classification on Chest X-rays
58. Multi-Label Long-Tailed Classification on Chest X-Rays by upgrading ML-GCN
59. Chest X-Ray Feature Pyramid Sum Model with Diseased Area Data Augmentation Method

Abstract: Over the last 7+ years, deep learning has been transforming medical imaging, from enhancing acquisition to maximizing downstream utility of scans. Much of this progress relies on supervised learning approaches with “black box” models. In this talk, I will show an example of recent work from my group where we move beyond this traditional paradigm to develop tools designed for the unique considerations of medical imaging, in a classic problem setting: multi-modal image registration. I will describe a novel architecture we developed, coined KeyMorph, that affords the user robustness, control, and interpretability.

Abstract: For multiple diseases, early detection significantly improves patient outcomes. This motivates considerable investments in population-wide screening programs, such as mammography for breast cancer and low-dose CT for lung cancer. To be effective and economically viable, these programs must find the right balance between early detection and overscreening. This capacity builds on two complementary technologies: (1) the ability to accurately assess patient risk at a given time point and (2) the ability to design screening regimens based on this risk. Moreover, these tools must obtain consistent performance across diverse populations and adapt to new clinical requirements while learning from limited datasets. In this talk, I’ll discuss approaches to address these challenges in cancer risk assessment from imaging and personalized screening policy design. We’ve demonstrated that these clinical models offer significant improvements over the current standard of care across globally diverse patient populations, and our image-based tools now underly prospective trials.