Bridging the Modality Gap: A Task-Oriented Generative Framework for Robust Medical Image Analysis with Incomplete Data

Abstract

Modern medical imaging relies on multi-modal data (e.g., T1, T2, FLAIR MRI) to support diagnosis, prognosis, and AI-driven image analysis. However, in real-world clinical practice, imaging protocols are frequently incomplete due to time constraints, scanner availability, or patient-specific limitations. This mismatch between idealized research datasets and routine clinical data significantly limits the deployment of state-of-the-art AI models, which typically require a fixed and complete set of imaging modalities. 

This project addresses the missing-modality problem through a task-oriented generative learning framework. Using large-scale multi-modal MRI datasets, we systematically simulate missing modalities and quantify the resulting degradation in diagnostic model performance. We then employ deep learning-based conditional generative models to synthesize the missing MRI sequences from available modalities and reintegrate them into the diagnostic pipeline. By demonstrating statistically significant improvements across multiple missing-modality scenarios, the project establishes a pragmatic and clinically relevant paradigm for handling incomplete imaging data. Ultimately, this work aims to maximize the utility of real-world clinical datasets and to facilitate the robust translation of advanced AI methods into routine radiological practice.

Dr. Zhe Wang

Zhe Wang, PhD is currently a Biologist Engineer/Postdoctoral Researcher at Inria Centre at Université Côte d’Azur (Nice, France), where his work focuses on artificial intelligence and medical image analysis. He received his PhD from the University of Orléans, CNRS UMR 7013 (Orléans, France), with research centered on deep learning methods for medical imaging.

He was previously a research fellow at Harvard Medical School (Boston, USA), serving as Principal Investigator of the Ralph Schlaeger Fellowship, focusing on AI-driven medical image analysis and generative modeling for clinical applications at Massachusetts General Hospital.