Integration of Learning Coalitions and Vision-Language Models for Cytological Analysis of Thyroid Cancer

Details

Supervisors

Macq, Benoît

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en informatique, à finalité spécialisée

Abstract

enThe deployment of deep learning in medical diagnostics, particularly in thyroid cytology, is constrained by a fundamental tension between the high computational cost of state-of-the-art models and the requirement for rigorous statistical reliability. Standard neural networks often produce miscalibrated confidence scores, providing no formal guarantees in safety-critical clinical environments. This thesis addresses these challenges by proposing a family of Conformal Adaptive Decision Systems (CADS), a modular progression of multi-expert cascades that leverage Conformal Prediction (CP) to achieve cost-efficient inference with mathematically grounded uncertainty quantification. We introduce four evolving architectures to resolve specific structural limitations: 1. CADS: A foundational framework where the size of a conformal prediction set (|C(x)|) acts as a principled signal for model escalation. 2. CADS-VLM: An extension that integrates Vision-Language Models (VLMs) (e.g., BiomedCLIP, PLIP) into the expert pool via Low-Rank Adaptation (LoRA), capturing semantic depth for complex pathologies. 3. Hybrid CADS-VLM: A heterogeneous pool fusing CNN and VLM experts, allowing the system to resolve easy cases at sub-GFLOP costs (e.g., 0.06 GFLOPs via MobileNet) while reserving high-capacity VLMs for difficult samples. 4. H-CADS: A final clinical iteration that replaces heuristic aggregation with a learned meta-classifier (stacker) and adds a secondary conformal layer to provide three-way triage decisions (auto/verify/refer). Evaluated on cervical cytology benchmarks as a methodological proxy for thyroid cancer, H-CADS achieves accuracy gains over the standard VLM cascade baseline, reaching up to +28pp on the most challenging few-shot configuration (HiCervix, 1-shot), driven primarily by a learned meta-classifier that breaks the heuristic ensemble ceiling by up to 31pp at the most resource-constrained clinical profile. By lowering the computational floor from 1.30 GFLOPs to 0.06 GFLOPs, the hybrid system further demonstrates the viability of high-performance, statistically reliable AI for resource-constrained clinical triage. This work provides a scalable roadmap for integrating diverse learning coalitions into the next generation of medical decision-support tools.