Sensitivity of explanations to training randomness in vision transformers

Details

Supervisors

Standaert, François-Xavier

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en mathématiques appliquées, à finalité spécialisée

Abstract

enDeep Neural Networks like Vision Transformers (ViT) achieve nowadays remarkable performances in image classification, but their opacity raises a fundamental question for their deployment in high-stakes domains: can we trust the explanations they produce? This master thesis studies the Layer-wise Relevance Propagation method applied to Vision Transformers, that aims to explain the model's decision by highlighting which areas of an image contributed the most to the prediction. By training a large number of models functionally equivalent differing only by the training seed, on two image classification tasks (facial emotion recognition and glaucoma diagnosis), this master thesis focuses on how these explanations are sensitive to the randomness inherent to the training phase, and shows how the explanations vary between models using the Mean Correlation With a Mean Explanation metric. The results reveal that models showing no significant difference in terms of accuracy can produce different explanations for an identical image. This phenomenon varies with the training regime: models fully fine-tuned show a higher instability of explanations but a better focus on discriminating areas, showing a trade-off between stability and quality of explanations. These results raise significant concerns about the use of artificial intelligence in critical contexts.