Attention-Based feature selection in high-dimensional datasets

Details

Supervisors

Dupont, Pierre

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en informatique, à finalité spécialisée

Abstract

enFeature selection is a crucial task in the analysis of microarray datasets, where a substantial number of features can be redundant or irrelevant. However, some traditional methods struggle with the challenges imposed by the high dimensionality of the data and the low amount of samples available. In recent years, the introduction of attention mechanisms in deep learning has sparked considerable interest, as they provide a means of focusing on the most pertinent information within large datasets. Previous publications have successfully adapted the mechanism to feature selection, demonstrating state-of-the-art performance for image classification tasks. In this work, we extend these ideas by integrating an attention framework specifically tailored to high-dimensional microarray data. We also use the insights the attention mechanism offers into the inner workings of the selection and compare the method to other commonly used methods. We show that the resulting feature ranking can perform competitively with other methods on high-dimensional datasets.