Leveraging AI and LLMs to Analyse and Automate Job Applications

Details

Supervisors

Kim Mens ; Brice Thomas

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en sciences informatiques, à finalité spécialisée

Abstract

This thesis investigates the use of large language models (LLMs) for automating parts of the job application evaluation process. The work was carried out in collaboration with ERTP, an early-stage company building a software-as-a-service (SAAS) platform for assessments of candidates applying for a job. The existing platform initially supported only multiple-choice questions. The goal of this thesis project was to extend it with three new assesment modules: (1)~open-question grading where answers receive proportional scores based on correctness, with AI detection; (2)~a template-driven CV parsing tool where candidates can edit the parsed result and use AI to optimise specific sections; and (3)~a prototype personality test based on situation-based free-text responses. An evaluative approach was formulated to measure the performance of nine large language models (LLM) against a set of 300 questions on various topics including computer science, mathematics, chemistry, and physics. This process will be evaluated based on their accuracy rate, cost per million tokens, latency rate, and multilingual capability. It has been found out that the top-performing LLM models are the flagship models of Claude, GPT, Perplexity, and Kimi with an accuracy rate of (96\%-99.0\%). Mistral Medium 3 scores an accuracy rate of 85.3\% but at a significantly cheaper cost. The three assessment modules were implemented within a modular, asynchronous backend architecture designed so that AI providers can be swapped through a single adapter layer. All modules were integrated into the company's existing platform and validated through administrator-as-candidate testing. We also discuss legal, ethical and practical risks of delegating evaluation to AI, and propose design measures including human oversight, transparency toward candidates, audit logging and conservative handling of low-confidence outputs, to mitigate these risks.