A comparison of program query languages to detect Python programming misconceptions

Details

Supervisors

Mens, Kim ; Liénard, Julien

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en sciences informatiques, à finalité spécialisée

Abstract

enDetecting and addressing common misconceptions in beginner programmer's code is key to improve their learning experience. Such misconceptions often lead to recurring patterns of incorrect reasoning that are difficult to detect with traditional testing techniques. This thesis evaluates the effectiveness of four static program query languages and tools: Flake8, Regex, CodeQL and Pyttern, for identifying such misconceptions in Python code submitted by students on the INGInious platform. We collected and classified common programming misconceptions from various educational sources and implemented a subset of 20 misconceptions using each language and compared them on a variety of criteria, including accuracy, performance, expressiveness, learning curve and query readability. Our analysis reveals trade-offs between tools. Some offer greater expressiveness but require a steeper learning curve, where others are easy to use but limited in their ability to detect more complex patterns. This thesis provides guidance on selecting the most appropriate tool depending on the use case.