Design and implementation of a modular executable packer: Experimenting with packing techniques and static detection

Details

Supervisors

Sadre, Ramin ; Bertrand Van Ouytsel, Charles-Henry

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en cybersécurité, à finalité spécialisée: conception et analyse de systèmes
Master [120] en cybersécurité, à finalité spécialisée: conception et analyse de systèmes

Abstract

enExecutable packing is a long-established technique used to alter binary programs, often for software protection but also for evading detection in malware. Techniques such as compression, encryption, and virtualization modify the structure and contents of an executable, making reverse engineering and static analysis significantly more difficult. While this presents challenges for digital forensics, antivirus engines, and detection systems, it also creates opportunities for research into how such transformations impact detection robustness. Recent advances have improved our understanding of packing through systematic taxonomies and tools like the Packing Box, which provide structured datasets and detection benchmarks. However, existing tools fall short when it comes to flexibility and control. Many rely on a limited set of packers or lack configurability, making it difficult to isolate the impact of specific packing techniques or to produce diverse and realistic datasets. This thesis presents PackBin, a modular and extensible packing framework. Designed for research purposes, PackBin introduces a plugin-based architecture that allows users to apply controlled transformations while automatically generating the corresponding loader logic. Its architecture decouples transformation pipelines, payload embedding, and runtime unpacking, paving the way for future extensions such as support of more executable formats. We use PackBin to conduct a series of experiments evaluating how different packing techniques affect detection by static analysis tools. Through entropy profiling, comparative visualizations, and static detection benchmarks using the Packing Box, we assess the resilience of existing tools. We also validate results using real-world antivirus engines, illustrating how PackBin can improve dataset diversity and enable fine-grained analysis of packing features. These findings not only highlight gaps in current static detection methods but also position PackBin as a useful tool for reproducible packed dataset generation and research in binary obfuscation and detection.