Development of a RAG-System Blueprint for Businesses by Leveraging a Prototype with Book Queries

Details

Supervisors

Van Oirbeek, Robin

Faculty

Faculté des sciences

Degree label

Master [120] en science des données, orientation statistique, à finalité spécialisée

Abstract

This thesis explores the development and optimization of Retrieval Augmented Generation (RAG) Systems with a focus on Metric Driven Development (MDD) methodologies. The research emphasizes the critical importance of key performance metrics, such as context recall, entity recall, answer correctness, and faithfulness, in guiding the iterative enhancement of RAG-Systems. By analyzing the relative impact of different components within the RAG architecture, particularly the retrieval mechanism, the study identifies the retrieval process as a vital area for optimization. Simple and cost-effective retrieval strategies are compared against more complex methods, highlighting the effectiveness of straightforward approaches in enhancing system performance. Additionally, the thesis addresses the challenges inherent in working with Large Language Models (LLMs) and the complexities of textual data, proposing solutions to manage implementation costs and improve data quality. The thesis advocates for a structured and incremental approach to system development, starting with simpler methods and evolving toward more sophisticated models as a foundation for future advancements in the field. Potential avenues for further research are also discussed, including expanding the knowledge base, enhancing retrieval mechanisms, and conducting real-world user testing to refine the system’s applicability and effectiveness.