← All theses

Department of Computer Science and Engineering

An LLM-Based RAG System for Question Answering over User-Provided Documents

Islam, S. M. Shufol

Abstract

The rapid growth of digital documents has created a need for efficient methods of accessing and understanding information contained within them. Traditional document-reading approaches require users to manually search through large amounts of text, which can be time-consuming and inefficient. This project presents an LLM-based Retrieval-Augmented Generation (RAG) system for question answering over user-provided documents, designed to provide users with relevant and context-aware answers from their uploaded documents. The proposed system combines semantic information retrieval with a Large Language Model (LLM) to improve document-based question answering. Users can upload supported documents to the system, after which the document content is extracted and divided into smaller text chunks. These chunks are transformed into numerical vector representations using a sentence-transformer-based embedding model and stored in ChromaDB, a vector database. When a user submits a question, the system converts the query into an embedding and performs semantic similarity search against the stored document chunks. The most relevant chunks are then retrieved and provided as contextual information to the LLM. The LLM generates an answer based primarily on the retrieved context, allowing the system to respond to questions without requiring the entire document to be supplied to the model for every query. The system consists of a React-based frontend, a FastAPI backend, a ChromaDB vector store, a sentence-transformer embedding model, and an LLM accessed through the Groq API. The application also provides source information associated with retrieved document content, allowing users to identify the document from which the answer was generated. The separation between document processing, semantic retrieval, and answer generation provides a modular architecture that can be extended with different language models or retrieval configurations. The developed system demonstrates the practical application of Retrieval-Augmented Generation for document-based question answering while reducing dependence on manually searching through lengthy documents. Although the current implementation has limitations related to document formats, retrieval accuracy, context size, and dependence on external LLM services, the architecture provides a strong foundation for future improvements such as multi-document knowledge management, conversation memory, improved retrieval techniques, document management interfaces, authentication, advanced source citation, and more sophisticated question-answering capabilities.