THESIS

Development of Large Language Model to Answer Academic Related Questions at Syiah Kuala University

using Fine-Tuning and Retrieval-Augmented Generation Methods

ABSTRACT

Right now, academic information at Universitas Syiah Kuala (USK) is distributed on a website or summarized in the form of Frequently Asked Questions (FAQ). Information in the form of a website and FAQ is not interactive. Certain information must be searched from the web or FAQ list. Therefore, a more interactive way to get information using a chatbot is needed. Chatbots can be built using a Large Language Model (LLM) such as Mistral 7B. Mistral 7B is a large language model that can be applied to answer questions such as academic information using data collected from universities. The fine-tuning method with the QLoRA and RAG techniques can be used to train the model and retrieve relevant information from external document sources. The results are then evaluated using the ROUGE score. The answers from the USK Mistral 7B model gave results with a score of >0.5 on 15 out of 56 questions using the RAG method, and the fine-tuning method was tested on 20 questions, producing a value with a score of >0.5. Testing was also conducted with different questions that had the same meaning, and response results were obtained with a ROUGE score of 0.4-0.5 from the questions asked. Using the USK Mistral 7B model in a chatbot, academic information at USK can be shared interactively.

PUBLICATION

• Fine-Tuning Large Language Model (LLM) to Answer Basic Questions for Prospective New Students at Syiah Kuala University Using the Retrieval-Augmented Generation (RAG) Method (link)