THESIS

Development of Bert and Hybrid Models for Sentiment Analysis using AcehXfine-Tuning

and Tokenizer Adaptation

ABSTRACT

In the digital era. sentiment analvsis has become one of the kev areas in natural language processing (NLP). NLP development for regional languages in Indonesia remains very limited, including for the Acehnese language which possesses rich lexical diversity and unique morphological structures. One of the main challenges in developing sentiment analysis for Acehnese is the lack of a representative dataset for sentiment analvsis tasks. Moreover, there is currently no BERT-based model utilizing the Masked Language Modeling (MLM) approach that has been specifically optimized for the Acehnese language. Existing pretrained models such as IndoBERT still relv on Indonesian-language data and have yet to fully capture the distinctive linguistic characteristics of Acehnese. Therefore, this studv aims to construct an AcehX Sentiment dataset in the Acehnese language ana develop the AcehXBERT model by re-training the IndoBERT-base model using the MLM approach on the AcehX corpus. This aims to enhance the model’s semantic and contextual understanding of the Acehnese language. The study also investigates the fine-tuning process for sentiment classification tasks on Acehnese text to evaluate the model’s comprehension of local context and its sentiment classification performance. Experimental results on the Acehx dataset using test data show that the AcehXBERT model for sentiment classification successfully achieved an F1- macro of 82.50% and the AcehXBERT+BiLSTM model achieved an F1-macro of 81.62% while for the NusaX dataset using test data, AcehXBERT achieved an F1- macro of 81.89% and AcehXBERT+BiLSTM achieved an F1-macro of 82.29% outperforming the model from NusaBERT. This study shows that an adaptive approach to pre-trained models and tokenizers is very important in the developmen of NLP for regional languages, especially in efforts to support the preservation and utilization of the Acehnese language in modern technology.

PUBLICATION

• Development of Acehx for Sentiment Analysis Using a Bert-Based Model (link)