Evaluasi Kualitas Rangkuman Teks Otomatis Menggunakan Algoritma ROUGE Studi Komparatif Model Google Gemini dan OpenAI ChatGPT - Ridwan Mahenra; Reza Pajriansyah (Universitas Stikubank, 2026)
The rapid development of large language models has significantly improved the performance of automatic text summarization systems. However, each model may demonstrate different levels of effectiveness depending on its architectural characteristics and training approach. This study aims to evaluate the quality of Indonesian news text summarization generated by Google Gemini and OpenAI ChatGPT. A total of fifty news articles were used as the dataset, each accompanied by a manually written summary to serve as the ground truth. The research procedure involved web scraping, text preprocessing, automated summarization using both models, and performance evaluation through ROUGE metrics, including ROUGE-1, ROUGE-2, and ROUGE-L. The results show that ChatGPT consistently achieves higher ROUGE-1 and ROUGE-2 scores compared with Gemini, indicating a better ability to preserve key terms and lexical relations within the text. Meanwhile, ROUGE-L scores for both models are relatively close, suggesting that Gemini remains competitive in maintaining overall summary structure. Distribution analysis also reveals that ChatGPT demonstrates more stable performance across various text types. Overall, this study concludes that ChatGPT provides more accurate and consistent automatic summarization results than Gemini for Indonesian news texts.