Dinamik
Vol 31 No 2 (2026)

Evaluasi Kualitas Rangkuman Teks Otomatis Menggunakan Algoritma ROUGE Studi Komparatif Model Google Gemini dan OpenAI ChatGPT

Ridwan Mahenra (Universitas Teknokrat Indonesia)
Reza Pajriansyah (Universitas Teknokrat Indonesia)



Article Info

Publish Date
01 Jul 2026

Abstract

The rapid development of large language models has significantly improved the performance of automatic text summarization systems. However, each model may demonstrate different levels of effectiveness depending on its architectural characteristics and training approach. This study aims to evaluate the quality of Indonesian news text summarization generated by Google Gemini and OpenAI ChatGPT. A total of fifty news articles were used as the dataset, each accompanied by a manually written summary to serve as the ground truth. The research procedure involved web scraping, text preprocessing, automated summarization using both models, and performance evaluation through ROUGE metrics, including ROUGE-1, ROUGE-2, and ROUGE-L. The results show that ChatGPT consistently achieves higher ROUGE-1 and ROUGE-2 scores compared with Gemini, indicating a better ability to preserve key terms and lexical relations within the text. Meanwhile, ROUGE-L scores for both models are relatively close, suggesting that Gemini remains competitive in maintaining overall summary structure. Distribution analysis also reveals that ChatGPT demonstrates more stable performance across various text types. Overall, this study concludes that ChatGPT provides more accurate and consistent automatic summarization results than Gemini for Indonesian news texts.

Copyrights © 2026






Journal Info

Abbrev

fti1

Publisher

Subject

Computer Science & IT

Description

The Jurnal DINAMIK aims to: Promote a comprehensive approach to informatics engineering and management incorporating viewpoints of different applications (computer graphics, computer networks and security, computer vision, computational intelligence, databases, big data, IT project management, and ...