Article Per Year (5 Year)
p-Index From 2021 - 2026
1.286
P-Index
This Author published in this journals
All Journal Journal of Information System Format : Jurnal Imiah Teknik Informatika JURSIMA (Jurnal Sistem Informasi dan Manajemen) Journal of Applied Data Sciences Jurnal Komtekinfo Jurnal Sistem Informasi dan Manajemen
Habibie, Dedi Rahman
Unknown Affiliation
Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Economics, Econometrics & Finance Library & Information Science

Published : 11 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 11 Documents
Search

 1   2 
Improving Evaluation Metrics for Text Summarization: A Comparative Study and Proposal of a Novel Metric Junadhi, Junadhi; Agustin, Agustin; Efrizoni, Lusiana; Okmayura, Finanta; Habibie, Dedi Rahman; Muslim, Muslim
Journal of Applied Data Sciences Vol 6, No 2: MAY 2025
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v6i2.547

Abstract

This research evaluates and compares the effectiveness of various evaluation metrics in text summarization, focusing on the development of a new metric that holistically measures summary quality. Commonly used metrics, including ROUGE, BLEU, METEOR, and BERTScore, were tested on three datasets: CNN/DailyMail, XSum, and PubMed. The analysis revealed that while ROUGE achieved an average score of 0.65, it struggled to capture semantic nuances, particularly for abstractive summarization models. In contrast, BERTScore, which incorporates semantic representation, performed better with an average score of 0.75. To address these limitations, we developed the Proposed Metric, which combines semantic similarity, n-gram overlap, and sentence fluency. The Proposed Metric achieved an average score of 0.78 across datasets, surpassing conventional metrics by providing more accurate assessments of summary quality. This research contributes a novel approach to text summarization evaluation by integrating semantic and structural aspects into a single metric. The findings highlight the Proposed Metric's ability to capture contextual coherence and semantic alignment, making it suitable for real-world applications such as news summarization and medical research. These results emphasize the importance of developing holistic metrics for better evaluation of text summarization models.
Co-Authors Agustin Agustin Amalia Shifa Aldila Andema, Henky Andhika Arifin, Yoyon Army, Widya Lelisa Asyahri Hadi Nasyuha Deborah Kurniawati Efrizoni, Lusiana Finanta Okmayura Handayani, Ika Fitri Hariselmi, Hariselmi Junadhi, Junadhi Lawrence Adi Supriyono Lestari, Widya Jati Miftahul Ilmi Miftahul Ilmi, Miftahul Muslim Muslim Nursaka Putra Previana, Cantika Nur Pulut Suryati, Pulut Rahimah