Claim Missing Document
Check
Articles

Found 13 Documents
Search

Improving Evaluation Metrics for Text Summarization: A Comparative Study and Proposal of a Novel Metric Junadhi Junadhi; Agustin Agustin; Lusiana Efrizoni; Finanta Okmayura; Dedi Rahman Habibie; Muslim Muslim
Journal of Applied Data Sciences Vol 6, No 2: MAY 2025
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v6i2.547

Abstract

This research evaluates and compares the effectiveness of various evaluation metrics in text summarization, focusing on the development of a new metric that holistically measures summary quality. Commonly used metrics, including ROUGE, BLEU, METEOR, and BERTScore, were tested on three datasets: CNN/DailyMail, XSum, and PubMed. The analysis revealed that while ROUGE achieved an average score of 0.65, it struggled to capture semantic nuances, particularly for abstractive summarization models. In contrast, BERTScore, which incorporates semantic representation, performed better with an average score of 0.75. To address these limitations, we developed the Proposed Metric, which combines semantic similarity, n-gram overlap, and sentence fluency. The Proposed Metric achieved an average score of 0.78 across datasets, surpassing conventional metrics by providing more accurate assessments of summary quality. This research contributes a novel approach to text summarization evaluation by integrating semantic and structural aspects into a single metric. The findings highlight the Proposed Metric's ability to capture contextual coherence and semantic alignment, making it suitable for real-world applications such as news summarization and medical research. These results emphasize the importance of developing holistic metrics for better evaluation of text summarization models.
Adaptive Neural Collaborative Filtering with Textual Review Integration for Enhanced User Experience in Digital Platforms Lusiana Efrizoni; Edwar Ali; Hadi Asnal; Junadhi Junadhi
Journal of Applied Data Sciences Vol 6, No 4: December 2025
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v6i4.944

Abstract

This research proposes a hybrid rating prediction model that integrates Neural Collaborative Filtering (NCF), Long Short-Term Memory (LSTM), and semantic analysis through Natural Language Processing (NLP) to enhance recommendation accuracy. The main objective is to improve alignment between system predictions and actual user preferences by leveraging multi-source information from the Amazon Movies and TV dataset, which includes explicit user–item ratings and textual reviews. The core idea is to combine three complementary processing paths—(1) user–item interaction modeling via NCF, (2) temporal dynamics capture through LSTM, and (3) semantic understanding of reviews using NLP—into a unified deep learning-based adaptive architecture. Experimental evaluation demonstrates that this multi-input approach outperforms the baseline collaborative filtering model, with the Mean Absolute Error (MAE) reduced from 1.3201 to 1.2817 (a 2.91% improvement) and the Mean Squared Error (MSE) reduced from 2.2315 to 2.1894 (a 1.89% improvement). Training metrics visualization further shows a stable convergence pattern, with the MAE gap between training and validation consistently below 0.03, indicating minimal overfitting. The findings confirm that integrating cross-dimensional signals significantly enhances predictive performance and can contribute to increased user satisfaction and engagement in recommendation platforms. The novelty of this work lies in the simultaneous integration of interaction, temporal, and semantic dimensions into a single adaptive recommendation framework, a configuration not jointly explored in prior studies. Moreover, the flexible architecture enables adaptation to other domains such as e-commerce, music, or online learning, broadening its practical applicability.
Edukasi Manajemen Ekspor Impor bagi Siswa SMA sebagai Pengenalan Bisnis Internasional: Tinjauan Literatur dan Rancangan Model di SMA IT Al Fityah Pekanbaru Arya Setiawan; Lusiana Efrizoni; Daffa Al Haq; Nasywa Syakiroh Muthmainnah
SCIENTIFIC JOURNAL OF REFLECTION : Economic, Accounting, Management and Business Vol. 9 No. 4 (2026): SCIENTIFIC JOURNAL OF REFLECTION: Economic, Accounting, Management, & Business
Publisher : Sekolah Menengah Kejuruan (SMK) Pustek

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.37481/sjr.v9i4.1773

Abstract

International trade education remains insufficiently explored at the senior high school level, particularly in relation to the practical management of export and import activities. This study aims to map existing research on export-import education, describe the educational context of SMA IT Al Fityah Pekanbaru, and develop a learning model suited to the school’s characteristics. A structured narrative literature review of 13 core publications published between 2021 and 2026 was combined with a descriptive case study based on school observations, documentation, and an initial presentation and discussion activity. The findings indicate that existing programs primarily target micro, small, and medium enterprises (MSMEs), university students, and teachers, with no specific studies on general senior high school students identified in the reviewed literature. Reported knowledge scores increased across studies, although the magnitude varied and the study designs lacked comparison groups. Practical activities or simulations appeared in at least seven studies, while mentoring was reported in three. Documentation was the most frequently addressed topic, whereas financing received the least attention. These findings informed a four-stage learning model Know, Understand, Practice, and Reflect integrating six export-import management functions into school-based learning activities. The model provides a preliminary framework for export-import education, but its effectiveness requires further empirical testing.