Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Digital Zone: Jurnal Teknologi Informasi dan Komunikasi
Rendika
Unknown Affiliation
Computer Science & IT Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Towards an Automated Essay Evaluation System NLP Based Text Embeddings and Similarity Metrics Permata, Regita; Rendika; Julianty, Luh Candra
Digital Zone: Jurnal Teknologi Informasi dan Komunikasi Vol. 16 No. 1 (2025): Digital Zone: Jurnal Teknologi Informasi dan Komunikasi
Publisher : Publisher: Fakultas Ilmu Komputer, Institution: Universitas Lancang Kuning

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.31849/digitalzone.v16i1.26541

Abstract

This study aims to develop an automatic essay answer assessment system based on Natural Language Processing (NLP) to reduce the time and effort required for evaluation. The system uses Cosine Similarity and Manhattan Distance as evaluation metrics and implements two text embedding methods—Term Frequency-Inverse Document Frequency (TF-IDF) and Bag of Words (BoW)—to represent the user’s answer text. The methodology begins with text processing and pre-processing, followed by embedding and similarity calculation between the user’s answer and the reference text to generate an evaluation score categorized into three levels: good, sufficient, and poor. Based on Cohen’s Kappa analysis, the kappa value for Cosine Similarity reaches 0.78, indicating high agreement between the Cosine TF-IDF and Cosine BoW methods. In contrast, Manhattan Distance yields a kappa value of -0.05, indicating a discrepancy between the two Manhattan-based methods. The evaluation results suggest that Cosine Similarity is more suitable, whereas Manhattan Distance is not relevant for this task. At the modeling stage, the best classification models are Decision Tree and Random Forest, each achieving an accuracy of 96.67%. Although Random Forest demonstrates a higher AUC than Decision Tree, it requires a longer training time. Overall, the system is considered effective for assessing essay answers with both purpose and consistency, offering potential applications in the field of education
Co-Authors Julianty, Luh Candra Permata, Regita