Text Generation untuk Profil Mata Kuliah pada Penilaian Outcome-Based Education Menggunakan Text-to-Text Transfer Transformers

Nurrohman, Nurrohman (UIN Sunan Gunung Djati Bandung)
Maylawati, Dian Sa'adillah (UIN Sunan Gunung Djati Bandung)
Al-Amin, Muhammad Insan (UIN Sunan Gunung Djati Bandung)

Jutisi: Jurnal Ilmiah Teknik Informatika dan Sistem Informasi
Vol 14, No 1: April 2025

The evaluation of Course Learning Outcomes (CPMK) in Outcome-Based Education (OBE) is still conducted manually, making it time-consuming and prone to errors. Additionally, the achievement profile of CPMK is often overlooked. This study aims to automate the generation of course profiles based on CPMK using Text Generation technology. The method employed is Transformers with the T5 (Text-to-Text Transfer Transformer) algorithm. Experiments were conducted using three variants of the T5 model: T5 Base, T5 Base with fine-tuning, and T5 XL, evaluated using the Bilingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results show that T5 XL achieved the best performance, with an average BLEU score of 0.592 and a ROUGE-L score of 0.721. T5 Base with fine-tuning recorded a BLEU score of 0.417 and a ROUGE-L score of 0.468, while T5 Base without fine-tuning had a BLEU score of 0.327 and a ROUGE-L score of 0.246. Additionally, more structured prompts yielded better evaluation results. This study demonstrates that T5 XL enhances the efficiency and accuracy of CPMK evaluation in OBE.Keywords: Outcome Based Education; Text Generation; Text-To-Text Transfer Transformers; Penilaian AbstrakEvaluasi capaian pembelajaran mata kuliah (CPMK) dalam Outcome-Based Education (OBE) masih dilakukan secara manual, memakan waktu, dan rentan terhadap kesalahan. Selain itu, profil pencapaian CPMK sering diabaikan. Penelitian ini bertujuan mengotomasi pembuatan profil mata kuliah berbasis CPMK menggunakan teknologi Text Generation. Metode yang digunakan adalah Transformers dengan algoritma T5 (Text-to-Text Transfer Transformers). Eksperimen dilakukan dengan tiga varian model T5: T5 Base, T5 Base dengan fine-tuning, dan T5 XL, dievaluasi menggunakan metrik Bilingual Evaluation Understudy (BLEU) dan Recall-Oriented Understudy for Gisting Evaluation (ROUGE). Hasil menunjukkan T5 XL memiliki performa terbaik dengan BLEU rata-rata 0,592 dan ROUGE-L 0,721. T5 Base dengan fine-tuning mencatat BLEU 0,417 dan ROUGE-L 0,468, sedangkan T5 Base tanpa fine-tuning memiliki BLEU 0,327 dan ROUGE-L 0,246. Selain itu, prompt yang lebih terstruktur menghasilkan evaluasi lebih baik. Penelitian ini membuktikan bahwa T5 XL meningkatkan efisiensi dan akurasi evaluasi CPMK dalam OBE.