Text Similarity Analysis for Evaluating Alignment Between Lesson Plans and Teaching Reports - Rachmat Chrismanto, Antonius; Sudiarto Raharjo, Willy; Gilang Purnajati, Oscar (Fakultas Ilmu Komputer, Universitas Klabat, 2025)
RPS (Rencana Pembelajaran Semester, or called Lesson Plans) is a class activity planning document in the higher education learning process that includes learning outcomes, methods, learning strategy, and evaluation criteria. It is created by the lecturers in charge of the course and coordinated with the relevant department. This document needs to be monitored throughout the semester for its conformity with the implementation document (Borang Pelaksanaan Perkuliahan (BPP)). It was done manually through our eRPS system, but it requires a lot of effort and precision and is not time-efficient. This research focused on evaluating the effectiveness of several content-based text similarity methods to detect RPS conformity compared with the BPP, or called Teaching Reports document. The Boyer-Moore (B), Rabin-Karp (R), Jaccard (JC), Jaro-Winkler (JW), Smith-Waterman (SW), Knuth-Morris-Pratt (K), Levenehtein cosine similarity (C), Dice (D), Jaro (J), and Soundex (S) algorithms were evaluated in this paper. In the vector-based similarity method, TF-IDF was used. The evaluation of 11 string-matching algorithms across four scenarios demonstrated clear performance trends. Fuzzy algorithms (SW with accuracy 0,845–0,870, and JW with accuracy 0,840-0,850) achieved the highest accuracy in a single row of lecturer scenario, while exact/pattern-based algorithms (B, K, and S with accuracy 0,8625–0,8725) on a combination of all rows of lectures with minimal variance (≈0,005–0,015). Pre-processing benefits fuzzy algorithms (+2.5%) but is neutral for exact/pattern-based algorithms. The combined scenario improves the exact/phonetic algorithms (+6–7%) but reduces the fuzzy performance algorithm (−10–14%). The optimal thresholds were generally 40–50%, except for JW and J, which were 65%.