JURNAL TEKNOLOGI DAN OPEN SOURCE
Vol. 9 No. 1 (2026): Jurnal Teknologi dan Open Source, June 2026

Semantic Diversity In The Formation Of Story Questions From The Big Language Model

Andra Havid Andra Ramadhon (andra)
Agung Cahyono (Unknown)
Taufiq Agung Cahyono (Unknown)



Article Info

Publish Date
16 Jun 2026

Abstract

The development of Large Language Models (LLMs) has opened new opportunities for the automatic generation of math word problems (MWPs). However, many existing approaches still produce repetitive and template-based problems due to limited variation in context, narrative structure, and semantic relationships. This limitation reduces the effectiveness of such problems in assessing students’ conceptual understanding. This study aims to develop a semantic diversity pipeline based on context-aware generation to produce more varied, meaningful, and curriculum-aligned math word problems for Indonesian elementary education. The proposed method involves building an Indonesian MWP dataset, fine-tuning an LLM using Low-Rank Adaptation (LoRA), and designing a generation pipeline consisting of context retrieval, prompt diversification, semantic evaluation, and a solvability filtering mechanism. Evaluation was conducted using automated metrics, including Self-BLEU, Jaccard Similarity, and Cosine Similarity based on Sentence-BERT, as well as qualitative assessments from mathematics teachers. The results show that the proposed approach successfully improves lexical, semantic, and contextual diversity in generated problems. The Self-BLEU score of 4.79 indicates low repetition, the Jaccard Similarity score of 0.194 reflects high vocabulary variation, and the Cosine Similarity score of 0.424 demonstrates balanced semantic diversity while maintaining mathematical consistency. Teacher evaluations further confirm that the generated problems are relevant to the curriculum, appropriately challenging, and more natural compared to conventional methods. Overall, this research contributes to the development of more adaptive and diverse LLM-based math word problem generation systems for mathematics learning in Indonesia.

Copyrights © 2026






Journal Info

Abbrev

JTOS

Publisher

Subject

Computer Science & IT

Description

Jurnal Teknologi dan Open Source menerbitkan naskah ilmiah. yang berkaitan dengan sistem informasi, teknologi informasi dan aplikasi open source secara berkala (2 kali setahun). Jurnal ini dikelola dan diterbitkan oleh Program Studi Teknik Informatika Fakultas Teknik, Universitas Islam Kuantan ...