Article Per Year (5 Year)
p-Index From 2021 - 2026
0.778
P-Index
This Author published in this journals
All Journal CommIT (Communication & Information Technology) Jurnal Telematika Jurnal Teknik Informatika (JUTIF) Engineering Science Letter Jurnal INFOTEL
Martina, Inge
Unknown Affiliation
Computer Science & IT Control & Systems Engineering Electrical & Electronics Engineering Engineering Industrial & Manufacturing Engineering Materials Science & Nanotechnology

Published : 11 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 11 Documents
Search

 1   2 
Analisis Pengaruh Karakteristik Masukan Teks terhadap Kinerja MiniLMv2-L6-H384 dan BERT-Base-Uncased pada Quora Question Pairs Wardani, Ken Ratri; Martina, Inge; Fong Xin Wern, Jimmy
Jurnal Telematika Vol. 20 No. 2 (2025)
Publisher : Yayasan Petra Harapan Bangsa

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.61769/telematika.v20i2.775

Abstract

Knowledge distillation is a technique for simplifying large language models into more concise models while maintaining accuracy. Bidirectional encoder representations from transformers (BERT) offer strong performance but require significant computational resources, whereas mini language models (MiniLM) are five times smaller. This study aims to compare the performance of these two models on a Quora question-pair dataset, focusing on the effects of sequence length and token rarity on classification accuracy. Both models were trained using identical training parameters. Test results show that BERT achieves 91.22% accuracy and 88.17% F1-score, slightly outperforming MiniLM, which achieves 90.12% accuracy and 86.73% F1-score. However, MiniLM provides 5.3 times faster inference speed. These findings provide empirical guidance for model optimisation in environments with limited computational resources or real-time response requirements, where MiniLM's efficiency is acceptable with a slight decrease in accuracy. Future research is recommended to explore hybrid systems that delegate complex tasks to large models and general tasks to smaller models.
Co-Authors Afifudi, Irfin Angela, Dina Chrisandy, Nicholas Fong Xin Wern, Jimmy Gamaliel, Yoyok Yusman Hartanto, Harry Hery Heryanto Hutagalung, Maclaurin Indriati, Ellen Irfin Afifudin Ken Ratri Retno Wardani Nugroho, Tunggul Arief Pontolaeng, Hanna Ribkah Pratama, Dionisius Rengku, Alwin Silitonga, Roland Y. H. The Houw Liong Tindaon, Phila Delphia Wardani, Ken Ratri Yulian, Hans