Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Journal of Electrical Technology
Indra Listiawan
Universitas AMIKOM Yogyakarta
Automotive Engineering Computer Science & IT Control & Systems Engineering Electrical & Electronics Engineering Energy Engineering Industrial & Manufacturing Engineering Materials Science & Nanotechnology

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Temporal Gradient Oscillation with Accuracy Recovery Mechanism for Efficient BERT-Based Text Classification Indra Listiawan; Ema Utami; Kusrini Kusrini; Arief Setyanto
JET (Journal of Electrical Technology) Vol 11, No 2 (2026): : Edisi June
Publisher : Universitas Islam Sumatera Utara

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.30743/jet.v11i2.13472

Abstract

Large Language Models (LLMs) such as BERT have demonstrated impressive performance across various NLP tasks, yet their high computational cost poses challenges for deployment in resource-constrained environments. This paper proposes a dynamic temporal token pruning approach based on gradient oscillation monitoring, where token importance is estimated from the temporal variability of gradient signals during training. Tokens exhibiting low gradient oscillation are selectively pruned to reduce effective input length. To mitigate potential performance degradation caused by aggressive pruning, an accuracy recovery mechanism based on lightweight re-finetuning is introduced. Experimental results on benchmark sentiment classification datasets, including IMDB and SST-2, demonstrate that the proposed method substantially reduces the number of input tokens while maintaining or recovering predictive performance. These results indicate that gradient oscillation provides a viable signal for token-level efficiency, achieving a favorable trade-off between input sparsity and model accuracy without modifying the underlying Transformer architecture.
Co-Authors Arief Setyanto Ema Utami Kusrini