<![CDATA[Large Language Models (LLMs) such as BERT have demonstrated impressive performance across various NLP tasks, yet their high computational cost poses challenges for deployment in resource-constrained environments. This paper proposes a dynamic temporal token pruning approach based on gradient oscillation monitoring, where token importance is estimated from the temporal variability of gradient signals during training. Tokens exhibiting low gradient oscillation are selectively pruned to reduce effective input length. To mitigate potential performance degradation caused by aggressive pruning, an accuracy recovery mechanism based on lightweight re-finetuning is introduced. Experimental results on benchmark sentiment classification datasets, including IMDB and SST-2, demonstrate that the proposed method substantially reduces the number of input tokens while maintaining or recovering predictive performance. These results indicate that gradient oscillation provides a viable signal for token-level efficiency, achieving a favorable trade-off between input sparsity and model accuracy without modifying the underlying Transformer architecture.]]>
Copyrights © 2026
