Sentiment classification using gradient modulation and layered attention - Natarajan, Bagiyalakshmi; Veeramakali, T. (Institute of Advanced Engineering and Science, 2025)
Sentiment analysis is a technique for evaluating text to ascertain whether a statement is positive, negative, or neutral. Currently, transformer-based models capture the contextual relationships among words in a phrase and accomplish sentiment analysis in a nuanced manner via multi-head attention. This approach, with a fixed number of layers and heads, struggles to find the complex relationships between phrases and their semantic structures. To mitigate this issue, the suggested technique incorporates the graded multi head attention model (GMHA) at the base of the distilled bidirectional encoder representations from transformers (DistilBERT) model. It is employed to augment the layers and heads progressively, capturing the relationships between sentences in a sophisticated manner. By increasing the layers and heads the proposed model extracts long-term and hierarchical relationships from the sentence. Additionally, the attention sentient optimization technique is introduced, which improves model learning by giving more weight to important words in a sentence. During training, the process checks to see which words (“amazing