Claim Missing Document
Check
Articles

Found 3 Documents
Search

Ensemble of winning tickets: pruning bidirectional encoder from the transformers attention heads for enhanced model efficiency Smarts, Nyalalani; Selvaraj, Rajalakshmi; Kuthadi, Venu Madhav
International Journal of Electrical and Computer Engineering (IJECE) Vol 15, No 2: April 2025
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijece.v15i2.pp2070-2080

Abstract

The advanced models of deep neural networks like bidirectional encoder from the transformers (BERT) and others, poses challenges in terms of computational resources and model size. In order to tackle these issues, techniques of model pruning have surfaced as the most useful methods in addressing the issues of model complexity. This research paper explores the concept of pruning BERT attention heads across the ensemble of winning tickets in order to enhance the efficiency of the model without sacrificing performance. Experimental evaluations show how effective the approach is, in achieving significant model compression while still maintaining competitive performance across different natural language processing tasks. The key findings of this study include model size that has been reduced by 36%, with our ensemble model reaching greater performance as compared to the baseline BERT model on both Stanford Sentiment Treebank v2 (SST-2) and Corpus of Linguistic Acceptability (CoLA) datasets. The results further show a F1-score of 94% and 96%, respectively, and accuracy scores of 95% and 96% on the two datasets. The findings of this research paper contribute to the ongoing efforts in enhancing the efficiency of large-scale language models.
A review on ischemic heart disease prediction frameworks using machine learning Bhende, Kabo Clifford; Sigwele, Tshiamo; Mokgethi, Chandapiwa; Maenge, Aone; Kuthadi, Venu Madhav
International Journal of Advances in Applied Sciences Vol 14, No 2: June 2025
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijaas.v14.i2.pp361-372

Abstract

Ischemic heart disease (IHD) is a leading cause of mortality worldwide, calling for advanced predictive models for timely intervention. Current literature reviews on machine learning (ML)-based IHD prediction frameworks often focus on predictive accuracy but lack depth in areas like dataset diversity, model interpretability, and privacy considerations. Existing IHD prediction frameworks face limitations, including reliance on small, homogenous datasets, limited critical analysis, and issues with model transparency, reducing their clinical utility. This review addresses these gaps through a systematic, comparative analysis of popular ML models, such as random forest (RF) and support vector machines (SVM), noting their strengths and limitations. Key contributions include a qualitative examination of prevalent tools, datasets, and evaluation metrics, identification of gaps in dataset diversity and interpretability; and recommendations for improving model transparency and data privacy. Major findings reveal a trend toward ensemble models for accuracy but highlight the need for explainable artificial intelligence (AI) to support clinical decisions. Future directions include using federated learning to enhance data privacy, integrating unstructured data for comprehensive prediction, and advancing explainable AI to build trust among healthcare providers. By addressing these areas, this review aims to guide future research toward developing robust, transparent ML frameworks that can be more effectively deployed in clinical settings.
Adaptive sentiment analysis for stock markets using deep learning Mawere, Talent; Rajalakshmi, Selvaraj; Kuthadi, Venu Madhav; Dinekanyane, Othlapile
International Journal of Advances in Applied Sciences Vol 15, No 1: March 2026
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijaas.v15.i1.pp416-426

Abstract

Stock markets are highly volatile, making price prediction very difficult. One of the factors influencing the volatility of financial markets is rapidly changing news sentiment. This study presents a novel adaptive deep learning (DL) framework for sentiment analysis with concept drift capabilities. The proposed model combines convolutional neural networks (CNN), bidirectional long short-term memory (BiLSTM), and attention mechanisms in its processing architecture. The model inputs preprocessed news headlines into both the CNN and BiLSTM-Attention networks to extract local features, model contextual dependencies, and prioritizes important sentiment cues in its prediction mechanism. We use FastText and Word2Vec for word embeddings, while incremental learning is used to manage concept drift. One key advantage of handling concept drift is that the model can continuously learn new patterns in data streams without needing to fully retrain the model. The model is validated on a curated dataset from various sources with superior performance across all metrics, like accuracy (0.9753) and an F1-score (0.98). It significantly outperforms benchmarks like distilled bidirectional encoder representations from transformers (DistilBERT), LSTM, and valence aware dictionary and sentiment reasoner (VADER). A run of ten iterations validated that the real-time pipeline did not exceed 200 ms in processing and classifying headlines. This signifies the practical viability of our model in fintech applications such as algorithmic trading and risk management.