Claim Missing Document
Check
Articles

Found 2 Documents
Search

Ensemble of winning tickets: pruning bidirectional encoder from the transformers attention heads for enhanced model efficiency Smarts, Nyalalani; Selvaraj, Rajalakshmi; Kuthadi, Venu Madhav
International Journal of Electrical and Computer Engineering (IJECE) Vol 15, No 2: April 2025
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijece.v15i2.pp2070-2080

Abstract

The advanced models of deep neural networks like bidirectional encoder from the transformers (BERT) and others, poses challenges in terms of computational resources and model size. In order to tackle these issues, techniques of model pruning have surfaced as the most useful methods in addressing the issues of model complexity. This research paper explores the concept of pruning BERT attention heads across the ensemble of winning tickets in order to enhance the efficiency of the model without sacrificing performance. Experimental evaluations show how effective the approach is, in achieving significant model compression while still maintaining competitive performance across different natural language processing tasks. The key findings of this study include model size that has been reduced by 36%, with our ensemble model reaching greater performance as compared to the baseline BERT model on both Stanford Sentiment Treebank v2 (SST-2) and Corpus of Linguistic Acceptability (CoLA) datasets. The results further show a F1-score of 94% and 96%, respectively, and accuracy scores of 95% and 96% on the two datasets. The findings of this research paper contribute to the ongoing efforts in enhancing the efficiency of large-scale language models.
A review on ischemic heart disease prediction frameworks using machine learning Bhende, Kabo Clifford; Sigwele, Tshiamo; Mokgethi, Chandapiwa; Maenge, Aone; Kuthadi, Venu Madhav
International Journal of Advances in Applied Sciences Vol 14, No 2: June 2025
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijaas.v14.i2.pp361-372

Abstract

Ischemic heart disease (IHD) is a leading cause of mortality worldwide, calling for advanced predictive models for timely intervention. Current literature reviews on machine learning (ML)-based IHD prediction frameworks often focus on predictive accuracy but lack depth in areas like dataset diversity, model interpretability, and privacy considerations. Existing IHD prediction frameworks face limitations, including reliance on small, homogenous datasets, limited critical analysis, and issues with model transparency, reducing their clinical utility. This review addresses these gaps through a systematic, comparative analysis of popular ML models, such as random forest (RF) and support vector machines (SVM), noting their strengths and limitations. Key contributions include a qualitative examination of prevalent tools, datasets, and evaluation metrics, identification of gaps in dataset diversity and interpretability; and recommendations for improving model transparency and data privacy. Major findings reveal a trend toward ensemble models for accuracy but highlight the need for explainable artificial intelligence (AI) to support clinical decisions. Future directions include using federated learning to enhance data privacy, integrating unstructured data for comprehensive prediction, and advancing explainable AI to build trust among healthcare providers. By addressing these areas, this review aims to guide future research toward developing robust, transparent ML frameworks that can be more effectively deployed in clinical settings.