Infolitika Journal of Data Science
Vol. 4 No. 1 (2026): May 2026

Ensemble Variable Importance: Combining Random Forest, Neural Network, and Support Vector Machine via Genetic Algorithm (Case Study: Student Productivity)

Asep Rusyana (Department of Statistics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)
Marzuki Marzuki (Department of Statistics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia
Department of Mathematics, Universiti Malaysia Terengganu, Kuala Nerus, Malaysia)

Siti Rusdiana (Department of Mathematics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)
Fitriana AR (Department of Statistics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)
Nurhasanah Nurhasanah (Department of Statistics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)
Nany Salwa (Department of Statistics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)
Mahmudi Mahmudi (Department of Mathematics, Universitas Syiah Kuala, Banda Aceh 23111, Indonesia)



Article Info

Publish Date
30 May 2026

Abstract

This study proposes and evaluates an ensemble variable‑importance framework that integrates permutation‑based importance scores from three distinct supervised learning algorithms: Random Forest, Neural Network, and Support Vector Machine, using a genetic‑algorithm optimizer. The approach addresses the well‑known problem that algorithm‑specific importance diagnostics can yield divergent feature rankings, complicating substantive interpretation and downstream decision‑making. Using a large publicly available student‑productivity dataset (N = 20,000), predictors describing study behavior, digital‑media use, lifestyle, and academic indicators were normalized with Min–Max scaling, and permutation variable importance (PVI) was estimated repeatedly within each model to obtain stable mean PVI values and standard errors. A genetic algorithm was then employed to search the space of ensemble weightings (rank‑aggregation solutions) that maximize a chosen fitness criterion—Spearman rank concordance with out‑of‑sample predictive relevance—thereby producing a consensus ranking of predictors. Empirical results indicate rapid GA convergence (fitness ≈ 0.82 within 20–30 generations) and strong cross‑model agreement for a small core of predictors: study hours (X3) and focus score (X15) consistently emerged as the most salient features across individual models and in the ensemble ranking. A secondary set of variables (e.g., sleep hours, phone usage, attendance, and stress level) displayed moderate importance, while several features exhibited model‑dependent variability in ranks. The ensemble procedure thereby yields stable, model‑agnostic importance estimates that enhance interpretability and reduce dependence on any single algorithm’s idiosyncrasies. We discuss implications for educational analytics and recommend external validation, targeted feature engineering, and sensitivity analyses (alternate scalings and GA settings) to assess robustness and to support reliable, actionable inferences from machine‑learning models in applied settings.

Copyrights © 2026






Journal Info

Abbrev

ijds

Publisher

Subject

Computer Science & IT Decision Sciences, Operations Research & Management Engineering

Description

Infolitika Journal of Data Science is a distinguished international scientific journal that showcases high caliber original research articles and comprehensive review papers in the field of data science. The journals core mission is to stimulate interdisciplinary research collaboration, facilitate ...