Knowledge Engineering and Data Science
Vol 7, No 1 (2024)

Exploring LSTM-based Attention Mechanisms with PSO and Grid Search under Different Normalization Techniques for Energy demands Time Series Forecasting

Pranolo, Andri (Unknown)
Zhou, Xiaofeng (Unknown)
Mao, Yingchi (Unknown)
Pratolo, Bambang Widi (Unknown)
Wibawa, Aji Prasetya (Unknown)
Utama, Agung Bella Putra (Unknown)
Ba, Abdoul Fatakhou (Unknown)
Muhammad, Abdullahi Uwaisu (Unknown)



Article Info

Publish Date
16 Apr 2024

Abstract

Advanced analytical approaches are required to accurately forecast the energy sector's rising complexity and volume of time series data.  This research aims to forecast the energy demand utilizing sophisticated Long Short-Term Memory (LSTM) configurations with Attention mechanisms (Att), Grid search, and Particle Swarm Optimization (PSO). In addition, the study also examines the influence of Min-Max and Z-Score normalization approaches in the preprocessing stage on the accuracy performances of the baselines and the proposed models. PSO and Grid Search techniques are used to select the best hyperparameters for LSTM models, while the attention mechanism selects the important input for the LSTM. The research compares the performance of baselines (LSTM, Grid-search-LSTM, and PSO-LSTM) and proposes models (Att-LSTM, Att-Grid-search-LSTM, and Att-PSO-LSTM) based on MAPE, RMSE, and R2 metrics into two scenarios normalization: Min-Max, and Z-Score. The results show that all models with Min-Max normalization have better MAPE, RMSE, and R2 than those with Z-Score. The best model performance is shown in Att-PSO-LSTM MAPE 3.1135, RMSE 0.0551, and R2 0.9233, followed by Att-Grid-search-LSTM, Att-LSTM, PSO-LSTM, Grid-search-LSTM, and LSTM. These findings emphasize the effectiveness of attention mechanisms in improving model predictions and the influence of normalization methods on model performance. This study's novel approach provides valuable insights into time series forecasting in energy demands.

Copyrights © 2024






Journal Info

Abbrev

keds

Publisher

Subject

Computer Science & IT Engineering

Description

Knowledge Engineering and Data Science (2597-4637), KEDS, brings together researchers, industry practitioners, and potential users, to promote collaborations, exchange ideas and practices, discuss new opportunities, and investigate analytics frameworks on data-driven and knowledge base ...