International Journal of Electrical and Computer Engineering
Vol 13, No 2: April 2023

Enhancing the stability of the deep neural network using a non-constant learning rate for data stream

Hussein Abdul Ameer Abbas Al-Khamees (University of Babylon)
Nabeel Al-A'araji (University of Babylon)
Eman Salih Al-Shamery (University of Babylon)



Article Info

Publish Date
01 Apr 2023

Abstract

The data stream is considered the backbone of many real-world applications. These applications are most effective when using modern techniques of machine learning like deep neural networks (DNNs). DNNs are very sensitive to set parameters, the most prominent one is the learning rate. Choosing an appropriate learning rate value is critical because it is able to control the overall network performance. This paper presents a new developing DNN model using a multi-layer perceptron (MLP) structure that includes network training based on the optimal learning rate. Thereupon, this model consists of three hidden layers and does not adopt the stability of the learning rate but has a non-constant value (varying over time) to obtain the optimal learning rate which is able to reduce the error in each iteration and increase the model accuracy. This is done by deriving a new parameter that is added to and subtracted from the learning rate. The proposed model is evaluated by three streaming datasets: electricity, network security layer-knowledge discovery in database (NSL-KDD), and human gait database (HuGaDB) datasets. The results proved that the proposed model achieves better results than the constant model and outperforms previous models in terms of accuracy, where it achieved 88.16%, 98.67%, and 97.63% respectively.

Copyrights © 2023






Journal Info

Abbrev

IJECE

Publisher

Subject

Computer Science & IT Electrical & Electronics Engineering

Description

International Journal of Electrical and Computer Engineering (IJECE, ISSN: 2088-8708, a SCOPUS indexed Journal, SNIP: 1.001; SJR: 0.296; CiteScore: 0.99; SJR & CiteScore Q2 on both of the Electrical & Electronics Engineering, and Computer Science) is the official publication of the Institute of ...