A Survey of Machine Learning Techniques for Self-tuning Hadoop Performance

Md. Armanur Rahman (Multimedia University)
J. Hossen (Multimedia University)
Venkataseshaiah C (Multimedia University)
CK Ho (Multimedia University)
Tan Kim Geok (Multimedia University)
Aziza Sultana (Dhaka International University)
Jesmeen M. Z. H. (Multimedia University)
Ferdous Hossain (Multimedia University)

International Journal of Electrical and Computer Engineering
Vol 8, No 3: June 2018

The Apache Hadoop framework is an open source implementation of MapReduce for processing and storing big data. However, to get the best performance from this is a big challenge because of its large number configuration parameters. In this paper, the concept of critical issues of Hadoop system, big data and machine learning have been highlighted and an analysis of some machine learning techniques applied so far, for improving the Hadoop performance is presented. Then, a promising machine learning technique using deep learning algorithm is proposed for Hadoop system performance improvement.