Earlier stage for straggler detection and handling using combined CPU test and LATE methodology

Anwar H. Katrawi (Universiti Sains Malaysia)
Rosni Abdullah (Universiti Sains Malaysia)
Mohammed Anbar (Universiti Sains Malaysia)
Ammar Kamal Abasi (Universiti Sains Malaysia)

International Journal of Electrical and Computer Engineering
Vol 10, No 5: October 2020

Using MapReduce in Hadoop helps in lowering the execution time and power consumption for large scale data. However, there can be a delay in job processing in circumstances where tasks are assigned to bad or congested machines called "straggler tasks"; which increases the time, power consumptions and therefore increasing the costs and leading to a poor performance of computing systems. This research proposes a hybrid MapReduce framework referred to as the combinatory late-machine (CLM) framework. Implementation of this framework will facilitate early and timely detection and identification of stragglers thereby facilitating prompt appropriate and effective actions.