Benchmarking open source deep learning frameworks

Ghadeer Al-Bdour (Jordan University of Science and Technology)
Raffi Al-Qurran (Jordan University of Science and Technology)
Mahmoud Al-Ayyoub (Jordan University of Science and Technology)
Ali Shatnawi (Jordan University of Science and Technology)

International Journal of Electrical and Computer Engineering
Vol 10, No 5: October 2020

Deep Learning (DL) is one of the hottest fields. To foster the growth of DL, several open source frameworks appeared providing implementations of the most common DL algorithms. These frameworks vary in the algorithms they support and in the quality of their implementations. The purpose of this work is to provide a qualitative and quantitative comparison among three such frameworks: TensorFlow, Theano and CNTK. To ensure that our study is as comprehensive as possible, we consider multiple benchmark datasets from different fields (image processing, NLP, etc.) and measure the performance of the frameworks' implementations of different DL algorithms. For most of our experiments, we find out that CNTK's implementations are superior to the other ones under consideration.