The recent breakthroughs in the field of Artificial Intelligence (AI) have brought deep learning to the forefront. Today, even more organizations are employing deep learning technologies for analyzing their data, which is often voluminous in nature. Hence, it's imperative that deep learning frameworks such as TensorFlow can be combined with big data platforms and pipelines.
The 2017 Facebook paper regarding training ImageNet in one hour using 256 GPUs spread over 32 servers (https://research.fb.com/wp-content/uploads/2017/06/imagenet1kin1h5.pdf) and a recent paper by Hong Kong Baptist University where they train ImageNet in four minutes using 2,048 GPUs (https://arxiv.org/pdf/1807.11205.pdf) prove that distributed AI can be a viable solution.
The main idea behind distributed AI is that the task can be divided into different processing clusters. A large number of frameworks have been proposed for distributed AI. We can use either...