Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying a broad class of distributed tensor computations. The purpose of Mesh TensorFlow is to formalize and implement distribution strategies for your computation graph over your hardware/processors. For example: {Split  the batch over rows of processors and split the  units in the hidden layer across columns of  processors.} Mesh TensorFlow is implemented as a layer over TensorFlow.

WWW: https://github.com/tensorflow/mesh
