MagmaDNN: Accelerated Deep Learning Using MAGMA
Machine Learning/Artificial Intelligence
TimeWednesday, July 311:30pm - 2pm
DescriptionMagmaDNN is a deep learning framework driven using the highly optimized MAGMA dense linear algebra package. The library offers comparable performance to other popular frameworks, such as TensorFlow, PyTorch, and Theano. C++ is used to implement the framework providing fast memory operations, direct cuda access, and compile time errors. Common neural network layers such as Fully Connected, Convolutional, Pooling, Flatten, and Dropout are included. Hyperparameter tuning is performed with a parallel grid search engine.
MagmaDNN uses several techniques to accelerate network training. For instance, convolutions are performed using the Winograd algorithm and FFTs. Another approach is MagmaDNNs custom memory manager, which is used to reduce expensive memory transfers. Finally, training is accelerated by distributing batches across GPU nodes.
This paper provides an overview of the MagmaDNN framework and how it leverages the MAGMA library to attain speed increases. This paper also addresses how deep networks are accelerated by training in parallel and further challenges with parallelization.