Implementation in Julia
There are many good and tested libraries for deep learning in popular programming languages:
Theano (Python) can utilize both CPU and GPU (from the MILA Lab at the University of Montreal)
Torch (Lua) is a Matlab-like environment (from Ronan Collobert, Clement Farabet, and Koray Kavukcuoglu)
Tensorflow (Python) makes use of data flow graphs
MXNet (Python, R, Julia, C++)
Caffe is the most popular and widely used
Keras (Python) based on Theano
Mocha (Julia) by Chiyuan Zhang
We will mainly go through Mocha for Julia, which is an amazing package written by Chiyuan Zhang, a PhD student at MIT.
To start, add the package as follows:
Pkg.update()
Pkg.add("Mocha")
Network architecture
Network architecture in Mocha refers to a set of layers:
data_layer = HDF5DataLayer(name="data", source="data-list.txt", batch_size=64, tops=[:data]) ip_layer = InnerProductLayer(name="ip", output_dim=500, tops=[:ip], bottoms=[:data])
The input of the
ip_layer
has the same name...