Building arbitrarily deep models of Spiking Convolutional Neural Networks on SpiNNaker
For an easy and simple implementation of Spiking Convolutional Neural Networks on SpiNNaker this additional Python package SpikingConvNet can be used. This package contains the necessary infrastructure and algorithms to build an arbitrarily deep network described as a sequential model.
Install MNIST dataset into main directory:
$ git clone https://github.com/sorki/python-mnist
$ cd python-mnist
$ bash get_data.sh
```
$ python main.py --mode loaddata
$ python main.py --mode training --layer 1
$ python main.py --mode training --layer 2
...
$ python main.py --mode training --layer n
$ python main.py --mode training --layer svm
$ python main.py --mode testing
```
contains the object classes for creating models of the Spiking Neural Networks and the necessary infrastructure for implementing such networks on SpiN- Naker. A SCNN model is built by sequentially adding of convolutional layers to the input layer and a classifier at last. The Spinnaker Network class holds instances and methods to interact with the SpiNNaker board. The structure is based on PyNNs procedure of creating Spiking Networks. Neurons belonging to a particular layer of the neural net- work and share common properties are packed into populations. Projections establish connections between populations. The strength of a connection is expressed by a numerical value, the synaptic weight that either increases the membrane potential of the post-neuron (exhibitory) or lessens the membrane potential (inhibitory). The model is defined on the host-computer and then transferred to the SpiN- Naker board, where the simulation is processed. When the simulation finishes, data are retrieved back to the host-computer and post-processed.
provides functions for generating sparse connections between neural popula- tions. The so-called projections between populations are employed by con- nection lists, which are automatically built by considering the tensors of the layers to be connected. Stride and kernel size specify the particular tensor of the posterior layer by Tn+1 = (Tn − shape)/stride + 1. In addition, the algorithms module supports the transformation of spiketrains. Training deeper layers requires the network of the previous layers to be rebuilt to obtain previous spiketrains. These spiketrains are transformed into plain times to assign them to a spiking source array and train the actual layer with the STDP update rule by windowing the input pattern over several time in- stances. The training of the network is done layer by layer, hence the input spikes of the currently trained layer depend on the previous layer. So a new simulation cycle is started when a deeper layer is trained. Because the synap- tic weights of previous layers are already determined, the previous layers are rebuild as parallel entities and therefore profit in terms of computational speed.
Supporting functions to visualise and handle data. The training and test set of the Spiking Neural Network is obtained by the MNIST dataset. Loading a specified subset of MNIST digits defined in parameters.py and supplies the network algorithms a shuffled set of data points. Processed data is visualised with the plotting functions: convolutional kernels can be plotted with heatmaps, spike times of each neuron along the time axis and the membrane voltages over time.