AI Hardware & Software Stack
Why is it hard to train spiking neural networks (SNNs), the models run in neuromorphic computing, directly with the backpropagation used for ordinary neural networks?
- ASNNs have no weights
- BSpiking is a discontinuous operation decided by whether a threshold is crossed, so it is not differentiable
- CSNNs can only perform inference and have no concept of training
Show answer and explanation
Answer: B. Spiking is a discontinuous operation decided by whether a threshold is crossed, so it is not differentiable
Each SNN neuron accumulates incoming spikes as a membrane potential and emits a spike when it crosses a threshold. Because this output switches discontinuously between 0 and 1, the gradient that backpropagation needs is zero almost everywhere or undefined. Researchers therefore use gradients that approximate the spike function with a smooth function during training, or train a conventional network and convert it into an SNN. Establishing training methods is one of the challenges of neuromorphic computing.
- A. SNNs have synaptic weights, and those are what is trained.
- B. Correct. Discontinuous firing makes gradients hard to compute.
- C. SNNs can be trained, but special methods are needed.