AI Hardware & Software Stack

Why is it hard to train spiking neural networks (SNNs), the models run in neuromorphic computing, directly with the backpropagation used for ordinary neural networks?

  1. ASNNs have no weights
  2. BSpiking is a discontinuous operation decided by whether a threshold is crossed, so it is not differentiable
  3. CSNNs can only perform inference and have no concept of training
Show answer and explanation

Answer: B. Spiking is a discontinuous operation decided by whether a threshold is crossed, so it is not differentiable

Each SNN neuron accumulates incoming spikes as a membrane potential and emits a spike when it crosses a threshold. Because this output switches discontinuously between 0 and 1, the gradient that backpropagation needs is zero almost everywhere or undefined. Researchers therefore use gradients that approximate the spike function with a smooth function during training, or train a conventional network and convert it into an SNN. Establishing training methods is one of the challenges of neuromorphic computing.

  • A. SNNs have synaptic weights, and those are what is trained.
  • B. Correct. Discontinuous firing makes gradients hard to compute.
  • C. SNNs can be trained, but special methods are needed.
Practice this topic Back to topic

Related calculators

Books on AI Hardware & Software Stack Ad

As an Amazon Associate I earn from qualifying purchases.

More AI Hardware & Software Stack questions

  1. What is the difference between training and inference for a neural network?
  2. Which is NOT a reason training needs much more memory than inference for the same model?
  3. A 7-billion-parameter (7B) model is trained with weights, gradients and Adam's two states all held in FP32 (4 bytes). Excluding activations, roughly how much memory is needed?
  4. If a 7-billion-parameter (7B) model's weights are held in FP16 (2 bytes), roughly how much memory do the weights take?
  5. If the 7B model (about 14 GB in FP16) is quantized to 4-bit integers (INT4), roughly how much memory do the weights take?

Comparisons in this topic