AI Hardware & Software Stack

Which orders the AI software stack from the layer closest to users down to the layer closest to hardware?

  1. AFramework (such as PyTorch) → compiler or runtime → operation libraries (such as cuBLAS) → driver → hardware
  2. BHardware → framework → driver → libraries
  3. CDriver → framework → hardware → compiler
Show answer and explanation

Answer: A. Framework (such as PyTorch) → compiler or runtime → operation libraries (such as cuBLAS) → driver → hardware

Users write models in a framework such as PyTorch. The framework or an AI compiler or runtime optimizes the computational graph, and operations such as matrix products and convolutions call kernels from highly optimized libraries. The libraries send commands to the GPU or NPU through the driver. For new hardware to be widely adopted, this whole software stack must be mature, not just the raw arithmetic performance.

  • A. Correct. Each layer hides differences in the layers below.
  • B. The order is wrong.
  • C. The order is wrong.
Practice this topic Back to topic

More AI Hardware & Software Stack questions

  1. What is the difference between training and inference for a neural network?
  2. Which is NOT a reason training needs much more memory than inference for the same model?
  3. A 7-billion-parameter (7B) model is trained with weights, gradients and Adam's two states all held in FP32 (4 bytes). Excluding activations, roughly how much memory is needed?
  4. If a 7-billion-parameter (7B) model's weights are held in FP16 (2 bytes), roughly how much memory do the weights take?
  5. If the 7B model (about 14 GB in FP16) is quantized to 4-bit integers (INT4), roughly how much memory do the weights take?