AI Hardware & Software Stack

What do AI compilers such as XLA, TVM or PyTorch's compile feature mainly do?

  1. ATranslate an AI model's answers into human language
  2. BAutomatically collect training data
  3. CAnalyze the model's computational graph, optimize fusion, data layout and tiling, and generate code for the target hardware
Show answer and explanation

Answer: C. Analyze the model's computational graph, optimize fusion, data layout and tiling, and generate code for the target hardware

An AI compiler takes the model as a computational graph of operations and data flows, fuses operations where possible, removes unnecessary computation, chooses memory layouts and tunes tile sizes, then generates kernels suited to the target, such as a GPU or NPU. It automatically produces optimized code for each hardware target from the same model, greatly reducing manual optimization work.

  • A. It has nothing to do with natural-language translation.
  • B. They do not collect data.
  • C. Correct. They exist to extract hardware performance.
Practice this topic Back to topic

More AI Hardware & Software Stack questions

  1. Which describes the computational graph used by AI frameworks and compilers?
  2. Which describes ONNX?
  3. Why can inference runtimes such as TensorRT or ONNX Runtime run faster than using the training framework directly for inference?
  4. Why do element-wise operations such as ReLU or layer normalization tend to take a lot of time relative to their operation count in AI workloads?
  5. Why are convolutions in CNNs sometimes converted into matrix multiplications, for example with im2col?