GPU Programming

What is a characteristic of the tensor cores (matrix units) in recent GPUs?

  1. AThey compute products and sums of small matrices in a single instruction, greatly accelerating AI mainly with low-precision numbers
  2. BThey are sensors that measure a physical quantity called a tensor
  3. CThey are circuits dedicated to drawing shadows in graphics
Show answer and explanation

Answer: A. They compute products and sums of small matrices in a single instruction, greatly accelerating AI mainly with low-precision numbers

A tensor core multiplies and accumulates small matrices, such as 4 × 4 or 16 × 16, in a single instruction. For the same area and power it performs many times more work than ordinary units doing one multiply and add at a time. It is mostly used with low precisions such as FP16, BF16, FP8 and INT8, greatly speeding up the matrix products that dominate neural network training and inference.

  • A. Correct. They are units specialized for matrix multiplication.
  • B. They are not sensors.
  • C. They are not for shadow rendering.
Practice this topic Back to topic

More GPU Programming questions

  1. In GPU mixed-precision computation, why are multiplications often done in a low precision such as FP16 while accumulation is done in FP32?
  2. Summing the same floating-point data on a GPU gives results whose last digits differ slightly from run to run. What is the most likely cause?
  3. What is a common guideline for choosing the number of threads per block in CUDA?
  4. What is the advantage of writing a kernel with a loop like the one above (a grid-stride loop)?
  5. For what kind of access does CUDA constant memory perform especially well?