English
Menu

Machine Learning Basics

Which of the following correctly describes recurrent neural networks (RNNs) and LSTMs?

  1. AAn RNN applies the same filter at every position of an image to extract local features, and an LSTM improves its pooling layer to make it robust to shifts in position
  2. BAn RNN treats all elements of a sequence independently and processes them simultaneously in parallel without using results from earlier time steps, so longer sequences train faster
  3. CAn RNN processes a sequence in order, carrying the hidden state from one time step to the next, and an LSTM uses gates to control whether information is kept or forgotten, making vanishing gradients less likely on long sequences
Show answer and explanation

Answer: C. An RNN processes a sequence in order, carrying the hidden state from one time step to the next, and an LSTM uses gates to control whether information is kept or forgotten, making vanishing gradients less likely on long sequences

An RNN processes sequence data such as text, audio or sensor time series one element at a time, passing the information so far to the next time step as a hidden state. Because errors are propagated back through time, gradients readily vanish or explode on long sequences, making it hard to learn information from the distant past. An LSTM has a cell that stores information and input, forget and output gates, so it can keep needed information for a long time. However, both must compute in time order and are hard to parallelize, and the Transformer improved on this point.

  • A. This describes a convolutional neural network (CNN).
  • B. An RNN uses the result of the previous time step, so it must compute in time order and is hard to parallelize along the sequence.
  • C. Correct. It processes the sequence by carrying the hidden state forward, and the LSTM's gates make long-range dependencies easier to handle.
Practice this topic Back to topic

Related calculators

Confusion matrix calculatorCompute accuracy, precision, recall, F1, specificity, MCC and other metrics from TP, FN, FP and TNCNN output size calculatorStack convolution, pooling and fully connected layers and get each layer's output size, parameter count and multiply-accumulate count

Books on Machine Learning Basics Ad

As an Amazon Associate I earn from qualifying purchases.

More Machine Learning Basics questions

  1. Which of the following describes the self-attention used in the Transformer?
  2. Which of the following describes transfer learning, in which a model trained on a large amount of data is used for a different task with little data?
  3. Which of the following correctly describes models that generate images and similar data?
  4. Which of the following describes how a large language model (LLM) processes text?
  5. When building a model that predicts hiring decisions from application documents, using past hiring outcomes as the correct answers, which of the following should be kept in mind?