English
Menu

What is Recurrent neural network (RNN)?

A neural network that processes sequence data one element at a time and passes the information so far to the next time step as a hidden state. The same weights are reused at every time step, so sequences of different lengths can be handled. Because vanishing gradients readily occur on long sequences, LSTM and GRU, which control the retention and forgetting of information with gates, were devised. Because it must compute in time order and is hard to parallelize, it has been replaced by the Transformer in many language-processing applications.

Japanese name: 再帰型ニューラルネットワーク(RNN)

Recurrent neural network (RNN) appears in 2 practice questions across 1 topic, including Machine Learning Basics.

Field: Machine Learning Basics

Try it: questions about Recurrent neural network (RNN)

Which of the following correctly describes recurrent neural networks (RNNs) and LSTMs?

  1. AAn RNN applies the same filter at every position of an image to extract local features, and an LSTM improves its pooling layer to make it robust to shifts in position
  2. BAn RNN treats all elements of a sequence independently and processes them simultaneously in parallel without using results from earlier time steps, so longer sequences train faster
  3. CAn RNN processes a sequence in order, carrying the hidden state from one time step to the next, and an LSTM uses gates to control whether information is kept or forgotten, making vanishing gradients less likely on long sequences
Show answer and explanation

Answer: C. An RNN processes a sequence in order, carrying the hidden state from one time step to the next, and an LSTM uses gates to control whether information is kept or forgotten, making vanishing gradients less likely on long sequences

An RNN processes sequence data such as text, audio or sensor time series one element at a time, passing the information so far to the next time step as a hidden state. Because errors are propagated back through time, gradients readily vanish or explode on long sequences, making it hard to learn information from the distant past. An LSTM has a cell that stores information and input, forget and output gates, so it can keep needed information for a long time. However, both must compute in time order and are hard to parallelize, and the Transformer improved on this point.

Recurrent neural network (RNN) in the explanations

Questions that mention Recurrent neural network (RNN) (2)

  1. Which of the following correctly describes recurrent neural networks (RNNs) and LSTMs?
  2. Which of the following describes the self-attention used in the Transformer?

Books on Machine Learning Basics Ad

As an Amazon Associate I earn from qualifying purchases.

Related terms

Attention mechanism Activation function Confusion matrix Convolutional neural network (CNN) Cross-validation Gradient descent k-means clustering Overfitting Precision and recall Principal component analysis (PCA)

Back to the glossary