English
Menu

What is Self-information?

The amount of information obtained on being told that an event of probability p has occurred. It is I = −log₂ p, measured in bits. It is larger for less likely events and 0 for an event that is certain (p = 1). An event of probability 1/2 carries 1 bit and one of probability 1/1024 carries 10 bits. The information from two independent events both occurring is the sum of their individual amounts. Averaging the information per symbol of a source, weighted by probability, gives its entropy. Also called information content.

Japanese name: 情報量

Self-information appears in 5 practice questions across 1 topic, including Information Theory and Coding Basics.

Field: Information Theory and Coding Basics

Try it: questions about Self-information

Let the amount of information obtained on being told that an event of probability p has occurred be −log₂ p bits. Which of the following is the amount of information obtained on being told that an event of probability 1/8 has occurred?

  1. A0.125 bits
  2. B0.375 bits
  3. C3 bits
Show answer and explanation

Answer: C. 3 bits

The amount of information is −log₂(1/8) = log₂ 8 = 3 bits. Since 1/8 = 1/2³, it is the same amount of information as being told the answer to a "one of two" question of probability 1/2 three times over. The less likely an event, the more information we receive on being told that it has occurred.

Which of the following correctly describes a property of the amount of information (−log₂ p on being told that an event of probability p has occurred)?

  1. AThe amount of information on being told that an event of probability 1, which always occurs, has occurred is 1 bit
  2. BThe amount of information on being told that two independent events have both occurred is the sum of their individual amounts of information
  3. CThe more probable an event, the more information we receive on being told that it has occurred
Show answer and explanation

Answer: B. The amount of information on being told that two independent events have both occurred is the sum of their individual amounts of information

Two independent events both occur with probability p₁ × p₂, so the amount of information is −log₂(p₁ × p₂) = (−log₂ p₁) + (−log₂ p₂), the sum of the individual amounts. Information is defined with a logarithm precisely to give it this additive property. Being told that a certain event has occurred tells us nothing new, so its amount of information is −log₂ 1 = 0.

Self-information in the explanations

Questions that mention Self-information (5)

  1. Let the amount of information obtained on being told that an event of probability p has occurred be −log₂ p bits. Which of the following is the amount of information obtained on being told that an event of probability 1/8 has occurred?
  2. Which of the following correctly describes a property of the amount of information (−log₂ p on being told that an event of probability p has occurred)?
  3. A source emits the symbols A, B, C and D independently with probabilities 1/2, 1/4, 1/8 and 1/8. Which of the following is the entropy (average amount of information per symbol) of this source?
  4. A source that emits 8 kinds of symbols has an entropy of 2.4 bits. Defining redundancy as 1 − (entropy ÷ maximum entropy), which of the following is the redundancy of this source?
  5. For two random variables X and Y, which of the following correctly describes the conditional entropy H(X|Y) and the mutual information I(X;Y)?

Books on Information Theory and Coding Basics Ad

As an Amazon Associate I earn from qualifying purchases.

Related terms

Information entropy Lossless compression Mutual information Average code length Binary symmetric channel Forward error correction (FEC) Hamming code Hamming distance Huffman coding Kraft inequality

Back to the glossary