What is Gain margin?

Answer: B. The gain margin is the difference of the gain from 0 dB at the frequency where the phase is −180°, and the phase margin is the difference of the phase from −180° at the frequency where the gain is 0 dB

The gain margin is how many decibels the gain is below 0 dB at the phase crossover frequency, where the phase reaches −180°. The phase margin is how many degrees the phase still is from −180° at the gain crossover frequency, where the gain is 0 dB. Both measure how close the loop transfer function comes to the point −1 (gain 0 dB, phase −180°), and they are read from the loop transfer function, not the closed loop. For servo systems, a gain margin of about 10 to 20 dB and a phase margin of about 40 to 60° are common guidelines.

Answer: B. About 9.5 dB

The phase is −90° − tan⁻¹ω − tan⁻¹(ω/2); the two angles add up to 90° when ω × (ω/2) = 1, so the phase reaches −180° at ω = √2 rad/s. The gain there is |L(j√2)| = 2/(√2 × √3 × √6) = 1/3, so the gain margin is 20 log₁₀3 ≈ 9.5 dB. Multiplying the gain by 3 (K = 6) would reach the limit, which agrees with the stable range 0 < K < 6 found with the Routh–Hurwitz stability criterion.