The random variable $X$ takes values in $\{-1, 0, 1\}$ with probabilities $P(X = -1) = P(X = 1)$ and $\alpha$ and $P(X = 0) = 1 - 2\alpha$, where $0 < \alpha < \frac{1}{2}$. Let $g(\alpha)$ denote the entropy of $X$ (in bits), parameterized by $\alpha$. Which of the following statements is/are TRUE?
The problem involves calculating the entropy of a random variable \(X\) with a given probability distribution. Entropy measures the uncertainty or randomness of a random variable. The random variable \(X\) can take values in \(\{-1, 0, 1\}\) with probabilities \(P(X = -1) = \alpha\), \(P(X = 1) = \alpha\), and \(P(X = 0) = 1 - 2\alpha\).
The entropy \(g(\alpha)\) is calculated using the formula:
\(g(\alpha) = - \sum P(x) \log_2 P(x)\)
Breaking it down for this problem:
- Entropy for \(X = -1\): \(-\alpha \log_2(\alpha)\)
- Entropy for \(X = 0\): \(-(1 - 2\alpha) \log_2(1 - 2\alpha)\)
- Entropy for \(X = 1\): \(-\alpha \log_2(\alpha)\)
Thus, the total entropy \(g(\alpha)\) is:
\(g(\alpha) = -2\alpha \log_2(\alpha) - (1 - 2\alpha) \log_2(1 - 2\alpha)\)
Now, let's evaluate the given options using this formula:
- Check for \(g(0.3) \gt g(0.4)\):
Calculate \(g(0.3)\):
\(g(0.3) = -2(0.3) \log_2(0.3) - (1 - 0.6) \log_2(0.4)\)
\(g(0.3) \approx 1.4855\)
Calculate \(g(0.4)\):
\(g(0.4) = -2(0.4) \log_2(0.4) - (1 - 0.8) \log_2(0.2)\)
\(g(0.4) \approx 1.3219\)
Therefore, \(g(0.3) \gt g(0.4)\).
- Check for \(g(0.3) \gt g(0.25)\):
Previously calculated \(g(0.3) \approx 1.4855\).
Calculate \(g(0.25)\):
\(g(0.25) = -2(0.25) \log_2(0.25) - (1 - 0.5) \log_2(0.5)\)
\(g(0.25) \approx 1.5\)
Therefore, \(g(0.3) \gt g(0.25)\).
Therefore, the correct answers are
$g(0.3) > g(0.4)$
and
$g(0.3) > g(0.25)$
.
Important Questions from Information Theory
The main processing functions of information system are given below. Arrange them in sequencing order:
(A) Process transaction
(B) Maintain master file
(C) Process enquiry
(D) Process report
(E) Process interactive supper applications
Choose the correct answer from the options given below:
- Consider a discrete memoryless source with an alphabet of four source symbols.
$s(t)$ is a multi-level (-1, 0, +1, +2) signal representing a long sequence of random symbols from the above source which is generating $10^4$ symbols per second.
Which of the following options is the correct value of equivalent Nyquist bandwidth of $s(t)$? - Match items in List – I with items in List – II and select the correct answer from the code given below :
List – I List – II a. The Screening Hypothesis 1. George A. Akerlof b. Job Market Signalling 2. K.J. Arrow c. The problem of moral hazard in the case of medical insurance 3. M. Spencer d. The market for lemons 4. Paul W. Miller and Paul A. Volcker
Codes : A source transmits symbols from an alphabet of size 16. The value of maximum achievable entropy (in bits) is _______
An analog baseband signal, bandlimited to 100 Hz, is sampled at the Nyquist rate. The samples are quantized into four message symbols that occur independently with probabilities $p_1 = p_4 = 0.125$ and $p_2 = p_3$. The information rate (bits/sec) of the message source is ____________
