An analog baseband signal, bandlimited to 100 Hz, is sampled at the Nyquist rate. The samples are quantized into four message symbols that occur independently with probabilities $p_1 = p_4 = 0.125$ and $p_2 = p_3$. The information rate (bits/sec) of the message source is ____________
The problem asks for the information rate (bits/sec) of a message source based on signal sampling and quantization.
1. Determine Sampling Rate (Nyquist Rate)
The analog baseband signal is bandlimited to 100 Hz. According to the Nyquist theorem, the minimum sampling rate required to avoid aliasing is twice the bandwidth.
Sampling Rate = Nyquist Rate = $2 \times \text{Bandwidth}$
Sampling Rate = $2 \times 100 \text{ Hz} = 200 \text{ samples/sec}$
2. Calculate Symbol Probabilities
There are four message symbols with probabilities $p_1, p_2, p_3, p_4$. We are given:
- $p_1 = 0.125$
- $p_4 = 0.125$
- $p_2 = p_3$
Since the sum of probabilities must be 1:
$p_1 + p_2 + p_3 + p_4 = 1$
$0.125 + p_2 + p_2 + 0.125 = 1$
$0.25 + 2p_2 = 1$
$2p_2 = 1 - 0.25 = 0.75$
$p_2 = 0.75 / 2 = 0.375$
Therefore, the probabilities are: $p_1 = 0.125$, $p_2 = 0.375$, $p_3 = 0.375$, $p_4 = 0.125$.
3. Calculate Entropy (Average Information per Symbol)
Entropy ($H$) measures the average information content per symbol. It is calculated using the formula:
$H = -\sum_{i=1}^{N} p_i \log_2(p_i)$
For N=4 symbols:
$H = -[ p_1 \log_2(p_1) + p_2 \log_2(p_2) + p_3 \log_2(p_3) + p_4 \log_2(p_4) ]$
$H = -[ 0.125 \log_2(0.125) + 0.375 \log_2(0.375) + 0.375 \log_2(0.375) + 0.125 \log_2(0.125) ]$
$H = -2 \times [ 0.125 \log_2(0.125) ] - 2 \times [ 0.375 \log_2(0.375) ]$
We know that $\log_2(0.125) = \log_2(1/8) = -3$.
And $\log_2(0.375) = \log_2(3/8) = \log_2(3) - \log_2(8) \approx 1.585 - 3 = -1.415$.
$H \approx -2 \times [ 0.125 \times (-3) ] - 2 \times [ 0.375 \times (-1.415) ]$
$H \approx -2 \times [-0.375] - 2 \times [-0.5306]$
$H \approx 0.75 + 1.0612 = 1.8112$ bits/symbol
4. Calculate Information Rate
The information rate is the product of the sampling rate and the entropy per symbol.
Information Rate = Sampling Rate $\times H$
Information Rate $\approx 200 \text{ samples/sec} \times 1.8112 \text{ bits/symbol}$
Information Rate $\approx 362.24$ bits/sec
This value lies between 360 and 363.
Important Questions from Information Theory
- Match items in List – I with items in List – II and select the correct answer from the code given below :
List – I List – II a. The Screening Hypothesis 1. George A. Akerlof b. Job Market Signalling 2. K.J. Arrow c. The problem of moral hazard in the case of medical insurance 3. M. Spencer d. The market for lemons 4. Paul W. Miller and Paul A. Volcker
Codes : A source transmits symbols from an alphabet of size 16. The value of maximum achievable entropy (in bits) is _______
The random variable $X$ takes values in $\{-1, 0, 1\}$ with probabilities $P(X = -1) = P(X = 1)$ and $\alpha$ and $P(X = 0) = 1 - 2\alpha$, where $0 < \alpha < \frac{1}{2}$. Let $g(\alpha)$ denote the entropy of $X$ (in bits), parameterized by $\alpha$. Which of the following statements is/are TRUE?
- Consider a discrete memoryless source with an alphabet of four source symbols.
$s(t)$ is a multi-level (-1, 0, +1, +2) signal representing a long sequence of random symbols from the above source which is generating $10^4$ symbols per second.
Which of the following options is the correct value of equivalent Nyquist bandwidth of $s(t)$? An information system uses the following daily workflow. Each function can start only after the input it explicitly requires has been produced. Arrange the functions in execution order.
(A) Process and validate the day’s sales transactions.
(B) Update the inventory master file using the validated transactions.
(C) Process managers’ enquiries using the figures in the completed daily report.
(D) Generate the daily report from the updated inventory master file.
(E) Run interactive decision-support scenarios using the results returned by those enquiries.
Choose the correct answer from the options below.
