An analog baseband signal, bandlimited to 100 Hz, is sampled at the Nyquist rate. The samples are quantized into four message symbols that occur independently with probabilities $p_1 = p_4 = 0.125$ and $p_2 = p_3$. The information rate (bits/sec) of the message source is ____________
The problem asks for the information rate (bits/sec) of a message source based on signal sampling and quantization.
1. Determine Sampling Rate (Nyquist Rate)
The analog baseband signal is bandlimited to 100 Hz. According to the Nyquist theorem, the minimum sampling rate required to avoid aliasing is twice the bandwidth.
Sampling Rate = Nyquist Rate = $2 \times \text{Bandwidth}$
Sampling Rate = $2 \times 100 \text{ Hz} = 200 \text{ samples/sec}$
2. Calculate Symbol Probabilities
There are four message symbols with probabilities $p_1, p_2, p_3, p_4$. We are given:
- $p_1 = 0.125$
- $p_4 = 0.125$
- $p_2 = p_3$
Since the sum of probabilities must be 1:
$p_1 + p_2 + p_3 + p_4 = 1$
$0.125 + p_2 + p_2 + 0.125 = 1$
$0.25 + 2p_2 = 1$
$2p_2 = 1 - 0.25 = 0.75$
$p_2 = 0.75 / 2 = 0.375$
Therefore, the probabilities are: $p_1 = 0.125$, $p_2 = 0.375$, $p_3 = 0.375$, $p_4 = 0.125$.
3. Calculate Entropy (Average Information per Symbol)
Entropy ($H$) measures the average information content per symbol. It is calculated using the formula:
$H = -\sum_{i=1}^{N} p_i \log_2(p_i)$
For N=4 symbols:
$H = -[ p_1 \log_2(p_1) + p_2 \log_2(p_2) + p_3 \log_2(p_3) + p_4 \log_2(p_4) ]$
$H = -[ 0.125 \log_2(0.125) + 0.375 \log_2(0.375) + 0.375 \log_2(0.375) + 0.125 \log_2(0.125) ]$
$H = -2 \times [ 0.125 \log_2(0.125) ] - 2 \times [ 0.375 \log_2(0.375) ]$
We know that $\log_2(0.125) = \log_2(1/8) = -3$.
And $\log_2(0.375) = \log_2(3/8) = \log_2(3) - \log_2(8) \approx 1.585 - 3 = -1.415$.
$H \approx -2 \times [ 0.125 \times (-3) ] - 2 \times [ 0.375 \times (-1.415) ]$
$H \approx -2 \times [-0.375] - 2 \times [-0.5306]$
$H \approx 0.75 + 1.0612 = 1.8112$ bits/symbol
4. Calculate Information Rate
The information rate is the product of the sampling rate and the entropy per symbol.
Information Rate = Sampling Rate $\times H$
Information Rate $\approx 200 \text{ samples/sec} \times 1.8112 \text{ bits/symbol}$
Information Rate $\approx 362.24$ bits/sec
This value lies between 360 and 363.
Important Questions from Information Theory
For any binary (n, h) linear code with minimum distance (2t + 1) or greater \(n - h \ge {\log _2}\left[ {\mathop \sum \limits_{i = 0}^α \left( {\begin{array}{*{20}{c}} n\\ i \end{array}} \right)} \right]\) where α is:
The main processing functions of information system are given below. Arrange them in sequencing order:
(A) Process transaction
(B) Maintain master file
(C) Process enquiry
(D) Process report
(E) Process interactive supper applications
Choose the correct answer from the options given below:
The random variable $X$ takes values in $\{-1, 0, 1\}$ with probabilities $P(X = -1) = P(X = 1)$ and $\alpha$ and $P(X = 0) = 1 - 2\alpha$, where $0 < \alpha < \frac{1}{2}$. Let $g(\alpha)$ denote the entropy of $X$ (in bits), parameterized by $\alpha$. Which of the following statements is/are TRUE?
A source transmits symbols from an alphabet of size 16. The value of maximum achievable entropy (in bits) is _______
- Consider a discrete memoryless source with an alphabet of four source symbols.
$s(t)$ is a multi-level (-1, 0, +1, +2) signal representing a long sequence of random symbols from the above source which is generating $10^4$ symbols per second.
Which of the following options is the correct value of equivalent Nyquist bandwidth of $s(t)$?
