Let X and Y be random variables with joint distribution : \(\Pr(X = i, Y = j) = \begin{cases}1/6 & \text{for } i = 5,6 \text{ and } j = 3,6,9\\0 & \text{otherwise}\end{cases}\) Then the mutual information I (X, Y) = .................. .
0
Mutual information measures how much knowing one random variable reduces uncertainty about the other: \(I(X,Y) = \sum_{i,j} p(i,j)\log\dfrac{p(i,j)}{p(i)\,p(j)}\). It is zero exactly when X and Y are independent, because then every log term is log 1 = 0.
Step 1 — Marginal of X: each value i = 5, 6 pairs with three values of j, so \(p(i) = 3 \times \tfrac{1}{6} = \tfrac{1}{2}\).
Step 2 — Marginal of Y: each value j = 3, 6, 9 pairs with two values of i, so \(p(j) = 2 \times \tfrac{1}{6} = \tfrac{1}{3}\).
Step 3 — For every pair, \(p(i)\,p(j) = \tfrac{1}{2} \times \tfrac{1}{3} = \tfrac{1}{6} = p(i,j)\), so X and Y are independent and each term is \(\tfrac{1}{6}\log 1 = 0\).
log(2) is the entropy H(X) and log(3) is H(Y); log(6) is the joint entropy H(X, Y). Since \(I = H(X) + H(Y) - H(X,Y) = \log 2 + \log 3 - \log 6 = 0\), none of those non-zero values is the mutual information.
0 option is correct.
Similar Questions
Let \(\varphi, \psi\) be the formula, \(\varphi \models \Psi\), if and only if \(\varphi \models \Psi\) is a :
Which of the following relations defined on X = {1, 2, 3} is an equivalent relation ?
A rod is divided into six segment is to be colored with one or more of 4 different colors. In how many ways can this be done ?
Consider the following linear programming problem : Maximize \(z = 5x_{1} + 2x_{2} + 3x_{3} - x_{4} + x_{5}\) Subject to : \(x_{1} + 2x_{2} + 2x_{3} + x_{4} = 8\), \(3x_{1} + 4x_{2} + x_{3} + x_{5} = 7\), \(x_{1}, x_{2}, x_{3}, x_{4}, x_{5} \geq 0\) What will be the basic feasible solution after iteration 1 ?
The thickness of a graph G is defined as minimum number of (edge-disjoint) planar subgraphs into which G can be decomposed. The thickness of G is defined as \(\theta(G)\). What is the thickness of planar graph ?
A committee is to be selected from 5 candidates a, b, c, d and e. The selection must satisfy all the following conditions :
• Either a or b must be included, but not both
• Either c or e or both must be included
• If d is included then b must be included
• Either a or c are included or neither is included.
• If e is included, then c and d must be included. How should selection be made ?
Let us consider the letters from a through h, and all 5 letters are made using these letters. (words are just strings of letters). How many of these words contain no repeated letters ?
Let us consider the following primal problem : Maximize \(z = 2x_{1} + x_{2}\) Subject to \(x_{1} - x_{2} \leq 4\), \(x_{1} - x_{2} \leq 2\), \(x_{1}, x_{2} \geq 0\) What will be its dual problem ?
Identify an appropriate operation sequence with respect to execution of a machine level language instruction.
The “instruction decode” phase in CPU’s instruction cycle is responsible to _____ .
Important Questions from Mixed Topic (MH-SET)
.................... is a client server application that identifies each host on the internet with a unique user-friendly name.
Every OS has a ................ or .......... which primarily resides in the main memory of the computer to perform some of the basic functions of the OS.
(i) Kernel
(ii) Interface
(iii) Nucleus
(iv) Compiler
Codes:
Which of the following are referred to as low level language?
(i) Machine language
(ii) Assembly language
(iii) High level language
(iv) Natural language
Codes:
Match the following:
List I List II (a) New York Public Library (i) Shelv AR (b) Miami University Augmented Research Group (ii) Smart library (c) Oula University Library (iii) Wolf Walk (d) North Carolina State University (iv) Find the future Codes:
Which part of the QR code represents the encoded data?
