All Exams Test series for 1 year @ ₹349 only
Question

Consider a Markov chain having state space $S = \{1,2,3,4\}$ with transition probability matrix $P = (p_{i,j})$ given by 
$$\begin{matrix} & \begin{matrix} 1 & \quad 2 & \quad 3 & \quad 4 \end{matrix} \\ \begin{matrix} 1 \\ 2 \\ 3 \\ 4 \end{matrix} & \begin{bmatrix} 1/2 & 0 & 1/2 & 0 \\ 1/4 & 1/4 & 1/4 & 1/4 \\ 1/3 & 0 & 1/3 & 1/3 \\ 1/2 & 0 & 1/2 & 0 \end{bmatrix} \end{matrix}$$
Then

The correct answer is
$\lim_{n \to \infty} p_{2,2}^{(n)} = 0, \sum_{n=0}^\infty p_{2,2}^{(n)} < \infty$

We need to analyze the behavior of the Markov chain described by the transition probability matrix $P$ for state 2.

Markov Chain Analysis

The state space is $S = \{1, 2, 3, 4\}$ and the transition matrix is:

    $ P = \begin{bmatrix} 1/2 & 0 & 1/2 & 0 \\ 1/4 & 1/4 & 1/4 & 1/4 \\ 1/3 & 0 & 1/3 & 1/3 \\ 1/2 & 0 & 1/2 & 0 \end{bmatrix} $

We are interested in the properties of state 2, specifically $\lim_{n \to \infty} p_{2,2}^{(n)}$ and $\sum_{n=0}^\infty p_{2,2}^{(n)}$.

State 2 Transience Check

  • Reachability to State 2: Observe the columns of the transition matrix $P$. The only non-zero entry in the second column (transitions *to* state 2) is $p_{2,2} = 1/4$. This means only state 2 can transition back to state 2. No other state (1, 3, or 4) can transition to state 2.
  • Leaving State 2: From state 2, the probabilities of moving to other states are $p_{2,1}=1/4$, $p_{2,2}=1/4$, $p_{2,3}=1/4$, $p_{2,4}=1/4$. The probability of leaving state 2 is $p_{2,1} + p_{2,3} + p_{2,4} = 1/4 + 1/4 + 1/4 = 3/4$.
  • Conclusion on Transience: Since there is a non-zero probability ($3/4$) of leaving state 2, and no other state can lead back to state 2, the chain will eventually leave state 2 and never return. Therefore, state 2 is a transient state.

Limit Behaviour ($\lim_{n \to \infty} p_{2,2}^{(n)}$)

  • For any transient state $j$, the probability of being in that state after infinitely many steps approaches zero, regardless of the starting state $i$.
  • Since state 2 is transient, $\lim_{n \to \infty} p_{2,2}^{(n)} = 0$.

Sum Behaviour ($\sum_{n=0}^\infty p_{2,2}^{(n)}$)

  • For any transient state $j$, the sum of probabilities of being in state $j$ over all steps starting from any state $i$ is finite.
  • Since state 2 is transient, $\sum_{n=0}^\infty p_{2,2}^{(n)} < \infty$.

Final Result

Combining the findings:

  • $\lim_{n \to \infty} p_{2,2}^{(n)} = 0$
  • $\sum_{n=0}^\infty p_{2,2}^{(n)} < \infty$

This corresponds to the properties of a transient state in a Markov chain.

Was this answer helpful?

Important Questions from Probability (Notes)

  1. A stick of length L is broken into two pieces at random. What is the average length of the smaller piece?
  2. A fair die was thrown three times and the outcome was repeatedly six. If the die is thrown again what is the probability of getting six?
  3. A canal system is shown in the figure. 

    Water flows from A to B through two channels. Gates $G_1$ and $G_2$ are operated independently to regulate the flow. Probability of $G_1$ to be open is 10% while that of $G_2$ is 20%. The probability that water will flow from A to B is

  4. A standard fair die is rolled until some face other than 5 or 6 turns up. Let $X$ denote the face value of the last roll, and $A = \{X \text{ is even}\}$ and $B = \{X \text{ is at most 2}\}$. Then,
  5. Let $X$ and $Y$ be two random variables with joint probability density function $f(x, y) = \begin{cases} \frac{1}{\pi} & \text{if } 0 \leq x^2 + y^2 \leq 1 \\ 0 & \text{otherwise.} \end{cases}$ Which of the following statements are correct?
Need Expert Advice?

Start Your Preparation with Prepp Mobile App

Download the app from Google Play & App Store
Download the app from Google Play & App Store
Prepp Mobile App