We need to analyze the behavior of the Markov chain described by the transition probability matrix $P$ for state 2.

Markov Chain Analysis

The state space is $S = \{1, 2, 3, 4\}$ and the transition matrix is:

    $ P = \begin{bmatrix} 1/2 & 0 & 1/2 & 0 \\ 1/4 & 1/4 & 1/4 & 1/4 \\ 1/3 & 0 & 1/3 & 1/3 \\ 1/2 & 0 & 1/2 & 0 \end{bmatrix} $

We are interested in the properties of state 2, specifically $\lim_{n \to \infty} p_{2,2}^{(n)}$ and $\sum_{n=0}^\infty p_{2,2}^{(n)}$.

State 2 Transience Check

• Reachability to State 2: Observe the columns of the transition matrix $P$. The only non-zero entry in the second column (transitions *to* state 2) is $p_{2,2} = 1/4$. This means only state 2 can transition back to state 2. No other state (1, 3, or 4) can transition to state 2.
• Leaving State 2: From state 2, the probabilities of moving to other states are $p_{2,1}=1/4$, $p_{2,2}=1/4$, $p_{2,3}=1/4$, $p_{2,4}=1/4$. The probability of leaving state 2 is $p_{2,1} + p_{2,3} + p_{2,4} = 1/4 + 1/4 + 1/4 = 3/4$.
• Conclusion on Transience: Since there is a non-zero probability ($3/4$) of leaving state 2, and no other state can lead back to state 2, the chain will eventually leave state 2 and never return. Therefore, state 2 is a transient state.

Limit Behaviour ($\lim_{n \to \infty} p_{2,2}^{(n)}$)

• For any transient state $j$, the probability of being in that state after infinitely many steps approaches zero, regardless of the starting state $i$.
• Since state 2 is transient, $\lim_{n \to \infty} p_{2,2}^{(n)} = 0$.

Sum Behaviour ($\sum_{n=0}^\infty p_{2,2}^{(n)}$)

• For any transient state $j$, the sum of probabilities of being in state $j$ over all steps starting from any state $i$ is finite.
• Since state 2 is transient, $\sum_{n=0}^\infty p_{2,2}^{(n)} < \infty$.

Final Result

Combining the findings:

• $\lim_{n \to \infty} p_{2,2}^{(n)} = 0$
• $\sum_{n=0}^\infty p_{2,2}^{(n)} < \infty$

This corresponds to the properties of a transient state in a Markov chain.