i.i.d. Observations and Consistency

We are given independent and identically distributed (i.i.d.) observations \(X_i\) for \(i \ge 1\). We know that their expected value is \(E(X_i) = 0\) and their variance is \(Var(X_i) = \sigma^2 > 0\). A key property that follows from \(E(X_i) = 0\) and \(Var(X_i) = \sigma^2\) is \(E(X_i^2) = Var(X_i) + (E(X_i))^2 = \sigma^2 + 0^2 = \sigma^2\).

We need to determine which of the given estimators are consistent for \(\sigma^2\). An estimator \(T_n\) based on a sample of size \(n\) is said to be consistent for a parameter \(\theta\) if \(T_n\) converges in probability to \(\theta\) as \(n \to \infty\). That is, for every \(\epsilon > 0\), \(P(|T_n - \theta| > \epsilon) \to 0\) as \(n \to \infty\). This is often written as \(T_n \xrightarrow{P} \theta\).

A crucial tool for checking consistency is the Law of Large Numbers (LLN). For a sequence of i.i.d. random variables \(Y_i\) with finite mean \(E(Y_i) = \mu\), the sample mean \(\bar{Y} = \frac{1}{n}\sum_{i=1}^n Y_i\) converges in probability to \(\mu\), i.e., \(\bar{Y} \xrightarrow{P} \mu\). We will use this to analyze the given options.

Let's examine each statement:

Consistent Estimator Check - Option 1

The estimator is \(T_n^{(1)} = \frac{1}{n}\sum_{i=1}^{n}(X_{2i-1}-X_{2i})^{2}\). Let \(Y_i = X_{2i-1} - X_{2i}\). Since \(X_i\) are i.i.d., the pairs \((X_{2i-1}, X_{2i})\) are independent across \(i\), and thus \(Y_i\) are i.i.d. variables. We need to find the expected value of \(Y_i^2 = (X_{2i-1}-X_{2i})^{2}\).

• \(E(Y_i) = E(X_{2i-1} - X_{2i}) = E(X_{2i-1}) - E(X_{2i}) = 0 - 0 = 0\).
• \(Var(Y_i) = Var(X_{2i-1} - X_{2i})\). Since \(X_{2i-1}\) and \(X_{2i}\) are independent, \(Var(X_{2i-1} - X_{2i}) = Var(X_{2i-1}) + Var(X_{2i}) = \sigma^2 + \sigma^2 = 2\sigma^2\).
• \(E(Y_i^2) = Var(Y_i) + (E(Y_i))^2 = 2\sigma^2 + 0^2 = 2\sigma^2\).

The estimator \(T_n^{(1)} = \frac{1}{n}\sum_{i=1}^{n}Y_i^2\) is the sample mean of the i.i.d. variables \(Y_i^2\). By the Law of Large Numbers, \(T_n^{(1)} \xrightarrow{P} E(Y_i^2) = 2\sigma^2\).

This estimator converges in probability to \(2\sigma^2\), not \(\sigma^2\). Therefore, it is not a consistent estimator for \(\sigma^2\).

Consistent Estimator Check - Option 2

The estimator is \(T_n^{(2)} = \frac{1}{n}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2}\), where \({\bar{X}} = \frac{1}{n}\sum_{i=1}^n X_i\). This is the biased sample variance. We can expand this estimator:

\(T_n^{(2)} = \frac{1}{n}\sum_{i=1}^{n}(X_{i}^{2} - 2X_i\bar{X} + \bar{X}^2) = \frac{1}{n}\sum X_i^2 - \frac{2}{n}\sum X_i \bar{X} + \frac{1}{n}\sum \bar{X}^2\)

Since \(\bar{X}\) is constant with respect to the summation over \(i\), this simplifies to:

\(T_n^{(2)} = \frac{1}{n}\sum X_i^2 - 2\bar{X}\left(\frac{1}{n}\sum X_i\right) + \frac{1}{n}(n\bar{X}^2) = \frac{1}{n}\sum X_i^2 - 2\bar{X}^2 + \bar{X}^2 = \frac{1}{n}\sum X_i^2 - \bar{X}^2\)

Let's check the convergence of each term:

• For the term \(\frac{1}{n}\sum X_i^2\): \(X_i^2\) are i.i.d. variables. We calculated \(E(X_i^2) = \sigma^2\). By the Law of Large Numbers, \(\frac{1}{n}\sum_{i=1}^n X_i^2 \xrightarrow{P} E(X_i^2) = \sigma^2\).
• For the term \(\bar{X}^2\): By the Law of Large Numbers, \(\bar{X} = \frac{1}{n}\sum X_i \xrightarrow{P} E(X_i) = 0\). By the continuous mapping theorem (since \(f(x) = x^2\) is a continuous function), if \(\bar{X} \xrightarrow{P} 0\), then \(\bar{X}^2 \xrightarrow{P} 0^2 = 0\).

Using Slutsky's theorem (which states that if \(A_n \xrightarrow{P} a\) and \(B_n \xrightarrow{P} b\), then \(A_n - B_n \xrightarrow{P} a - b\)), we have:

\(T_n^{(2)} = \frac{1}{n}\sum X_i^2 - \bar{X}^2 \xrightarrow{P} \sigma^2 - 0 = \sigma^2\).

This estimator converges in probability to \(\sigma^2\). Therefore, it is a consistent estimator for \(\sigma^2\).

Consistent Estimator Check - Option 3

The estimator is \(T_n^{(3)} = \frac{1}{n}\sum_{i=1}^{n}X_{i}^{2}\). As discussed in the analysis for Option 2, \(X_i^2\) are i.i.d. variables with \(E(X_i^2) = \sigma^2\).

By the Law of Large Numbers, the sample mean of \(X_i^2\) converges in probability to its expected value:

\(T_n^{(3)} = \frac{1}{n}\sum_{i=1}^n X_i^2 \xrightarrow{P} E(X_i^2) = \sigma^2\).

This estimator converges in probability to \(\sigma^2\). Therefore, it is a consistent estimator for \(\sigma^2\).

Consistent Estimator Check - Option 4

The estimator is \(T_n^{(4)} = \frac{1}{n-1}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2}\). This is the unbiased sample variance, commonly denoted \(S^2\).

We can write \(T_n^{(4)}\) in terms of the biased sample variance \(T_n^{(2)}\) from Option 2:

\(T_n^{(4)} = \frac{1}{n-1}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2} = \frac{n}{n-1} \cdot \frac{1}{n}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2} = \frac{n}{n-1} T_n^{(2)}\).

Let's check the convergence of each part:

• The term \(\frac{n}{n-1}\) is a deterministic sequence that converges to 1 as \(n \to \infty\). Specifically, \(\frac{n}{n-1} = \frac{1}{1 - 1/n}\). As \(n \to \infty\), \(1/n \to 0\), so \(\frac{n}{n-1} \to 1\). Since deterministic convergence implies convergence in probability, \(\frac{n}{n-1} \xrightarrow{P} 1\).
• From the analysis of Option 2, we know that \(T_n^{(2)} = \frac{1}{n}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2} \xrightarrow{P} \sigma^2\).

Using Slutsky's theorem (which states that if \(A_n \xrightarrow{P} a\) and \(B_n \xrightarrow{P} b\), then \(A_n B_n \xrightarrow{P} ab\)), we have:

\(T_n^{(4)} = \frac{n}{n-1} T_n^{(2)} \xrightarrow{P} 1 \cdot \sigma^2 = \sigma^2\).

This estimator converges in probability to \(\sigma^2\). Therefore, it is a consistent estimator for \(\sigma^2\).

Summary of Consistency Results

Based on our analysis using the Law of Large Numbers and properties of convergence in probability:

• Option 1: \(\frac{1}{n}\sum_{i=1}^{n}(X_{2i-1}-X_{2i})^{2}\) is consistent for \(2\sigma^2\).
• Option 2: \(\frac{1}{n}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2}\) is consistent for \(\sigma^2\).
• Option 3: \(\frac{1}{n}\sum_{i=1}^{n}X_{i}^{2}\) is consistent for \(\sigma^2\).
• Option 4: \(\frac{1}{n-1}\sum_{i=1}^{n}(X_{i}-{\bar{X}})^{2}\) is consistent for \(\sigma^2\).

The statements that are correct are options 2, 3, and 4.