Evaluating Estimators $\hat{\beta}_1$ and $\hat{\beta}_2$
The question asks about the properties (unbiasedness and variance) of two estimators, $\hat{\beta}_1$ and $\hat{\beta}_2$, for the parameter $\beta$ in the linear model $Y_i = \beta X_i + \epsilon_i$, where $\epsilon_i \sim N(0, \sigma^2)$ are independent and identically distributed, and $X_i$ are nonrandom, positive, and distinct.
Estimator Unbiasedness Analysis
We need to check if both $\hat{\beta}_1$ and $\hat{\beta}_2$ are unbiased estimates of $\beta$. An estimator is unbiased if its expected value equals the true parameter value.
Unbiasedness of $\hat{\beta}_1$
The first estimator is the Ordinary Least Squares (OLS) estimator:
$ \hat{\beta}_1 = \frac{\sum_{i=1}^n (Y_i - \bar{Y})(X_i - \bar{X})}{\sum_{i=1}^n (X_i - \bar{X})^2} $Substitute $Y_i = \beta X_i + \epsilon_i$ and $\bar{Y} = \beta \bar{X} + \bar{\epsilon}$:
$ \hat{\beta}_1 = \frac{\sum (\beta(X_i - \bar{X}) + (\epsilon_i - \bar{\epsilon}))(X_i - \bar{X})}{\sum (X_i - \bar{X})^2} $ $ \hat{\beta}_1 = \frac{\beta \sum (X_i - \bar{X})^2 + \sum (\epsilon_i - \bar{\epsilon})(X_i - \bar{X})}{\sum (X_i - \bar{X})^2} $ $ \hat{\beta}_1 = \beta + \frac{\sum (\epsilon_i - \bar{\epsilon})(X_i - \bar{X})}{\sum (X_i - \bar{X})^2} $Taking the expectation:
$ E[\hat{\beta}_1] = \beta + \frac{\sum (E[\epsilon_i] - E[\bar{\epsilon}])(X_i - \bar{X})}{\sum (X_i - \bar{X})^2} $Since $E[\epsilon_i] = 0$ and $X_i$ are nonrandom, $E[\bar{\epsilon}] = 0$. Thus, $E[\hat{\beta}_1] = \beta$. $\hat{\beta}_1$ is unbiased.
Unbiasedness of $\hat{\beta}_2$
The second estimator is:
$ \hat{\beta}_2 = \frac{\sum_{i=1}^n Y_i X_i}{\sum_{i=1}^n X_i^2} $Substitute $Y_i = \beta X_i + \epsilon_i$:
$ \hat{\beta}_2 = \frac{\sum (\beta X_i + \epsilon_i) X_i}{\sum X_i^2} = \frac{\beta \sum X_i^2 + \sum \epsilon_i X_i}{\sum X_i^2} $ $ \hat{\beta}_2 = \beta + \frac{\sum \epsilon_i X_i}{\sum X_i^2} $Taking the expectation:
$ E[\hat{\beta}_2] = \beta + \frac{\sum E[\epsilon_i] X_i}{\sum X_i^2} $Since $E[\epsilon_i] = 0$ and $X_i$ are nonrandom, $E[\hat{\beta}_2] = \beta$. $\hat{\beta}_2$ is also unbiased.
Conclusion: Both $\hat{\beta}_1$ and $\hat{\beta}_2$ are unbiased estimates for $\beta$. Option A is correct.
Estimator Variance Comparison
Next, we compare the variances of $\hat{\beta}_1$ and $\hat{\beta}_2$. The variance depends on the variance of the error term, $\sigma^2$.
Variance of $\hat{\beta}_1$
Using the expression for $\hat{\beta}_1$:
$ Var(\hat{\beta}_1) = Var\left(\beta + \frac{\sum (\epsilon_i - \bar{\epsilon})(X_i - \bar{X})}{\sum (X_i - \bar{X})^2}\right) $ $ Var(\hat{\beta}_1) = \frac{1}{(\sum (X_i - \bar{X})^2)^2} Var\left(\sum (\epsilon_i - \bar{\epsilon})(X_i - \bar{X})\right) $Since $\sum (\epsilon_i - \bar{\epsilon})(X_i - \bar{X}) = \sum \epsilon_i (X_i - \bar{X})$ and $\epsilon_i$ are independent with $Var(\epsilon_i) = \sigma^2$:
$ Var(\hat{\beta}_1) = \frac{1}{(\sum (X_i - \bar{X})^2)^2} \sum Var(\epsilon_i) (X_i - \bar{X})^2 $ $ Var(\hat{\beta}_1) = \frac{\sigma^2}{(\sum (X_i - \bar{X})^2)^2} \sum (X_i - \bar{X})^2 = \frac{\sigma^2}{\sum (X_i - \bar{X})^2} $Variance of $\hat{\beta}_2$
Using the expression for $\hat{\beta}_2$:
$ Var(\hat{\beta}_2) = Var\left(\beta + \frac{\sum \epsilon_i X_i}{\sum X_i^2}\right) $ $ Var(\hat{\beta}_2) = \frac{1}{(\sum X_i^2)^2} Var\left(\sum \epsilon_i X_i\right) $Since $\epsilon_i$ are independent with $Var(\epsilon_i) = \sigma^2$:
$ Var(\hat{\beta}_2) = \frac{1}{(\sum X_i^2)^2} \sum Var(\epsilon_i) X_i^2 $ $ Var(\hat{\beta}_2) = \frac{\sigma^2}{(\sum X_i^2)^2} \sum X_i^2 = \frac{\sigma^2}{\sum X_i^2} $Variance Comparison
We compare $Var(\hat{\beta}_1) = \frac{\sigma^2}{\sum (X_i - \bar{X})^2}$ and $Var(\hat{\beta}_2) = \frac{\sigma^2}{\sum X_i^2}$.
We know that $\sum (X_i - \bar{X})^2 = \sum X_i^2 - n \bar{X}^2$. Since $X_i$'s are positive and distinct, and $n \ge 2$, we have $\bar{X}^2 > 0$ and $n \bar{X}^2 > 0$. Therefore,
$ \sum (X_i - \bar{X})^2 < \sum X_i^2 $Since the denominator for $Var(\hat{\beta}_1)$ is smaller than the denominator for $Var(\hat{\beta}_2)$, and $\sigma^2$ is positive, we have:
$ Var(\hat{\beta}_1) > Var(\hat{\beta}_2) $Conclusion: $\hat{\beta}_1$ has a larger variance than $\hat{\beta}_2$. Option B is correct.
Final Answer Summary
Based on the analysis:
- Statement A is correct: Both $\hat{\beta}_1$ and $\hat{\beta}_2$ are unbiased estimates.
- Statement B is correct: $\hat{\beta}_1$ has a larger variance than $\hat{\beta}_2$.
Therefore, the correct options are A and B.
Important Questions from Gauss-Markov Models
- Let $X_1, X_2,..., X_n$ ($n > 5$) be independent random variables such that
$X_t = \alpha + \alpha^2 t + \epsilon_t$, for $t = 1,..., n$,
where, $\epsilon_1, \epsilon_2,..., \epsilon_n$ are independent and identically distributed $N(0, \sigma^2)$ random variables. Here $\alpha \in R$ and $\sigma > 0$ are unknown parameters. Which of the following statements are true? Consider the linear statistical model
$y_{ij} = \mu + \tau_i + \varepsilon_{ij}; \quad i = 1, 2, \cdots, a; \quad j = 1, 2, \cdots, n$
where $\mu$ is unknown, $\tau_i$ are independently and identically distributed as $N(0, \sigma_\tau^2)$, $\varepsilon_{ij}$ are independently and identically distributed as $N(0, \sigma^2)$; $\tau_i$ and $\varepsilon_{ij}$ are independent for all $i$ and $j$. Note that $\tau_i$ is the $i^{th}$ treatment effect. Suppose $SS_{total}, SS_{treatment}, SS_{error}$ are total sum of squares, total treatment sum of squares and error sum of squares, respectively. To test :$H_0 : \sigma_\tau^2 = 0$ vs $H_A : \sigma_\tau^2 > 0$ which of the following statements is not true?
Consider the model $Y = X\beta + \epsilon$, where $Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, X = \left( (x_{ij}) \right)_{n \times p}$ and $\beta = \begin{bmatrix} \beta_1 \\ \vdots \\ \beta_p \end{bmatrix}, \epsilon = \begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{bmatrix}$. $E(\epsilon) = 0$ and $D(\epsilon) = \sigma^2 I_n, p < n$. Let $\hat{\beta}$ be the solution of $X^T X \beta = X^T Y$. Which of the following are true?
Let $X_1, X_2, \ldots$ be a random sample from uniform $(0, 3\theta), \theta > 0$. Define
$T_n = \frac{1}{3} \max\{X_1, X_2, \ldots, X_n\}$. Which of the following is NOT true?
- Consider the following regression problem $y_i = \beta_1 e^i + \beta_2 e^{-i} + \epsilon_i; \ i = 1, \ldots, n$. Here $\epsilon_1, \ldots, \epsilon_n$ are i.i.d. $N(0, \sigma^2)$ random variables. If $\hat{\beta}_1$ and $\hat{\beta}_2$ are the least square estimators of $\beta_1$ and $\beta_2$, respectively, then which of the following statements are correct?
