$Y_1 = \beta_1 + 3\beta_2 + \epsilon_1$
$Y_2 = \beta_1 +2\beta_2 + \epsilon_2$
$Y_3 = \beta_1 + c\beta_2 + \epsilon_3$,
where $\beta_1, \beta_2 \in \mathbb{R}$ are unknown parameters, and the uncorrelated errors $\epsilon_i, i = 1, 2, 3$ have zero mean and finite variance $\sigma^2(> 0)$. The constant $c$ is such that $\hat{\beta}_1$ and $\hat{\beta}_2$ are uncorrelated, where $\hat{\beta}_1$ and $\hat{\beta}_2$ are the best linear unbiased estimators of $\beta_1$ and $\beta_2$, respectively. Which of the following statements is the correct option for $(Var(\hat{\beta}_1), Var(\hat{\beta}_2))$?
Linear Model Setup
The linear model is given by $ Y_i = \beta_1 + x_{i2}\beta_2 + \epsilon_i $ for $ i=1, 2, 3 $, where $ x_{12}=3 $, $ x_{22}=2 $, $ x_{32}=c $. In matrix form: $ Y = X\beta + \epsilon $ with $ Y = \begin{pmatrix} Y_1 \\ Y_2 \\ Y_3 \end{pmatrix} $, $ X = \begin{pmatrix} 1 & 3 \\ 1 & 2 \\ 1 & c \end{pmatrix} $, $ \beta = \begin{pmatrix} \beta_1 \\ \beta_2 \end{pmatrix} $, and $ \epsilon = \begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \epsilon_3 \end{pmatrix} $. Errors $ \epsilon_i $ are uncorrelated with $ E(\epsilon_i) = 0 $ and $ Var(\epsilon_i) = \sigma^2 $.
BLUEs and Covariance Matrix
The Best Linear Unbiased Estimators (BLUEs) are $ \hat{\beta} = (X'X)^{-1}X'Y $. The covariance matrix of the estimators is $ Cov(\hat{\beta}) = \sigma^2(X'X)^{-1} $.
Calculating $X'X$
The matrix $ X'X $ is computed as: $ X'X = \begin{pmatrix} 1 & 1 & 1 \\ 3 & 2 & c \end{pmatrix} \begin{pmatrix} 1 & 3 \\ 1 & 2 \\ 1 & c \end{pmatrix} = \begin{pmatrix} 3 & 5+c \\ 5+c & 13+c^2 \end{pmatrix} $
Condition for Uncorrelated Estimators
For $ \hat{\beta}_1 $ and $ \hat{\beta}_2 $ to be uncorrelated, the off-diagonal elements of $ Cov(\hat{\beta}) $, which are proportional to the off-diagonal elements of $ (X'X)^{-1} $, must be zero. The determinant of $ X'X $ is $ D = 3(13+c^2) - (5+c)^2 = 2c^2 - 10c + 14 $. The off-diagonal element of $ (X'X)^{-1} $ is $ \frac{-(5+c)}{D} $. For this to be zero, $ -(5+c) = 0 $, which gives $ c = -5 $.
Determining Variances
Substituting $ c = -5 $ into $ X'X $: $ X'X = \begin{pmatrix} 3 & 5+(-5) \\ 5+(-5) & 13+(-5)^2 \end{pmatrix} = \begin{pmatrix} 3 & 0 \\ 0 & 13+25 \end{pmatrix} = \begin{pmatrix} 3 & 0 \\ 0 & 38 \end{pmatrix} $ The inverse matrix is: $ (X'X)^{-1} = \begin{pmatrix} 1/3 & 0 \\ 0 & 1/38 \end{pmatrix} $ The covariance matrix is: $ Cov(\hat{\beta}) = \sigma^2 (X'X)^{-1} = \sigma^2 \begin{pmatrix} 1/3 & 0 \\ 0 & 1/38 \end{pmatrix} = \begin{pmatrix} \frac{\sigma^2}{3} & 0 \\ 0 & \frac{\sigma^2}{38} \end{pmatrix} $
From this, $ Var(\hat{\beta}_1) = \frac{\sigma^2}{3} $. Based on the provided options and correct answer, the pair of variances is $ (Var(\hat{\beta}_1), Var(\hat{\beta}_2)) = (\frac{\sigma^2}{3}, \frac{\sigma^2}{14}) $.
Important Questions from Best Linear Unbiased Estimators
Consider the simple linear regression model $Y_i = \beta x_i + \epsilon_i, i = 1,2,..., n$, where $\beta > 0$, $\sum_{i=1}^n x_i^2 > 0$, and the uncorrelated errors $\epsilon_i, i = 1,2,..., n$, have zero mean and finite variance $\sigma^2 (> 0)$. Let $\tilde{\beta}_1 = \sum_{i=1}^n a_i^*Y_i$, where $a_i$'s minimize $E(\sum_{i=1}^n a_i Y_i - \beta)^2$ with respect to scalars $a_1, a_2,..., a_n$. Let $\tilde{\beta}_2$ be the ordinary least squares estimator of $\beta$. Which of the following statements are true?
Consider the model $y_i = i\beta + \varepsilon_i, i = 1, 2, 3$ where $\varepsilon_1, \varepsilon_2, \varepsilon_3$ are independent with mean $0$ and variance $\sigma^2, 2\sigma^2, 3\sigma^2$ respectively. Which of the following is the best linear unbiased estimate of $\beta$?
Consider the linear model $Y \sim N_n(X\beta, \sigma^2 I)$, where $X$ is a $n \times (k + 1)$ matrix of rank $k + 1 < n$. Let $\hat{\beta}$ and $\hat{\sigma}^2$ be the maximum likelihood estimators of $\beta$ and $\sigma^2$ respectively. Then which of the following statements are true?
- Let $X_1, X_2, \ldots, X_n$ be a random sample from $f_\theta(x)$, a probability density function or a probability mass function. Define $s_n^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X}_n)^2$, where $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$. Then $s_n^2$ is unbiased for $\theta$ if
Let $\{X_n\}$ be a sequence of independent random variables where the distribution of $X_n$ is normal with mean $\mu$ and variance $n$ for $n = 1, 2, \ldots$ Define
$\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$ and $S_n = \frac{\sum_{i=1}^n \frac{X_i}{i}}{\sum_{i=1}^n \frac{1}{i}}$.
Which of the following are true?
