Consider the model $Y = X\beta + \epsilon$, where $Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, X = \left( (x_{ij}) \right)_{n \times p}$ and $\beta = \begin{bmatrix} \beta_1 \\ \vdots \\ \beta_p \end{bmatrix}, \epsilon = \begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{bmatrix}$. $E(\epsilon) = 0$ and $D(\epsilon) = \sigma^2 I_n, p < n$. Let $\hat{\beta}$ be the solution of $X^T X \beta = X^T Y$. Which of the following are true?
Linear Model Estimability and BLUE Analysis
We analyze the properties of the Ordinary Least Squares (OLS) estimator $\hat{\beta}$ derived from the normal equations $X^T X \hat{\beta} = X^T Y$ for the linear model $Y = X\beta + \epsilon$, given $E(\epsilon) = 0$, $D(\epsilon) = \sigma^2 I_n$, and $p < n$. We evaluate each option based on established principles of linear regression theory.
Option A: Estimability and BLUE
Statement: If $C^T \beta$ is estimable then $C^T \hat{\beta}$ is the best linear unbiased estimator (BLUE) of $C^T \beta$.
Analysis: A linear parametric function $C^T \beta$ is estimable if $C$ lies in the row space of $X$. The Gauss-Markov theorem states that under the assumptions $E(\epsilon)=0$ and $D(\epsilon)=\sigma^2 I_n$, the OLS estimator $\hat{\beta}$ yields the BLUE for any estimable linear combination $C^T \beta$. The derivation of $\hat{\beta}$ from $X^T X \hat{\beta} = X^T Y$ ensures this property holds, regardless of whether $X$ has full column rank. Therefore, the statement is true.
Option B: Estimability Condition
Statement: All linear parametric functions are estimable if and only if $\text{Rank}(X) > p$.
Analysis: All linear parametric functions $C^T \beta$ are estimable if and only if the row space of $X$ spans $\mathbb{R}^p$, which means $\text{Rank}(X) = p$. The condition $\text{Rank}(X) > p$ is impossible because the rank of an $n \times p$ matrix cannot exceed $p$. Thus, the statement is false.
Option C: Partial Estimability
Statement: If $\text{Rank}(X) < p$ then some linear parametric functions are not estimable.
Analysis: A linear parametric function $C^T \beta$ is estimable if and only if $C$ is in the row space of $X$. The dimension of the row space is $\text{Rank}(X)$. If $\text{Rank}(X) < p$, the row space is a proper subspace of $\mathbb{R}^p$. This implies there exist vectors $C \in \mathbb{R}^p$ that are not in the row space of $X$. For such vectors, $C^T \beta$ is not estimable. Therefore, the statement is true.
Option D: Variance Estimator Bias
Statement: $(Y - X\hat{\beta})^T (Y - X\hat{\beta}) / (n - p)$ is an unbiased estimator of $\sigma^2$.
Analysis: The sum of squared residuals is $SSR = (Y - X\hat{\beta})^T (Y - X\hat{\beta})$. The unbiased estimator for $\sigma^2$ is $SSR / (n - \text{Rank}(X))$. The statement uses $n-p$ as the denominator. This is only correct if $\text{Rank}(X) = p$. Since it is possible that $\text{Rank}(X) < p$, this estimator is not generally unbiased. Therefore, the statement is false.
Conclusion
Based on the analysis, the true statements are A and C.
Important Questions from Gauss-Markov Models
- Let $X_1, X_2,..., X_n$ ($n > 5$) be independent random variables such that
$X_t = \alpha + \alpha^2 t + \epsilon_t$, for $t = 1,..., n$,
where, $\epsilon_1, \epsilon_2,..., \epsilon_n$ are independent and identically distributed $N(0, \sigma^2)$ random variables. Here $\alpha \in R$ and $\sigma > 0$ are unknown parameters. Which of the following statements are true? Consider the linear statistical model
$y_{ij} = \mu + \tau_i + \varepsilon_{ij}; \quad i = 1, 2, \cdots, a; \quad j = 1, 2, \cdots, n$
where $\mu$ is unknown, $\tau_i$ are independently and identically distributed as $N(0, \sigma_\tau^2)$, $\varepsilon_{ij}$ are independently and identically distributed as $N(0, \sigma^2)$; $\tau_i$ and $\varepsilon_{ij}$ are independent for all $i$ and $j$. Note that $\tau_i$ is the $i^{th}$ treatment effect. Suppose $SS_{total}, SS_{treatment}, SS_{error}$ are total sum of squares, total treatment sum of squares and error sum of squares, respectively. To test :$H_0 : \sigma_\tau^2 = 0$ vs $H_A : \sigma_\tau^2 > 0$ which of the following statements is not true?
Let $X_1, X_2, \ldots$ be a random sample from uniform $(0, 3\theta), \theta > 0$. Define
$T_n = \frac{1}{3} \max\{X_1, X_2, \ldots, X_n\}$. Which of the following is NOT true?
- Consider the following regression problem $y_i = \beta_1 e^i + \beta_2 e^{-i} + \epsilon_i; \ i = 1, \ldots, n$. Here $\epsilon_1, \ldots, \epsilon_n$ are i.i.d. $N(0, \sigma^2)$ random variables. If $\hat{\beta}_1$ and $\hat{\beta}_2$ are the least square estimators of $\beta_1$ and $\beta_2$, respectively, then which of the following statements are correct?
Consider the multiple linear regression model
$Y_i = \beta_1 x_{i1} + \dots + \beta_p x_{ip} + \epsilon_i$, with $E(\epsilon_i) = 0$, $Cov(\epsilon_i, \epsilon_k) = 0$,
if $i \ne k$ and $Var(\epsilon_i) = \sigma^2$, for $i, k = 1, \dots, n$. If $\hat{y}_i$ is the least squares fit of $y_i$ and $\hat{\epsilon}_i$ is the corresponding estimated residual for $i = 1, \dots, n$, then which of the following statements are always correct?
