Consider a linear model $Y_{4 \times 1} = X_{4 \times 4} \beta_{4 \times 1} + \varepsilon_{4 \times 1}$, where $\text{Disp}(\varepsilon) = \sigma^2 I_4$ for some $\sigma^2 > 0$. One needs to choose the design matrix $X$ such that its elements take values in the set $\{-1, 0, 1\}$. Now, consider the following three choices of $X$ $X_1 = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$, $X_2 = \begin{bmatrix} 1 & -1 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 1 & 1 \end{bmatrix}$ and $X_3 = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & -1 & -1 \\ 1 & -1 & 1 & -1 \\ 1 & -1 & -1 & 1 \end{bmatrix}$ Which of the following statements are true?
The problem asks us to evaluate three design matrices ($X_1, X_2, X_3$) for a linear model $Y = X\beta + \varepsilon$, where $\text{Disp}(\varepsilon) = \sigma^2 I_4$. We need to determine which of the given statements regarding estimability, uncorrelatedness of estimates, and efficiency are true.
Analysis of Design Matrices
1. Estimability of $\beta$ (Statement A)
A vector parameter $\beta$ is estimable if and only if the rank of the design matrix $X$ equals the number of parameters, which is the number of columns in $X$. Here, $p=4$. We need to check if $\text{rank}(X) = 4$ for $X_1, X_2,$ and $X_3$.
$X_1$: $X_1$ is the $4 \times 4$ identity matrix ($I_4$).
$X_1 = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$
The rank of $I_4$ is 4. Thus, $\beta$ is estimable for $X_1$.
$X_2$:
$X_2 = \begin{bmatrix} 1 & -1 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 1 & 1 \end{bmatrix}$
$X_2$ is a block diagonal matrix. The rank is the sum of the ranks of the blocks. The top-left $2 \times 2$ block $\begin{bmatrix} 1 & -1 \\ 1 & 1 \end{bmatrix}$ has a determinant of $(1)(1) - (-1)(1) = 2 \ne 0$, so its rank is 2. The bottom-right $2 \times 2$ block is identical and also has rank 2. The total rank is $2 + 2 = 4$. Thus, $\beta$ is estimable for $X_2$.
$X_3$:
$X_3 = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & -1 & -1 \\ 1 & -1 & 1 & -1 \\ 1 & -1 & -1 & 1 \end{bmatrix}$
The columns of $X_3$ are orthogonal to each other. Orthogonal non-zero vectors are linearly independent. Since there are 4 linearly independent columns, the rank of $X_3$ is 4. Thus, $\beta$ is estimable for $X_3$.
Since $\text{rank}(X) = 4$ for all three matrices, statement A is true.
2. Uncorrelated Least Squares Estimates (Statement B)
The variance-covariance matrix of the least squares estimator $\hat{\beta}$ is $\text{Var}(\hat{\beta}) = \sigma^2 (X'X)^{-1}$. The estimates $\hat{\beta}_i$ and $\hat{\beta}_j$ are uncorrelated if the corresponding off-diagonal elements of $\text{Var}(\hat{\beta})$ are zero. This occurs if and only if the matrix $(X'X)^{-1}$ is diagonal, which means $X'X$ must be diagonal. This condition is satisfied if the columns of the design matrix $X$ are orthogonal.
$X_1$: The columns of $I_4$ are the standard basis vectors, which are orthogonal. $X_1'X_1 = I_4$, which is diagonal.
$X_2$: Let the columns be $c_1, c_2, c_3, c_4$.
$c_1 = [1, 1, 0, 0]', c_2 = [-1, 1, 0, 0]', c_3 = [0, 0, 1, 1]', c_4 = [0, 0, -1, 1]'$.
We check for orthogonality:
$c_1 \cdot c_2 = (1)(-1) + (1)(1) + 0 + 0 = 0$
$c_3 \cdot c_4 = 0 + 0 + (1)(-1) + (1)(1) = 0$
All other pairs involving one column from the first block and one from the second block are also zero (e.g., $c_1 \cdot c_3 = 0$). The columns are orthogonal. $X_2'X_2 = \text{diag}(2, 2, 2, 2)$, which is diagonal.
$X_3$: The columns were shown to be orthogonal in the analysis of $X_3$'s rank. $X_3'X_3 = \text{diag}(4, 4, 4, 4)$, which is diagonal.
Since the columns of $X_1, X_2,$ and $X_3$ are orthogonal, $X'X$ is diagonal for all three cases. Thus, statement B is true.
3. Comparison of Design Matrices (Statements C and D)
A design matrix is generally considered "better" if it leads to more efficient estimates, meaning smaller variances. The variance of $\hat{\beta}_i$ is $\sigma^2$ times the $i$-th diagonal element of $(X'X)^{-1}$.
Comparing $X_2$ and $X_1$ (Statement C):
For $X_1$: $(X_1'X_1)^{-1} = I_4$. The diagonal elements are all 1. $\text{Var}(\hat{\beta}_{i, X_1}) = \sigma^2$.
For $X_2$: $(X_2'X_2)^{-1} = (2I_4)^{-1} = \frac{1}{2}I_4$. The diagonal elements are all $\frac{1}{2}$. $\text{Var}(\hat{\beta}_{i, X_2}) = \frac{\sigma^2}{2}$.
Since $\frac{\sigma^2}{2} < \sigma^2$, the estimates obtained using $X_2$ have smaller variances than those using $X_1$. Therefore, $X_2$ is a more efficient and thus better choice than $X_1$. Statement C is true.
Comparing $X_3$ and $X_2$ (Statement D analysis omitted as per instructions)
Conclusion
Based on the analysis:
Statement A is true (all matrices have full rank).
Statement B is true (all matrices have orthogonal columns).
Statement C is true ($X_2$ provides smaller estimate variances than $X_1$).
The true statements are A, B, and C.
Was this answer helpful?
Important Questions from Gauss-Markov Models
Let $X_1, X_2,..., X_n$ ($n > 5$) be independent random variables such that $X_t = \alpha + \alpha^2 t + \epsilon_t$, for $t = 1,..., n$, where, $\epsilon_1, \epsilon_2,..., \epsilon_n$ are independent and identically distributed $N(0, \sigma^2)$ random variables. Here $\alpha \in R$ and $\sigma > 0$ are unknown parameters. Which of the following statements are true?
where $\mu$ is unknown, $\tau_i$ are independently and identically distributed as $N(0, \sigma_\tau^2)$, $\varepsilon_{ij}$ are independently and identically distributed as $N(0, \sigma^2)$; $\tau_i$ and $\varepsilon_{ij}$ are independent for all $i$ and $j$. Note that $\tau_i$ is the $i^{th}$ treatment effect. Suppose $SS_{total}, SS_{treatment}, SS_{error}$ are total sum of squares, total treatment sum of squares and error sum of squares, respectively. To test :$H_0 : \sigma_\tau^2 = 0$ vs $H_A : \sigma_\tau^2 > 0$ which of the following statements is not true?
Consider the model $Y = X\beta + \epsilon$, where $Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, X = \left( (x_{ij}) \right)_{n \times p}$ and $\beta = \begin{bmatrix} \beta_1 \\ \vdots \\ \beta_p \end{bmatrix}, \epsilon = \begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{bmatrix}$. $E(\epsilon) = 0$ and $D(\epsilon) = \sigma^2 I_n, p < n$. Let $\hat{\beta}$ be the solution of $X^T X \beta = X^T Y$. Which of the following are true?
Consider the following regression problem $y_i = \beta_1 e^i + \beta_2 e^{-i} + \epsilon_i; \ i = 1, \ldots, n$. Here $\epsilon_1, \ldots, \epsilon_n$ are i.i.d. $N(0, \sigma^2)$ random variables. If $\hat{\beta}_1$ and $\hat{\beta}_2$ are the least square estimators of $\beta_1$ and $\beta_2$, respectively, then which of the following statements are correct?
