To solve the problem regarding the properties of the projection matrix \( H = X(X'X)^{-1}X' \) in a linear regression model, let us examine each statement:
- \(0 \le h_{ii} \le 1\), \(1 \le i \le n\).
The diagonal elements of the hat matrix \( H \), denoted as \( h_{ii} \), are also known as leverage values. They satisfy the condition \(0 \le h_{ii} \le 1\). This is because each \( h_{ii} \) represents the influence of the i-th observation on its own predicted value. The sum of all leverage values equals to the number of parameters estimated, i.e., \( \sum_{i=1}^n h_{ii} = \text{tr}(H) = p \). Therefore, this statement is correct. - If \( h_{ii} = 0 \) or 1 for some i, then \( h_{ij} = 0 \) for all \( j \ne i \).
If \( h_{ii} = 0 \), it means the i-th observation doesn't influence its prediction at all, which is not possible in practice for a non-empty model. However, if \( h_{ii} = 1 \), it implies that the i-th observation perfectly predicts itself, and thus, the i-th row of the projection matrix \( H \) would have all other elements \( h_{ij} = 0 \) for \( j \neq i \). Hence, this statement is true: if \( h_{ii} = 1 \), then \( h_{ij} = 0 \) for all \( j \neq i \). - The variance-covariance matrix of the vector of the predicted values \(\hat{Y}\) is \(\sigma^2 H\).
The predicted values \(\hat{Y}\) are given by \( \hat{Y} = HY \). The variance-covariance matrix of \(\hat{Y}\) is given by: \[ D(\hat{Y}) = D(HY) = H D(Y) H' \] Given \( D(Y) = \sigma^2 I \), we have: \[ D(\hat{Y}) = \sigma^2 H H' = \sigma^2 H \] Since \( H \) is idempotent (\( H^2 = H \)), this statement is correct. - For \(1 \le i \le n\), if \(e_i\) is the residual corresponding to \(Y_i\), i.e., \(e_i = Y_i - \hat{Y}_i\), then the variance of \(e_i\) equals \(\sigma^2(1 - h_{ii})\).
The residuals are given by \( e = Y - \hat{Y} = (I - H)Y \). The variance-covariance matrix of the residuals is: \[ D(e) = (I - H)D(Y)(I - H)' \] Given \( D(Y) = \sigma^2 I \), we have: \[ D(e) = \sigma^2 (I - H)(I - H)' \] Since \( (I - H) \) is also idempotent (\( (I - H)^2 = I - H \)), we have: \[ D(e) = \sigma^2 (I - H) \] Therefore, the variance of the i-th residual is \(\sigma^2(1 - h_{ii})\), which makes this statement correct.
Thus, all the given statements about the properties of the projection matrix \( H \) in the context of a linear regression model are correct.
Important Questions from Gauss-Markov Models
- Let $X_1, X_2,..., X_n$ ($n > 5$) be independent random variables such that
$X_t = \alpha + \alpha^2 t + \epsilon_t$, for $t = 1,..., n$,
where, $\epsilon_1, \epsilon_2,..., \epsilon_n$ are independent and identically distributed $N(0, \sigma^2)$ random variables. Here $\alpha \in R$ and $\sigma > 0$ are unknown parameters. Which of the following statements are true? Consider the linear statistical model
$y_{ij} = \mu + \tau_i + \varepsilon_{ij}; \quad i = 1, 2, \cdots, a; \quad j = 1, 2, \cdots, n$
where $\mu$ is unknown, $\tau_i$ are independently and identically distributed as $N(0, \sigma_\tau^2)$, $\varepsilon_{ij}$ are independently and identically distributed as $N(0, \sigma^2)$; $\tau_i$ and $\varepsilon_{ij}$ are independent for all $i$ and $j$. Note that $\tau_i$ is the $i^{th}$ treatment effect. Suppose $SS_{total}, SS_{treatment}, SS_{error}$ are total sum of squares, total treatment sum of squares and error sum of squares, respectively. To test :$H_0 : \sigma_\tau^2 = 0$ vs $H_A : \sigma_\tau^2 > 0$ which of the following statements is not true?
Consider the model $Y = X\beta + \epsilon$, where $Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, X = \left( (x_{ij}) \right)_{n \times p}$ and $\beta = \begin{bmatrix} \beta_1 \\ \vdots \\ \beta_p \end{bmatrix}, \epsilon = \begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{bmatrix}$. $E(\epsilon) = 0$ and $D(\epsilon) = \sigma^2 I_n, p < n$. Let $\hat{\beta}$ be the solution of $X^T X \beta = X^T Y$. Which of the following are true?
Let $X_1, X_2, \ldots$ be a random sample from uniform $(0, 3\theta), \theta > 0$. Define
$T_n = \frac{1}{3} \max\{X_1, X_2, \ldots, X_n\}$. Which of the following is NOT true?
- Consider the following regression problem $y_i = \beta_1 e^i + \beta_2 e^{-i} + \epsilon_i; \ i = 1, \ldots, n$. Here $\epsilon_1, \ldots, \epsilon_n$ are i.i.d. $N(0, \sigma^2)$ random variables. If $\hat{\beta}_1$ and $\hat{\beta}_2$ are the least square estimators of $\beta_1$ and $\beta_2$, respectively, then which of the following statements are correct?
