$E(Y_1) = \beta_1 + \beta_2 + \beta_3 = E(Y_2)$,
$E(Y_3) = \beta_1 - \beta_2 = E(Y_4)$,
where $\beta_1, \beta_2$ and $\beta_3$ are unknown parameters. Define $e_1 = \frac{1}{\sqrt{2}}(Y_1 - Y_2)$ and $e_2 = \frac{1}{\sqrt{2}}(Y_3 - Y_4)$. An unbiased estimator of $\sigma^2$ is
Solution: Unbiased Estimator for Variance $\sigma^2$
The problem asks for an unbiased estimator of the common unknown variance $\sigma^2$, given four uncorrelated observations $Y_1, Y_2, Y_3, Y_4$ with common variance $\sigma^2$ and specific expectations. Two new variables, $e_1$ and $e_2$, are defined as linear combinations of these observations.
Key Information and Definitions
- Observations $Y_1, Y_2, Y_3, Y_4$ are uncorrelated.
- $Var(Y_i) = \sigma^2$ for $i=1, 2, 3, 4$.
- Expectations: $E(Y_1) = E(Y_2) = \beta_1 + \beta_2 + \beta_3$ and $E(Y_3) = E(Y_4) = \beta_1 - \beta_2$.
- Derived variables: $e_1 = \frac{1}{\sqrt{2}}(Y_1 - Y_2)$ and $e_2 = \frac{1}{\sqrt{2}}(Y_3 - Y_4)$.
- Goal: Find $\hat{\sigma}^2$ such that $E(\hat{\sigma}^2) = \sigma^2$.
Calculating Expectations of $e_1$ and $e_2$
First, we find the expected values of $e_1$ and $e_2$. Since the expectations of $Y_1$ and $Y_2$ are equal, and the expectations of $Y_3$ and $Y_4$ are equal:
- $E(e_1) = E\left(\frac{1}{\sqrt{2}}(Y_1 - Y_2)\right) = \frac{1}{\sqrt{2}}(E(Y_1) - E(Y_2)) = \frac{1}{\sqrt{2}}(0) = 0$.
- $E(e_2) = E\left(\frac{1}{\sqrt{2}}(Y_3 - Y_4)\right) = \frac{1}{\sqrt{2}}(E(Y_3) - E(Y_4)) = \frac{1}{\sqrt{2}}(0) = 0$.
Calculating Variances of $e_1$ and $e_2$
Next, we calculate the variances of $e_1$ and $e_2$. Using the property that $Var(X - Y) = Var(X) + Var(Y)$ for uncorrelated variables X and Y, and $Var(aX) = a^2 Var(X)$:
- $Var(e_1) = Var\left(\frac{1}{\sqrt{2}}(Y_1 - Y_2)\right) = \left(\frac{1}{\sqrt{2}}\right)^2 Var(Y_1 - Y_2)$ $Var(e_1) = \frac{1}{2} (Var(Y_1) + Var(Y_2)) = \frac{1}{2} (\sigma^2 + \sigma^2) = \frac{1}{2} (2\sigma^2) = \sigma^2$.
- $Var(e_2) = Var\left(\frac{1}{\sqrt{2}}(Y_3 - Y_4)\right) = \left(\frac{1}{\sqrt{2}}\right)^2 Var(Y_3 - Y_4)$ $Var(e_2) = \frac{1}{2} (Var(Y_3) + Var(Y_4)) = \frac{1}{2} (\sigma^2 + \sigma^2) = \frac{1}{2} (2\sigma^2) = \sigma^2$.
Deriving the Unbiased Estimator
For any random variable $X$ with mean $E(X)$ and variance $Var(X)$, we know that $Var(X) = E(X^2) - (E(X))^2$. Therefore, $E(X^2) = Var(X) + (E(X))^2$. Applying this to $e_1$ and $e_2$:
- $E(e_1^2) = Var(e_1) + (E(e_1))^2 = \sigma^2 + 0^2 = \sigma^2$.
- $E(e_2^2) = Var(e_2) + (E(e_2))^2 = \sigma^2 + 0^2 = \sigma^2$.
Now, we check the expectation of each option to find the unbiased estimator:
- Option 1: $E\left(\frac{1}{2}(e_1^2 - e_2^2)\right) = \frac{1}{2}(E(e_1^2) - E(e_2^2)) = \frac{1}{2}(\sigma^2 - \sigma^2) = 0 \neq \sigma^2$.
- Option 2: $E\left(\frac{1}{2}(e_1^2 + e_2^2)\right) = \frac{1}{2}(E(e_1^2) + E(e_2^2)) = \frac{1}{2}(\sigma^2 + \sigma^2) = \frac{1}{2}(2\sigma^2) = \sigma^2$. This is the unbiased estimator.
- Option 3: $E\left(\frac{1}{4}(e_1^2 + e_2^2)\right) = \frac{1}{4}(E(e_1^2) + E(e_2^2)) = \frac{1}{4}(\sigma^2 + \sigma^2) = \frac{1}{4}(2\sigma^2) = \frac{1}{2}\sigma^2 \neq \sigma^2$.
- Option 4: $E(e_1^2 + e_2^2) = E(e_1^2) + E(e_2^2) = \sigma^2 + \sigma^2 = 2\sigma^2 \neq \sigma^2$.
Therefore, the unbiased estimator for $\sigma^2$ is $\frac{1}{2}(e_1^2 + e_2^2)$.
Important Questions from Best Linear Unbiased Estimators
Consider the simple linear regression model $Y_i = \beta x_i + \epsilon_i, i = 1,2,..., n$, where $\beta > 0$, $\sum_{i=1}^n x_i^2 > 0$, and the uncorrelated errors $\epsilon_i, i = 1,2,..., n$, have zero mean and finite variance $\sigma^2 (> 0)$. Let $\tilde{\beta}_1 = \sum_{i=1}^n a_i^*Y_i$, where $a_i$'s minimize $E(\sum_{i=1}^n a_i Y_i - \beta)^2$ with respect to scalars $a_1, a_2,..., a_n$. Let $\tilde{\beta}_2$ be the ordinary least squares estimator of $\beta$. Which of the following statements are true?
Consider the model $y_i = i\beta + \varepsilon_i, i = 1, 2, 3$ where $\varepsilon_1, \varepsilon_2, \varepsilon_3$ are independent with mean $0$ and variance $\sigma^2, 2\sigma^2, 3\sigma^2$ respectively. Which of the following is the best linear unbiased estimate of $\beta$?
Consider the linear model $Y \sim N_n(X\beta, \sigma^2 I)$, where $X$ is a $n \times (k + 1)$ matrix of rank $k + 1 < n$. Let $\hat{\beta}$ and $\hat{\sigma}^2$ be the maximum likelihood estimators of $\beta$ and $\sigma^2$ respectively. Then which of the following statements are true?
- Let $X_1, X_2, \ldots, X_n$ be a random sample from $f_\theta(x)$, a probability density function or a probability mass function. Define $s_n^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X}_n)^2$, where $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$. Then $s_n^2$ is unbiased for $\theta$ if
Let $\{X_n\}$ be a sequence of independent random variables where the distribution of $X_n$ is normal with mean $\mu$ and variance $n$ for $n = 1, 2, \ldots$ Define
$\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$ and $S_n = \frac{\sum_{i=1}^n \frac{X_i}{i}}{\sum_{i=1}^n \frac{1}{i}}$.
Which of the following are true?
