Understanding Unbiased Estimators
The sample variance is defined as $s_n^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X}_n)^2$, where $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$.
We want to find the distributions $f_\theta(x)$ for which $s_n^2$ is an unbiased estimator of the parameter $\theta$, meaning $E[s_n^2] = \theta$.
A key statistical property is that $s_n^2$ is always an unbiased estimator of the population variance, $\sigma^2$. Therefore, $s_n^2$ is an unbiased estimator for $\theta$ if and only if the population variance $\sigma^2$ equals $\theta$.
Distribution Variance Analysis
We examine the variance ($\sigma^2$) for each distribution listed in the options:
- Option A: Poisson Distribution
The probability mass function is $f_\theta(x) = e^{-\theta} \frac{\theta^x}{x!}$ for $x = 0, 1, 2, \ldots$. This represents a Poisson distribution with parameter $\lambda = \theta$.
For a Poisson($\lambda$) distribution, the variance is $Var(X) = \lambda$.
Here, $\sigma^2 = Var(X) = \theta$. Since $\sigma^2 = \theta$, $s_n^2$ is an unbiased estimator for $\theta$.
- Option B: Scaled Normal Distribution
The probability density function is $f_\theta(x) = \frac{1}{\sqrt{2\pi}{\sqrt\theta}} e^{-\frac{x^2}{2\theta}}$ for $-\infty < x < \infty$.
This is the PDF of a Normal distribution $N(\mu, \sigma^2)$ with mean $\mu = 0$ and variance $\sigma^2 = \theta$.
Thus, $\sigma^2 = Var(X) = \theta$. Since $\sigma^2 = \theta$, $s_n^2$ is an unbiased estimator for $\theta$.
- Option C: Exponential Distribution
The PDF is $f_\theta(x) = \frac{1}{\theta} e^{-\frac{x}{\theta}}$ for $x > 0$. This is an Exponential distribution with mean $E[X] = \theta$ and variance $Var(X) = \theta^2$.
Here, $\sigma^2 = \theta^2$. Since $\sigma^2 \neq \theta$ (unless $\theta = 1$), $s_n^2$ is not generally an unbiased estimator for $\theta$.
- Option D: Exponential Distribution
The PDF is $f_\theta(x) = \theta e^{-\theta x}$ for $x > 0$. This is an Exponential distribution with mean $E[X] = 1/\theta$ and variance $Var(X) = 1/\theta^2$.
Here, $\sigma^2 = 1/\theta^2$. Since $\sigma^2 \neq \theta$ (unless $\theta = 1$), $s_n^2$ is not generally an unbiased estimator for $\theta$.
Conclusion on Unbiasedness
The sample variance $s_n^2$ is an unbiased estimator for $\theta$ precisely when the population variance $\sigma^2$ is equal to $\theta$. This condition holds true for the distributions given in Option A (Poisson) and Option B (Scaled Normal).
Important Questions from Best Linear Unbiased Estimators
Consider the simple linear regression model $Y_i = \beta x_i + \epsilon_i, i = 1,2,..., n$, where $\beta > 0$, $\sum_{i=1}^n x_i^2 > 0$, and the uncorrelated errors $\epsilon_i, i = 1,2,..., n$, have zero mean and finite variance $\sigma^2 (> 0)$. Let $\tilde{\beta}_1 = \sum_{i=1}^n a_i^*Y_i$, where $a_i$'s minimize $E(\sum_{i=1}^n a_i Y_i - \beta)^2$ with respect to scalars $a_1, a_2,..., a_n$. Let $\tilde{\beta}_2$ be the ordinary least squares estimator of $\beta$. Which of the following statements are true?
Consider the model $y_i = i\beta + \varepsilon_i, i = 1, 2, 3$ where $\varepsilon_1, \varepsilon_2, \varepsilon_3$ are independent with mean $0$ and variance $\sigma^2, 2\sigma^2, 3\sigma^2$ respectively. Which of the following is the best linear unbiased estimate of $\beta$?
Consider the linear model $Y \sim N_n(X\beta, \sigma^2 I)$, where $X$ is a $n \times (k + 1)$ matrix of rank $k + 1 < n$. Let $\hat{\beta}$ and $\hat{\sigma}^2$ be the maximum likelihood estimators of $\beta$ and $\sigma^2$ respectively. Then which of the following statements are true?
Let $\{X_n\}$ be a sequence of independent random variables where the distribution of $X_n$ is normal with mean $\mu$ and variance $n$ for $n = 1, 2, \ldots$ Define
$\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$ and $S_n = \frac{\sum_{i=1}^n \frac{X_i}{i}}{\sum_{i=1}^n \frac{1}{i}}$.
Which of the following are true?
- Let $Y_1, Y_2, Y_3$ be uncorrelated observations with common variance $\sigma^2$ and expectations given by $\mathbb{E}(Y_1) = \beta_1, \mathbb{E}(Y_2) = \beta_2$ and $\mathbb{E}(Y_3) = \beta_1 + \beta_2,$ where $\beta_1, \beta_2$ are unknown parameters. The best linear unbiased estimator of $\beta_1 + \beta_2$ is
