Understanding the Maximum Likelihood Estimator (MLE)

We are given a random sample $X_1, \dots, X_n$ from a normal distribution $N(\theta, 1)$, where the parameter $\theta$ can only take two possible values: $\theta \in \{1, 2\}$. We need to find the Maximum Likelihood Estimator (MLE) for $\theta$.

Likelihood Function and Maximization

The probability density function (PDF) of the normal distribution is:

$ f(x|\theta) = \frac{1}{\sqrt{2\pi}} e^{-\frac{(x-\theta)^2}{2}} $

The likelihood function for the sample $X_1, \dots, X_n$ is the product of the PDFs:

$ L(\theta | x_1, \dots, x_n) = \prod_{i=1}^n f(x_i|\theta) = \left(\frac{1}{\sqrt{2\pi}}\right)^n \exp\left(-\sum_{i=1}^n \frac{(x_i-\theta)^2}{2}\right) $

To find the MLE, we need to find the value of $\theta \in \{1, 2\}$ that maximizes $L(\theta)$. Maximizing $L(\theta)$ is equivalent to minimizing the sum of the squared differences, $S(\theta)$:

$ S(\theta) = \sum_{i=1}^n (x_i-\theta)^2 $

Comparing Likelihoods for $\theta=1$ and $\theta=2$

Since $\theta$ can only be 1 or 2, we compare the likelihood values (or equivalently, the sum of squares) for these two possibilities:

• Calculate $S(1) = \sum_{i=1}^n (x_i-1)^2$.
• Calculate $S(2) = \sum_{i=1}^n (x_i-2)^2$.

The MLE, denoted by $\hat{\theta}$, is the value (1 or 2) that yields the smaller sum of squares:

• If $S(1) < S(2)$, then $\hat{\theta} = 1$.
• If $S(2) < S(1)$, then $\hat{\theta} = 2$.
• If $S(1) = S(2)$, both $\theta=1$ and $\theta=2$ maximize the likelihood. This occurs when the sample mean $\overline{x} = \frac{1}{n}\sum_{i=1}^n x_i = 1.5$.

This comparison can be simplified by looking at the sample mean $\overline{x}$. The MLE $\hat{\theta}$ is 1 if $\overline{x} < 1.5$ and 2 if $\overline{x} > 1.5$.

Evaluating the Options

• Existence: Because the parameter space $\theta \in \{1, 2\}$ is finite, a maximum likelihood value must exist for any observed sample. Thus, the MLE exists.
• Is it $\overline{X}$?: The MLE $\hat{\theta}$ is either 1 or 2. The sample mean $\overline{X}$ can take on values other than 1 or 2. For example, if the sample mean is 1.3, the MLE is $\hat{\theta}=1$, not 1.3. If the sample mean is 1.7, the MLE is $\hat{\theta}=2$, not 1.7. Therefore, the MLE is not $\overline{X}$.
• Unbiasedness: The MLE $\hat{\theta}$ is not necessarily unbiased. Its value depends discretely on whether $\overline{X}$ falls below or above 1.5. The expected value $E[\hat{\theta}]$ might not equal the true $\theta$.

Conclusion

The MLE exists because the parameter space is discrete and finite. However, the MLE takes the value 1 or 2 based on the data, and it is not equal to the sample mean $\overline{X}$.