Understanding the Maximum Likelihood Estimator (MLE)
We are given a random sample $X_1, \dots, X_n$ from a normal distribution $N(\theta, 1)$, where the parameter $\theta$ can only take two possible values: $\theta \in \{1, 2\}$. We need to find the Maximum Likelihood Estimator (MLE) for $\theta$.
Likelihood Function and Maximization
The probability density function (PDF) of the normal distribution is:
$ f(x|\theta) = \frac{1}{\sqrt{2\pi}} e^{-\frac{(x-\theta)^2}{2}} $
The likelihood function for the sample $X_1, \dots, X_n$ is the product of the PDFs:
$ L(\theta | x_1, \dots, x_n) = \prod_{i=1}^n f(x_i|\theta) = \left(\frac{1}{\sqrt{2\pi}}\right)^n \exp\left(-\sum_{i=1}^n \frac{(x_i-\theta)^2}{2}\right) $
To find the MLE, we need to find the value of $\theta \in \{1, 2\}$ that maximizes $L(\theta)$. Maximizing $L(\theta)$ is equivalent to minimizing the sum of the squared differences, $S(\theta)$:
$ S(\theta) = \sum_{i=1}^n (x_i-\theta)^2 $
Comparing Likelihoods for $\theta=1$ and $\theta=2$
Since $\theta$ can only be 1 or 2, we compare the likelihood values (or equivalently, the sum of squares) for these two possibilities:
- Calculate $S(1) = \sum_{i=1}^n (x_i-1)^2$.
- Calculate $S(2) = \sum_{i=1}^n (x_i-2)^2$.
The MLE, denoted by $\hat{\theta}$, is the value (1 or 2) that yields the smaller sum of squares:
- If $S(1) < S(2)$, then $\hat{\theta} = 1$.
- If $S(2) < S(1)$, then $\hat{\theta} = 2$.
- If $S(1) = S(2)$, both $\theta=1$ and $\theta=2$ maximize the likelihood. This occurs when the sample mean $\overline{x} = \frac{1}{n}\sum_{i=1}^n x_i = 1.5$.
This comparison can be simplified by looking at the sample mean $\overline{x}$. The MLE $\hat{\theta}$ is 1 if $\overline{x} < 1.5$ and 2 if $\overline{x} > 1.5$.
Evaluating the Options
- Existence: Because the parameter space $\theta \in \{1, 2\}$ is finite, a maximum likelihood value must exist for any observed sample. Thus, the MLE exists.
- Is it $\overline{X}$?: The MLE $\hat{\theta}$ is either 1 or 2. The sample mean $\overline{X}$ can take on values other than 1 or 2. For example, if the sample mean is 1.3, the MLE is $\hat{\theta}=1$, not 1.3. If the sample mean is 1.7, the MLE is $\hat{\theta}=2$, not 1.7. Therefore, the MLE is not $\overline{X}$.
- Unbiasedness: The MLE $\hat{\theta}$ is not necessarily unbiased. Its value depends discretely on whether $\overline{X}$ falls below or above 1.5. The expected value $E[\hat{\theta}]$ might not equal the true $\theta$.
Conclusion
The MLE exists because the parameter space is discrete and finite. However, the MLE takes the value 1 or 2 based on the data, and it is not equal to the sample mean $\overline{X}$.
Important Questions from Random Variables
- A mobile manufacturing company uses two brands of batteries for its mobiles. The life (in years) of batteries of Brand I follows an exponential distribution with the probability density function
$ f(x) = \begin{cases} e^{-x}, & \text{if } x>0, \\ 0, & \text{otherwise,} \end{cases} $
and that of Brand II follows a gamma distribution with the probability density function
$ g(x) = \begin{cases} \frac{x}{4} e^{-x/2}, & \text{if } x>0, \\ 0, & \text{otherwise.} \end{cases} $
The company uses the batteries of Brands I and II in proportion of $20\%$ and $80\%$ respectively, in its mobiles. The probability that a randomly selected mobile has the battery life more that $2$ years is - Consider a discrete random variable $X$ with the probability mass function
$ P(X = 0) = \frac{\theta}{3}, \ P(X = 1) = 1 - \frac{\theta}{2}, \ P(X = 2) = \frac{\theta}{6}, $
where $\theta \in (0,1)$ is an unknown parameter. In a random sample of size $90$ from this distribution, the observed counts for $X = 0, 1$ and $2$ are $20, 60$ and $10$, respectively. Then, the maximum likelihood estimate of $\theta$ is - Let $X$ be a random sample of size $1$ from the probability density function
$ f(x|\theta) = \begin{cases} \frac{3}{\theta^3} (\theta - x)^2, & \text{if } 0<x<\theta, \\ 0, & \text{otherwise.} \end{cases} $
If $ \left(\frac{X}{1-\lambda_1}, \frac{X}{1-\lambda_2}\right) $ is a confidence interval for $\theta$ with confidence coefficient $1 - \alpha$, where $\lambda_i \in (0,1), \ i = 1,2, \ \lambda_1<\lambda_2$, and $\alpha \in (0,1)$, then which of the following statements is true? - Let $X_1, X_2, . . ., X_n$ be a random sample from a continuous distribution with the common probability density function
$ f(x|\theta) = \begin{cases} \frac{2\theta^2}{x^{\theta+1}}, & \text{if } x>2, \\ 0, & \text{otherwise,} \end{cases} $
where $\theta (> 0)$ is an unknown parameter. Suppose $P(Y>\chi^2_{m,\beta}) = \beta$, where $Y \sim \chi^2_m$. For testing $H_0: \theta = 1$ against $H_1 : \theta>1$, a uniformly most powerful test of size $\alpha, \ 0<\alpha<1$, will reject $H_0$ if - Suppose we want to estimate the population mean $\bar{Y}$ of a variable for a finite population of size $85$, with $34$ Statisticians and $51$ Biologists. We consider the following sampling scheme:
A stratified random sample with $2$ strata of Statisticians (Stratum-1) and Biologists (Stratum-2), where $12$ Statisticians and $15$ Biologists are drawn from Stratum-1 and Stratum-2, respectively, using SRSWOR scheme.
Denote $\bar{y}_S, \bar{y}_B$, and $\bar{y}$ as the mean of the variable among the Statistician sample, Biologist sample, and the combined sample, respectively. Which of the following is an unbiased estimator of $\bar{Y}$?
