All Exams Test series for 1 year @ ₹349 only
Question

$X_1, X_2, \cdots, X_n$ are independent and identically distributed $N(\theta, 1)$ random variables, where $\theta$ takes only integer values i.e.
$\theta \in \{\cdots, -2, -1, 0, 1, 2, \cdots\}$.
Which of the following is the maximum likelihood estimator of $\theta$?

The correct answer is
Integer closest to $\bar{X}$

The problem asks for the Maximum Likelihood Estimator (MLE) of $\theta$ for $n$ independent and identically distributed (i.i.d.) random variables $X_1, X_2, \cdots, X_n$ from a Normal distribution $N(\theta, 1)$, with the constraint that $\theta$ must be an integer ($\theta \in \mathbb{Z}$).

Likelihood Function Derivation

  1. The probability density function (PDF) of a single observation $X_i$ from $N(\theta, 1)$ is: $f(x_i; \theta) = \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x_i - \theta)^2}{2}\right)$
  2. The likelihood function $L(\theta)$ for $n$ i.i.d. observations is the product of the individual PDFs: $L(\theta) = \prod_{i=1}^{n} f(x_i; \theta) = \prod_{i=1}^{n} \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x_i - \theta)^2}{2}\right)$ $L(\theta) = \left(\frac{1}{\sqrt{2\pi}}\right)^n \exp\left(-\frac{1}{2} \sum_{i=1}^{n} (x_i - \theta)^2\right)$
  3. To find the MLE, we typically maximize the log-likelihood function $\ln L(\theta)$: $\ln L(\theta) = -\frac{n}{2}\ln(2\pi) - \frac{1}{2} \sum_{i=1}^{n} (x_i - \theta)^2$

Maximizing the Likelihood

Maximizing $\ln L(\theta)$ is equivalent to minimizing the term $\sum_{i=1}^{n} (x_i - \theta)^2$. If there were no constraint on $\theta$, we would find the derivative with respect to $\theta$ and set it to zero:

$\frac{d}{d\theta} \ln L(\theta) = \frac{d}{d\theta} \left(-\frac{1}{2} \sum_{i=1}^{n} (x_i - \theta)^2\right) = -\frac{1}{2} \sum_{i=1}^{n} 2(x_i - \theta)(-1) = \sum_{i=1}^{n} (x_i - \theta)$

Setting the derivative to zero:

$\sum_{i=1}^{n} (x_i - \theta) = 0 \implies \sum_{i=1}^{n} x_i - n\theta = 0 \implies n\bar{X} - n\theta = 0 \implies \theta = \bar{X}$

This shows that the value of $\theta$ that maximizes the likelihood (without constraints) is the sample mean, $\bar{X}$.

Applying the Integer Constraint

However, $\theta$ must be an integer. The likelihood function $L(\theta)$ (and $\ln L(\theta)$) is maximized when the sum of squared differences, $\sum_{i=1}^{n} (x_i - \theta)^2$, is minimized. This sum represents the sum of squared deviations from $\theta$. Geometrically, we are looking for the integer $\theta$ that is closest to the data points $x_1, \cdots, x_n$. The value that minimizes the sum of squared differences is the mean, $\bar{X}$. Since $\theta$ must be an integer, the integer value that minimizes this sum is the integer closest to $\bar{X}$.

For example, if $\bar{X} = 2.3$, the integers are 2 and 3. The sum of squares is minimized when $\theta=2$, because $|2.3 - 2| < |2.3 - 3|$. If $\bar{X} = 2.8$, the sum of squares is minimized when $\theta=3$, because $|2.8 - 3| < |2.8 - 2|$. If $\bar{X}=2.5$, both integers 2 and 3 minimize the sum equally (assuming we round halves up or down consistently, the definition of "closest" handles this).

Conclusion

Therefore, the maximum likelihood estimator of $\theta$ under the integer constraint is the integer closest to the sample mean $\bar{X}$.

Was this answer helpful?

Important Questions from Elementary Bayesian Inference

  1. Suppose the distribution of $X$ given $\theta$ is normal with mean $\theta$ and variance $15$. Further, let the prior (improper) distribution of $\theta$ be proportional to $1, \ -\infty<\theta<\infty$. If the observed value of $X$ is $13$, then which of the following statements is true?
  2. Let $X_1, X_2, . . ., X_n$ be a random sample from $N(\theta, 1)$, $\theta \in R$. If $\hat{\theta}$ is the Bayes estimator of $\theta$ with respect to some prior $\pi(\theta)$ and loss function $L(\theta, d)$. Then, which of the following statements are true?
  3. Let $X|\theta \sim \text{Uniform}[0, \theta]$ and $\theta$ has an Exponential distribution with mean $\lambda$, where $\lambda > 3$ is known. If the realized value of $X$ is $2025$, then the posterior mode equals
  4. Suppose the probability mass function of a random variable X under the parameter $\theta = \theta_0$ and $\theta = \theta_1 (\ne \theta_0)$ are given by
    x0123
    $p_{\theta_0}(x)$0.010.040.50.45
    $p_{\theta_1}(x)$0.020.080.40.5

    Define a test $\phi$ such that $\phi(x) = 1$ if $x = 0, 1$, and $0$ if $x = 2, 3$.
    For testing $H_0: \theta = \theta_0$ against $H_1: \theta = \theta_1$, the test $\phi$ is
  5. $\theta$ is the probability of obtaining a head in the toss of a coin. The coin is tossed three times and we record
    $Y = 1$ if all the three tosses result in heads
    $Y = 2$ if all the three tosses result in tails
    $Y = 3$ otherwise
    If the prior density of $\theta$ is Beta $(\alpha, \beta)$, and $\hat{\theta}_i$ is the posterior mean of $\theta$ given $Y = i$, for $i = 1, 2$, then
Need Expert Advice?

Start Your Preparation with Prepp Mobile App

Download the app from Google Play & App Store
Download the app from Google Play & App Store
Prepp Mobile App