The observation $X$ has normal distribution with unknown mean $-\infty < \mu < \infty$ and variance $1$. Consider the problem of estimation of $\mu$ based on $X$ under the squared error loss. Which of the following statements are correct?
The problem asks about the bias properties of Bayes estimators for the mean ($\mu$) of a normal distribution $N(\mu, 1)$ under squared error loss, considering both proper and improper priors.
Analyzing Estimator Bias
Statement A: Bayes Estimator Bias with Proper Prior
Statement: The Bayes estimator of $\mu$ under a proper prior is always biased.
Reasoning: Let $X \sim N(\mu, 1)$ with prior $\pi(\mu)$ which is a proper prior (integrates to 1). The Bayes estimator under squared error loss is the posterior mean, $\delta(x) = E[\mu|x]$. The bias is $B(\mu) = E[\delta(X)] - \mu$. Unlike the Maximum Likelihood Estimator (MLE), $\hat{\mu}_{MLE} = x$, which is unbiased ($E[X] = \mu$), the Bayes estimator typically shrinks the estimate towards the prior mean. For example, if $\pi(\mu) \sim N(\mu_0, \sigma^2)$, the Bayes estimator is $\delta(x) = \frac{\sigma^2 x + \mu_0}{\sigma^2+1}$. Its bias is $B(\mu) = E[\delta(X)] - \mu = \frac{\sigma^2 \mu + \mu_0}{\sigma^2+1} - \mu = \frac{\mu_0 - \mu}{\sigma^2+1}$. This bias is non-zero unless $\mu_0 = \mu$. Since $\mu$ is unknown, this estimator is generally biased for $\mu \neq \mu_0$. A degenerate proper prior $\pi(\mu) = \delta_{\mu_0}(\mu)$ yields $\delta(x)=\mu_0$, which is biased unless $\mu = \mu_0$. Thus, the statement holds true as the Bayes estimator generally deviates from the unbiased MLE.
Conclusion: Statement A is correct.
Statement B: Unbiased Bayes Estimator with Proper Prior
Statement: There is a proper prior for which the Bayes estimator of $\mu$ is unbiased.
Reasoning: As established in the analysis of Statement A, Bayes estimators under proper priors for location parameters are typically biased because they involve shrinkage. An unbiased estimator for $\mu$ requires $E[\delta(X)] = \mu$. This generally does not happen with a proper prior unless the prior mean perfectly aligns with the unknown true mean $\mu$, which isn't possible in practice.
Conclusion: Statement B is incorrect.
Statement C: Bayes Estimator Bias with Improper Prior
Statement: The Bayes estimator of $\mu$ under an improper prior is always biased.
Reasoning: Consider the improper prior $\pi(\mu) = 1$ for all $\mu \in (-\infty, \infty)$. The posterior distribution $p(\mu|x)$ is proportional to the likelihood times the prior, $p(\mu|x) \propto \exp(-\frac{1}{2}(x-\mu)^2) \cdot 1$. This posterior is $N(x, 1)$. The posterior mean is $E[\mu|x] = x$. The resulting Bayes estimator is $\delta(x) = x$. This estimator is unbiased since $E[X] = \mu$. Therefore, the statement that it's *always* biased is false.
Conclusion: Statement C is incorrect.
Statement D: Unbiased Bayes Estimator with Improper Prior
Statement: There is an improper prior for which the Bayes estimator of $\mu$ is unbiased.
Reasoning: As demonstrated in the analysis of Statement C, using the improper prior $\pi(\mu) = 1$ results in the Bayes estimator $\delta(x) = x$, which is the unbiased MLE.
Conclusion: Statement D is correct.
Final Answer Determination
Based on the analysis, statements A and D are correct.
- Statement A is correct: Bayes estimators under proper priors are typically biased for location parameters.
- Statement D is correct: The uniform improper prior ($\pi(\mu)=1$) leads to the unbiased MLE as the Bayes estimator.
Important Questions from Elementary Bayesian Inference
- Let $X|\theta \sim \text{Uniform}[0, \theta]$ and $\theta$ has an Exponential distribution with mean $\lambda$, where $\lambda > 3$ is known. If the realized value of $X$ is $2025$, then the posterior mode equals
Let $X_1, X_2, \dots, X_7$ be a random sample from $N(\mu, \sigma^2)$ where $\mu$ and $\sigma^2$ are unknown. Consider the problem of testing $H_0: \mu = 2$ against $H_1: \mu > 2$. Suppose the observed values of $x_1, x_2, \dots, x_7$ are $1.2, 1.3, 1.7, 1.8, 2.1, 2.3, 2.7$. If we use the Uniformly Most Powerful test, which of the following is true?
Suppose $X_i \mid \theta_i \sim N(\theta_i, \sigma^2), i = 1, 2$ are independently distributed. Under the prior distribution, $\theta_1$ and $\theta_2$ are i.i.d $N(\mu, \tau^2)$, where $\sigma^2, \mu$ and $\tau^2$ are known. Then which of the following is true about the marginal distributions of $X_1$ and $X_2$?
Let $(X,Y)$ have the joint discrete distribution such that $X \mid Y=y \sim \text{Binomial} \quad (y, 0.5)$ and $Y \sim \text{Poisson}(\lambda), \lambda > 0$, where $\lambda$ is an unknown parameter. Let $T = T(X,Y)$ be any unbiased estimator of $\lambda$. Then
Consider the pdf
$f(x;\theta,\sigma) = \frac{0.9}{\sigma}\phi\left(\frac{x-\theta}{\sigma}\right) + 0.1\ \phi(x-\theta)$, where $-\infty < \theta < \infty$ and $\sigma > 0$ are unknown parameters and $\phi$ denotes the pdf of $N(0,1)$. Let $X_1, X_2, \cdots, X_n$ be a random sample from this probability distribution. Then which of the following is (are) correct?
