Random Variable Expectations Explained

In probability and statistics, a random variable, often denoted by X, represents the numerical outcome of a random phenomenon. For a real-valued random variable X, we often deal with its mean value or expected value, denoted as E[X], and the expected value of its square, denoted as E[X²]. Understanding the relationship between these two quantities is fundamental.

Variance and its Fundamental Relation

The variance of a random variable X, denoted as Var(X) or σ², is a measure of how far the values of a random variable are spread out from its expected value. The definition of variance is given by the formula:

$$\text{Var(X) = E[(X - E[X])^2]}$$

A more common and computationally useful formula for variance is:

$$\text{Var(X) = E[X^2] - (E[X])^2}$$

This formula connects E[X²] and (E[X])² directly to the variance.

Inequality Derivation: Why E[X²] ≥ (E[X])² Always Holds

One of the most important properties of variance is that it can never be negative. Since variance measures the squared deviation from the mean, it must always be greater than or equal to zero.

• The term $$(X - E[X])^2$$ represents a squared value. Any real number squared is always non-negative (greater than or equal to zero).
• Therefore, the expected value of a non-negative quantity must also be non-negative.
• This means, $$\text{Var(X) ≥ 0}$$.

Substituting the computational formula for variance into this inequality, we get:

$$\text{E[X^2] - (E[X])^2 ≥ 0}$$

By adding $$(E[X])^2$$ to both sides of the inequality, we arrive at the relation that always holds:

$$\text{E[X^2] ≥ (E[X])^2}$$

This fundamental statistical relation is a consequence of the fact that variance is non-negative.

Analysis of Given Options

Let's examine each option in light of our derivation:

• Option 1: $$(E[X])^2 > E(X^2)$$

  This is equivalent to $$E[X^2] < (E[X])^2$$. This would imply that $$E[X^2] - (E[X])^2 < 0$$, which means Var(X) < 0. As established, variance cannot be negative, so this relation does not always hold.

• Option 2: $$E(X^2) ≥ (E[X])^2$$

  This is the relation we derived directly from the non-negativity of variance. It means $$E[X^2] - (E[X])^2 ≥ 0$$, or Var(X) ≥ 0. This relation always holds for any real-valued random variable.

• Option 3: $$E[X^2] = (E[X])^2$$

  This relation implies that $$E[X^2] - (E[X])^2 = 0$$, which means Var(X) = 0. Variance is zero only if the random variable X is a constant (i.e., it takes only one value with probability 1). Since this is not true for all random variables (e.g., a fair coin flip where X can be 0 or 1), this relation does not always hold.

• Option 4: $$E[X^2] > (E[X])^2$$

  This relation implies that $$E[X^2] - (E[X])^2 > 0$$, which means Var(X) > 0. This is true for any random variable that is not a constant. However, if X is a constant, then Var(X) = 0, and the equality $$E[X^2] = (E[X])^2$$ holds. Since the question asks for the relation that always holds, including the case where X is a constant, the ">" strict inequality is not universally true. The "≥" (greater than or equal to) symbol accounts for both cases (constant or non-constant random variables).

Therefore, the relation $$E(X^2) ≥ (E[X])^2$$ is the one that always holds.