Pooled Linear Regression Coefficient Analysis
This problem examines the properties of a pooled least squares estimate ($\hat{\beta_0}$) for the slope coefficient in a simple linear regression model. We are given two independent samples, yielding slope estimates $\hat{\beta_1}$ and $\hat{\beta_2}$ respectively, with the condition $\hat{\beta_1} > \hat{\beta_2} > 0$. We need to determine the possible behavior of $\hat{\beta_0}$, the estimate obtained from the combined (pooled) sample.
Understanding Least Squares Estimates
The simple linear regression model is given by $Y = \alpha + \beta X + \epsilon$. The least squares estimate for the slope coefficient ($\beta$) in a sample of size $n$ is calculated as:
$ \hat{\beta} = \frac{\sum_{i=1}^{n} (X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n} (X_i - \bar{X})^2} $A positive slope estimate ($\hat{\beta} > 0$) indicates that, on average, $Y$ tends to increase as $X$ increases within that specific sample. The pooled estimate $\hat{\beta_0}$ is calculated from the combined dataset containing $n_1 + n_2$ observations.
Analyzing the Pooled Estimate ($\hat{\beta_0}$)
While individual sample estimates $\hat{\beta_1}$ and $\hat{\beta_2}$ are positive, the pooled estimate $\hat{\beta_0}$ is not necessarily bounded between $\hat{\beta_2}$ and $\hat{\beta_1}$. The pooled estimate reflects the overall relationship between $X$ and $Y$ across all observations. $\hat{\beta_0}$ can be negative if the covariance between $X$ and $Y$ in the pooled sample is negative. This occurs when the distribution of $X$ values and $Y$ values in the combined dataset results in $\sum (X_i - \bar{X})(Y_i - \bar{Y}) < 0$.
Scenario Illustrating a Negative Pooled Estimate
To demonstrate that $\hat{\beta_0}$ can be negative, consider the following hypothetical data points satisfying $\hat{\beta_1} > \hat{\beta_2} > 0$ (with $n_1, n_2 > 2$):
- Sample 1 ($n_1=3$): Data points $(1, 10), (1.5, 11.5), (2, 13)$. The least squares slope estimate is $\hat{\beta_1} = 3$.
- Sample 2 ($n_2=3$): Data points $(9, 10), (9.5, 10.5), (10, 11)$. The least squares slope estimate is $\hat{\beta_2} = 1$.
Here, the condition $\hat{\beta_1} = 3 > \hat{\beta_2} = 1 > 0$ is met.
Now, let's pool these six observations: $(1, 10), (1.5, 11.5), (2, 13), (9, 10), (9.5, 10.5), (10, 11)$.
The pooled sample means are $\bar{X} = 5.5$ and $\bar{Y} = 11$.
We calculate the pooled slope estimate $\hat{\beta_0}$ using the formula:
$ \hat{\beta_0} = \frac{\sum_{i=1}^{n_1+n_2} (X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n_1+n_2} (X_i - \bar{X})^2} $The sum of the products $(X_i - \bar{X})(Y_i - \bar{Y})$ for the pooled sample is:
- For points from Sample 1: $(1-5.5)(10-11) + (1.5-5.5)(11.5-11) + (2-5.5)(13-11) = (-4.5)(-1) + (-4)(0.5) + (-3.5)(2) = 4.5 - 2 - 7 = -4.5$.
- For points from Sample 2: $(9-5.5)(10-11) + (9.5-5.5)(10.5-11) + (10-5.5)(11-11) = (3.5)(-1) + (4)(-0.5) + (4.5)(0) = -3.5 - 2 + 0 = -5.5$.
The total sum of products is $-4.5 + (-5.5) = -10$.
The sum of the squared deviations $(X_i - \bar{X})^2$ for the pooled sample is:
- From Sample 1 points: $(-4.5)^2 + (-4)^2 + (-3.5)^2 = 20.25 + 16 + 12.25 = 48.5$.
- From Sample 2 points: $(3.5)^2 + (4)^2 + (4.5)^2 = 12.25 + 16 + 20.25 = 48.5$.
The total sum of squares is $48.5 + 48.5 = 97$.
Therefore, the pooled slope estimate is $\hat{\beta_0} = \frac{-10}{97}$.
Conclusion
The example clearly shows that even with positive individual regression coefficients ($\hat{\beta_1} > \hat{\beta_2} > 0$), the pooled regression coefficient ($\hat{\beta_0}$) can be negative. This demonstrates that the pooled estimate's sign and value depend on the aggregate distribution of the data, not just the relationships within subgroups. Thus, the statement "$\hat{\beta_0}$ can be negative" is true.
Important Questions from Ratio And Regression
- A simple random sample (without replacement) of size $n$ is drawn from a finite population of size $N (\ge 7)$. What is the probability that the $4^{\text{th}}$ population unit is included in the sample but the $6^{\text{th}}$ population unit is not included in the sample?
- For a data set $(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)$ the following two models were fitted using least square method.
Model 1: $y_i = \beta_0 + \beta_1 x_i \quad i = 1, 2, \dots n$
Model 2: $y_i = \beta_0^* + \beta_1^* x_i + \beta_2^* x_i^2 \quad i = 1, 2, \dots n$
Let $\hat{\beta}_0, \hat{\beta}_1$ be least square estimates of $\beta_0, \beta_1$ from model 1 and $\hat{\beta}_0^*, \hat{\beta}_1^*, \hat{\beta}_2^*$ be the least square estimates from model 2.
Let $A = \sum_1^n \left(y_i - (\hat{\beta}_0 + \hat{\beta}_1 x_i)\right)^2$,
$B = \sum_1^n \left(y_i - (\hat{\beta}_0^* + \hat{\beta}_1^* x_i + \hat{\beta}_2^* x_i^2)\right)^2$
Then - Suppose $\bar{Y}$ is the sample mean of the study variables corresponding to a sample of size n using simple random sampling with replacement scheme and $\bar{Y}_{st}$ is the sample mean of the study variables corresponding to a sample of size n using stratified random sampling with replacement scheme under proportional allocation. Which of the following is/are sufficient condition/conditions for $Var(\bar{Y}) = Var(\bar{Y}_{st})$?
- Consider the problem of drawing a sample of size 2 from a finite population of size 20. The sampling is done with replacement using probability proportional to size sampling scheme. The normed size measures $p_1, \cdots, p_{20}$ are given by $p_i = \frac{1}{40}$, $i = 1, \cdots, 10, \; p_i = \frac{3}{40}$, $i = 11, \cdots, 20$. The expected number of distinct units drawn is
- Consider a finite population of size $N$. Let $T_1$ be the sample mean based on a sample of size $n$ under simple random sampling with replacement (SRSWR) scheme. Let $T_2$ be the sample mean based on a stratified random sample of size $n$ where the samples are drawn from each of 4 strata using SRSWR scheme under proportional allocation. Then which of the following are sufficient conditions for $\text{Var}(T_1) = \text{Var}(T_2)$ to hold?
