Understanding Correlation Coefficients: Simple vs. Multiple

In statistics, correlation coefficients are used to measure the strength and direction of the linear relationship between variables. There are different types of correlation coefficients, including simple correlation and multiple correlation.

• Simple Correlation Coefficient: A simple correlation coefficient, denoted by $r_{ij}$, measures the linear relationship between two variables, $X_i$ and $X_j$. It ranges from -1 to +1. A value near +1 indicates a strong positive linear relationship, a value near -1 indicates a strong negative linear relationship, and a value near 0 indicates a weak or no linear relationship.
• Multiple Correlation Coefficient: A multiple correlation coefficient, denoted by $R_{1.23}$ (or $R_{1,23}$ as in the question), measures the linear relationship between a dependent variable ($X_1$ in this case) and a set of two or more independent variables ($X_2$ and $X_3$). It represents the correlation between the dependent variable and the best possible linear combination of the independent variables. The multiple correlation coefficient $R$ always ranges from 0 to 1, inclusive. A value of 1 means the dependent variable can be perfectly predicted by a linear combination of the independent variables, while a value of 0 means there is no linear relationship.

Comparing Multiple Correlation $R_{1,23}$ with Simple Correlations $r_{12}$, $r_{13}$, and $r_{23}$

The question asks how the multiple correlation coefficient $R_{1,23}$ compares to the simple correlation coefficients $r_{12}$, $r_{13}$, and $r_{23}$. Here, $r_{12}$ is the simple correlation between $X_1$ and $X_2$, $r_{13}$ is between $X_1$ and $X_3$, and $r_{23}$ is between $X_2$ and $X_3$. The multiple correlation $R_{1,23}$ specifically relates $X_1$ to the set $(X_2, X_3)$.

A key property of the multiple correlation coefficient is that adding more independent variables to a linear model will never decrease the multiple correlation coefficient. It will either stay the same or increase. This means the multiple correlation $R_{1,23}$ between $X_1$ and the set $(X_2, X_3)$ must be at least as large as the absolute value of the simple correlation between $X_1$ and any single variable from the set $(X_2, X_3)$ when considered alone.

Specifically, we have the following properties:

• $R_{1,23} \ge |r_{12}|$
• $R_{1,23} \ge |r_{13}|$

This tells us that $R_{1,23}$ is not less than the absolute value of $r_{12}$ and not less than the absolute value of $r_{13}$. Since $R_{1,23}$ is always non-negative (between 0 and 1), this property can be stated as $R_{1,23} \ge \max(|r_{12}|, |r_{13}|)$.

However, the multiple correlation $R_{1,23}$ is not directly comparable to $r_{23}$ (the correlation between the two independent variables) in the same way. There is no general rule that $R_{1,23} \ge |r_{23}|$. The value of $r_{23}$ influences the value of $R_{1,23}^2$ as per the formula:

$\qquad R_{1,23}^2 = \frac{r_{12}^2 + r_{13}^2 - 2 r_{12} r_{13} r_{23}}{1 - r_{23}^2}$

The formula shows that $r_{23}$ plays a role, but the relationship $R_{1,23} \ge |r_{23}|$ is not a general truth.

Analyzing the Given Options

Let's evaluate the provided options based on the properties discussed:

• Option 1: <p>always equal to the product of r₁₂, r₁₃&nbsp;and r₂₃</p>
  This is incorrect. The multiple correlation coefficient is not calculated by simply multiplying the simple correlation coefficients.
• Option 2: <p>less than any r₁₂, r₂₃, r₁₃</p>
  This is incorrect. As stated by the property, $R_{1,23}$ is generally not less than $|r_{12}|$ or $|r_{13}|$. In fact, it is typically greater than or equal to the larger of $|r_{12}|$ and $|r_{13}|$.
• Option 3: <p>not less than any r₁₂, r₁₃&nbsp;and r₂₃</p>
  This option states that $R_{1,23}$ is not less than any of $r_{12}$, $r_{13}$, and $r_{23}$. As we established, $R_{1,23} \ge |r_{12}|$ and $R_{1,23} \ge |r_{13}|$. Since $R$ is non-negative, this means $R_{1,23} \ge r_{12}$ and $R_{1,23} \ge r_{13}$ if $r_{12}$ and $r_{13}$ are non-negative. If they are negative, the comparison should be with the absolute value. The inclusion of $r_{23}$ in the comparison set in the option's wording is potentially misleading, as $R_{1,23}$ is not guaranteed to be greater than or equal to $|r_{23}|$. However, comparing the options, this statement is the closest to the correct statistical property, which is that the multiple correlation is not less than the absolute value of any simple correlation between the dependent variable and an independent variable included in the model. Interpreting the option in the context of the dependent variable ($X_1$), it implies $R_{1,23} \ge \max(|r_{12}|, |r_{13}|)$.
• Option 4: <p>always equal to the sum of r₁₂, r₁₃&nbsp;and r₂₃</p>
  This is incorrect. The multiple correlation coefficient is not the sum of the simple correlation coefficients.

Based on the statistical properties of multiple correlation, the multiple correlation coefficient $R_{1,23}$ is always greater than or equal to the absolute value of any simple correlation between the dependent variable ($X_1$) and any single independent variable ($X_2$ or $X_3$) included in the multiple regression. Option 3 aligns with this property, even if its wording could be more precise by specifying comparison with absolute values and focusing on $r_{12}$ and $r_{13}$. Therefore, $R_{1,23}$ is not less than any $r_{12}$ and $r_{13}$ (when considering absolute values for comparison with the non-negative $R$).

Summary Comparison

The multiple correlation coefficient captures the combined linear effect of the independent variables on the dependent variable. This combined effect cannot be weaker than the strongest individual effect of an independent variable on the dependent variable (in terms of linear association). Thus, $R_{1,23}$ is at least as large as the absolute value of $r_{12}$ or $r_{13}$.

Conclusion on Multiple Correlation Comparison

The multiple correlation coefficient $R_{1,23}$ between $X_1$ and the set of variables $(X_2, X_3)$ is not less than the absolute value of the simple correlation coefficients between $X_1$ and each of the independent variables ($|r_{12}|$ and $|r_{13}|$). Among the given options, option 3 best reflects this fundamental property, despite the potentially confusing inclusion of $r_{23}$.

Revision Table: Understanding Correlation

Additional Information on Multiple Correlation Properties

The multiple correlation coefficient $R$ provides a measure of the overall goodness of fit of a multiple linear regression model. $R^2$, the coefficient of determination, represents the proportion of the variance in the dependent variable that is predictable from the independent variables. The fact that $R_{1,23} \ge \max(|r_{12}|, |r_{13}|)$ highlights that using multiple predictors ($X_2$ and $X_3$) together can explain at least as much variance in $X_1$ as using the single best predictor ($X_2$ or $X_3$) alone.

If the independent variables ($X_2$ and $X_3$) are uncorrelated ($r_{23}=0$) and both are correlated with $X_1$, the formula for $R_{1,23}^2$ simplifies to $R_{1,23}^2 = r_{12}^2 + r_{13}^2$. In this specific case, $R_{1,23}$ would typically be greater than both $|r_{12}|$ and $|r_{13}|$ (unless one of them is zero).

If one of the simple correlations, say $r_{13}$, is zero, then $X_3$ does not linearly predict $X_1$ when considered alone. However, including $X_3$ in a multiple regression with $X_2$ might still increase the multiple correlation if $X_3$ helps to explain the variance in $X_1$ that is not explained by $X_2$ (e.g., if $X_3$ is correlated with $X_2$ and $X_1$ in a specific way).

The multiple correlation coefficient increases as the independent variables explain more unique variance in the dependent variable. It also increases if the independent variables are themselves less correlated with each other (lower $|r_{23}|$), provided they are correlated with the dependent variable ($X_1$).