7.1.7 MLR - Outliers and Influential Observations¶

Objective: This session addresses the concept of multicollinearity in multiple linear regression, focusing on its detection, implications, and strategies for resolution.
Context: Understanding multicollinearity is crucial for ensuring the reliability of regression models by highlighting issues that may arise from highly correlated independent variables.

Definition: Multicollinearity occurs when two or more independent variables in a regression model are highly correlated, potentially leading to unreliable and unstable estimates of regression coefficients.
Detection Methods:
Scatter Plots: Visual examination of scatter plots between pairs of independent variables.
Correlation Coefficients: Statistical measure that indicates the degree to which two variables are related. A correlation coefficient greater than 0.7 or less than -0.7 suggests potential multicollinearity.

Scenario:
Analyzing data with two independent variables: annual income (\(x_1\)) and household size (\(x_2\)).
Found a sample correlation coefficient of 0.32 between \(x_1\) and \(x_2\), indicating a moderate degree of linear association.

Setup:
Introducing a new variable, monthly income, highly correlated with annual income, to illustrate multicollinearity effects.
Regression model including both monthly and annual income demonstrates inflated variance in estimated coefficients, leading to misleading results.

Coefficient Instability: Changes in model coefficients from small variations in the data or model specification.
Impaired Hypothesis Testing:
T-tests may incorrectly fail to reject the null hypothesis for individual coefficients due to inflated standard errors.
This can mask the true impact of predictors on the dependent variable, suggesting no significant relationship where one may exist.

Removing Variables: Eliminating one or more correlated independent variables to reduce redundancy and enhance model interpretability.
Principal Component Analysis (PCA): Transforming correlated variables into a smaller number of uncorrelated variables.
Ridge Regression: A regularization method that introduces a bias term to the regression estimates, reducing variance at the expense of introducing some bias.

Summary: Multicollinearity is a significant issue in multiple regression that requires careful analysis and appropriate treatment to ensure the validity and reliability of the model.
Next Steps: The course will continue to explore diagnostic tools and advanced regression techniques to handle multicollinearity and other common issues in regression analysis.