Multiple Linear Regression is a statistical method used to model the relationship between a dependent variable and two or more independent variables. The basic form of multiple linear regression:
Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ + ε
- Y is the dependent variable.
- X₁, X₂, …, Xₖ are the independent variables.
- β₀ is the intercept, representing the expected value of Y when all independent variables are set to 0.
- β₁, β₂, …, βₖ are the coefficients that represent the effect of each independent variable on Y.
- ε is the error term. This term accounts for the variability in the data that is not explained by the model.
The goal of multiple linear regression is to estimate the values of the coefficients that best fit the data. The least squares method aims to estimate the coefficients in the multiple linear regression model by minimizing the sum of squared differences between the predicted values and the actual values of the dependent variable. This is often referred to as the “sum of squared residuals” or “sum of squared errors“.
The function we aim to minimize is: Minimize: Σ(Yᵢ - Ŷᵢ)²,
- Yᵢ is the actual observed value of the ith data point.
- Ŷᵢ is the predicted value of the ith data point based on the regression model.
Matrix Notation:
- Matrix notation in multiple linear regression simplifies the mathematical representation of complex relationships between dependent and independent variables.
- It offers computational advantages, making it suitable for handling large datasets and high-dimensional models.
- Its standardized format enhances compatibility with statistical software, enabling seamless implementation and analysis, etc.
The multiple linear regression model can be expressed using matrix notation.
Y = Xβ + ε
Where:
- Y is the vector of observed values of the dependent variable.
- X is the matrix of independent variables (including a column of 1s for the intercept).
- β is the vector of coefficients.
- ε is the vector of errors or residuals.
To estimate the coefficients β:
β = (XᵀX)⁻¹XᵀY