Regression Analysis

Linear regression models a dependent variable Y in terms of a linear combination of p independent variables X=[X1|...|Xp] and estimates the coefficients of the combination using independent observations (x_i,Y_i ),i=1,...,n. The Gauss-Markov conditions guarantees that the least squares estimate of the regression coefficients constitutes the best linear estimator. Under the assumption of white noise, it is possible to test the significance of each regression coefficient, evaluate the uncertainty/goodness of fit, and use the fitted model for predicting novel outcomes. When p>n, classical linear regression cannot be applied, and penalized approaches such as ridge regression, lasso or elastic net have to be used.