Linear regression

regression coefficientmultiple linear regressionregressionlinear regression modellinearleast squares regressionerror variablelinear weightsregression linemultiple regression
In statistics, linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more explanatory variables (or independent variables).wikipedia
463 Related Articles

Simple linear regression

simple regressioni.e. regression linelinear least squares regression with an intercept term and a single explanator
The case of one explanatory variable is called simple linear regression.
In statistics, simple linear regression is a linear regression model with a single explanatory variable.

Regression analysis

regressionmultiple regressionregression model
Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of the response given the values of the predictors, rather than on the joint probability distribution of all of these variables, which is the domain of multivariate analysis.
The most common form of regression analysis is linear regression, in which a researcher finds the line (or a more complex linear function) that most closely fits the data according to a specific mathematical criterion.

Linear predictor function

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data.
This sort of function usually comes in linear regression, where the coefficients are called regression coefficients.

Linear model

linear modelslinearlinear process
Such models are called linear models.
The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model.

Tikhonov regularization

ridge regressionregularizeda squared regularizing function
Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).
Also known as ridge regression, it is particularly useful to mitigate the problem of multicollinearity in linear regression, which commonly occurs in models with large numbers of parameters.

Ordinary least squares

OLSleast squaresOrdinary least squares regression
ordinary least squares): Multivariate analogues of ordinary least squares (OLS) and generalized least squares (GLS) have been developed.
In statistics, ordinary least squares (OLS) is a type of linear least squares method for estimating the unknown parameters in a linear regression model.

Generalized linear model

generalized linear modelslink functiongeneralised linear model
In statistics, the generalized linear model (GLM) is a flexible generalization of ordinary linear regression that allows for response variables that have error distribution models other than a normal distribution.

Lasso (statistics)

LASSOLasso methodLeast Absolute Shrinkage and Selection Operator
Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).
It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.

Segmented regression

Linear segmented regressionPiecewise regressionsegmented regression analysis
Segmented linear regression is segmented regression whereby the relations in the intervals are obtained by linear regression.

Bayesian linear regression

Bayesian regressionlinear Bayesian regression
In statistics, Bayesian linear regression is an approach to linear regression in which the statistical analysis is undertaken within the context of Bayesian inference.

Polynomial regression

cubic regressionPolynomial fittingregression
For this reason, polynomial regression is considered to be a special case of multiple linear regression.

Weighted least squares

Batch Least Squareslinear least squaresweighted
For example, weighted least squares is a method for estimating linear regression models when the response variables may have different error variances, possibly with correlated errors.
Weighted least squares (WLS), also known as weighted linear regression, is a generalization of ordinary least squares and linear regression in which the errors covariance matrix is allowed to be different from an identity matrix.

Heteroscedasticity-consistent standard errors

Huber–White standard errorEicker–WhiteHuber–White standard errors
(See also Weighted linear least squares, and Generalized least squares.) Heteroscedasticity-consistent standard errors is an improved method for use with uncorrelated but potentially heteroscedastic errors.
The topic of heteroscedasticity-consistent (HC) standard errors arises in statistics and econometrics in the context of linear regression and time series analysis.

Loss function

objective functioncost functionrisk function
Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).
Many common statistics, including t-tests, regression models, design of experiments, and much else, use least squares methods applied using linear regression theory, which is based on the quadratric loss function.

Generalized least squares

feasible generalized least squaresgeneralizedgeneralized (correlated)
(See also Weighted linear least squares, and Generalized least squares.) Heteroscedasticity-consistent standard errors is an improved method for use with uncorrelated but potentially heteroscedastic errors. Multivariate analogues of ordinary least squares (OLS) and generalized least squares (GLS) have been developed.
In statistics, generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in a regression model.

Multicollinearity

collinearitymulticollinearperfect multicollinearity
In this situation the coefficient estimates of the multiple regression may change erratically in response to small changes in the model or the data.

Statistics

statisticalstatistical analysisstatistician
In statistics, linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more explanatory variables (or independent variables).
Least squares applied to linear regression is called ordinary least squares method and least squares applied to nonlinear regression is called non-linear least squares.

Design matrix

data matrixdesign matricesdata matrices
The theory relating to such models makes substantial use of matrix manipulations involving the design matrix: see for example linear regression.

Partial least squares regression

partial least squarespartial least-squarespartial least squares ('''PLS''')
Partial least squares regression (PLS regression) is a statistical method that bears some relation to principal components regression; instead of finding hyperplanes of maximum variance between the response and independent variables, it finds a linear regression model by projecting the predicted variables and the observable variables to a new space.

Median

averagesample medianmedian-unbiased estimator
Most commonly, the conditional mean of the response given the values of the explanatory variables (or predictors) is assumed to be an affine function of those values; less commonly, the conditional median or some other quantile is used.
The Theil–Sen estimator is a method for robust linear regression based on finding medians of slopes.

Dummy variable (statistics)

dummy variableindicator variabledummy variables
Care must be taken when interpreting regression results, as some of the regressors may not allow for marginal changes (such as dummy variables, or the intercept term), while others cannot be held fixed (recall the example from the introduction: it would be impossible to "hold t i fixed" and at the same time change the value of t i 2 ).
A dummy independent variable (also called a dummy explanatory variable) which for some observation has a value of 0 will cause that variable's coefficient to have no role in influencing the dependent variable, while when the dummy takes on a value 1 its coefficient acts to alter the intercept.

Commonality analysis

Commonality analysis may be helpful in disentangling the shared and unique impacts of correlated independent variables.
Commonality analysis is a statistical technique within multiple linear regression that decomposes a model's R 2 statistic (i.e., explained variance) by all independent variables on a dependent variable in a multiple linear regression model into commonality coefficients.

Variance

sample variancepopulation variancevariability
Various models have been created that allow for heteroscedasticity, i.e. the errors for different response variables may have different variances.
In linear regression analysis the corresponding formula is

General linear model

multivariate linear regressionmultivariate regressionGLM
This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.
The general linear model incorporates a number of different statistical models: ANOVA, ANCOVA, MANOVA, MANCOVA, ordinary linear regression, t-test and F-test.

Multilevel model

Hierarchical linear modelingMultilevel modelshierarchical linear model
Hierarchical linear models (or multilevel regression) organizes the data into a hierarchy of regressions, for example where A is regressed on B, and B is regressed on C.
These models can be seen as generalizations of linear models (in particular, linear regression), although they can also extend to non-linear models.