# Linear regression

**regression coefficientmultiple linear regressionregressionlinear regression modellinearleast squares regressionerror variablelinear weightsregression linemultiple regression**

In statistics, linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more explanatory variables (or independent variables).wikipedia

463 Related Articles

### Simple linear regression

**simple regressioni.e. regression linelinear least squares regression with an intercept term and a single explanator**

The case of one explanatory variable is called simple linear regression.

In statistics, simple linear regression is a linear regression model with a single explanatory variable.

### Regression analysis

**regressionmultiple regressionregression model**

Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of the response given the values of the predictors, rather than on the joint probability distribution of all of these variables, which is the domain of multivariate analysis.

The most common form of regression analysis is linear regression, in which a researcher finds the line (or a more complex linear function) that most closely fits the data according to a specific mathematical criterion.

### Linear predictor function

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data.

This sort of function usually comes in linear regression, where the coefficients are called regression coefficients.

### Linear model

**linear modelslinearlinear process**

Such models are called linear models.

The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model.

### Tikhonov regularization

**ridge regressionregularizeda squared regularizing function**

Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).

Also known as ridge regression, it is particularly useful to mitigate the problem of multicollinearity in linear regression, which commonly occurs in models with large numbers of parameters.

### Ordinary least squares

**OLSleast squaresOrdinary least squares regression**

ordinary least squares): Multivariate analogues of ordinary least squares (OLS) and generalized least squares (GLS) have been developed.

In statistics, ordinary least squares (OLS) is a type of linear least squares method for estimating the unknown parameters in a linear regression model.

### Generalized linear model

**generalized linear modelslink functiongeneralised linear model**

In statistics, the generalized linear model (GLM) is a flexible generalization of ordinary linear regression that allows for response variables that have error distribution models other than a normal distribution.

### Lasso (statistics)

**LASSOLasso methodLeast Absolute Shrinkage and Selection Operator**

Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).

It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.

### Segmented regression

**Linear segmented regressionPiecewise regressionsegmented regression analysis**

Segmented linear regression is segmented regression whereby the relations in the intervals are obtained by linear regression.

### Bayesian linear regression

**Bayesian regressionlinear Bayesian regression**

In statistics, Bayesian linear regression is an approach to linear regression in which the statistical analysis is undertaken within the context of Bayesian inference.

### Polynomial regression

**cubic regressionPolynomial fittingregression**

For this reason, polynomial regression is considered to be a special case of multiple linear regression.

### Weighted least squares

**Batch Least Squareslinear least squaresweighted**

For example, weighted least squares is a method for estimating linear regression models when the response variables may have different error variances, possibly with correlated errors.

Weighted least squares (WLS), also known as weighted linear regression, is a generalization of ordinary least squares and linear regression in which the errors covariance matrix is allowed to be different from an identity matrix.

### Heteroscedasticity-consistent standard errors

**Huber–White standard errorEicker–WhiteHuber–White standard errors**

(See also Weighted linear least squares, and Generalized least squares.) Heteroscedasticity-consistent standard errors is an improved method for use with uncorrelated but potentially heteroscedastic errors.

The topic of heteroscedasticity-consistent (HC) standard errors arises in statistics and econometrics in the context of linear regression and time series analysis.

### Loss function

**objective functioncost functionrisk function**

Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).

Many common statistics, including t-tests, regression models, design of experiments, and much else, use least squares methods applied using linear regression theory, which is based on the quadratric loss function.

### Generalized least squares

**feasible generalized least squaresgeneralizedgeneralized (correlated)**

(See also Weighted linear least squares, and Generalized least squares.) Heteroscedasticity-consistent standard errors is an improved method for use with uncorrelated but potentially heteroscedastic errors. Multivariate analogues of ordinary least squares (OLS) and generalized least squares (GLS) have been developed.

In statistics, generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in a regression model.

### Multicollinearity

**collinearitymulticollinearperfect multicollinearity**

In this situation the coefficient estimates of the multiple regression may change erratically in response to small changes in the model or the data.

### Statistics

**statisticalstatistical analysisstatistician**

In statistics, linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more explanatory variables (or independent variables).

Least squares applied to linear regression is called ordinary least squares method and least squares applied to nonlinear regression is called non-linear least squares.

### Design matrix

**data matrixdesign matricesdata matrices**

The theory relating to such models makes substantial use of matrix manipulations involving the design matrix: see for example linear regression.

### Partial least squares regression

**partial least squarespartial least-squarespartial least squares ('''PLS''')**

Partial least squares regression (PLS regression) is a statistical method that bears some relation to principal components regression; instead of finding hyperplanes of maximum variance between the response and independent variables, it finds a linear regression model by projecting the predicted variables and the observable variables to a new space.

### Median

**averagesample medianmedian-unbiased estimator**

Most commonly, the conditional mean of the response given the values of the explanatory variables (or predictors) is assumed to be an affine function of those values; less commonly, the conditional median or some other quantile is used.

The Theil–Sen estimator is a method for robust linear regression based on finding medians of slopes.

### Dummy variable (statistics)

**dummy variableindicator variabledummy variables**

Care must be taken when interpreting regression results, as some of the regressors may not allow for marginal changes (such as dummy variables, or the intercept term), while others cannot be held fixed (recall the example from the introduction: it would be impossible to "hold t i fixed" and at the same time change the value of t i 2 ).

A dummy independent variable (also called a dummy explanatory variable) which for some observation has a value of 0 will cause that variable's coefficient to have no role in influencing the dependent variable, while when the dummy takes on a value 1 its coefficient acts to alter the intercept.

### Commonality analysis

Commonality analysis may be helpful in disentangling the shared and unique impacts of correlated independent variables.

Commonality analysis is a statistical technique within multiple linear regression that decomposes a model's R 2 statistic (i.e., explained variance) by all independent variables on a dependent variable in a multiple linear regression model into commonality coefficients.

### Variance

**sample variancepopulation variancevariability**

Various models have been created that allow for heteroscedasticity, i.e. the errors for different response variables may have different variances.

In linear regression analysis the corresponding formula is

### General linear model

**multivariate linear regressionmultivariate regressionGLM**

This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

The general linear model incorporates a number of different statistical models: ANOVA, ANCOVA, MANOVA, MANCOVA, ordinary linear regression, t-test and F-test.

### Multilevel model

**Hierarchical linear modelingMultilevel modelshierarchical linear model**

Hierarchical linear models (or multilevel regression) organizes the data into a hierarchy of regressions, for example where A is regressed on B, and B is regressed on C.

These models can be seen as generalizations of linear models (in particular, linear regression), although they can also extend to non-linear models.