# Lasso (statistics)

**LASSOLasso methodLeast Absolute Shrinkage and Selection Operator\ell_1 penaltyAbsolute Shrinkage and Selection OperatorElastic netLASSO estimatorLasso regression**

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.wikipedia

68 Related Articles

### Feature selection

**variable selectionfeaturesselecting**

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

### Robert Tibshirani

**Robert J. TibshiraniTibshirani, RobertRob Tibshirani**

It was originally introduced in geophysics literature in 1986, and later independently rediscovered and popularized in 1996 by Robert Tibshirani, who coined the term and provided further insights into the observed performance.

Lasso method, which proposed the use of L 1 penalization in regression and related problems, and Significance Analysis of Microarrays.

### Regularization (mathematics)

**regularizationregularizedregularize**

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

In the case of least squares, this problem is known as LASSO in statistics and basis pursuit in signal processing.

### Linear regression

**regression coefficientmultiple linear regressionregression**

It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.

Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).

### Lp space

**L'' ''p'' spaceL ''p'' spacesL'' ''p'' spaces**

where is the standard \ell^p norm, and 1_N is an N \times 1 vector of ones.

Techniques which use an L1 penalty, like LASSO, encourage solutions where many parameters are zero.

### Basis pursuit denoising

The LASSO is closely related to basis pursuit denoising.

Basis pursuit denoising has potential applications in statistics (c.f. the LASSO method of regularization), image compression and compressed sensing.

### Proportional hazards model

**proportional hazards modelsCox proportional hazards modelCox model**

Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

In high-dimension, when number of covariates p is large compared to the sample size n, the LASSO method is one of the classical model-selection strategies.

### Elastic net regularization

**elastic netelastic net regularized regressionElastic nets**

Elastic net regularization adds an additional ridge regression-like penalty which improves performance when the number of predictors is larger than the sample size, allows the method to select strongly correlated variables together, and improves overall prediction accuracy.

In statistics and, in particular, in the fitting of linear or logistic regression models, the elastic net is a regularized regression method that linearly combines the L 1 and L 2 penalties of the lasso and ridge methods.

### Least-angle regression

**LARSleast angle regression**

Lasso regularized models can be fit using a variety of techniques including subgradient methods, least-angle regression (LARS), and proximal gradient methods.

### Coefficient of determination

**R-squaredR'' 2 R 2**

Assuming that \beta_{0}=0, the solution path can then be defined in terms of the famous accuracy measure called R^2:

As Hoornweg (2018) shows, several shrinkage estimators – such as Bayesian linear regression, ridge regression, and the (adaptive) lasso – make use of this decomposition of R^2 when they gradually shrink parameters from the unrestricted OLS solutions towards the hypothesized values.

### Cross-validation (statistics)

**cross-validationcross validationLeave-one-out cross-validation**

Determining the optimal value for the regularization parameter is an important part of ensuring that the model performs well; it is typically chosen using cross-validation.

Click on the lasso for an example.

### Laplace distribution

**Laplacedouble exponentialLaplace distributed**

Just as ridge regression can be interpreted as linear regression for which the coefficients have been assigned normal prior distributions, lasso can be interpreted as linear regression for which the coefficients have Laplace prior distributions.

### Tikhonov regularization

**ridge regressionregularizeda squared regularizing function**

Lasso was originally formulated for least squares models and this simple case reveals a substantial amount about the behavior of the estimator, including its relationship to ridge regression and best subset selection and the connections between lasso coefficient estimates and so-called soft thresholding.

### Statistics

**statisticalstatistical analysisstatistician**

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

### Machine learning

**machine-learninglearningstatistical learning**

### Regression analysis

**regressionmultiple regressionregression model**

### Statistical model

**modelprobabilistic modelstatistical modeling**

### Least squares

**least-squaresmethod of least squaresleast squares method**

Lasso was originally formulated for least squares models and this simple case reveals a substantial amount about the behavior of the estimator, including its relationship to ridge regression and best subset selection and the connections between lasso coefficient estimates and so-called soft thresholding.

### Collinearity

**collinearcollinear pointscolinear**

It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.

### Generalized linear model

**generalized linear modelslink functiongeneralised linear model**

Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

### Generalized estimating equation

**generalized estimating equationsweighted estimating equations**

Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

### M-estimator

**M-estimationM-estimatorsestimation**

### Geometry

**geometricgeometricalgeometries**

Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.

### Bayesian statistics

**BayesianBayesian methodsBayesian analysis**

Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.

### Convex analysis

**convex-analyticconvexity**

Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.