Lasso (statistics)

LASSOLasso methodLeast Absolute Shrinkage and Selection Operator\ell_1 penaltyAbsolute Shrinkage and Selection OperatorElastic netLASSO estimatorLasso regression
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.wikipedia
68 Related Articles

Feature selection

variable selectionfeaturesselecting
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

Robert Tibshirani

Robert J. TibshiraniTibshirani, RobertRob Tibshirani
It was originally introduced in geophysics literature in 1986, and later independently rediscovered and popularized in 1996 by Robert Tibshirani, who coined the term and provided further insights into the observed performance.
Lasso method, which proposed the use of L 1 penalization in regression and related problems, and Significance Analysis of Microarrays.

Regularization (mathematics)

regularizationregularizedregularize
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.
In the case of least squares, this problem is known as LASSO in statistics and basis pursuit in signal processing.

Linear regression

regression coefficientmultiple linear regressionregression
It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.
Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L 2 -norm penalty) and lasso (L 1 -norm penalty).

Lp space

L'' ''p'' spaceL ''p'' spacesL'' ''p'' spaces
where is the standard \ell^p norm, and 1_N is an N \times 1 vector of ones.
Techniques which use an L1 penalty, like LASSO, encourage solutions where many parameters are zero.

Basis pursuit denoising

The LASSO is closely related to basis pursuit denoising.
Basis pursuit denoising has potential applications in statistics (c.f. the LASSO method of regularization), image compression and compressed sensing.

Proportional hazards model

proportional hazards modelsCox proportional hazards modelCox model
Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.
In high-dimension, when number of covariates p is large compared to the sample size n, the LASSO method is one of the classical model-selection strategies.

Elastic net regularization

elastic netelastic net regularized regressionElastic nets
Elastic net regularization adds an additional ridge regression-like penalty which improves performance when the number of predictors is larger than the sample size, allows the method to select strongly correlated variables together, and improves overall prediction accuracy.
In statistics and, in particular, in the fitting of linear or logistic regression models, the elastic net is a regularized regression method that linearly combines the L 1 and L 2 penalties of the lasso and ridge methods.

Least-angle regression

LARSleast angle regression
Lasso regularized models can be fit using a variety of techniques including subgradient methods, least-angle regression (LARS), and proximal gradient methods.

Coefficient of determination

R-squaredR'' 2 R 2
Assuming that \beta_{0}=0, the solution path can then be defined in terms of the famous accuracy measure called R^2:
As Hoornweg (2018) shows, several shrinkage estimators – such as Bayesian linear regression, ridge regression, and the (adaptive) lasso – make use of this decomposition of R^2 when they gradually shrink parameters from the unrestricted OLS solutions towards the hypothesized values.

Cross-validation (statistics)

cross-validationcross validationLeave-one-out cross-validation
Determining the optimal value for the regularization parameter is an important part of ensuring that the model performs well; it is typically chosen using cross-validation.
Click on the lasso for an example.

Laplace distribution

Laplacedouble exponentialLaplace distributed
Just as ridge regression can be interpreted as linear regression for which the coefficients have been assigned normal prior distributions, lasso can be interpreted as linear regression for which the coefficients have Laplace prior distributions.

Tikhonov regularization

ridge regressionregularizeda squared regularizing function
Lasso was originally formulated for least squares models and this simple case reveals a substantial amount about the behavior of the estimator, including its relationship to ridge regression and best subset selection and the connections between lasso coefficient estimates and so-called soft thresholding.

Statistics

statisticalstatistical analysisstatistician
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

Machine learning

machine-learninglearningstatistical learning
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

Regression analysis

regressionmultiple regressionregression model
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

Statistical model

modelprobabilistic modelstatistical modeling
In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.

Least squares

least-squaresmethod of least squaresleast squares method
Lasso was originally formulated for least squares models and this simple case reveals a substantial amount about the behavior of the estimator, including its relationship to ridge regression and best subset selection and the connections between lasso coefficient estimates and so-called soft thresholding.

Collinearity

collinearcollinear pointscolinear
It also reveals that (like standard linear regression) the coefficient estimates do not need to be unique if covariates are collinear.

Generalized linear model

generalized linear modelslink functiongeneralised linear model
Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

Generalized estimating equation

generalized estimating equationsweighted estimating equations
Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

M-estimator

M-estimationM-estimatorsestimation
Though originally defined for least squares, lasso regularization is easily extended to a wide variety of statistical models including generalized linear models, generalized estimating equations, proportional hazards models, and M-estimators, in a straightforward fashion.

Geometry

geometricgeometricalgeometries
Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.

Bayesian statistics

BayesianBayesian methodsBayesian analysis
Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.

Convex analysis

convex-analyticconvexity
Lasso’s ability to perform subset selection relies on the form of the constraint and has a variety of interpretations including in terms of geometry, Bayesian statistics, and convex analysis.