sklearn.linear model.MultiTaskLasso

A sklearn.linear model.MultiTaskLasso is an Multi-Task Lasso System within sklearn.linear_model class.

• Context:
  • Usage:

1) Import MultiTaskLasso model from scikit-learn : from sklearn.linear_model import MultiTaskLasso

2) Create design matrix X and response vector Y

3) Create MultiTaskLasso object: model= MultiTaskLasso([alpha=1.0, fit_intercept=True, normalize=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, random_state=None, selection=’cyclic])

4) Choose method(s):

• fit(X, y), fits MultiTaskElasticNet model with coordinate descent.
• get_params([deep]), gets parameters for this estimator.
• path(X, y[, l1_ratio, eps, n_alphas, ...]), computes elastic net path with coordinate descent.
• predict(X), predicts using the linear model.
• score(X, y[, sample_weight]), returns the coefficient of determination R^2 of the prediction.
• set_params(**params),, sets the parameters of this estimator.

• Example(s):
  • Joint feature selection with multi-task Lasso
• Counter-Example(s):
  • sklearn.linear model.LinearRegression
  • sklearn.linear_model.Lasso
  • sklearn.linear_model.Ridge.
  • sklearn.linear_model.ARDRegression.
  • sklearn.linear_model.ElasticNet.
  • sklearn.linear_model.HuberRegressor.
  • sklearn.linear_model.Lars.
• See: Regression System, L1 Norm, L2 Norm Cross-Validation Task, Ridge Regression Task, Bayesian Analysis.

---

---

References

2017A

• (scikit-learn.org, 2017) ⇒ http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.MultiTaskLasso.html
  • QUOTE: class sklearn.linear_model.MultiTaskLasso(alpha=1.0, fit_intercept=True, normalize=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, random_state=None, selection=’cyclic’)

Multi-task Lasso model trained with L1/L2 mixed-norm as regularizer

The optimization objective for Lasso is:

(1 / (2 * n_samples)) * ||Y - XW||^2_Fro + alpha * ||W||_21

Where:

||W||_21 = \sum_i \sqrt{\sum_j w_{ij}^2}

i.e. the sum of norm of each row.

2017B

• (scikit-learn.org, 2017) ⇒ "1.1.4. Multi-task Lasso" http://scikit-learn.org/stable/modules/linear_model.html#multi-task-lasso Retrieved: 2017-11-05
  • QUOTE: The MultiTaskLasso is a linear model that estimates sparse coefficients for multiple regression problems jointly: y is a 2D array, of shape (n_samples, n_tasks). The constraint is that the selected features are the same for all the regression problems, also called tasks.

The following figure compares the location of the non-zeros in W obtained with a simple Lasso or a MultiTaskLasso. The Lasso estimates yields scattered non-zeros while the non-zeros of the MultiTaskLasso are full columns.

(...)

Mathematically, it consists of a linear model trained with a mixed [math]\displaystyle{ \ell_1 }[/math] [math]\displaystyle{ \ell_2 }[/math] prior as regularizer. The objective function to minimize is:

[math]\displaystyle{ \underset{w}{min\,} { \frac{1}{2n_{samples}} ||X W - Y||_{Fro} ^ 2 + \alpha ||W||_{21}} }[/math]

where Fro indicates the Frobenius norm:

[math]\displaystyle{ ||A||_{Fro} = \sqrt{\sum_{ij} a_{ij}^2} }[/math]

and \ell_1 \ell_2 reads:

[math]\displaystyle{ ||A||_{2 1} = \sum_i \sqrt{\sum_j a_{ij}^2} }[/math]

The implementation in the class MultiTaskLasso uses coordinate descent as the algorithm to fit the coefficients.

---