2010 JointCovariateSelectionandJoint

(Obozinski et al., 2010) ⇒ Guillaume Obozinski, Ben Taskar, and Michael I. Jordan. (2010). “Joint Covariate Selection and Joint Subspace Selection for Multiple Classification Problems.” In: Statistics and Computing Journal, 20(2). doi:10.1007/s11222-008-9111-x

Subject Headings:

Notes

Cited By

Quotes

Author Keywords

Block norm; Group Lasso; Lasso; Multitask learning; Random projections; Regularization path; Subspace selection; Supervised dimensionality reduction; Trace norm; Variable selection

Abstract

We address the problem of recovering a common set of covariates that are relevant simultaneously to several classification problems. By penalizing the sum of [math]\displaystyle{ l_2 }[/math] norms of the blocks of coefficients associated with each covariate across different classification problems, similar sparsity patterns in all models are encouraged. To take computational advantage of the sparsity of solutions at high regularization levels, we propose a blockwise path-following scheme that approximately traces the regularization path. As the regularization coefficient decreases, the algorithm maintains and updates concurrently a growing set of covariates that are simultaneously active for all problems. We also show how to use random projections to extend this approach to the problem of joint subspace selection, where multiple predictors are found in a common low-dimensional subspace. We present theoretical results showing that this random projection approach converges to the solution yielded by trace-norm regularization. Finally, we present a variety of experimental results exploring joint covariate selection and joint subspace selection, comparing the path-following approach to competing algorithms in terms of prediction accuracy and running time.

References

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2010 JointCovariateSelectionandJoint	Guillaume Obozinski			Joint Covariate Selection and Joint Subspace Selection for Multiple Classification Problems				10.1007/s11222-008-9111-x