- (Cai et al., 2013) ⇒ Xiao Cai, Chris Ding, Feiping Nie, and Heng Huang. (2013). “On the Equivalent of Low-rank Linear Regressions and Linear Discriminant Analysis based Regressions.” In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ISBN:978-1-4503-2174-7 doi:10.1145/2487575.2487701
- Learning; linear discriminant analysis; low-rank regression; low-rank ridge regression; sparse low-rank regression
The low-rank regression model has been studied and applied to capture the underlying classes / tasks correlation patterns, such that the regression / classification results can be enhanced. In this paper, we will prove that the low-rank regression model is equivalent to doing linear regression in the linear discriminant analysis (LDA) subspace. Our new theory reveals the learning mechanism of low-rank regression, and shows that the low-rank structures exacted from classes / tasks are connected to the LDA projection results. Thus, the low-rank regression efficiently works for the high-dimensional data.
Moreover, we will propose new discriminant low-rank ridge regression and sparse low-rank regression methods. Both of them are equivalent to doing regularized regression in the regularized LDA subspace. These new regularized objectives provide better data mining results than existing low-rank regression in both theoretical and empirical validations. We evaluate our discriminant low-rank regression methods by six benchmark datasets. In all empirical results, our discriminant low-rank models consistently show better results than the corresponding full-rank methods.
|2013 OntheEquivalentofLowRankLinearR||Xiao Cai|
|On the Equivalent of Low-rank Linear Regressions and Linear Discriminant Analysis based Regressions||10.1145/2487575.2487701||2013|
|Author||Xiao Cai +, Chris Ding +, Feiping Nie + and Heng Huang +|
|proceedings||Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining +|
|title||On the Equivalent of Low-rank Linear Regressions and Linear Discriminant Analysis based Regressions +|