2014 ClassDistributionRegularizedCon

• (Xie et al., 2014) ⇒ Sihong Xie, Jing Gao, Wei Fan, Deepak Turaga, and Philip S. Yu. (2014). “Class-distribution Regularized Consensus Maximization for Alleviating Overfitting in Model Combination.” In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2014) Journal. ISBN:978-1-4503-2956-9 doi:10.1145/2623330.2623676

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222014%22+Class-distribution+Regularized+Consensus+Maximization+for+Alleviating+Overfitting+in+Model+Combination
• http://dl.acm.org/citation.cfm?id=2623330.2623676&preflayout=flat#citedby

Quotes

Author Keywords

• Data mining; ensemble; generalization error; large margin

Abstract

In data mining applications such as crowdsourcing and privacy-preserving data mining, one may wish to obtain consolidated predictions out of multiple models without access to features of the data. Besides, multiple models usually carry complementary predictive information, model combination can potentially provide more robust and accurate predictions by correcting independent errors from individual models. Various methods have been proposed to combine predictions such that the final predictions are maximally agreed upon by multiple base models. Though this maximum consensus principle has been shown to be successful, simply maximizing consensus can lead to less discriminative predictions and overfit the inevitable noise due to imperfect base models. We argue that proper regularization for model combination approaches is needed to alleviate such overfitting effect. Specifically, we analyze the hypothesis spaces of several model combination methods and identify the trade-off between model consensus and generalization ability. We propose a novel model called Regularized Consensus Maximization (RCM), which is formulated as an optimization problem to combine the maximum consensus and large margin principles. We theoretically show that RCM has a smaller upper bound on generalization error compared to the version without regularization. Experiments show that the proposed algorithm outperforms a wide spectrum of state-of-the-art model combination methods on 11 tasks.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2014 ClassDistributionRegularizedCon	Jing Gao Wei Fan Philip S. Yu Deepak Turaga Sihong Xie			Class-distribution Regularized Consensus Maximization for Alleviating Overfitting in Model Combination				10.1145/2623330.2623676		2014