2009 ConstrainedOptimizationforValid

• (Chen et al., 2009) ⇒ Minmin Chen, Yixin Chen, Michael R. Brent, and Aaron E. Tenney. (2009). “Constrained Optimization for Validation-guided Conditional Random Field Learning.” In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2009). doi:10.1145/1557019.1557046

Subject Headings:

Notes

• Categories and Subject Descriptors: I.2.6 Artificial Intelligence: Learning — Parameter learning; J.3 Life and Medical Sciences: Biology and genetics; J.1 Administrative Data Processing: Financial.
• General Terms: Algorithm, Experimentation

Cited By

• http://scholar.google.com/scholar?q=%22Constrained+optimization+for+validation-guided+conditional+random+field+learning%22+2009
• http://portal.acm.org/citation.cfm?doid=1557019.1557046&preflayout=flat#citedby

Quotes

Author Keywords

Conditional Random Fields, Constrained Optimization, Cross Validation, Extended Saddle Points

Abstract

Conditional random fields(CRFs) are a class of undirected graphical models which have been widely used for classifying and labeling sequence data. The training of CRFs is typically formulated as an unconstrained optimization problem that maximizes the conditional likelihood. However, maximum likelihood training is prone to overfitting. To address this issue, we propose a novel constrained nonlinear optimization formulation in which the prediction accuracy of cross-validation sets are included as constraints. Instead of requiring multiple passes of training, the constrained formulation allows the cross-validation be handled in one pass of constrained optimization.
The new formulation is discontinuous, and classical Lagrangian based constraint handling methods are not applicable. A new constrained optimization algorithm based on the recently proposed extended saddle point theory is developed to learn the constrained CRF model. Experimental results on gene and stock-price prediction tasks show that the constrained formulation is able to significantly improve the generalization ability of CRF training.

---

---

References

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2009 ConstrainedOptimizationforValid	Minmin Chen Yixin Chen Michael R. Brent Aaron E. Tenney			Constrained Optimization for Validation-guided Conditional Random Field Learning		KDD-2009 Proceedings		10.1145/1557019.1557046		2009