2006 CombinMarkovRandomFields
- (Bekkerman et al., 2006) ⇒ Ron Bekkerman, Mehran Sahami and Erik Learned-Miller. (2006). “Combinatorial Markov Random Fields.” In: Proceedings of the 17th European Conference on Machine Learning (ECML 2006) doi:10.1007/11871842_8
Subject Headings:
Notes
Cited By
Quotes
Abstract
A combinatorial random variable is a discrete random variable defined over a combinatorial set (e.g., a power set of a given set). In this paper we introduce combinatorial Markov random fields (Comrafs), which are Markov random fields where some of the nodes are combinatorial random variables. We argue that Comrafs are powerful models for unsupervised and semi-supervised learning. We put Comrafs in perspective by showing their relationship with several existing models. Since it can be problematic to apply existing inference techniques for graphical models to Comrafs, we design two simple and efficient inference algorithms specific for Comrafs, which are based on combinatorial optimization. We show that even such simple algorithms consistently and significantly outperform Latent Dirichlet Allocation (LDA) on a document clustering task. We then present Comraf models for semi-supervised clustering and transfer learning that demonstrate superior results in comparison to an existing semi-supervised scheme (constrained optimization).
References
- McGurk, H., MacDonald, J.: Hearing lips and seeing voices. Nature 264(5588) (1976) 746{748
- de Sa, V.: Unsupervised Classi¯cation Learning from Cross-Modal Environmental Structure. PhD thesis, University of Rochester (1994)
- Friedman, N., Mosenzon, O., Slonim, N., Tishby, N.: Multivariate information bottleneck. In: Proceedings of UAI-17. (2001)
- Bickel, S., Sche®er, T.: Multi-view clustering. In: Proceedings of ICDM-4. (2004)
- Bekkerman, R., El-Yaniv, R., McCallum, A.: Multi-way distributional clustering via pairwise interactions. In: Proceedings of ICML-22. (2005) 41{48
- Li, S.: Markov random ¯eld modeling in computer vision. Springer Verlag (1995)
- Besag, J.: Spatial interaction and statistical analysis of lattice systems. Journal of the Royal Statistical Society 36(2) (1974) 192{236
- Blei, D., Ng, A., Jordan, M.: Latent Dirichlet allocation. JMLR 3 (2003) 993{1022
- Tishby, N., Pereira, F., Bialek, W.: The information bottleneck method (1999) Invited paper to the 37th Annual Allerton Conference.
- Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. In: Proceedings of SIGKDD-9. (2003) 89{98
- Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 48(3) (1986)
- Wagsta®, K., Cardie, C.: Clustering with instance-level constraints. In: Proceedings of ICML-17. (2000)
- McCallum, A., Corrada-Emmanuel, A., Wang, X.: Topic and role discovery in social networks. In: Proceedings of IJCAI-19. (2005) 786{791
- Bekkerman, R., Sahami, M.: Semi-supervised clustering using combinatorial MRFs. In: Proceedings of ICML-23 Workshop on Learning in Structured Out-put Spaces. (2006)
- Bekkerman, R., El-Yaniv, R., Tishby, N., Winter, Y.: Distributional word clusters vs. words for text categorization. JMLR 3 (2003) 1183{1208,
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2006 CombinMarkovRandomFields | Mehran Sahami Erik Learned-Miller | Combinatorial Markov Random Fields | http://www.cs.umass.edu/~ronb/papers/ecml06.pdf |