- (Yang et al., 2014) ⇒ Shuang-Hong Yang, Alek Kolcz, Andy Schlaikjer, and Pankaj Gupta. (2014). “Large-scale High-precision Topic Modeling on Twitter.” In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2014) Journal. ISBN:978-1-4503-2956-9 doi:10.1145/2623330.2623336
- Concept learning; data mining; large-scale machine learning; social media; text classification; topic modeling
We are interested in organizing a continuous stream of sparse and noisy texts, known as “tweets ", in real time into an ontology of hundreds of topics with measurable and stringently high precision. This inference is performed over a full-scale stream of Twitter data, whose statistical distribution evolves rapidly over time. The implementation in an industrial setting with the potential of affecting and being visible to real users made it necessary to overcome a host of practical challenges. We present a spectrum of topic modeling techniques that contribute to a deployed system. These include non-topical tweet detection, automatic labeled data acquisition, evaluation with human computation, diagnostic and corrective learning and, most importantly, high-precision topic inference. The latter represents a novel two-stage training algorithm for tweet text classification and a close-loop inference mechanism for combining texts with additional sources of information. The resulting system achieves 93% precision at substantial overall coverage.
|2014 LargeScaleHighPrecisionTopicMod||Shuang-Hong Yang|
|Large-scale High-precision Topic Modeling on Twitter||10.1145/2623330.2623336||2014|
|Author||Shuang-Hong Yang +, Alek Kolcz +, Andy Schlaikjer + and Pankaj Gupta +|
|proceedings||Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining +|
|title||Large-scale High-precision Topic Modeling on Twitter +|