2017 LearningtoGenerateReviewsandDis
- (Radford et al., 2017) ⇒ Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. (2017). “Learning to Generate Reviews and Discovering Sentiment.” doi:10.48550/arXiv.1704.01444
Subject Headings: Byte-Level RLM.
Notes
- It explores the properties of byte-level recurrent language models (RLMs). The authors found that when given sufficient capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, they found that a single unit in the model can be used to perform sentiment analysis. These representations, learned in an unsupervised manner, achieve state-of-the-art results on the binary subset of the Stanford Sentiment Treebank. They are also very data efficient, meaning that they can achieve good results with only a small amount of labeled data. The authors also demonstrate that the sentiment unit has a direct influence on the generative process of the model. Simply fixing its value to be positive or negative generates samples with the corresponding positive or negative sentiment.
Cited By
Quotes
Abstract
We explore the properties of byte-level recurrent language models. When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, we find a single unit which performs sentiment analysis. These representations, learned in an unsupervised manner, achieve state of the art on the binary subset of the Stanford Sentiment Treebank. They are also very data efficient. When using only a handful of labeled examples, our approach matches the performance of strong baselines trained on full datasets. We also demonstrate the sentiment unit has a direct influence on the generative process of the model. Simply fixing its value to be positive or negative generates samples with the corresponding positive or negative sentiment.
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2017 LearningtoGenerateReviewsandDis | Ilya Sutskever Rafal Jozefowicz Alec Radford | Learning to Generate Reviews and Discovering Sentiment | 10.48550/arXiv.1704.01444 | 2017 |