2019 SuperGLUEAStickierBenchmarkforG

• (Wang, Pruksachatkun et al., 2019) ⇒ Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. (2019). “SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.” In: Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). arXiv:1905.00537

Subject Headings: SuperGLUE Benchmark, GLUE Benchmark, Natural Language Understanding System; Natural Language Inference System, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019).

Notes

Computing Resource(s):

• Repository and other information available at https://super.gluebenchmark.com

Other Link(s):

• NIPS Proceedings Beta: http://papers.nips.cc/paper/8589-superglue-a-stickier-benchmark-for-general-purpose-language-understanding-systems
• DBLP: https://dblp.org/rec/html/conf/nips/WangPNSMHLB19
• AleX Wang Gitpage: https://w4ngatang.github.io/static/papers/superglue.pdf

Pre-print(s):

• ArXiv: https://arxiv.org/abs/1905.00537

Related Paper(s):

• (Wang et al., 2019) ⇒ Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. (2019). “GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.” In: Proceedings of the 7th International Conference on Learning Representations (ICLR 2019).

Cited By

• Google Scholar: ~70 Citations
• Semantic Scholar: ~ 113 Citations
• MS Academic: ~ 79 Citations.

Quotes

Abstract

In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at: https://super.gluebenchmark.com/

BibTeX

@inproceedings{2019_SuperGLUEAStickierBenchmarkforG,
  author    = {Alex Wang and
               Yada Pruksachatkun and
               Nikita Nangia and
               [[Amanpreet Singh]] and
               Julian Michael and
               Felix Hill and
               [[Omer Levy]] and
               Samuel R. Bowman},
  editor    = {Hanna M. Wallach and
               Hugo Larochelle and
               Alina Beygelzimer and
               Florence d'Alch{\'{e}}{-}Buc and
               Emily B. Fox and
               Roman Garnett},
  title     = {SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding
               Systems},
  booktitle = {Advances in Neural Information Processing Systems 32: Annual Conference
               on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14
               December 2019, Vancouver, BC, Canada},
  pages     = {3261--3275},
  year      = {2019},
  url       = {http://papers.nips.cc/paper/8589-superglue-a-stickier-benchmark-for-general-purpose-language-understanding-systems},
}

---

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2019 SuperGLUEAStickierBenchmarkforG	Omer Levy Alex Wang Amanpreet Singh Julian Michael Felix Hill Yada Pruksachatkun Nikita Nangia Samuel R. Bowman			SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems						2019