Software Code Large Language Model (LLM)

A Software Code Large Language Model (LLM) is an LLM for software code generation.

• <B<>Example(s):
  • Codex.
  • ...
• See: CoderEval Benchmark, LeetCode Benchmark.

---

---

References

2023

• (Shen et al., 2023) ⇒ Bo Shen, Jiaxin Zhang, Taihong Chen, Daoguang Zan, Bing Geng, An Fu, Muhan Zeng, Ailun Yu, Jichuan Ji, Jingyang Zhao, Yuenan Guo, and Qianxiang Wang. (2023). “PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback.” doi:10.48550/arXiv.2307.14936
  • ABSTRACT: ... Large Language Models for Code (Code LLM) are flourishing. New and powerful models are released on a weekly basis, demonstrating remarkable performance on the code generation task. Various approaches have been proposed to boost the code generation performance of pre-trained Code LLMs, such as supervised fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework, which can effectively and efficiently boost pre-trained large language models for code generation. Under this framework, we present PanGu-Coder2, which achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through an extensive evaluation on CoderEval and LeetCode benchmarks, we show that PanGu-Coder2 consistently outperforms all previous Code LLMs.

---