Toolformer LLM

A Toolformer LLM is a external-tools large language model t(through simple APIs in a self-supervised way).

• Context:
  • It can be trained to decide which APIs to call, when to call them, what arguments to pass, and how to integrate the results into future token predictions.
  • It can annotate a large corpus of data with API calls embedded in text, which it then uses for finetuning.
  • It can use multiple external tools with the potential to integrate more.
  • It can represent both input and outputs for these tools as text sequences.
  • It can learn to predict the appropriate tool for each task based on its training and task context.
  • ...
• Example(s):
  • ...
• Counter-Example(s):
  • Traditional language models without the capability to autonomously use external tools.
  • Models that require extensive human annotations for tool usage.
• See: Language Model, Natural Language Processing, API, Self-Supervised Learning.

---

---

References

2023

• (Schick et al., 2023) ⇒ Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. (2023). “Toolformer: Language Models Can Teach Themselves to Use Tools.” In: arXiv preprint arXiv:2302.04761. doi:10.48550/arXiv.2302.04761

---