BLOOM Large Language Model (LLM)

From GM-RKB
(Redirected from BLOOM)
Jump to navigation Jump to search

A BLOOM Large Language Model (LLM) is a foundation transformer-based LLM.



References

2023

2023

  • (Wikipedia, 2023) ⇒ https://en.wikipedia.org/wiki/BLOOM_(language_model) Retrieved:2023-5-7.
    • BigScience Large Open-science Open-access Multilingual Language Model (BLOOM) is a transformer-based large language model. It was created by over 1000 AI researchers to provide a free large language model for everyone who wants to try. Trained on around 366 billion tokens over March through July 2022, it is considered an alternative to OpenAI's GPT-3 with its 176 billion parameters. BLOOM uses a decoder-only transformer model architecture modified from Megatron-LM GPT-2. The BLOOM project was started by a co-founder of Hugging Face. Six main groups of people were involved, including HuggingFace's BigScience team, the Microsoft DeepSpeed team, the NVIDIA Megatron-LM team, the IDRIS/GENCI team, the PyTorch team, and the volunteers in the BigScience Engineering workgroup.BLOOM was trained using data of 46 natural languages and 13 programming languages. In total, 1.6 TeraByte pre-processed text was converted into 350 billion unique tokens as BLOOM's training datasets.


2023

2023

List of large language models
Name Release dateTemplate:Efn Developer Number of parametersTemplate:Efn Corpus size LicenseTemplate:Efn Notes
BLOOM July 2022 Large collaboration led by Hugging Face 175 billion 350 billion tokens (1.6TB)[1] Responsible AI Essentially GPT-3 but trained on a multi-lingual corpus (30% English excluding programming languages)
  1. huggingface.co, bigscience/bloom · Hugging Face, n.d., Hugging Face, webpage