Bloom

BLOOM is an open multilingual large language model created by the BigScience Workshop, a collaborative open science project in which researchers pooled time and resources. Its architecture is an autoregressive transformer for next token prediction, similar in design to GPT3, and it was trained on text in 46 natural languages and 13 programming languages, so it can generate prose in many languages as well as code. The listed page is the model's documentation in the Hugging Face Transformers library, which suits researchers and developers who want to load, fine tune or study an open model rather than call a hosted assistant. BLOOM comes in a family of sizes trained on the same dataset, from about 560 million parameters through 1.1, 1.7, 3 and 7.1 billion up to the full 176 billion parameter model. Transformers provides ready classes for causal language modeling, question answering, token classification and sequence classification, and the documentation links example scripts, notebooks and task guides for text generation. The weights are free to download from Hugging Face under the BigScience RAIL License 1.0, a responsible AI license that permits use, modification and redistribution subject to a list of use restrictions. There is no charge for the model itself; the cost of running it depends on the user's own hardware or on whichever paid hosting service they choose, and the largest version needs substantial GPU memory. BLOOM was contributed to Transformers in June 2022 and its paper appeared that November. It is best understood as an important open research model from that period rather than a current coding assistant, and newer open models generally outperform it on code.