AudioCraft is a code base from Meta AI Research for generative audio. It brings together models that produce music and sound effects from text and that compress raw audio, so researchers and developers can build on one consistent stack instead of stitching together separate projects.
The stack has three parts. MusicGen generates music from a text description and can also be conditioned on a melody, producing mono or stereo output. AudioGen generates environmental sounds and sound effects from text. EnCodec is the neural audio codec that converts a waveform into streams of discrete tokens and back again. MusicGen and AudioGen each use a single autoregressive language model that works over those compressed tokens, with a token interleaving pattern that avoids cascading several models together. A pretrained text encoder handles the conditioning for text to audio generation.
It is aimed at machine learning researchers, audio engineers and developers who want to train, run or study audio generation models. The site links to the code, model cards, technical details, research papers and a Meta AI blog post, and offers audio samples for both music and sound generation.
The site publishes no pricing and no hosted product. AudioCraft is distributed as code you run yourself, so the hardware and compute you use are your own responsibility. It is a research release rather than a consumer app, and it is not the same product as the separate music service that uses the name Audiogen.
