Meta ups the ante with AudioCraft, a new generative AI tool for audio and music

Image source: Generated by Unbounded AI

Source: Wall Street News

Author: Cao Zexi

On Wednesday, August 2, Meta launched a new generative AI tool for audio and music called AudioCraft, which helps users create music and audio based on text prompts.

This AI tool combines the three models or technologies of AudioGen, EnCodec and MusicGen into one, and can generate high-quality, almost human-created audio and music from text content.

Among them, MusicGen has received Meta-owned and specially authorized music training, and can generate music from text prompts; AudioGen has received public sound effect training, and can generate audio from text prompts, such as simulating dog barking or footsteps; coupled with EnCodec codec With an improved version of the player, users can generate higher quality music more efficiently.

According to Meta, the AudioCraft line of models produces high-quality audio with long-term consistency and is easy to use:

With AudioCraft, we simplify the overall design of audio generative models compared to previous work in the field - giving people a complete way to use existing models that Meta has developed over the past few years, while also enabling them to push the limits and develop your own models.

Meta points out that AudioCraft is suitable for compression and generation of music, sound, and audio files. Because it's so easy to build and reuse, someone who wants to build a better sound generator, compression algorithm, or music generator can do it all in the same codebase and build on what others have done.

Meta name:

Having a solid open source foundation will foster innovation and complement the way we make and listen to audio and music in the future. With more control, we think MusicGen can become a new kind of instrument - just like synthesizers did when they first came out.

All Facebook users can install AudioCraft, and Meta specifically invites researchers and music professionals to use the tool:

We see the AudioCraft collection of models as an inspiring tool for musicians and sound designers, helping people quickly brainstorm and iterate on their compositions in new ways. We can't wait to see what people create with Audiocraft.

Meta launched its first version of EnCodec in October 2022 as an AI tool for compressing and decompressing audio files without loss of sound quality, allowing users to quickly and easily share audio documents. Its purpose is to improve the quality of all audio files, not just music files. At the time, it was specifically aimed at improving the quality of voice calls and voice messages, especially in adverse situations such as poor network connections. The model has since evolved and is now introduced with AudioGen and SoundGen as a tool to help synthesized sounds and music appear more realistic when actually played.

While some artists have embraced AI-generated tools for more creativity, others have been critical of copyright infringement.