30 years of .mp3

It all started with an idea. Prof. Dieter Seitzer, who later became the first director of Fraunhofer IIS, wanted to transmit music over telephone lines.

“In the mid-1990s, our goal became to play back mp3 files on small, portable devices,”  recalls Bernhard Grill, a core member of the mp3 development team alongside Karlheinz Brandenburg, Harald Popp, Jürgen Herre, and others. What seems obvious today was considered far-fetched back then. Still, the team pressed on – and succeeded. For early internet users, the benefits of mp3 were immediately obvious, and music in this format quickly spread. The Fraunhofer IIS team partnered with a German company to develop the first mp3 decoder chip, enabling the creation of the first mp3 player without moving parts right in Erlangen. With that, all the pieces for a global success story were in place, and the vision of a music library in your pocket became reality. 

A Clever Trick Optimizes Storage

The mp3 format cleverly takes advantage of the characteristics of human hearing. Audio signals always contain components that are inaudible to us. These irrelevant parts are stored with less precision – using fewer bits. The resulting files are up to ten times smaller, with virtually no audible difference from the original. This groundbreaking combination paved the way for mp3’s global triumph.

The format was standardized in the 1990s by the ISO-MPEG committee. The first standard, MPEG-1, included three layers of audio coding: Layer 1, 2, and 3. The name “mp3” is short for “MPEG-1 Audio Layer-3” and was chosen on July 14, 1995, through an internal email poll at Fraunhofer IIS.

A Growth Engine for Fraunhofer IIS

The development of the codec proved to be a stroke of luck for Fraunhofer IIS. The technology became a global success and resulted in significant profits for the entire Fraunhofer Society – generating hundreds of millions of euros in licensing revenue over the patent’s lifetime. This allowed Fraunhofer IIS to expand its team in Erlangen and grow into the largest institute of the Fraunhofer-Gesellschaft. And the work on audio codecs didn’t stop with mp3.

Today, the institute remains globally renowned for its audio technologies. With over 400 employees in the Audio and Media Technologies division, it is the world’s largest and most successful center for audio technology development.

mp3: The Foundation for Present Success

The development of mp3 laid the groundwork for four generations of audio codecs that are indispensable today. Apple’s iPod and the iTunes Music Store helped propel mp3’s successor, MPEG Advanced Audio Coding (AAC), into the mainstream. Subsequent generations brought further refinements, reducing bitrates even more. Whether in streaming, digital radio, TV, or video calls like Apple’s FaceTime – AAC codecs are everywhere.

“We’re constantly working to continue this success story,” says Bernhard Grill, now Director of Audio and Media Technologies division and one of the institute’s managing directors. “We were able to compensate for the expiration of mp3 patents and continue growing the team. The following codec generations have been just as economically successful as mp3.”

Today’s highlights include the MPEG-H Audio System, which brings immersive sound into living rooms and introduces interactive features. Users can choose between several audio mixes – for example, selecting from two commentators during a football match or entirely turning off commentary. The EVS speech codec, used in all 4G and 5G mobile calls, was also co-developed at Fraunhofer IIS. It enables Full-HD voice quality calls that sound like the person you’re talking to is right next to you. The Low Complexity Communication Codec (LC3) has become the Bluetooth standard, delivering excellent audio quality with minimal energy consumption for example on Bluetooth headsets. 

"We’re constantly working to continue this success story. The following codec generations have been just as economically successful as mp3"

Billions of Hours Streamed Audio and Video Content

Most people rarely think about audio coding in their daily lives. Yet we all interact with these groundbreaking technologies multiple times a day. Just one of mp3’s many descendants – the xHE-AAC codec – is used by three billion people every month to stream two billion hours of content. This latest generation of the AAC format is ideal for streaming movies, music, podcasts, and more. It’s built into all major operating systems and powers platforms like Netflix, Instagram, and Audible. xHE-AAC not only delivers highly efficient compression but also includes features that ensure consistent volume and clear audio even in noisy environments.

AI Unlocks New Potential

Fraunhofer IIS continues to lead the way in audio innovation. The fifth generation of codecs uses AI-based methods to achieve unprecedentedly low bitrates of under 3 kbit/s for speech. This new codec could, for instance, be used for mobile satellite communication. Another AI-driven project is noise suppression in the upHear Microphone Processing range, which ensures clear audio on smartphones while conserving battery life.

But the researchers don’t stop at audio. A new frontier is opening up with generative AI and large language models. One early result is “Teuken-7B,” an efficiently trained European language model that supports all 24 official EU languages and complies with local data protection standards. In the future, Teuken-7B and similar models will be adapted for use in government, justice, and industry. To support this, Fraunhofer IIS is currently building AI computing infrastructure in collaboration with Friedrich-Alexander University Erlangen-Nuremberg. Just like in the early days of mp3, limited computing power remains a challenge – but the Fraunhofer team still has plenty of brilliant ideas, just like back then.

30 years of .mp3 - Nothing is too hard

When creating pioneering audio technologies, the work is harder for some content types than for others, which can present teams at Fraunhofer IIS with major challenges.

Prof. Dr.-Ing. Jürgen Herre, now a professor at the AudioLabs in Erlangen, reported in 2006 on how the team struggled with Suzanne Vega's a cappella song “Tom's Diner” during the development of mp3.

Dr. Richard Füg, head of the Audio and Speech Coding group at Fraunhofer IIS, and his team still face challenges in codec development today. For example, how do you ensure the best sound for media playback on mobile devices, even in noisy environments? Our 4th generation audio codec, xHE-AAC, does this by default.

30 years of .mp3 - The listening test environment

There are no revolutionary audio technologies without listening tests. Fortunately, the possibilities at Fraunhofer IIS have improved enormously from the development of the mp3 to the present day.

Prof. Dr.-Ing. Karlheinz Brandenburg, head of the Fraunhofer IDMT back in 2006, recalled how the numerous listening tests during the development of the mp3 strained the nerves of his boss, Prof. Seitzer. Daniela Rieger, deputy head of the Soundlab group at Fraunhofer IIS, shows “Mozart”, one of the most modern sound laboratories in the world. Here, advanced technologies are developed and tested, such as the immersive and interactive format MPEG-H Audio or IVAS – a codec that turns telephone calls in 5G mobile networks into a spatial experience. 

30 years of .mp3 - celebrating successes

We at Fraunhofer IIS celebrate our successes - and there have been a lot of them over the past 30 years.

Harald Popp, head of the Audio and Media Technology department until 2018, and Guillaume Fuchs, Chief Scientist in the Speech and Audio Research department, describe a custom that has endured at Fraunhofer IIS from the development of the mp3 to the present day. In the NESC project, which is led by Guillaume and others, the team is working on a 5th generation audio codec that achieves extremely low bit rates thanks to AI-based methods. This makes it ideal for mobile communication in non-terrestrial networks (so-called NTNs). Skyl, a provider of such services, is now testing our codec in its first applications. A reason to celebrate!