Interactive AI Music

Generative AI is unlocking new interactive music experiences.

AI music doesn’t have to be ‘slop’. It can be Interactive.

I’ve been conducting research in deep learning and music for many years now, and recently I’ve begun to question our work’s impact on music.

Have we contributed with ‘slop’ machines? I hope not. Yet, I cannot ignore the disconnect between my enthusiasm and the public’s perception:

• I’m excited about AI’s artistic potential.

• The world perceives AI output as ‘slop’.

For those who aren’t familiar, ‘slop’ is a popular term used to describe AI-generated art as disposable or low-value content.

Feeling both worried and uncertain about this, I stepped back last year to explore:

• The current state of AI music and emerging artistic trends.

• The broader history of algorithmic and generative music, long before AI.

To realize that the most interesting AI music outlets fall into the realm of Interactive AI Music:

• Generative albums, such as Kunst Kaputt (SENAIDA), which listeners can actively interact with.

• Live performances, like Prompt Jokeys (Dadabots) or MusicFX DJ (Magenta), where the audience or performers generate music live with AI.

• Interactive radios, exemplified by Suno’s or Sonauto’s AI radios.

• New streaming services, as seen in Udio’s recent move toward an interactive, AI-based streaming model.

• Interactive installations, such as Space DJ (Magenta) or TECHNE: The Vivid Unknown (John Fitzgerald), where music (and visuals) respond to audience input.

• Or even new bands! Like BRNRT Collective, our Interactive AI Music band :)

What do these projects have in common? They use AI to create new, interactive musical experiences that were previously not possible.

In this post I define Interactive AI Music, a term that brings together my favorite artistic projects in AI and music.

Definition

Interactive AI Music involves three important characteristics:

• Interactivity: it has the ability to change in response to user input, allowing audiences or performers to influence music in real-time.

• Artistic agency: musicians compose both the music and the interaction, ensuring that the resulting AI experience remains artistically meaningful.

• Generative AI: instead of playing pre-recorded tracks, AI is used to create new melodies, rhythms, harmonies, or sounds on-the-fly.

Artistic agency

AI music is often criticized for being soulless and threatening to musicians’ livelihoods. But AI music does not need to be like that.

I strongly believe that it can be used to create art that was previously not possible, and that it will be equally appealing as previous art forms.

However, the narrative and tools have to evolve:

• soulless → AI to tell stories

• replace musicians → AI in service of artistic agency

For this reason, storytelling and artistic agency lie at the heart of Interactive AI Music. As such, artists do not only compose music—but also compose the interaction.

Interactive AI Music is composed with interactivity in mind.

The result? An immersive, interactive experience fully directed and composed by the artist to tell a story with the help of AI.

Generative

Generative music is also known as indeterminate music, chance music, stochastic music, or aleatoric music. In essence, random processes shape generative music.

“There are three alternatives: live music, recorded music and generative music. Generative music enjoys some of the benefits of both its ancestors. Like live music, it is always different. Like recorded music, it is free of time-and-place limitations.” — Brian Eno.

Accordingly, no two interactive experiences are the same—as generative models allow each interaction to unfold uniquely.

In Interactive AI Music, generativity and interactivity are closely intertwined. Modern generative AI systems unlock forms of interaction that were previously not possible.

Yet, generative music is not new and interactive systems can exist without AI.

Generative modular setups is one example.

Distribution formats

Legacy distribution formats feel obsolete as new playback ideas emerge.

• MP3s made digital audio easy to store and share in the early days of the internet.

• Dolby Atmos facilitates spatial audio that works across different speaker layouts.

AI can be seen as a new format where instead of distributing a fixed audio file, artists distribute generative AI models or applications that make music on-the-fly.

With this new paradigm, a musical piece now becomes a generative system rather than a single recording.

Such new formats would allow music to adapt culturally and contextually, giving music a longer lifespan by making each piece more flexible and reusable:

• Share personalized remixes on social media.

• Start your day with acoustic versions.

• Energize workouts with EDM remixes.

Accordingly, the listener shifts from being a passive consumer to an active co-creator.

This opens the door to new super-fan experiences, aligning with the industry’s growing focus on deeper, more intimate relationships between artists and audiences.

With new media distribution formats, art consumers are becoming co-creators.

New artistic medium

What’s an artistic medium? It’s the thing artists use to make art. Like canvas for painters, clay for sculptors, video for directors, or recordings for musicians.

A new artistic medium → AI
One that’s alive → generative

Artists are releasing AI models or interactive applications or generative albums. For inviting fans to co-create with them.

Music designed for interaction—opening up new interactive, artistic possibilities.

Interacting with SENAIDA’s Kunst Kaputt generative album (video above) feels like playing a game. Hence, Interactive AI Music is new artistic medium where you don’t play music, you play with music.

Nao Tokui@naotokui_en

Google Magenta @GoogleMagenta RealTime + music sentiment analysis + prompt block game = A twist on the lo-fi anime girl. 👉 You need to play to keep her studying. If the music gets too quiet → she falls asleep 😴 If it’s too danceable → she starts dancing 💃

7:18 AM · Sep 18, 2025 · 5.65K Views

---

2 Replies · 14 Reposts · 52 Likes

Analogy

Think of movies and games. Movies are a passive experience, games are interactive. Music is typically like a movie, but Interactive AI Music is more like a game.

A director creates a movie designed to be consumed in a very specific way. Every shot, sound, and edit is crafted so you experience the story exactly as intended.

A game studio, on the other hand, creates a space for you to explore. The story is there, but it’s flexible. You make decisions and experience it in your own way.

Interactive AI Music works the same way.

It might not be the default way to experience music, just as games aren’t the default way to consume stories. But when you engage with it, it offers something movies and traditional albums can’t: exploration and a sense of co-creation.

“Music that you play with, not music that you play” — Mikey Shulman (CEO at Suno).

Not a new concept

Interactive, adaptive, generative, or algorithmic music is not new.

For example, musical improvisation has existed since the 9th century, with performers generatively shaping music in real-time interacting with other performers.

Adaptive music is also common in video games. Music can change in volume, tempo, or arrangement, often shifting the tone and intensity during events like combats.

One of the earliest examples is Space Invaders (1978), where a simple repeating four-note motif speeds up as the aliens descend.

In recent years, interactive music has evolved from early tape and computer experiments into sophisticated digital systems. Today, we are exploring AI for combining art and technology to reimagine how we experience music.

Limitations

Interactive AI Music faces two main challenges:

• New listening habits are required: from passive to active listeners.

• Both interactive formats and distribution remain undefined.

Music consumption has historically been passive. Most listeners want to relax when they listen to music, just as they do when watching a movie or reading a book.

Interactive AI Music, which demands active participation, conflicts with how most people prefer to experience music.

At the same time, today’s music streaming platforms are highly consolidated and built around passive listening, and changing global listening habits will be difficult.

Interactive AI music may not become the default way people experience music, but it can coexist with traditional formats, like videogames do with movies and books.

Also, standardized formats and distribution have yet to be defined.

It’s unclear whether current streaming platforms will support interactive experiences, as Netflix has begun doing with video games. Or whether new platforms will emerge to disrupt the industry, similar to how Steam did in gaming.

For now, musicians rely on their own websites to share interactive online installations. This works for frontier artists, but limits access to wider audiences due to format and distribution gaps.

Possibilities

A musical medium is emerging through Interactive AI Music, enabling new ideas like generative albums and radios, installations, or interactive performances.

Its value lies in unlocking interactions that were previously unattainable, from new gamified experiences or installations to innovative live formats that are interactive.

Music can thus become a participatory and collective process rather than a fixed outcome.

Much like digitalization democratized music production (bedroom producers) or enabled unconventional live show formats (DJ sets), Interactive AI Music can offer a fresh perspective on how music is created, performed, and experienced.

This moment offers a unique opportunity to shape a nascent artistic scene. Musicians, startups, researchers, and corporations are now exploring how AI-driven interactive music can help expand audiences and inspire unexpected artistic directions.

A provocation

Finally, I want to leave you, the reader, with a provocation.

The future of AI music won’t shape itself. It’s up to us to imagine what it can become. If its current state feels like ‘slop,’ consider creating something different yourself.

Explore, experiment, and propose original ideas. While this essay has focused on Interactive AI Music, I’m sure there’s still much more to discover.

Acknowlegment. Thanks to Zack Zukowsky (Dadabots) for the analogy comparing videogames and cinema.

Disclaimer. The views expressed are my own and do not reflect the opinions or positions of my employer.