IE 11 is not supported. For an optimal experience visit our site on another browser.

Machine-Made Melodies: How Humans Are Creating Artistic Partnerships with AI

So far computers have been mere tools for helping humans become creators. But artificially intelligent machines are now becoming our collaborators — and maybe even artists in their own right.
Neil Davidge Portrait Shoot
A composer at work.Joseph Branston / Computer Music Magazine

Computers have revolutionized the art of making music. Using software like Audacity or GarageBand, even symphonic neophytes can learn to record, arrange, and edit their own songs.

So far computers have been mere tools for helping humans become creators. But artificially intelligent machines are now becoming our collaborators — and maybe even artists in their own right.

Meet ALYSIA, the latest song-writing AI. Short for Automatic LYrical SongwrIting Application, ALYSIA is a “co-creative system.” Unlike other recent song-writing AIs, ALYSIA is designed for the human and computer to engage in a true creative partnership. Humans supply ALYSIA with lyrics. ALYSIA turns those lyrics into melodies.

Related: How Virtual Reality is Redefining Live Music

“It can make meaningful contributions in the same way as any other artist,” says ALYSIA co-creator Maya Ackerman, a professor at San Jose State University in California.

ALYSIA’s output won't chart anytime soon, but its first songs aren’t bad for an amateur songwriter — especially a machine. Just listen to the trio of pop songs ALYSIA created recently with help from Ackerman and David Loker, lead data scientist at ION, who helped co-create the system.

Just as musicians are inspired by what they listen to, ALYSIA hones its craft by taking in existing songs. Ackerman and Loker programmed the features of two dozen pop songs into ALYSIA: lyrics, key signature, time signature, and rhythm. When a human feeds lyrics to ALYSIA, it analyzes the songs in its database to generate a rhythm and melody to accompany the words. From those key elements, human artists can then flesh out details like harmonies and instrumentation.

Image; Joshua Palkki
Joshua Palkki, a professor of Vocal/Choral Music Education at California State University in Long Beach, uses ALYSIA.Maya Ackerman

Ackerman and Loker were ALYSIA’s first collaborators, but in recent months other artists have signed on to work with the machine. One new song, written with California State University in Long Beach music professor Josh Palkki, uses a melody ALYSIA created based on words from the Emily Dickinson poem “Hope is the Thing With Feathers.”

Other new collaborations use an ALYSIA-based system that was trained with songs from Puccini operas instead of top-40 hits. Dubbed Robocini, the system is currently working with two San Francisco-based artists: Davide Verotta is a composer using ALYSIA to create an aria and James Morgan is working with the system to craft the first collaborative computer-human opera.

Rafael Pérez y Pérez, an expert in the emerging field of computational creativity at Autonomous Metropolitan University in Mexico City calls ALYSIA’s contributions “interesting and original.”

Related: How Computers Are Learning to Predict the Future

“From a practical point of view, a system like ALYSIA has a great potential for supporting the development of musical skills in humans,” he says. “Particularly, I am thinking of those people that do not have the opportunity to have formal studies but nevertheless have the necessity of expressing themselves through music.”

Hang Chu, a graduate student in computer vision at the University of Toronto who has built his own song-writing AI system, points out that music made with ALYSIA raises some interesting creative rights issues.

“In its current stage, the system isn't fully automatic yet,” he says. “Because of this, it is difficult to tell who should get the credit of composing the amazing songs.” Ackerman and Loker built ALYSIA, but how much should we credit them for the music it creates? This question will be even more pressing as we develop music-writing AI systems equipped to generate songs without humans adding the finishing touches.

“We are so used to thinking of computers as tools, with which we don’t need to share credit,” she said. “It’s new and, hopefully, exciting to think of sharing credit with them.”

For more of the breakthroughs changing our lives, follow NBC MACH.