IE 11 is not supported. For an optimal experience visit our site on another browser.

AI can now compose pop music and even symphonies. Here's how composers are joining in.

Artificial intelligence is giving composers new tools for composing and collaborating, but some see a downside.
Image: Composer Lucas Cantor
Composer Lucas Cantor oversees a 67-piece orchestra rehearsing new movements of Schubert's Symphony No. 8 in London.Archie Brooksbank

For the past two centuries, composers, academics and fans of classical music have puzzled over Franz Schubert’s Symphony No. 8, which he abandoned in 1822 after writing the first two movements.

Why did the Austrian composer walk away from his “unfinished symphony” six years before his death? Did he associate it with illness? Was he distracted by other work? And if he had completed the final two movements, what would they sound like?

Now Lucas Cantor, a Los Angeles-based composer for film and television, has finished the work Schubert started — with help from artificial intelligence that runs on a smartphone.

"I jumped at the opportunity to use AI to complete the symphony; I think I responded yes before I even finished reading the email when Huawei reached out," Cantor told NBC News MACH. "I've always been excited about the intersection of technology and music."

Cantor is not alone. Composers he has worked with are eager to use new technology as a tool for pushing their work into new areas. And it's not just in classical music. With several new musical programs emerging that encourage musicians to compose with AI, tech enthusiast and former American Idol contestant Taryn Southern released the first pop album to be co-written and co-produced with artificial intelligence.

Turning patterns into music

AI is essentially a pattern-recognition system. Feed it enough data, and it will find patterns within that information that it can use to make decisions. In the case of the Schubert symphony, the decisions were about which musical notes should be placed where.

And there are other examples of AI's creativity, from other disciplines.

Artists like the Munich-based Mario Klingemann have used AI to recognize patterns across thousands of 14th to 17th century European oil portraits to create an ever-changing single work, highlighting a “greatest hits” medley of images. (The work sold at auction in March for more than $50,000.)

Similarly, software from Open AI, a San Francisco-based artificial intelligence company, can write a portion of a news story and then have AI write the rest. (The results are so convincing that Open AI, fearful of building a fake-news machine that any propaganda outlet or hostile nation could use to spread disinformation, released only a limited version of the software to the public.)

For Cantor and Huawei, the Chinese telecommunications giant that built the software used to complete the Schubert symphony, the question was which dataset to use to train the software. All of Western music? All classical music?

In the end, Cantor and engineers from Huawei fed as much of Schubert’s catalog as they could find — roughly 2,000 pieces of piano music — into the software inside the company's new Mate 20 phone. The goal was to teach the AI to think like Schubert and to compose new passages, including what Cantor calls the “heart and soul of any piece of music”: the melody.

“I was impressed that a computer could generate musical ideas, so everything that it sent me back I found to be kind of incredible," Cantor says. "It’s like having a collaborator that has an endless stream of ideas, that never gets tired, that never runs out of ideas, that never has a bad attitude and never needs to take a break.”

Once the AI suggested a series of new melodies, Cantor used his professional expertise to choose one. Then he elaborated on the software's notes, adding instruments and harmonies to flesh out the AI's contribution into a full movement.

Can algorithms replace composers?

Economists, behavioral scientists and other experts have long argued that AI could one day automate entire industries, from delivery driving to dry cleaning. Critics of AI have concluded, however, that even the most sophisticated AI software can’t replace our most quintessentially human capacity: the ability to find inspiration in the world around us and use our innate creativity to make art.

But even the most beautiful music is, ultimately, just code. Algorithms can be taught to write code, and when an algorithm has seen enough of the musical code it's being asked to emulate, it can compose music that sounds, well, human.

Does this mean artificial intelligence can now take the place of a skilled composer? Cantor demurs, calling AI “just another tool in the toolbox” when he’s composing and saying he exploits all sorts of technology during the creative process. “The guitar is a piece of technology that you would collaborate with," he says. "None of these things can make music on their own. When put in the hands of a capable human, they can really augment the human potential. And AI is not different.”

But what happens when AI is used to create entire compositions?

Southern, the Los Angeles-based pop artist, says the technology — which she says makes the process of composing more like shopping for a melody — is perfect for someone like her: a music lover who grew up writing code but who lacks formal training in either discipline.

“Music was always a fun hobby for me, but I always felt like I didn’t actually have the skill set to make it as a musician,” Southern said. “But understanding the language of data and how music can translate into data made so much sense to me.”

Image; Taryn Southern
Pop artist and former American Idol contestant Taryn Southern sits down with NBC News Tech Correspondent Jacob Ward to chat about her album, co-produced and co-written with artificial intelligence.Chiara Sottile

For her new album, "I Am AI," Southern composed songs using the artificial intelligence platform Amper. But it's just one of several similar programs musicians can use to compose songs, including Google’s Magenta, Sony's Flow Machines and Jukedeck. Each of these programs makes it possible for a musician to pick the key of music, specific instruments, the beats per minute of the rhythm as well as the point where the music builds to a climax. Then the artificial intelligence offers up a range of options that the musician can choose from — picking one, combining several or altering their initial input for an alternative result.

Difficult questions

The creators of Amper, who imagined that it would be used to replace musicians typically hired to compose and record promotional music, fed it a variety of genres. Choose “documentary,” “futuristic” and “hopeful,” and up comes a spare, glitchy track with an upbeat bass line. Choose “trap bombastic,” and you can imagine the result rattling the windows of your car.

The music created by Amper and similar programs is essentially indistinguishable from what you can hear on any radio station, and it raises difficult questions. For example, if we use pattern-recognition software to create only music that is similar to what we’ve enjoyed in the past, will we wind up trapped in a feedback loop? How will musicians find new inspiration if they are only sampling what’s already been done? Will music lovers of future generations wind up listening only to variations on compositions from the past?

Southern says that, for better or for worse, the need to produce nearly endless amounts of new work will drive the market for music-composition AI software. “There is a tremendous amount of pressure for any content creator to be constantly producing faster, better and stronger,” she says.

And if the result sounds as good as anything composed by a human, is that necessarily a bad thing? “These tools will actually just fortify creativity,” she says. “And ultimately, it’s not really about, ‘Is the computer being creative?’ It’s about, ‘Does the person who’s listening to the music or looking at the art think it’s being creative?’”

Want more stories about technology?