Excerpted from The Musician's A.I. Handbook with permission from the author and publisher.
A.I. music production encompasses both music composition and music production, and they both need a bit of a clarification.
The A.I. Music Composition section consists of sections on A.I. text-to-music platforms, which actually generate music and MIDI files; A.I. music idea platforms, which are more for melody and chord change suggestions, how to use ChatGPT as a compositional tool; and A.I. lyric generation, again to give you ideas for words and phrases when you’re stuck.
The section on A.I. Production Tools covers platforms that create new sounds, transform current sounds into different ones, some A.I.’s that are smart enough to play along with you, A.I. song analysis, and A.I. voice cloning, a hot topic for many.
My courses cover all sorts of A.I. audio plugins and standalone apps, from A.I. DAWS, track separation tools, noise reduction tools, plugins like A.I.-driven compressors, limiters, EQs, gates and reverbs, to where to place them in the signal path of your DAW.
A.I. Composition
While many think that using A.I. for writing a song is simply typing in a prompt that says, “Create a song that sounds like Drake,” there’s much more to it than that. A music A.I. like Loudly or Boomy might output something that might be in the ballpark, but the surprise is that you’ll probably only get about 30 seconds and not a full-length song. If you’re a musician or songwriter, that’s not what you’re looking for.
A.I. composition is really more than that. It’s asking one of the many A.I. tools to help you generate a melody, a lyric or a chord progression that you might find useful in your daily songwriting. It’s learning how to form the exact prompt that will give you an idea for a new beat or melody twist that you never would have thought of otherwise.
As you might imagine, there are now dozens of tools that can help us do that, with more coming online all the time. Some are platforms that you access in a browser, some are standalone apps, and some are DAW plugins. They range from being remarkably simple to deeply intricate, some requiring a steep learning curve to navigate.
This is not about mastering a specific platform or plugin so we won’t go down the rabbit hole of learning a particular A.I., although I will point out the features and nuances of some of the ones that I find more interesting.
Instead, we’ll look at how we can have these help us in our songwriting, so that no matter which one you use or if a new one comes along, your approach can remain consistent in order to get the best results.
A.I. Text-To-Music Platforms
There are a number of platforms that are user-friendly and don’t require much music skill, and these fall into the “consumer” platforms category. They tend to respond to simple prompts (in fact, they won’t allow you to get too complicated) and produce results that impress consumers or music beginners, but aren’t likely to be useful in a professional setting. They can be very helpful in certain situations though, as long as you don’t expect a result that’s beyond the capabilities of the platform. Unfortunately, the music that gets generated is disappointing much of the time unfortunately, and it’s all because of one song.
The Viral Track That Changed Everything
The song that significantly heightened A.I. awareness within the music industry was a track created by the anonymous ghostwriter977 called “Heart On My Sleeve,” released in March of 2023. This would be just another track by an unknown composer except that it featured superstars Drake and The Weeknd, or so it seemed.
It turned out that neither of these artists actually performed on the track. Their voices were A.I.-generated. That didn’t stop the song from becoming a viral hit though, as “Heart On My Sleeve” quickly accumulated 600,000 streams on Spotify, 275,000 views on YouTube, 6.9 million views on Twitter, and 15 million views on TikTok before a take-down was forced by Universal Music Group, the parent company of these artists' record labels.
The quick success of the track sent ripples through the music industry, as a slew of new superstar imitation songs soon flooded social media and online distributors. It also misled casual music fans into thinking that all they had to do was sign into a music A.I. and input a prompt that simply said, “Make a song like Drake,” and out would pop a flawless imitation.
What most people didn’t understand was that “Heart On My Sleeve” and all the other similar tracks that followed were actually meticulously crafted mostly without the use of A.I.
While ghostwriter977 never revealed the A.I. that he used, he did state that he was a ghostwriter for a record label with a great deal of music production experience. He crafted the song manually, and only used A.I. to clone the voices of Drake and The Weeknd, which then replaced his vocal performance.
The fact of the matter is that the various Drake, John Lennon, Michael Jackson, Kanye West, and Ariana Grande (and others) A.I. songs that you can find online require far more than just a prompt to create, as all of them involve good old-fashioned music production to create the backing track first.
The effect of these “fake” songs was that millions flocked to A.I. music generators like Boomy and Loudly only to be disappointed that their creations weren’t even in the same universe as the originals. On the positive side, these same creators were exposed to producing music via A.I., but the downside was that online music streamers were suddenly flooded with millions of these tracks.
The music industry’s reaction was fast and furious, as record labels issued take-down notices on A.I.-generated fake tracks, and music distributors like Spotify and Tunecore deleted and rejected not only obvious fake tracks, but 100 percent A.I.-generated tracks as well.
Regardless of its limitations, consumer-level A.I. generated music is here to stay.
A.I. Music Generation Prompts
Regardless of the platform you use, crafting a
prompt is your first step to generating music. Like with everything that’s A.I.-centric, the better the prompt, the better the outcome. Let’s look at some prompt creation practices that will help you get the result you’re looking for faster. These are the elements to consider in a prompt:
1. The genre of music. While generic terms like jazz, classical, or rock are a starting point, getting more specific like "upbeat, '80s-style synth-pop with a joyful melody” zeros in on a better result.
2. The mood you’re aiming for. Using words like “melancholy,” “upbeat,” “dramatic,” ”dreamy,” "intense," "mellow," or "rhythmic" can help guide the A.I. in a specific direction.
3. The instruments you’d like to be prominent. If you have certain instruments in mind, spell it out. For example, "A soft ballad featuring a piano, cello, and female vocals.” Also, what instruments you’d like to exclude (example: “A jazz combo without a saxophone”).
4. The tempo and rhythm. If you have a specific tempo in mind, indicate it via beats per minute (BPM). Likewise you’re looking for a rhythmic pattern, include that. For instance, "A lively salsa at 180 BPM" or "A slow, melancholic waltz."
5. Reference existing music. If you have a specific sound or artist in mind, referencing them can be helpful.
6. Structure or progression. If you want a song with a specific structure (intro, verse, chorus, bridge, etc.), specify that in the prompt. For example, "A track with an ethereal intro leading into a powerful chorus, followed by a soft bridge and ending with an echoing outro."
7. The duration of the song. If you need a specific length for the music, state it. "A 2-minute cinematic intro for a sci-fi film."
8. Refine the result. A.I.-generated music might not always match your vision on the first try. You might have to tweak your prompt or provide feedback on the generated output to refine the results. Provide feedback on what you liked and what you'd like to change for the next iteration.
9. Test different prompts. If the A.I. allows it, run multiple prompts to see which one provides the desired result.
Remember that the punch your prompt packs can also be swayed by the capabilities and design of the specific A.I. music generator you’re working with. Some A.I.’s might vibe better with certain types of prompts than others. A dash of experimentation and iteration will go a long way in sharpening your prompts and bagging the best results.
A.I. Music Composition
Idea Platforms
The second type of composition platform is more oriented to a professional musician, composer, or songwriter. In this case you’re looking more for chord pattern or melody ideas than a finished track, hence the name “Idea Platform.” That means that the prompt approach is different since we’re not looking for a entire arrangement, only bits of the skeleton.
A.I. Composition General Prompt Guidelines
When using one of these tools, if you were to create a simple prompt like "create a memorable melody,” you’d most likely get a repeating sequence of quarter notes from the C major scale that you most likely won’t find very inspiring. To get something usable, try adding the following to the prompt:
• Ask for "a combination of quarter, eighth, and sixteenth notes" when prompting for a melody.
• Tell it what scale or key signature you want it to use.
• Try asking it for specific chord progressions using roman numerals.
• Ask for chord progressions that fit a genre's style, like a jazzy chord progression or a punk rock progression.
A.I. Composition Idea Platforms
A.I. composition idea platforms come in different flavors: some are browser-based, others are stand-alone apps, and some come as plugins. The following platforms vary in their focus—some are for fresh melody ideas, others are for discovering new chord patterns, while some are jacks of both trades. A select few can almost morph into a DAW.
Because some of the follow platforms can get complex, I suggest finding one thing that it does well, and stick to that if you’re not up for learning another new platform. A good example of this would be Scaler 2, which is an excellent platform for finding new chord substitutions and changes. As above, there are new ones coming online all the time, and some on this list might have folded or been acquired by the time you read this.
A.I. Lyric Generation
If you have trouble coming up with lyrics for songs, you’ll be pleased to know that writing lyrics is one area that A.I. excels at. While you can’t expect it to come up with lyrics on the level of Bob Dylan or Bernie Taupin, you will get something rather serviceable that just might contain a word or phrase that will spark your own lyric writing breakthrough.
Just like in every other A.I. case, the better the prompt the better the results. Here are some prompt tips to follow for best results:
• The more you set the stage for the song, the better your results will be (“A boy away in the Navy longing for his girlfriend in New York City”)
• Ask the A.I. to imagine it’s the songwriter (“Imagine you’re a songwriter. . .”)
• Add the mood, the concept, the intended audience, any related emotions, the hero and the goal (“Imagine you’re a country music songwriter. Write a sad song aimed at teen-age girls about a boy that looks like Kid Laroi who’s away in the Navy”)
• Ask the A.I. to become a person in the song (“Imagine you’re a cowboy on a cool night on the open range”)
• Remember to tell it the form of the song you want (example: “The song should have three verses, two chorus, a bridge”)
A.I. Lyric Prompts
While you can ask the A.I. to generate an entire song, sometimes you only need help in a particular section. Here are some examples:
• "Write a chorus about a school romance that ends at graduation."
• "Craft a verse that describes the feeling of wandering through a city that you don’t know late at night."
• "Write a bridge about the bittersweet feeling of a relationship breaking up that you knew had to end.”
• "Compose a song that captures the essence of a day at the beach where the sand pebbles that are like memories in the style of a 1980s power ballad.”
• "Write a ballad about two stars in the sky that are destined to never meet."
You often get better results if you ask the A.I. to act as a songwriter or coach. Here are some examples.
• “I want you to act as a songwriter. Write a song about a sad young boy trapped in New York City, longing to return to his country home. The song is aimed at teenagers. It should contain an intro, three verses, two different choruses, a bridge and an outro.”
• “I want you to act as a songwriting coach and help me determine the specific aspect of my song that is causing writer's block.”
• “I want you to act as my songwriting mentor, offering suggestions related to a particular song section. I will provide the lyrics of the song I'm working on and identify the part or parts I'm struggling with. Offer specific recommendations for revisions, alternative approaches, or exercises that can help me address the issue in that section. Share your insights on why you believe these suggestions will improve the overall song.”
A.I. Music Production Tools
Now that you’ve written the song it’s time to produce it. That means putting together the arrangement and the instrumentation that’s appropriate for the style of music that you’re working in. While there’s no A.I. that will arrange a song for you yet, there are a lot of A.I.’s that will help you not only come up with new and unusual sounds, but suggest new musical parts that fit your song harmonically and rhythmically as well.
We’re going to break these down into five categories: A.I. Sounds, A.I. Tonal Morphing, A.I. Accompaniment, A.I. Voice Cloning and A.I. Song Analyzers. Each one has a particular focus on music production. You might only need one of the four, but don’t be surprised if you find a need for all of them at one time or another.
A.I. Sound Generation
Producers are often faced with a decision—“Do I use vintage sounds that everyone knows or do I look for sounds that no one has heard before?” Often a new and exciting sound can be a big part of why a song is a hit, so more and more producers are looking beyond sample libraries to something that’s completely cutting edge, and maybe never even heard before. As you might imagine, there’s an A.I. for that.
A.I. Accompaniment
We’ve now entered into the realm of A.I. becoming your band member and playing along with you in real time. One of the most interesting A.I.’s in this category is EZDrummer 3, where its Bandmate feature will find the right drum part for your song without you having to program it. This is a very exciting development for those that aren’t good at programming drums or simply don’t want to take the time. Likewise, Playbeat 3 generates drum patterns and variations based on how you train it during your everyday songwriting life.
Regardless of the production category of A.I. that you use, you’ll find that they’ll keep getting better at what they do as they’re trained more, or new ones utilizing larger neural networks come on the market.
Plus, many singers will find that licensing their voice to one of these platforms will be yet another revenue source. While there’s no standard on what kind of revenue is possible yet, the concept is new enough that everyone is still learning about what’s possible, especially on the business side.
On the other hand, voice cloning has the potential for abuse, as it could easily be used to commit fraud, spread misinformation and generate fake audio evidence. As we’ve seen in the case of ghostwriter977 and “Heart On My Sleeve,” the fact that a superstar’s voice has been illegally cloned doesn’t matter much to the general public when it comes to consuming a song. Look for legislation concerning illegal voice cloning soon, as multiple bills are pending in both the California legislature and United States Congress.
Commercial Use Policies
This sounds like a broken record, but be sure to always check the terms and conditions of the platform to find out its commercial use policies.
In some instances you’re free to use anything you generate, but there may be commercial use restrictions with some virtual vocalists databases.
In other words, you can use it for experimentation, but you may have to pay extra to release the song that’s using virtual vocalist commercially.
In some instances you must provide attribution to the virtual vocalist in order to use the song.
You’re never allowed to use the virtual vocalist’s name and likeness without prior agreement, which is negotiated separately.
Be sure to read the fine print to understand what’s required to use the app, platform or database.
A.I. Audio Tools
While the book touches on all sorts of A.I.’s, some readers may be more focused on the many A.I. audio tools than the other areas. In fact, A.I. audio plugins shine in that they cut to the chase and make the setup of compressors, EQs and reverbs much easier by automatically setting the perfect parameters for the song.
While these audio tools definitely get you into the ball park, it should be noted that they require some tweaking in order to contribute to a professional mix. On the other hand, A.I. track separation tools do what was once almost unthinkable by taking a full-mixed stereo track and separating it into individual elements, while A.I. noise reduction tools can take what a few years ago would have been an usable recording and make it noise-free.
Before we get into those tools, let’s take a look at what many consider the most important audio tool to a modern musician - the digital audio workstation.
A.I. DAWs
Combining artificial intelligence with a digital audio workstation results in a new kind of DAW called a Generative Audio Workstation, or GAW. In this case, you get an A.I. composition tool capable of generating audio from text, along with the elements of a DAW such as an editing timeline, audio processing and effects, and a digital mixer.
What’s interesting here is the wide variety of approaches that different developers take. For instance, Aiva is touted for its music generation, but once the music is generated it can be manipulated via a browser-based DAW. Since the number of tracks that it can work with is limited, for anything more advanced you have to export the MIDI file, import it into your favorite DAW, and manipulate it via virtual instruments and plugin processors from there.
A.I. Audio Plugins
The idea behind A.I.-enabled compressors, limiters, equalizers and reverbs is to take the work out of setting them up. While there are traditional manual methods for setup for all of these processors (see my Mixing Engineer’s Handbook or Music Mixing Primer course for how that’s done), it does take some time and know-how to do so. Using an A.I. audio plugin means the setup is done for you without the guesswork if you don’t know how to do it.
In most cases, setup couldn’t be simpler. You select a Profile (kick, snare, guitar, bass, etc.), hit the Learn button, play about eight seconds of the track, and you’re good to go as the A.I. selects what it believes to be the most appropriate settings based on its training. If you have an instrument that’s not on the profile list, select the Universal profile and you’re likely to get a setting that will work.
The A.I. does this by first training on a dataset of samples of that particular instrument. The larger the dataset of really good sounding samples, the more likely it will automatically dial in a setting that works.
This automatic setup is great for fast mixes, but a pro mixer will take these automatic settings just as a starting point and tweak them as he or she listens to the track in the mix.
Producer/engineer Bobby Owsinski is one of the best selling authors in the music industry with 23 books that are now staples in audio recording, music, and music business programs in colleges around the world, including the "Deconstructed Hits" series, "Social Media Promotion For Musicians," The Mixing Engineer’s Handbook, "Music 4.0: A Survival Guide For Making Music In the Internet Age," and more. More at bobbyowsinskicourses.com, bobbyoinnercircle.com
