AI Voice Studio Workflow: Custom Narration and Music Beds

Why Audio Decides Whether an AI Video Feels Finished

Every creator who ships AI-generated video learns the same lesson the hard way: viewers forgive a lot of visual imperfection, but they almost never forgive bad audio. A slightly melted face in the background of a wide shot goes unnoticed. A narration track that clips, hisses, or lands half a beat behind the mouth it belongs to will get the video muted, skipped, or scrolled past in under two seconds.

That asymmetry is worth internalizing before you touch a single slider. Video models have become genuinely good at producing motion, lighting, and camera language. Audio, by comparison, is still the layer where amateur outputs announce themselves. The good news is that audio is also the easiest layer to fix, because the rules are far more deterministic than they are for visuals. There is no equivalent of 'the hands look weird' in a well-mixed voiceover. If the script is written for speech, the voice is directed properly, the music sits under the voice instead of on top of it, and the final loudness is normalized, the result sounds professional even when the visuals are obviously synthetic.

Treat audio as the spine of the piece, not as the last step you squeeze in before export. Narrate first, cut picture to the narration, and you will save yourself hours of nudging clips to fit a voice that was always going to win the argument.

The Three Audio Layers Every Clip Needs

Most videos live or die on three layers, and each has a distinct job.

Narration: the information layer

Narration carries meaning: the claim, the explanation, the story beat. It is the only layer the audience is consciously processing, which means it deserves the most attention and the highest place in the mix. If a viewer cannot parse a sentence, nothing else in the timeline matters.

Music: the emotional layer

Music tells the audience how to feel about what they are looking at. The same drone shot of a city at dusk reads as melancholic, triumphant, or ominous depending on what plays underneath. Music is also the layer most often overused, because a track that would be great in isolation can bury the voice and flatten the pacing of a script.

Ambience and sound design: the realism layer

Room tone, footsteps, traffic, wind, keyboard clicks, whooshes on transitions. These are subtle and almost never noticed consciously, which is exactly why they work. A scene with none of them feels like a slideshow; a scene with too many feels like a Foley demo reel.

Priority order for attention: narration first, music second, ambience third. Priority order for loudness is the exact reverse.

Writing Narration Scripts That Synthesize Cleanly

Synthesized speech fails in predictable places. Write around them.

Short sentences win. A 35-word sentence with three subordinate clauses will trip up even a good voice model, and it will exhaust a human listener at the same time. Aim for 8-18 words per sentence, with a mix of lengths so the rhythm does not turn robotic.

Punctuation is direction. Commas create micro-pauses, periods create full stops, em dashes create a beat of hesitation, and question marks lift the final contour. If a line reads flat in the output, adding punctuation is usually faster than regenerating with a different voice.

Numbers and symbols need to be written as spoken. '1,200' might be read as 'one thousand two hundred' or 'twelve hundred' — pick one and spell it out. The same applies to dates, units, version numbers, and currency signs.

Keep names consistent. If a brand is pronounced two different ways across a video, the audience notices and starts doubting everything else. Build a small pronunciation sheet for recurring names, product terms, and acronyms, and reuse it across every asset in a campaign.

Avoid tongue-twisters and unintended homographs. Words like 'read', 'lead', and 'live' can be resolved by the model or not; if a mispronunciation would change meaning, rewrite the sentence.

Plan by duration. Spoken English averages roughly 140-160 words per minute at a comfortable documentary pace. A 45-second segment therefore holds about 105-120 words. Writing to word count beats writing long and cutting, because cuts leave awkward breath gaps and the pacing starts to feel clipped.

Read every draft aloud, or have a text-to-speech tool read it back. Anything your own mouth stumbles over, a synthetic voice will stumble over too.

Choosing and Directing an AI Voice

Voice selection criteria

Filter candidates on six axes: language and regional accent, timbre (warm, bright, neutral, gravelly), perceived age, gender presentation, baseline energy (calm, conversational, energetic), and articulation clarity.

Then match voice to content rather than to personal taste. A hardware review benefits from a grounded, mid-energy baritone; a fast-paced product teaser benefits from a brighter, quicker read; a meditation video wants low energy and slow pacing. If you are building a channel, pick one voice and stay with it. Consistency builds recognition faster than variety.

Directing prosody

A raw text-to-speech pass is a first draft. The controls that matter most:

• Rate: slow down for technical explanations, speed up for lists and recaps.
• Pitch and contour: a small lift on the final phrase of a section signals 'new topic' without saying it.
• Emphasis: mark the one or two words per sentence that carry the meaning.
• Pause insertion: add a 300-600 ms pause between sections so the audience can breathe.
• Emotion: use sparingly. Full-send 'excited' for 60 seconds is tiring; a 5-10% shift reads as natural.

The three-take rule

Generate at least three versions of any important passage with different settings, then listen to all three on a phone speaker at half volume. Phone speakers reveal intelligibility problems that studio headphones hide. Pick the take that stays clear, not the one with the most character.

Generating Background Music That Fits the Cut

Music prompts work best when they specify instrumentation, genre, tempo, mood, and energy curve instead of adjectives alone. 'Warm lo-fi hip hop, 80 BPM, mellow Rhodes piano, soft brushed drums, light vinyl texture, sparse arrangement that leaves space for a voiceover' is a usable prompt. 'Emotional and inspiring' is not.

Useful prompt ingredients:

• Instrumentation: which instruments carry the melody, which are texture.
• Tempo: BPM, or a range.
• Mood and mode: major for optimism, minor for tension, modal for ambience.
• Energy curve: does it build, hold steady, or decay?
• Density: how much is happening in the 1-4 kHz range where speech lives.
• Duration and structure: 30 seconds, loopable, with a clean ending.

Two practical rules. First, generate one track per scene or per 30-60 second block and crossfade between them, rather than stretching one track across five minutes. Long AI tracks tend to drift or develop abrupt structural seams. Second, prefer instrumental-only beds under narration. Lyrics compete for the same cognitive channel as speech, and even a single repeated vocal hook will pull attention away from the message.

If you have access to stems, use them. A drums-and-bass stem under dialogue plus the full mix during transitions gives you far more control than volume automation alone.

Mixing and Mastering: Levels, Ducking, Loudness

Starting levels

• Narration: peaks around -6 to -3 dBFS, average around -12 dBFS.
• Music bed under narration: -24 to -18 dBFS, roughly 12-18 dB below the voice.
• Music in narration-free sections: -12 to -8 dBFS.
• Ambience: -30 to -24 dBFS, present but never identifiable.

Ducking (sidechain compression)

Set the music to duck 6-10 dB when the voice enters. Use an attack of 100-200 ms so the first syllable is not clipped, and a release of 300-500 ms so the recovery sounds gradual. If you can hear the music pumping, your release is too fast or your duck depth is too deep.

EQ carving

High-pass the narration around 80-100 Hz to remove rumble. Then dip the music 2-4 dB in the 1-4 kHz intelligibility band using a wide, gentle curve. This is the single most effective trick for making a voice sit on top of a busy track without raising its level.

Loudness normalization

Platforms normalize playback, so deliver a consistent target rather than the loudest file you can produce:

• Social video and most streaming platforms: about -14 LUFS integrated.
• Podcast-style spoken content: -16 LUFS integrated.
• True peak ceiling: -1 to -2 dBTP to survive lossy encoding.
• Dynamic range: keep the narration-to-music contrast; over-compression sounds cheap and tiring.

De-ess if sibilance ('s' and 'sh' sounds) stings. A gentle 4-8 kHz reduction on the voice, triggered only on harsh syllables, is usually enough.

Syncing Audio to Generated Shots

There are two directions to work, and picking the right one early saves the most time.

Narration-first: generate and lock the voiceover, then build the visual timeline to match it. Cut shots on sentence boundaries, hold a shot through an important clause, and change the image when the idea changes. This works best for explainers, tutorials, and anything scripted.

Picture-first: lock the visual edit (often because the generated clips are expensive to redo), then trim, stretch, or rewrite the narration to fit. This works best for montages and mood pieces where the visuals lead.

Practical timing techniques for both:

• Map the beat. Find the music's downbeats and place shot transitions on or just before them.
• Use micro-pauses as cut points. A 200-400 ms silence is a natural place to change scenes.
• Fix short shots with slow motion, a punch-in, or an inserted cutaway, not by speeding narration.
• Fix long shots with a music-only beat or an added line of narration, not with dead air.
• Check sync tolerance. Anything under about 80 ms of offset reads as in-sync; beyond 120 ms it looks wrong.

A Repeatable End-to-End Audio Workflow

1. Write the script to duration. Target 140-160 words per minute.
2. Split the script into sections. Each section becomes one audio block, so mistakes stay contained.
3. Generate three voice takes per section. Vary only one setting at a time.
4. Choose takes and assemble the voice track. Add 300-600 ms gaps between sections.
5. Normalize the voice. Get a consistent level before you add anything else.
6. Generate music per scene. Prompt with instrumentation, tempo, mood, and density.
7. Place the bed and set ducking. The voice always wins.
8. Add ambience and accents. Transition whooshes, room tone, one or two accents per scene.
9. Master to your platform target. About -14 LUFS with a true peak near -1 dBTP.
10. Generate captions and check timing against the spoken words.
11. Listen on three systems: phone speaker, laptop speakers, headphones.

Multi-language versions

Generate narration in each target language from the same script rather than translating a finished audio file. Keep sentence boundaries identical across languages so the visual cut still lands on the same beats. Expect translations to run 10-30% longer in some languages, so re-check duration before locking picture. For captions, keep reading speed between 15 and 20 characters per second and let captions run slightly ahead of the voice rather than behind it.

Quality Control Checklist and Common Mistakes

Checklist before export:

• Does the voice stay intelligible on a phone speaker at low volume?
• Is any syllable clipped by ducking, compression, or an early music entry?
• Does the music enter and exit cleanly, with no abrupt cut at the end?
• Are sibilants harsh anywhere?
• Does loudness hold steady across scenes?
• Do captions match the spoken words exactly, including numbers?
• Is there any point where two audio layers fight for the same frequency range?
• Do the first two seconds sound intentional?

Common mistakes and their fixes:

• Music too loud in the mix. It sounds fine on headphones and drowns the voice on a phone. Fix: increase duck depth and carve 1-4 kHz.
• Narration written for reading, not speaking. Long subordinate clauses create flat, breathless delivery. Fix: split sentences.
• One music track stretched across the whole video. Energy flatlines by the halfway point. Fix: one track per scene, crossfade between them.
• Everything at maximum. No dynamic contrast means no impact. Fix: let quiet sections be quiet.
• Ambience overdone. Fix: if you can consciously identify a sound effect, it is too loud.
• Ignoring the opening. Most viewers decide whether to keep watching in the first two seconds, before any visual has time to develop.

FAQ

How long does an AI narration workflow take?

For a 60-second video, scripting takes 20-30 minutes, voice generation about 10 minutes, music 10 minutes, mixing 20-30 minutes, and QC another 10. That is roughly 1.5-2 hours for a first pass, and about half that once you have a reusable template and a saved set of voice presets.

Should I use one voice for an entire channel?

Yes, if brand recognition matters. Consistency in timbre, pacing, and loudness does more for retention than switching voices for variety. If you need differentiation across series, vary the music and the pacing rather than the voice itself.

Can I use AI narration for client work?

Check the tool's license terms for commercial use and what rights you get to generated audio. Keep records of the model, version, and prompt used for each deliverable in case a client asks how a line was produced or wants a revision months later.

How do I stop music from fighting the voice?

Three steps, in order: duck the music 6-10 dB under speech, high-pass the voice at 80-100 Hz, and dip the music 2-4 dB between 1 and 4 kHz. Only raise the voice level if those three fail.

What loudness should I target?

About -14 LUFS integrated with a true peak around -1 dBTP for social and streaming platforms. Podcasts often sit closer to -16 LUFS because listeners are on headphones for longer stretches.

Do I need a separate sound design pass?

For talking-head and explainer content, no: narration and music carry it. For narrative or product films, yes. Ambience plus one or two well-placed accents adds more perceived production value per minute of work than anything else in the pipeline.

How many words fit in 30 seconds?

Roughly 70-80 at a documentary pace, and 90 or more if the read is quick and the content is list-like. Write to that number instead of cutting afterwards.

What if the generated voice mispronounces a word every time?

Rewrite it phonetically or restructure the sentence. Chasing a fix through settings rarely works as reliably as changing the spelling or swapping in a synonym, and it costs far less time.