Why Audio Is Still the Giveaway in AI Video
Text-to-video generation now produces frames that hold up surprisingly well on a phone screen. Audio is where most AI-assisted projects lose the audience, often within the first ten seconds. Three failure modes appear again and again: narration with flat prosody and emphasis landing on the wrong syllable, background music that swells and stops without any relationship to the edit, and a voice track carrying broadband noise that the music then amplifies.
Viewers forgive a slightly soft render. They do not forgive a hiss under dialogue, a music bed that fights the narrator, or a voice that mispronounces the product name. Audio also carries most of the emotional signal in a scene. When narration and score point in different directions, the sequence reads as flat even if the visuals are strong.
The practical consequence is a shift in how you budget post time. On a typical two-minute explainer, a reasonable split is roughly 30% script and voice direction, 40% audio assembly and mixing, and 30% visual polish. That is a very different balance from the one most first drafts use, where audio gets whatever time is left.
Write for the Ear Before You Generate Voice
Synthesis engines reward clean text. They do not understand intent, irony, or a sentence that only works when a human reads it. Editing the script for the ear is the single highest-leverage step in the whole workflow.
Plain sentences, short clauses
Write as if you are reading aloud to someone across a table. Replace stacked clauses with separate sentences. A useful target is 10 to 18 words per sentence for narration, with a hard ceiling around 25. If a sentence needs two commas and a dash to survive on the page, split it into three.
Read the draft out loud with a timer. Every place your voice stumbles is a place a synthetic voice will stumble too, and it will not recover as gracefully.
Numbers, names, acronyms, and units
Text-to-speech will read "3.5 GB" as something you did not intend, and it will guess at acronyms. Keep two versions of the script: one for on-screen text with numerals and abbreviations, and one for narration where numbers are spelled out ("three point five gigabytes") and acronyms are expanded or respelled phonetically.
Proper nouns deserve their own pass. Build a short pronunciation list for product names, place names, and people, then test each one in isolation before you generate the full script. Fixing a single mispronounced word later is cheap; discovering it after the mix is done is not.
Breath marks and pause control
Punctuation is your pause tool. Commas give a short beat, periods a longer one, and paragraph breaks let an engine reset its prosody. If a pause still sounds too short, add an ellipsis or split the line into two separate generation requests and place a deliberate gap between them in the editor.
Mark emphasis explicitly in your script notes. Underline the word that carries the meaning in each sentence. Most engines let you influence stress through capitalization, punctuation, or SSML-style tags, and even a rough marking pass measurably improves intelligibility.
Directing an AI Voice Instead of Just Selecting One
Choosing a voice is the visible part of the job. Direction is what separates a competent narration from one that sounds like a person who understands the subject.
Cast by format
Match the voice to the content type rather than to your personal taste:
- Fast tutorial: clear, neutral, mid-range, roughly 150 to 165 words per minute.
- Documentary or brand film: lower register, 130 to 145 words per minute, longer pauses at section breaks.
- Product ad: tighter pacing, more forward energy, shorter sentences.
- Character dialogue: distinct timbre per character; avoid casting two similar voices in the same scene.
Pace, pauses, and expression controls
Most engines expose two or three knobs that trade consistency against expressiveness. Raising stability produces steadier output that can sound flat over a long read. Raising style produces more color but more variance between takes. Generate two or three takes of the same paragraph with different settings and keep the best one per section rather than committing to a single global preset.
Consistency across a series
If you are producing episodes, save the exact voice, model version, and settings for each speaker. Voice drift between episodes is one of the most common complaints from viewers of AI-narrated series, and it usually comes from a silent model update or a forgotten slider position. Version your audio settings the same way you version a design file.
Cleanup: Making Synthetic Speech Sit in a Real Mix
What noise reduction can and cannot fix
Noise reduction is built for steady-state noise: hum, air conditioning rumble, computer fan whine, mains buzz. It cannot fix awkward phrasing, a clipped syllable, or a word that was never generated correctly. Overusing it introduces watery artifacts and erodes consonants, which makes speech harder to follow even though the track looks cleaner on a meter.
Apply reduction in small amounts and listen for consonant damage. If a line still sounds synthetic after cleanup, regenerate it rather than processing harder.
Text-based audio editing
Editing audio by editing text is the fastest way to tighten a synthetic narration. Delete filler words, collapse overly long gaps, and regenerate single sentences instead of whole paragraphs when a word is wrong. When you splice a regenerated line into the original take, crossfade at a zero crossing and match the level, or you will hear a small click or a level jump.
De-essing, plosives, and breath control
Synthetic voices often carry harder sibilance than human recordings. A dynamic reduction of 2 to 4 dB in the 5 to 8 kHz range, applied only when the sibilant fires, tames it without dulling the whole track. High-pass the voice around 80 to 100 Hz to remove rumble that eats headroom. Manual breath control is usually unnecessary with synthetic voices, but generated breaths can be useful for realism in long-form narration.
Choosing Music That Follows the Edit
BPM and cut rhythm
Music tempo and edit rhythm should agree. At 120 BPM a beat lands every half second, so eight beats is four seconds. If your average shot length is around three seconds, a bed near 90 to 100 BPM will often lock naturally to your cuts. Cut on the beat at section changes and let shots run across beats inside a section. Rigid beat-matching everywhere feels mechanical; matching only at structural moments feels intentional.
Structure and emotional continuity
Pick music whose arc matches the arc of the scene, or use a loopable bed plus separate intro and outro cues. Abrupt genre shifts mid-scene break immersion faster than any visual mismatch. When you must move between two cues, overlap them by one to two beats at a transition point and let one fade under the other rather than cutting hard.
Loopable beds versus fully scored cues
Loops give you control and predictable length but risk monotony. Keep a loop interesting with automation: bring one layer in during the problem statement, add percussion at the turn, strip back to a single element for the closing line. That is a five-minute job that reads as a composed score.
Music that sits under narration should be chosen for how it behaves in the mid-range, not just how it sounds on its own. A dense, busy mix will fight speech no matter how far you duck it.
Mixing Fundamentals for a Voice-Plus-Music Mix
Anchor the voice first
Set your voice track first and mix everything around it. Peak the narration around -6 dBFS during assembly, then lower the whole mix later when you master. Working the other way around, starting with music that sounds great solo, guarantees you will spend the rest of the session fighting it.
Ducking and sidechain automation
A static duck of 12 to 18 dB under narration is a reasonable starting point. Set attack around 10 to 20 milliseconds so the consonants are not clipped, and release around 250 to 400 milliseconds so the music breathes back in naturally instead of pumping. For short-form content with dense speech, manual volume automation often sounds better than a ducking processor, because you can keep the music up in the gaps between sentences.
EQ carving instead of volume fighting
The frequencies that carry speech intelligibility live roughly between 1 and 4 kHz. A gentle 2 to 3 dB dip in the music in that band lets you keep the bed louder without burying the voice. Do the same on the low end: high-pass the music around 60 to 80 Hz unless it is carrying the emotional weight of the scene.
Loudness targets by destination
For web video, an integrated loudness around -16 to -14 LUFS with a true peak ceiling near -1 dBTP is a safe, widely accepted target. Vertical short-form platforms normalize aggressively, so keeping your master consistent and free of clipping matters more than hitting an exact number. Check the mix on a phone speaker before you export; if the narration survives that, it will survive anywhere.
A Repeatable Workflow From Script to Export
Stage 1: Lock the script and pronunciation list
Finalize wording first. Every later stage gets more expensive if the script moves.
Stage 2: Generate section by section
Generate in paragraphs, not in one long pass. You get better control, and a mistake only costs you one paragraph.
Stage 3: Comp the takes
Choose the best take per section and assemble a rough narration track before doing any cleanup. Do not polish a line you might replace.
Stage 4: Clean and level
Apply noise reduction, de-essing, high-pass filtering, and clip gain. Aim for consistent perceived loudness across the whole read, not identical peaks.
Stage 5: Spot the music
Lay the bed against the assembled narration and mark the moments where the music should breathe, drop out, or lift. Spotting before mixing prevents you from rebuilding the mix later.
Stage 6: Mix, check, export
Balance voice and music, apply ducking, verify loudness, listen on phone speakers and headphones, then export a clean master plus a music-free version for reuse.
Sound Design Layers That Add Polish
Beyond voice and music, a thin ambience bed and a handful of transition sounds do most of the perceived production value. Keep ambience low, somewhere around -30 dB, so it glues sections together without becoming audible as a layer. Add a soft swell or reverse effect at major scene changes. Above all, use silence deliberately: pulling everything down for one second before a key line makes the line land.
Common Mistakes and a QA Checklist
Recurring problems worth designing around:
- Generating the entire narration in one pass, then discovering one bad paragraph at the end.
- Choosing music before the edit is locked, so the cue no longer fits.
- Ducking the music so far that the video feels empty between sentences.
- Applying heavy noise reduction to a track that only needed a high-pass filter.
- Exporting without listening on a small speaker.
Before delivery, confirm: narration intelligible at low volume, music consistent in level from start to finish, no clipping on plosives, no audible splice clicks, loudness within target, and a music-only stem exported alongside the full mix.
Choosing Tools Without Locking Your Workflow In
Four categories matter: voice generation, audio cleanup, music sourcing, and a mixing environment. Evaluate each on the same criteria: uncompressed WAV export at 48 kHz, stem export, clear commercial usage terms, batch processing for series work, and the ability to work offline. Prefer tools that export standard files over tools that only exist inside one editor, so you can change one piece of the chain without rebuilding everything.
FAQ
How long should narration be for a two-minute video?
At a comfortable 145 words per minute, roughly 260 to 300 words, leaving room for pauses and a few seconds of music-only opening and closing.
Should music start at the very beginning of a video?
Usually not at full level. Start the bed low or with a single instrument and bring it up after the first line so the opening sentence is never contested.
Can I mix dialogue and music without a full editing suite?
Yes. Any editor that supports volume automation, EQ, and loudness metering is enough. Ducking processors are convenient but optional.
How do I keep an AI voice from sounding robotic?
Vary sentence length, mark emphasis, generate multiple takes, and edit pauses deliberately. Prosody problems are almost always script problems before they are engine problems.
What level should background music sit at under narration?
Start around 15 dB below the voice and adjust by ear. If you can follow the melody without effort while someone is speaking, it is too loud.
Is a music-free export really necessary?
Yes, for repurposing. Localized versions, platform-specific edits, and accessibility variants all become much easier when you have stems rather than a single flattened file.
