Why audio decides whether a video feels professional
Viewers forgive imperfect framing, slightly soft focus, even a shaky phone shot. They rarely forgive bad sound. A hollow room, a narrator who mispronounces the product name, music that swells over the key sentence, or a mix that lurches in volume between cuts — any one of those breaks attention within seconds, and the viewer leaves before the visuals get a chance to work.
Audio is also the layer most creators handle last, after the story, the edit, and the color are locked. By then the schedule is gone, and the temptation is to drop in whatever music loops at roughly the right tempo. AI voice and music generation removes most of the friction from that final step, which is genuinely useful, but convenience is not craft. A synthetic narrator reading an unedited script over a stock-sounding bed produces a video that feels machine-made even when every frame is beautiful.
The remedy is a workflow: decide what each sound layer is doing, prepare the script so a speech model can perform it, choose voices and music against explicit criteria, and mix in a fixed order. Everything below is that workflow, from planning to export.
The three layers of an AI-assisted soundtrack
Every soundtrack is a stack of three layers, and each has a different job. Confusing their roles is the most common reason an AI-generated soundtrack sounds cluttered.
Dialogue carries meaning
Narration and on-screen speech deliver the information. Dialogue must be intelligible at low volume, on a phone speaker, in a noisy room. Nothing else in the mix is permitted to compromise that. If a viewer has to rewind to catch a sentence, the video has failed regardless of how good the score is.
Music carries emotion
Score tells the audience how to feel about what they are watching: hopeful, tense, playful, reflective. Music should support the edit, not narrate it. If the score is busy explaining the joke, the joke is dead. If it swells at the exact moment the narrator delivers the payoff line, the line is buried.
Ambience and effects carry place
Room tone, footsteps, wind, keyboard clicks, a soft whoosh on a transition. These details make a scene feel located rather than floating in a vacuum. Silence between them matters just as much: removing ambience entirely is a legitimate choice, and it reads as deliberate when the rest of the mix keeps it.
A practical rule keeps the stack from collapsing: in any eight-second window, one layer leads and the other two recede. If you cannot name which layer is leading at a given moment, the mix is probably fighting itself.
Writing a narration script that speech synthesis reads correctly
Most disappointing AI narration is not the model's fault. It is the script's. Speech synthesis reads exactly what you give it, including the ambiguities a human narrator would have resolved by instinct. Ten minutes of preparation removes most of that risk.
Normalize numbers, symbols, and acronyms
Write out anything a model might mangle. "1,200" may be read as "one comma two hundred." "2024" might become "two thousand twenty-four" when you wanted "twenty twenty-four." Acronyms are worse: API, ROI, and CEO are usually spelled out letter by letter, but NATO and GIF are spoken as words. Write the pronunciation you want, then remove the crutch text before generating. Currency, units, file sizes, and decimal points all belong in a normalization pass.
Use punctuation as a performance instruction
A period is a full stop, a comma is a short breath, an em dash is a sharper turn, and a paragraph break is a scene change. Ellipses often produce exactly the hesitant trailing-off you want — or a weird mechanical pause, depending on the engine. Test one paragraph before committing to a hundred. If a sentence keeps coming out flat, split it rather than adding emphasis markers.
Break long sentences into breathable beats
Human narrators breathe. Models generate long uninterrupted strings if you let them, and the result sounds uncanny. Keep most sentences under about twenty words, and split compound ideas into two lines even when a single line would be grammatically fine. Short lines are also easier to regenerate individually when one moment lands badly.
Add delivery notes before generating, not after
Decide the tone for each section: warm explainer, urgent promo, calm documentary. Then write to that tone. A script written conversationally will sound conversational. A script written in corporate passive voice will sound like a press release no matter which voice you cast.
Casting and cloning voices without creating problems
Evaluation criteria for a synthetic voice
Run every candidate voice through the same 30-second test paragraph and score it on five things:
- Intelligibility at low volume. Play it through a phone speaker at 30 percent. If consonants smear, reject it.
- Prosody range. Does it vary pitch and pace, or is it a monotone with a smile stapled on?
- Emotional fit. Read the same line three ways and see which the voice handles best; cast the voice for the emotion you need most often.
- Accent and locale. A generic "English" voice is often American. If your audience is elsewhere, check regional variants before assuming the model does not have them.
- Consistency across takes. Generate the same sentence three times. Wide variation means you will fight inconsistency across a long edit.
Consent, contracts, and disclosure
Cloning a real person's voice — a colleague, an actor, a public figure — requires written permission that specifies the project, the duration, and the permitted contexts. A verbal yes in a hallway is not enough. When a client supplies a reference recording, confirm they hold the rights to it. For advertisement and broadcast work, check local rules on synthetic voice disclosure; several jurisdictions require an on-screen or spoken notice, and platforms increasingly enforce their own labeling for realistic synthetic media.
Keep a simple voice bible for every project: chosen voice, settings, reference files, consent documents, and the exact generation parameters used. Reproducing a voice six weeks later without that record is painful.
Generating music that actually fits the edit
Map scenes to emotional targets
Before generating anything, write a one-line emotional target per scene: "quiet curiosity," "building momentum," "relief after resolution." This turns music generation from a slot machine into a search with an answer key. You will know within five seconds whether a candidate fits, instead of clicking through dozens hoping something sticks.
Ask for structure, not just genre
"Lo-fi hip hop" is a search tag, not a brief. Stronger prompts describe instrumentation, tempo, energy curve, and the absence of distracting elements: sparse piano and low strings, 72 BPM, slowly rising energy, no drums until the midpoint, no vocals, no prominent melody that competes with narration. Explicitly requesting "no lead melody" is one of the highest-value phrases for any video with a voiceover.
Loops, stems, and edit points
Generate longer than you need and cut to the music rather than looping a short clip into obvious repetition. Where a tool offers separated stems — drums, bass, harmony, melody — you gain fine control: drop the drums during dialogue, bring them back on the cut. Even a two-stem split between rhythm and everything else is enough to fix a mix that would otherwise be unsalvageable.
Watch for mood whiplash
A hard cut between two very different cues is jarring unless the edit itself is jarring. Crossfade over a beat, or hold one sustained pad underneath both cues so the transition has a floor. Two seconds of overlap fixes what ten minutes of level automation cannot.
Mixing: dialogue first, everything else serves it
Set dialogue as the reference
Bring narration to a comfortable level and mix everything against it. In a typical spoken-word mix, dialogue sits around -12 to -6 dBFS on the meter, with music 12 to 18 dB below it during narration and rising in gaps. If you start with music and push dialogue on top, you will end up with a loud, muddy result every time.
Carve space instead of just lowering volume
High-pass the music — often around 100 to 200 Hz — so it stops competing with the fundamental frequencies of the human voice. A gentle dip of 2 to 4 dB somewhere between 1 and 3 kHz in the music, applied only while narration plays, clears the intelligibility band. Compression or sidechain ducking on the music track, triggered by the dialogue, does this automatically and keeps the music present rather than distant.
Match the acoustic space
If the voice sounds like it was recorded in a small padded room and the ambience sounds like a cathedral, the illusion breaks. Add a touch of the same short reverb to both, or remove reverb from both. Consistency matters more than the specific amount.
Loudness and true peak by destination
Broadcast and streaming targets differ, and a mix that is right for one may be quiet or crunchy on another:
- Web and social: roughly -14 LUFS integrated, true peak no higher than -1 dBTP.
- Podcast and spoken audio: roughly -16 LUFS, with peaks well controlled.
- Broadcast television: around -23 LUFS, with strict true peak limits.
Measure the finished file, not the timeline preview. If a platform normalizes loudness automatically, hitting the target yourself preserves your dynamic choices instead of having them flattened by someone else's algorithm.
A step-by-step production workflow
- Lock the picture first. Generate audio against a finished edit, not a moving one. Every later cut invalidates sync work.
- Normalize the script. Numbers, acronyms, units, and pronunciation notes, in one pass.
- Generate a scratch narration. One voice, no music. Listen for clarity and pace before polishing anything.
- Time the read to the picture. Lines that are two seconds too long are easier to fix by rewriting than by speeding up the model, which usually sounds unnatural.
- Cast the final voice and regenerate section by section. Keep the takes you like; regenerate only the lines that misfired.
- Generate music per scene against your emotional targets. Export stems where possible.
- Lay ambience and effects. Keep them subtle. If you notice them consciously on first listen, they are probably too loud.
- Mix dialogue first, then duck music, then add reverb consistently across layers.
- Check on three systems: good headphones, a phone speaker, and a laptop speaker. The phone check catches buried consonants; the laptop check catches boomy low end.
- Measure loudness and true peak, then export a clean, unclipped master.
Localization: one video, several languages
Multi-language versions are often the strongest argument for AI narration, but they need their own pass. Do not translate word for word. Localized scripts should be rewritten to natural sentence length in the target language, because German compounds, Japanese politeness levels, and Spanish rhythm all change how long a line takes to say. Re-time each version to the picture rather than forcing the picture to fit.
The voice should change too — or at least the accent and locale variant should. Keeping the same synthetic voice across eight languages is convenient and rarely convincing. Finally, check every localized number, date, unit, and proper noun by ear. Models handle them inconsistently, and a mispronounced city name is the fastest way to lose a regional audience.
Common mistakes and how to fix them
- Music overpowers the voice. Duck it, high-pass it, and delete any lead melody that competes with the narration band.
- Every line at the same energy. Vary pace and pitch section by section; flat delivery is a script and direction problem more often than a model problem.
- Robotic pacing in long sentences. Split them. Two short lines almost always beat one long line.
- Obvious loop points. Generate a longer bed and cut to the music instead of repeating a short clip.
- Sudden loudness jumps between scenes. Normalize narration clips to a consistent target before mixing, not after.
- No headroom on export. Leave at least 1 dB of true peak margin, or encoders will distort.
- Generic stock-sounding score. Describe instrumentation and energy curve, not just genre, and favor sparse arrangements for dialogue-heavy sections.
FAQ
How long does an AI-assisted soundtrack take?
A five-minute explainer typically takes two to three hours for a careful first pass: script normalization, narration generation and regeneration, music selection per scene, mixing, and loudness checks. Rushing to fifteen minutes costs far more time in fixes later.
Should I always use AI narration instead of a human voice?
No. Use a human narrator when performance, humor, or subtle emotion is the point, and use synthetic narration for scale, iteration, multilingual versions, and information-dense explainers. Many strong videos combine both: a human host on camera, synthetic voice for inserts and updates.
Can I fix a bad narration take by speeding it up?
Rarely. Time-stretching tends to introduce unnatural artifacts. Rewrite the line shorter, change the punctuation, or split it into two sentences and regenerate.
Do I need a special audio editor?
Free or low-cost editors handle everything described here: multitrack mixing, EQ, compression, ducking, loudness metering, and true peak limiting. Skip anything that cannot display an integrated loudness reading.
How do I keep quality consistent across a series?
Write down the voice, settings, music prompt style, dialogue level, and loudness target in a one-page template, then reuse it for every episode. Consistency across episodes is what makes a series feel professional, more than any single episode's polish.
What about tracking usage limits and rights?
Check the terms of whichever generation tools you use for commercial use, attribution, and the scope of any voice cloning you perform, and keep the records with the project files. Rights questions are easiest to answer when the documentation already exists.
Good sound does not draw attention to itself. When the workflow above is running, viewers simply understand the message, feel the intended emotion, and stay to the end — which is the only test that matters.
