Why audio decides whether an AI video feels professional
Audiences forgive visuals far more readily than they forgive sound. A slightly soft render, a background that warps for two frames, a hand that briefly resembles a rake — most viewers either miss it or shrug it off. Audio does not receive that grace. Harsh sibilance, a hollow room tone, robotic phrasing, or a music bed that fights the narration will pull someone out of a scene instantly, often before they can explain what felt wrong.
Part of the reason is biological. The ear is an extraordinarily sensitive timing instrument. It detects a few milliseconds of drift between a mouth shape and a syllable, and it registers level changes of a decibel or two without conscious effort. Vision is spatial and forgiving; hearing is temporal and precise. When you generate video with AI tools, you are handing the viewer a stream of images they will judge loosely and a stream of sound they will judge strictly.
Synthesis also changes where audio problems originate. In a traditional shoot, dialogue noise comes from the room: air conditioning, traffic, a refrigerator hum, a microphone brushing against fabric. In a generated clip, dialogue arrives surgically clean in isolation and then breaks down at the seams. Concatenated takes have inconsistent tone. A voice model may shift timbre between sentences. Ambience generated separately may not match the scene's implied space — a cathedral reverb on a shot of a small office, or a dry closet sound on a wide outdoor vista.
Most audio failures in AI production fall into three buckets:
- Artifacts — metallic de-essing, watery noise reduction, clicks where segments were joined, and clipping from over-loud exports.
- Mismatch — ambience that contradicts the visual space, music that fights the emotional beat, or a voice that does not fit the character on screen.
- Level and dynamics — dialogue buried under music, whisper-quiet narration, or a mix that sounds fine on headphones and inaudible on a phone speaker.
Each bucket has a distinct fix, and the fixes are cheap compared with regenerating visuals. That is the core argument for treating audio as a first-class stage rather than an afterthought.
Build the audio plan before you generate the first frame
Retrofitting good audio onto a finished cut is possible, but it costs time and forces compromises. The cheaper path is to make audio decisions at the same moment you decide on shot composition.
Script, pacing, and voice casting
Read your script aloud at the intended pace before you generate anything. Narration that looks tidy on the page often runs long when spoken, and AI voice models are unforgiving about awkward constructions. Short sentences with clear clauses synthesize better than long sentences with nested asides, because the model has fewer chances to place a stress in the wrong place.
When casting a synthetic voice, decide on three attributes and write them down: register (low, mid, high), pace (measured, conversational, brisk), and texture (warm, neutral, crisp). Then audition at least four options reading the same 30-second passage. Judge them on the hardest line in the script, not the easiest. Numbers, acronyms, and proper nouns are where weak voice models fall apart.
Keep a pronunciation sheet for anything ambiguous: brand names, place names, technical terms, and initialisms. Feeding the same pronunciation notes into every generation keeps consistency across a series, which matters more than most creators expect. A voice that shifts its treatment of the same product name in episode three reads as careless.
Technical baseline: sample rate, headroom, and file hygiene
Set one project standard and never deviate mid-project. A 48 kHz sample rate at 24-bit depth is the safe default for video work, since it matches delivery standards for most platforms and broadcast pipelines. Voices generated at 44.1 kHz can be resampled, but resampling adds a step where things go wrong silently.
Leave headroom. Peaks should sit around -6 dBFS before mixing, never at 0. If a generated voice clip arrives normalized to the ceiling, pull it down before you process it. Compression and saturation behave badly on material that is already pinned to the top, and clipped synthesis sounds noticeably worse than clipped live dialogue because there is no natural detail to mask the distortion.
Organize your audio assets into four folders from day one: voice, ambience, music, and effects. Name files with scene and take numbers, for example s03_voice_take2.wav. When you are 40 minutes into a mix and need the alternate take of a single line, that naming discipline saves the session.
Dialogue cleanup: from synthetic speech to production-ready narration
Noise reduction and de-reverb
Synthetic voices rarely have broadband noise, but they do carry artifacts: faint digital buzz, a hiss on sustained consonants, and occasionally a subtle room simulation baked in by the model. Apply noise reduction gently. Two passes at 30 percent reduction almost always sound better than one pass at 80 percent, because aggressive reduction introduces a watery, phasey quality that is far more distracting than the noise it removed.
If a voice clip has baked-in reverb, treat it as a decision point rather than a repair job. Light de-reverb can pull a clip back toward the front of the mix, but heavy processing on synthetic speech tends to hollow out the lower midrange and make the voice sound thin. If de-reverb is not working after two attempts, regenerate the line with a drier voice setting. Regeneration is usually faster than restoration.
Always work on a copy. Keep the untouched original in your voice folder so you can A/B against it in context rather than in solo.
Plosives, sibilance, and harsh consonants
Plosives — the pops on P, B, and T sounds — show up in generated speech less as true pops and more as low-frequency thumps that make the mix feel muddy. A high-pass filter set between 70 and 90 Hz on the dialogue track removes most of that energy without affecting intelligibility.
Sibilance is the more common problem. De-ess with a narrow dynamic band around 5 to 8 kHz rather than a static EQ cut. A static cut dulls every S in the piece; a dynamic de-esser only acts when the energy crosses a threshold. Aim for 3 to 5 dB of reduction at the peak, then check on phone speakers, where sibilance is exaggerated.
Breath, pause, and rhythm editing
Synthetic narration often has unnatural pause placement. The fix is not to remove every pause but to redistribute them. Cut 100 to 200 milliseconds out of a mid-sentence gap and add it after the sentence instead. The line becomes easier to follow, and the rhythm starts to feel intentional rather than mechanical.
If your voice model inserts audible breaths, keep roughly one in three. Completely breathless narration sounds uncanny over more than about 30 seconds. Where breaths are missing entirely, you can layer in a soft sample at low level, but do it sparingly.
Loudness targets and delivery specs by platform
Integrated loudness, true peak, and dynamic range
The most common technical rejection reason for a video is a mix that is either too quiet or too hot for the platform. Integrated loudness targets have converged around well-known values:
- Broadcast and most streaming platforms: around -24 to -23 LUFS integrated.
- YouTube and similar video platforms: roughly -14 LUFS integrated.
- Podcast distribution: approximately -16 LUFS integrated for stereo, -19 for mono.
- Short-form social: often normalized aggressively, so aim for -14 LUFS with a true peak no higher than -1 dBTP.
True peak matters more than integrated loudness for avoiding distortion after encoding. Set a ceiling limiter at -1 dBTP for any lossy delivery format. That extra decibel of headroom is what prevents crackle when the platform re-encodes your file.
Dynamic range is a creative decision, not a spec. A documentary narration can breathe across 10 LU. A punchy product ad usually wants 4 to 6 LU. Both can hit the same integrated target.
Mono or stereo: choose deliberately
Dialogue is mono. Ambience and music are stereo. That is the standard structure and it exists for a reason: a mono dialogue track sits solidly in the center of the stereo field and stays legible on a phone speaker that sums everything to one channel. Widening dialogue with a stereo imager is a common mistake — it sounds impressive on headphones and collapses badly everywhere else.
Test every mix in mono at least once. If a music bed masks the narration in mono, the arrangement is wrong, not the level.
Ambience, music, and sound design that match the image
Building an ambience bed that sits underneath dialogue
Ambience establishes space. A wide outdoor shot needs air, low-frequency wind, and distant movement. An interior needs a room tone with a defined early reflection. When you generate ambience separately from the visuals, the mismatch is the thing viewers notice, even if they cannot name it.
Start by describing the space in concrete terms: size, surface materials, distance to the nearest wall, and what is happening outside. Then choose the closest matching ambience layer and cut it to the shot length with a gentle 300 to 500 millisecond crossfade at each end. Ambience that starts and stops abruptly is one of the most noticeable amateur mistakes in AI video.
Level ambience 18 to 24 dB below dialogue. It should be felt more than heard.
Music selection, ducking, and pacing
Choose music before you finalize the edit if you can. Cutting to a beat produces a video that feels deliberate, while cutting first and dropping music on top produces a video that feels like two separate things happening at once.
Ducking is essential. Use a sidechain compressor keyed to the dialogue track, with a threshold that engages only when narration is present and a release of 200 to 400 milliseconds. Aim for 4 to 6 dB of ducking. More than that and the music pumps audibly.
Watch for frequency collisions. If the music has a busy synth pad in the 1 to 3 kHz range and the narration sits there too, no amount of ducking will fully solve it. Carve a 2 to 3 dB dip in the music in that band instead.
Spot effects and transitions
Spot effects sell physical reality: a cup touching a table, fabric shifting, footsteps on gravel. Generated clips almost never include them. Add them at 10 to 15 dB below dialogue and align them to the exact frame of contact.
Use transition sounds sparingly. A whoosh on every cut becomes noise. Reserve them for scene changes and structural beats, and vary the pitch so repeated transitions do not feel copy-pasted.
Sync: lips, beats, and visual rhythm
Diagnosing drift and micro-sync errors
Drift is when audio and video gradually separate over the length of a clip. Micro-sync error is when they are consistently offset by a small fixed amount. They look similar and have different fixes. Drift requires time-stretching a segment; a fixed offset requires simply nudging the audio by a few frames.
To diagnose, find a hard consonant — a T or a K — and step through frame by frame. If the mouth closes two frames after the sound, it is a fixed offset. If the offset at the start of the clip is zero and grows to six frames by the end, it is drift.
For generated dialogue, keep clips short. Twenty to thirty seconds per generated segment limits how much drift can accumulate and makes resynchronization trivial. Long single takes are convenient until they are not.
Visual rhythm matters as much as literal sync. Cuts that land on the downbeat feel intentional; cuts that land a quarter-second late feel sloppy even when nothing is technically out of sync.
Quality control checklist before publishing
Run the same checks every time, in the same order. Consistency catches more errors than attention does.
- Listen once at normal volume on studio headphones, taking no notes.
- Listen again on a phone speaker at moderate volume, writing down every moment that is hard to understand.
- Play the whole piece in mono.
- Check integrated loudness and true peak against your delivery target.
- Verify the first two seconds and last two seconds for abrupt entrances or cutoffs.
- Confirm captions match the final audio, not the script.
- Check that no ambience or music layer ends mid-phrase.
- Export, then listen to the exported file rather than the timeline.
That last step catches export-specific problems, including sample rate mismatches and limiter artifacts that only appear after encoding.
Common mistakes and how to catch them early
Over-processing dialogue. Stacking noise reduction, de-essing, EQ, compression, and saturation on the same clip produces a thin, artificial voice. Process in stages and A/B against the original after each one.
Mixing at one volume. A mix that only works loud will fail on mobile. Check at low volume, where balance problems become obvious because quiet elements vanish entirely.
Ignoring the first three seconds. Short-form platforms decide reach based on early retention. If your opening line is buried under an intro music swell, you lose viewers before the content begins.
Uniform ambience across scenes. Using the same room tone for an interior and an exterior flattens the whole piece. Every distinct space deserves its own ambience.
Never checking mono. Dialogue that is stereo-widened or phase-shifted disappears or thins dramatically in mono, and a large share of viewers listen in mono.
Exporting at the wrong sample rate. A 44.1 kHz export delivered into a 48 kHz pipeline can introduce a subtle pitch shift that is hard to identify but easy to feel.
Tool stack: what to use at each stage
A workable stack does not need to be expensive. It needs to cover five jobs: generation, editing, restoration, loudness, and verification.
- Voice generation: any modern text-to-speech tool with adjustable pace and stability controls. Prioritize consistent timbre across a series over exotic voice options.
- Editing and mixing: a DAW or a video editor with a capable audio page. Timeline-based editing is usually faster than waveform-only tools for sync work.
- Restoration: a dedicated repair suite handles de-reverb and spectral cleanup better than a general-purpose EQ chain.
- Loudness: a loudness meter with true peak measurement, plus a limiter set to -1 dBTP for lossy delivery.
- Verification: a phone, a cheap pair of earbuds, and a mono sum. These three reveal more real-world problems than any analyzer.
Build a template session with your dialogue chain, ambience bus, music bus, and metering already configured. Rebuilding the same chain for every project is where consistency dies.
FAQ
How much time should audio take relative to video?
For a two-minute piece, budget roughly 30 to 50 percent of total production time on audio. It sounds excessive until you compare it with the cost of regenerating visuals because the narration did not land.
Can I fix bad generated audio instead of regenerating it?
Sometimes. Light noise, a small offset, or an inconsistent level is fixable. Baked-in reverb, a wrong voice character, or clipped synthesis usually is not. Regenerate when the problem is fundamental.
Why does my mix sound great on headphones and terrible on a phone?
Phone speakers reproduce very little below 200 Hz and sum everything to mono. If your dialogue sits close to the music in the low midrange, it disappears. High-pass the music and check in mono.
Do I need a separate ambience track for every scene?
Yes, if the scenes take place in different spaces. A shared ambience bed is one of the fastest ways to make a polished video feel templated.
What is the single highest-impact audio improvement?
Getting dialogue intelligibility right: consistent level, gentle de-essing, and enough space in the music bed. Most other audio work is refinement on top of that foundation.
Should I master to the loudest possible level?
No. Platforms normalize on playback, so pushing level only costs you headroom and dynamic contrast. Hit the target, protect the true peak, and keep the dynamics you want.
The iteration loop that keeps quality high
Treat audio as a loop rather than a final step. Generate, clean, place, check, and then adjust the generation if the check fails. Each pass should be short: listen, note the single worst moment, fix that moment, listen again. Fixing one problem at a time produces better results than trying to correct everything in a single marathon session, because the ear adapts to the mix and stops hearing its own flaws after about 20 minutes.
Take breaks. Fifteen minutes away from the mix resets your perception more effectively than any plugin, and the problems you could not hear before the break will be obvious after it. That single habit — reset, listen, fix one thing — will do more for the perceived quality of an AI-generated video than any individual tool in your stack.
