Generative video has crossed the line from novelty to production tool. Small teams now build entire ad campaigns, explainer series, and short narrative films from a handful of reference images and a script. Yet two stubborn problems still consume the largest share of production time: the character changes subtly between shots, and the same footage refuses to fit cleanly into the many aspect ratios a campaign demands.
Both problems share a root cause. Most pipelines treat every shot as an independent creative act rather than a frame inside a continuous visual system. The character in shot three gets described again from scratch, and the 16:9 master is squeezed into a 9:16 slot with a blunt crop. The result looks almost right, which is the worst possible outcome — viewers notice the uncanny shifts even when they cannot name them.
This guide lays out a practical, tool-agnostic workflow for solving both problems together. It covers how to build a reference kit, how to choose between text-to-video and image-to-video paths, how to structure master frames so resizing stops being destructive, and what to do when drift has already crept into a render. It is written for solo creators and lean studios that need repeatable results rather than one-off lucky generations.
The Two Failure Modes, Defined
Before fixing anything, it helps to name what is actually breaking. "The AI is bad at faces" is not a diagnosis. It is a symptom description that leads to random tinkering.
What image resizing actually breaks
Resizing in AI video is not the same as resizing a photograph. When you force a generated sequence into a new aspect ratio, you are usually doing one of four things: cropping, letterboxing, outpainting, or regenerating at the target ratio. Each one damages something different.
Cropping destroys composition. A two-shot framed for widescreen loses one actor entirely when cut to vertical. Letterboxing destroys immersion on mobile, where black bars eat a third of the screen. Outpainting preserves the subject but can hallucinate new background details that contradict the previous shot. Regenerating at the target ratio is the cleanest option visually, but it re-rolls the dice on the character's face, hair, and wardrobe.
The real damage, though, is relational. When shot one is framed wide and shot two is framed close because of a ratio change, the viewer's sense of continuity collapses. Scale relationships — how big a character is relative to the room, how much headroom they have, where their eyeline sits — carry more continuity information than facial detail does.
Why character consistency fails
Character drift comes from three sources that are frequently confused with each other.
The first is model variance. Every generation is a sampling event. Even with an identical prompt and seed, small changes to resolution, motion strength, or frame count can shift facial geometry. This is not a bug; it is how diffusion-based systems work.
The second is reference dilution. If you feed the model a different reference image for every shot, you are effectively asking it to draw three different people who happen to share a name. Reference images must be reused, not reinvented.
The third is prompt drift. Writers naturally vary language to avoid repetition: "the detective walks into the room" becomes "the investigator enters the office." To a language-conditioned model, those are meaningfully different prompts, and it will render meaningfully different clothing, lighting, and posture.
How the two problems feed each other
Resizing and consistency amplify one another. Say you generate a strong vertical clip for a social cut. Later you need a horizontal version for a website hero banner. You outpaint the sides, and the model extends the background but also subtly reinterprets the character's jawline at the new resolution. Now the character looks slightly different between the two deliverables of the same campaign.
The fix is architectural, not cosmetic: generate once at the highest practical fidelity, derive all other formats from that single master, and never let a downstream format become a new source of truth.
Build a Reference Kit Before You Generate Anything
Most consistency problems are decided before the first frame renders. A reference kit turns a vague character idea into a fixed visual contract.
The four documents every project needs
Start with a character sheet: front, three-quarter, and profile views of the same person in neutral lighting, plus a close-up of the face. These images become the anchors you reuse in every shot.
Second, a wardrobe and prop sheet. If the character wears a red jacket with a specific zipper pull, that detail needs to appear in the reference set. Models reconstruct clothing from whatever evidence they are given, and verbal descriptions of garments are notoriously unreliable.
Third, a lighting and palette board. Film stills or your own renders that establish key light direction, contrast ratio, and the dominant colors of the scene. This is what stops episode one from looking like a sunny afternoon and episode three from looking like a cloudy dusk.
Fourth, an expression and pose board for any shot where the character does something emotionally specific — crying, laughing, sprinting, sleeping. Asking a model to invent a complex expression from text alone is one of the fastest routes to a face that no longer matches.
Reference hygiene rules
Keep every reference at the same resolution and the same color space. Mixed source resolutions are a frequent hidden cause of resizing artifacts, because the model receives contradictory scale cues. Trim backgrounds away from character references; a busy background in the anchor image will leak into unrelated scenes.
Label files consistently and version them. A kit that lives as hero_v3b_final_final.png will not survive a week of collaboration.
When to commission vs. generate references
If your character appears for more than thirty seconds of screen time across multiple shots, generate the sheet yourself using an image model and then clean it up in a raster editor. Spending an hour retouching eyes and hairline in a reference image saves many hours of rejected video renders later.
Choosing the Right Generation Path for Consistency
Not every shot deserves the same technique. Matching the method to the shot type prevents most drift.
Text-to-video: fast, flexible, fragile
Text-to-video is excellent for establishing shots, landscapes, abstract transitions, and anything where a specific human face is not the subject. It is the worst choice for close-ups of a recurring character. Keep it for scenes where identity is not the payload.
Image-to-video and first/last frame control
Image-to-video anchors the first frame, which locks the character's appearance at the start of the clip. Even better is explicit first-and-last-frame control: you supply the beginning and ending pose, and the model interpolates motion. This gives you two identity anchors per clip instead of one and dramatically reduces mid-clip drift.
For dialogue scenes, generate a high-quality still of the character in the exact framing you want, then animate it. It is slower per shot but far more predictable.
Motion transfer and performance reuse
When a shot depends on precise body movement — a dance, a fight, a specific gesture — motion transfer from a reference performance keeps the choreography intact while another model supplies the appearance. Use it when the movement matters more than photoreal skin detail.
The hybrid rule of thumb
Use the cheapest path that satisfies the shot's identity requirement. Establishing shot with no face: text-to-video. Character medium shot: image-to-video from a kit reference. Character close-up with dialogue: first/last frame with a retouched still as the anchor.
Fixing Resizing: Master Frames, Safe Zones, and Upscaling Order
Resizing stops being painful once you stop treating each aspect ratio as a separate creative project.
Generate a master, derive the rest
Pick one primary aspect ratio — usually the widest one you will deliver — and generate everything there at the highest resolution your workflow can sustain. Every other format is derived from that master. A 21:9 or 16:9 master can be safely cropped to 1:1 and 9:16 if the framing was planned for it. A 9:16 master cannot be expanded into widescreen without inventing content.
Design for the narrowest crop
Before you lock a composition, overlay the narrowest target ratio on your viewport. Keep faces, hands, and key props inside that inner rectangle. This is the same safe-zone discipline broadcast editors have used for decades, and it works identically for AI video. If the subject's eyes sit outside the vertical safe zone, the social cut will decapitate them.
Outpaint before you upscale
Order of operations matters enormously. If you upscale first and outpaint second, the model extends the frame at high resolution and often produces mismatched grain, sharpness, and noise patterns along the seam. Outpaint first at native resolution, then upscale the completed frame. The resulting edge is seamless because both regions were generated under the same conditions.
Consistency across derived formats
When you outpaint, reuse the same character reference and the same seed where the tool allows it. Then compare the derived frame to the master side by side at 100% zoom, focusing on the jawline, the ear, and the hairline. Those three zones reveal drift faster than anything else.
Handling resolution mismatch between sources
If your reference images are 1024 pixels wide and your target is 4K, do not simply feed the small image in and hope. Upscale the reference first with a detail-preserving upscaler, then generate. Models inherit the sharpness of their input; a soft anchor produces a soft character, and no amount of final upscaling recovers detail that was never described.
A Step-by-Step Workflow From Script to Multi-Format Delivery
Here is the sequence that consistently produces stable results.
Step 1: Script breakdown by identity load
Go through the script and tag every shot as low, medium, or high identity load. Low means no recurring face. Medium means the character appears but is not the focus. High means a close-up, a hero moment, or a dialogue beat. This tag determines the generation path for each shot.
Step 2: Lock the look with a single test shot
Before generating a whole scene, produce one high-identity shot and approve it. That approved frame becomes your visual contract. Every subsequent shot is compared against it, not against your memory of the idea.
Step 3: Generate at master ratio and resolution
Render all shots at the master ratio. Resist the temptation to preview in vertical just because the client wants social cuts; previews become sources of truth by accident.
Step 4: Assemble a rough cut before polishing
Edit the sequence together at low resolution first. Continuity errors are far easier to spot in motion than in isolated clips. Watch the rough cut twice: once for story, once purely for face and wardrobe continuity.
Step 5: Repair, then derive formats
Only after the master cut is approved should you begin deriving alternative aspect ratios. Derive them in batch so that the same reference set and settings apply to every clip.
Step 6: Deliver with a format matrix
Maintain a simple matrix listing each deliverable, its ratio, resolution, duration cap, and safe-zone rules. This prevents the classic late-stage discovery that the vertical cut needs two extra seconds of headroom that the master never contained.
Prompt, Seed, and Keyframe Discipline
Small habits prevent most drift, and they cost nothing.
Freeze your character description
Write one canonical description of your character and paste it verbatim into every prompt. Do not paraphrase. If the description says "short dark curly hair, olive skin, faded green canvas jacket," never write "curly-haired man in a green coat." Consistency in language produces consistency in pixels.
Reuse seeds and settings
When a tool exposes a seed, record it alongside the shot. Reusing a seed with a slightly adjusted prompt produces far less drift than a fresh seed with the same prompt. Keep a settings log with resolution, motion strength, frame count, and guidance values for every approved shot.
Keyframe injection for long sequences
For anything longer than a few seconds, break it into segments and inject keyframes at each boundary. Generating a ten-second clip in one pass invites mid-clip identity decay; generating three overlapping segments and blending them keeps the face stable.
Negative prompts as guardrails
Maintain a standing negative prompt list for recurring problems: extra fingers, plastic skin, warped jewelry, changing eye color, drifting hair length. Update it whenever a new artifact appears twice.
Post-Production Repairs When Drift Already Happened
Sometimes you inherit footage rather than generate it. Here is how to salvage a sequence.
Face replacement and detail compositing
If one shot has the wrong face, isolate it and replace the head region with a matched frame from an approved shot. Match color temperature, grain, and motion blur before blending. This is faster than regenerating and preserves the original performance.
Color matching across shots
Drift is often mistaken for identity drift when it is really color drift. Apply a consistent grade — a shared LUT or a match-to-reference tool — before you decide a face is wrong. Many "wrong face" problems resolve after the grade is unified.
Warp and stabilize for resized footage
When a derived crop shifts the subject's position between shots, use a stabilization pass with subject tracking to lock the eyeline to a consistent screen position. Stable eyelines read as continuity even when the underlying geometry has changed.
Knowing when to regenerate
Regenerate when the identity error is structural — wrong face shape, wrong age, wrong wardrobe. Repair when the error is tonal — slightly different lighting, minor color shift, small framing difference. Repairs are cheap; regeneration resets everything downstream.
Common Mistakes and Tool Selection Criteria
Mistakes that cost the most time
Generating in the final delivery ratio instead of a master ratio is the single most expensive habit. So is using fresh reference images per shot, describing the character differently in each prompt, and upscaling before outpainting. Another frequent error is approving a shot on a laptop screen at 50% zoom; drift hides at low magnification and reveals itself on a phone.
Teams also over-correct. They clamp motion to near-zero to protect the face, producing lifeless footage. A better trade is to allow expressive motion in low-identity-load shots and reserve rigid control for close-ups.
Criteria for choosing your stack
Evaluate tools on five axes: reference fidelity (how well it respects an anchor image), frame control (first frame, last frame, keyframe injection), ratio handling (native multi-ratio output versus cropping), seed stability, and export flexibility. Prioritize reference fidelity and frame control above raw visual quality, because a beautiful model that ignores your anchor image will cost more time than a plainer one that obeys it.
Also check licensing for commercial delivery and whether the tool stores your references, since character sheets often contain talent likenesses.
FAQ
How many reference images does a character need?
Three to five is the practical sweet spot: a clean frontal portrait, a three-quarter view, a profile, and one full-body shot showing wardrobe and proportions. More than eight rarely improves results and slows every generation.
Should I generate vertical first for social media?
Only if vertical is your sole deliverable. If you need both, always generate the widest master first and derive vertical from it. Expanding a vertical master into widescreen requires outpainting large regions that frequently disagree with the original scene.
Why does my character's face change after upscaling?
Upscalers reconstruct detail from statistical priors rather than from your reference. If the source frame is soft or low-resolution, the upscaler invents features. Outpaint and finalize the frame at native resolution, then upscale once as the last step.
Can I fix consistency without regenerating anything?
Yes, up to a point. Face replacement from an approved shot, unified grading, and eyeline stabilization solve minor drift. Structural errors like a changed face shape or wardrobe cannot be repaired convincingly and should be regenerated.
How long should a single generated clip be?
Keep individual generations short — two to five seconds — and assemble longer sequences in the edit. Shorter clips drift less, give you more control points, and let you swap an individual segment without rebuilding the whole scene.
Does a fixed seed guarantee the same character?
No. A fixed seed reduces variance but does not eliminate it, because changing the prompt, resolution, or clip length changes the sampling path. Seeds are a variance reducer, not a guarantee; the reference image remains your primary anchor.
How do I keep dialogue scenes consistent?
Generate a still of the character in the exact framing first, retouch it until it matches your approved look, then animate it with first-frame or first-and-last-frame control. Keep camera movement minimal in dialogue coverage, and let cuts carry the pacing instead.
Treating consistency and resizing as one planning problem rather than two technical emergencies is what separates a smooth production from a week of re-rolls. Lock your references, generate wide, derive narrow, and repair only what is genuinely broken.
