Multi-Image Fusion: Keep Characters Consistent Across Scenes

Why Characters Drift in AI Video

A generated frame is a fresh guess. Even with a locked seed, every new shot hands the model a new set of conditions to satisfy: new camera angle, new lighting, new background, new pose. Text is a lossy way to describe a person. A line like late twenties, olive skin, short dark hair, small scar above the left eyebrow leaves thousands of decisions to the sampler, and the sampler makes those decisions differently on every render.

The result is character drift: the slow or sudden mutation of a face across frames and cuts. A jawline widens. Freckles vanish. A jacket shifts from olive to teal between two shots that are supposed to be seconds apart. Viewers read faces faster than they read anything else in a frame, so drift is the single most damaging artifact in AI video. It never reads as a stylistic choice. It reads as a continuity error.

Drift has three root causes worth separating, because each one has a different fix:

Prompt ambiguity. The model invents details you never specified, and invents different ones each time.
Independent generation. Each shot is conditioned on its own prompt, with nothing anchoring it to the shots around it.
Competing pressures. Style, lighting, motion, and camera conditioning all pull on the same latent space, and identity is usually the weakest signal of the group.

Multi-image fusion attacks the second and third causes directly and reduces the first by handing the model evidence instead of adjectives. That is the entire premise: stop describing the character, start showing the character.

What Multi-Image Fusion Really Does

Multi-image fusion means conditioning a generation on several reference images of the same subject at once, rather than on a single image or on text alone. Instead of one anchor frame competing against a long prompt, the model receives a small cluster of the same face from different angles and blends the identity cues it extracts from all of them into the output.

The practical effect is that identity stops being a suggestion and becomes a constraint. You are no longer asking for someone who looks approximately like this. You are supplying the visual evidence the sampler needs to reconstruct the same person under new conditions.

Reference Images vs. Text-Only Conditioning

Approach	Identity stability	Setup effort	Flexibility
Text-only prompt	Low to moderate	Minimal	Very high
Single reference image	Moderate	Low	High
Multi-image fusion	High	Moderate	Medium-high

The trade-off is real: as you tighten identity, you loosen the model's freedom to reinterpret the scene. That is usually the right trade for narrative work and usually the wrong trade for abstract or experimental sequences where the face is not the point.

The Three Layers: Identity, Style, Motion

The cleanest mental model is to treat identity, style, and motion as three separate layers that you control independently and merge at the end.

Identity comes from the reference set. It should be as consistent as possible in lighting and wardrobe so the model averages a stable face rather than a blurred average of several different looks.
Style comes from a look-dev frame or a color grade applied after generation, not from the identity references. If your references are shot in flat daylight and your scene is night neon, do not fix that with references. Fix it with lighting language and a grade.
Motion comes from keyframes, depth or pose guidance, and camera instructions. Keep the motion budget proportional to how strongly you are holding identity.

What Fusion Does Not Fix

Be honest about the edges of the technique. Multi-image fusion will not rescue hands in complex occlusion, extreme profile angles where no reference covers that side of the face, or wardrobe changes mid-shot that you never modeled. It also will not match a character to a real person's likeness with any legal safety, which is a rights problem rather than a technical one.

Building a Reference Set That Holds Up

The reference set is the foundation. A weak set produces a character who looks like a sibling of your character, which is worse than obvious drift because it is easy to miss until the edit is nearly done.

Coverage Checklist

8 to 15 images total, all of the same person under the same lighting setup
Front, three-quarter left, three-quarter right, and at least one profile
Four to six close portraits for facial detail
Two to four medium shots for shoulder line, hair volume, and posture
One or two full-body frames for proportion and silhouette
Neutral expression plus the two expressions the character uses most
One wardrobe set per scene group, kept in separate folders

Aim for images at 1024 pixels on the short side or larger, with clean edges and no heavy filters, watermarks, or compression mush. If a detail matters in the final video, it must be visible in the references.

Preprocessing and Cleanup

Spend twenty minutes here and save hours later. Crop references to a consistent aspect ratio so the model is not learning framing as identity. Remove busy backgrounds where you can, since a strong background pattern in the references can bleed into generated scenes. Deduplicate near-identical frames, because ten copies of the same selfie teach the model nothing new and skew the average toward one angle. Finally, name files by angle and expression so you can swap subsets quickly when a scene needs a different emphasis.

Common Reference Mistakes

Mixing lighting setups. References from a sunny afternoon and a dim indoor shoot average into a face that matches neither.
Frontal-only sets. Profiles collapse because the model has never seen the character from the side.
Low-resolution sources. Detail the model cannot see is detail it invents.
Third-party faces. Reference images of a real person you do not have rights to use are a licensing problem waiting to surface.
Too many expressions at once. If every reference is mid-laugh, every generated shot will look mid-laugh.

The Fusion Workflow, Step by Step

This is a repeatable pipeline you can run on any project, from a thirty-second teaser to a multi-episode series.

Step 1: Lock the Character Sheet

Build the reference set, review it as a contact sheet, and remove any frame that does not feel like the same person. Generate a handful of test portraits with the set before you commit. If the test portraits already drift, the set is the problem, not the prompt.

Step 2: Produce a Look-Dev Frame Per Scene

Before animating anything, generate one still that captures the location, time of day, lens character, palette, and key light direction. Approve it. That frame becomes the style reference for every shot in that scene and the visual baseline for your grade at the end.

Step 3: Build the Shot List as Keyframes

Generate still keyframes for every story beat before generating a single second of motion. This is the most important habit in the whole workflow. Stills are cheap and fast; video is slow and expensive. If a keyframe does not look like your character, no amount of motion settings will save it.

Step 4: Generate Short Clips

Three to six seconds per shot is the sweet spot. Use the approved keyframes as start and end frames where the tool supports it, keep camera movement modest, and keep the number of moving elements in frame low. Short clips stay coherent; long clips give drift more time to accumulate.

Step 5: Stitch, Compare, Repair

Assemble the timeline, then run a continuity pass at full speed and again frame by frame on every cut. Fix outliers locally with inpainting, face restoration, or regeneration of that single shot. Do not re-render the whole sequence because one shot drifted; you will lose the shots that were already working.

Keyframe Control and Motion Direction

Keyframes do two jobs at once: they define the pose and composition the shot must respect, and they give the model a fixed target to fuse identity toward. When a shot has both a start frame and an end frame, the model has a corridor to travel through, which dramatically reduces identity wander.

Practical guidance:

Prefer start and end frames over start-only frames for any shot with dialogue or a large pose change.
Use depth or pose guidance when the character must interact with an object or another character.
Keep the motion budget small per shot. A slow push-in holds a face far better than a whip pan.
When identity strength is high, reduce motion amplitude. High identity weight plus large motion produces warping and rubbery faces.
Lock the seed per shot so a re-render with a tweaked prompt changes one variable at a time.

If a shot needs a big camera move, consider cutting it into two shorter shots with an edit rather than generating one long sweeping take.

Cross-Scene Continuity: Wardrobe, Light, Props

Identity is only one continuity thread. Audiences forgive a slightly different shadow far more readily than a different shirt, so build a continuity table before you render and check every element against it.

Element	Where it lives	How to control it
Face and hair	Reference set	Multi-image fusion, identity weight
Wardrobe	Separate reference folder	Its own reference set or locked descriptive token
Lighting	Look-dev frame	Style reference plus final grade
Location	Set references	Environment plate per scene
Props	Prop references	Separate small reference set per hero object
Color and grain	Post-production	LUT and grain applied after generation, never before

Apply the grade after generation for two reasons. First, baking a look into the render forces the model to fight for identity through a color shift. Second, a single grade across all scenes is what makes disparate AI shots feel like one film.

Choosing Between Fusion, Fine-Tuning, and Text-Only

Method	Setup cost	Identity strength	Flexibility	Best for
Text-only prompting	Minutes	Low	Very high	Mood pieces, backgrounds, one-off shots
Single reference image	Minutes	Moderate	High	Quick tests, secondary characters
Multi-image fusion	Hours	High	Medium-high	Recurring characters, dialogue, series work
Subject fine-tuning	Days plus compute	Very high	Medium	Long-running characters used across many projects

A pragmatic sequence for most teams: start with fusion, and only invest in fine-tuning when the character will appear in dozens of shots across multiple projects. Fine-tuning gives the most stable identity but is the least forgiving when you want to change wardrobe, age, or style. Fusion gives you ninety percent of the stability at a fraction of the setup time.

Troubleshooting Common Failure Modes

The face morphs between clips

Usually a seed and reference-weight problem rather than a model limitation. Lock the seed per shot, shorten clips to three seconds, and raise identity weight while lowering style weight. Then check whether the references themselves are inconsistent.

The character looks like a relative, not the same person

Prune the reference set down to a single lighting setup and select for facial structure rather than wardrobe or pose. A tight, consistent set of eight images beats a sprawling set of thirty every time.

Wardrobe flickers

Move wardrobe out of the identity layer and into its own reference set or a strongly weighted descriptive token. If a costume change matters, treat it as a separate character variant with its own folder.

Style drifts scene to scene

Use one approved look-dev frame per scene as the style anchor and unify everything with a single grade at the end. Mixed styles across scenes almost always come from mixed style references.

Motion looks stiff or warped

Reduce identity weight or reduce motion amplitude in that shot. High identity weight leaves little latent room for movement, so the model warps the face to satisfy both demands.

Hands and props break

Shorten the shot, slow the motion, and inpaint the problem area. Hands and small props are still the weakest part of any generative video pipeline, and local repair is almost always faster than regeneration.

Worked Example: A Three-Scene Teaser

Suppose you are producing a thirty-second teaser with one lead character, Maya, across three locations: a coffee shop in daylight, a rainy street at night, and a rooftop at dusk.

Character build. Twelve references: six portraits across three angles, four medium shots in the neutral wardrobe, two full-body frames. One consistent soft daylight setup.
Look development. Three approved stills, one per location, each establishing palette, key light direction, and lens feel.
Shot list. Fourteen keyframes mapped to the beats: six in the coffee shop, five on the street, three on the rooftop.
Clip generation. Fourteen clips of two to four seconds, seeds locked per shot, identity weight high in the coffee shop and slightly lower on the rooftop where atmospheric haze needs room.
Continuity pass. Freeze on every cut and check face landmarks, hair volume, jacket, and screen direction. Repair three shots locally.
Grade. One LUT, one grain pass, one output. The result reads as a single film even though fourteen separate generations produced it.

Total generation time matters less than the order of operations. Stills first, motion second, grade last. Every shortcut that skips a stage shows up as drift somewhere in the final cut.

FAQ and Pre-Render Checklist

How many reference images do I actually need? Eight to fifteen well-chosen frames is the practical range. Below eight, coverage gaps appear at profile angles. Above twenty, you are mostly adding redundancy that blurs the identity average.

Can I get away with one reference image? Yes for short, low-stakes shots. Expect visible drift the moment the character turns or the lighting changes.

Does multi-image fusion work for stylized or animated characters? It works well, and sometimes better than photorealism, because stylized faces have fewer micro-details to lose. Keep the art style of your references consistent, or the fusion will blend styles as well as identity.

Do I need to fine-tune a model to get real consistency? Not for most projects. Fusion handles recurring characters well across a single production. Move to fine-tuning when the same character must look identical across many separate projects over a long period.

How do I keep a character consistent between two different tools? Treat the reference set as the source of truth and export it in both directions, then normalize the look with your grade. Cross-tool matching is never pixel-perfect; the grade is what hides the seam.

What about rights and likeness? Only generate people you have permission to depict. Using reference images of a real public figure, a client, or a stranger without consent is a legal exposure regardless of how convincing the output is.

Pre-render checklist:

Reference set reviewed as a contact sheet and pruned
One look-dev frame approved per scene
Keyframes generated and approved before any video render
Seed locked per shot, one variable changed per iteration
Clips kept to three to six seconds
Continuity table checked at every cut
Grade and grain applied after generation, not baked in
Local repair used instead of full re-renders

Run that checklist on every project and character consistency stops being a gamble. The technique is not a magic switch; it is a production discipline that treats the face as a fixed asset and everything else as variable. Once that framing clicks, multi-image fusion becomes the default starting point for any AI video work with a recurring cast.