A character who looks like the same person in shot one and shot forty is not a rendering trick. It is a pipeline. Multi-image fusion is the part of that pipeline that turns a handful of stills into a stable identity the video model can reuse, shot after shot, without drifting into a different face halfway through the scene.
This guide walks through the whole workflow: how identity information is extracted from multiple references, how to build a reference set that survives different camera angles and lighting setups, how to control keyframes so motion stays believable, how to choose models that respect identity, and how to diagnose the specific failures that break continuity.
Why character drift happens in the first place
Most generative video models do not have a persistent memory of a person. They generate each frame or each short clip as a fresh inference problem, conditioned on whatever the prompt, the reference image, and the previous frames happen to contain. When those conditioning signals are thin or contradictory, the model fills the gap with plausible invention. That invention is exactly what drift looks like on screen.
Three forces cause most continuity failures:
Under-specification. A single reference photo encodes one angle, one lighting condition, and one expression. The moment the script asks for a profile view at dusk, the model has no data to work with and improvises.
Competing signals. If your reference set includes a soft studio headshot and a harsh flash photo, the model receives contradictory information about skin tone, shadow depth, and bone structure. It averages them into a face that matches neither.
Temporal accumulation. Small errors compound. A two percent shift in eye spacing in the first clip becomes a visibly different person by clip six, especially when clips are generated independently and stitched together later.
The practical conclusion is that consistency is a data problem before it is a model problem. Fix the inputs and most drift disappears.
What multi-image fusion actually does
Multi-image fusion is the process of combining several reference images of the same subject into a single identity representation that conditions generation. Different tools implement it differently, but the underlying stages are consistent.
Feature extraction across multiple views
The system analyzes each reference image and pulls out identity-relevant features: facial geometry, interpupillary distance, jawline shape, hairline, eyebrow density, skin texture, and persistent marks like freckles or scars. With multiple angles, the extracted feature set becomes three-dimensional rather than flat, which means the model can rotate the identity in its latent space instead of guessing.
Weighted merging and conflict resolution
Not all references deserve equal weight. A sharp, evenly lit, front-facing image should dominate. A blurry candid should contribute texture information at most. Good fusion gives you control over weighting, either through explicit reference ordering or through prompt syntax that signals which image matters most.
Identity conditioning at generation time
Once merged, the identity representation is injected into the generation process alongside your motion prompt. This is the difference between "a woman in a red coat walking" and "this woman in a red coat walking." The first produces a stranger; the second produces your character.
Persistence across shots
The final benefit is reuse. A well-built identity representation can be applied to dozens of shots, different environments, and different camera distances without rebuilding the character from scratch. That is what makes serialized content — episodic series, product stories with a recurring presenter, narrative shorts — practical rather than theoretical.
Building a reference set that survives every shot
The quality ceiling of your entire project is set here. A weak reference set cannot be rescued by prompt engineering later.
Cover the angles you intend to shoot
Ask what the script actually requires, then collect references that match. A typical minimum viable set:
- One straight-on, neutral-expression hero shot with even lighting
- One three-quarter view, slightly turned, same lighting
- One full-body shot showing proportions and default wardrobe
- One profile or near-profile view for turning shots
- One shot in the dominant scene lighting (interior warm, exterior overcast, night neon)
If your story includes a character running, add a dynamic reference. If it includes them at a desk under a monitor glow, add a reference with cool top-light on the face. Every lighting condition you plan to shoot is a condition the fusion model should have seen at least once.
Keep wardrobe and hair deliberately boring in the reference set
Busy patterns, reflective jewelry, and flyaway hair create high-frequency detail that models reproduce inconsistently. Lock a simple silhouette in the references, then describe wardrobe changes in the prompt or swap in wardrobe-specific references per scene. This separates identity from costume, which is much easier to control.
Normalize the inputs
Before fusion, run a short preparation pass on every image:
- Crop tightly enough that the face occupies a meaningful share of the frame.
- Fix exposure so no reference is dramatically darker or brighter than the others.
- Remove other people and distracting background elements.
- Upscale low-resolution images rather than feeding blurry inputs.
- Convert to a consistent aspect ratio to avoid letterboxing artifacts.
Ten minutes of preparation routinely saves hours of regeneration.
Watch for conflicting identities
If two references are actually two different people — a model, a stunt double, a stock photo mixed in by mistake — fusion will produce a hybrid face. Audit the set visually by placing all references side by side at thumbnail size. If one does not look like the same human, remove it.
The fusion workflow, step by step
Here is a repeatable sequence you can run on any identity-capable video tool.
Step 1: Write a character bible
Before touching the tool, write down the non-negotiables: age range, ethnicity, hair color and length, eye color, build, default wardrobe, and two or three distinguishing features. This document becomes your audit checklist. When drift appears in output, you compare against the bible rather than against memory.
Step 2: Generate a base identity sheet
If you do not have real reference photos, generate a consistent character sheet first using an image model. Ask for a grid of the same person in six views. Iterate until all six cells read as one human. Then split the grid into individual reference images.
Step 3: Fuse the references and test cheaply
Load the reference set into your fusion step. Then run the cheapest possible validation: still images. Generate five stills of the character in different poses and lighting. Look at the eyes, the nose bridge, and the hairline. If those three hold, video generation will usually hold too.
Step 4: Keyframe the sequence
Do not generate a two-minute video in one pass. Generate key poses — the visual beats of the scene — as stills using the fused identity. This gives you a storyboard that already contains the correct face. Only then move to motion.
Step 5: Generate in short blocks
Render motion in clips of a few seconds each, using the previous clip's final frame as the starting frame of the next where your tool supports it. Short blocks limit how far drift can compound before you catch it.
Step 6: Re-fuse between scenes
When the scene changes location or lighting dramatically, rebuild the identity conditioning using scene-appropriate references. A character lit by candlelight needs a candlelight reference, not a studio headshot.
Keyframe discipline and scene-level continuity
Keyframes are the strongest lever you have after the reference set. Think of them as anchor points that constrain interpolation. The model is free to invent motion between anchors, but the anchors themselves are fixed.
Practical rules that prevent most continuity breaks:
- Place a keyframe whenever the character changes direction, changes distance from camera, or changes emotional state.
- Never place two keyframes so far apart that the model has to invent an entire body rotation.
- Keep camera movement and character movement in separate clips when possible; simultaneous complex motion increases the chance both degrade.
- Inspect the first and last frame of every clip against the keyframe before accepting the clip.
Scene-level continuity also includes things fusion does not cover: wardrobe state, prop position, and time of day. Keep a simple continuity sheet per scene listing what has changed. Models will not remember that the jacket came off in the previous scene unless you say so.
Choosing models and tools for identity work
Not every model respects reference conditioning equally. Evaluate candidates against these criteria rather than against marketing copy.
Reference count and weighting. How many images can you supply, and can you control their relative influence? Two-image support is often too thin for a full scene.
Identity strength versus motion freedom. Some models lock identity so hard that motion becomes stiff. Others prioritize motion and let faces slide. Test both extremes on the same prompt and pick your balance per project.
Frame-to-frame stability. Generate a static shot and watch for flicker. A model that jitters on a locked-off shot will jitter worse on a moving one.
Resolution and aspect ratio flexibility. Vertical social formats and widescreen narrative formats stress models differently. Confirm both.
Keyframe input support. The ability to supply a start frame, an end frame, or both is worth more than most prompt features.
Iteration speed. You will regenerate many times. A slower model with marginally better quality can cost you the whole schedule.
A reasonable approach is to keep two tools: one for identity-heavy dialogue and close-up work, one for wide environmental motion. Trying to force a single model to excel at both usually produces compromise everywhere.
Troubleshooting common fusion failures
Face drift that appears gradually
Symptom: the character is correct in clip one and subtly wrong by clip five. Cause: accumulating error plus thin conditioning. Fix: regenerate clips in shorter blocks, re-anchor with a keyframe from the good clip, and if drift persists, add a reference that matches the current scene lighting.
Face morphing at cut points
Symptom: an obvious jump in facial structure when two clips are joined. Cause: the two clips used different conditioning or the last frame of clip A was never fed into clip B. Fix: always chain the previous final frame forward, or regenerate clip B with the same fused identity and lighting reference.
Wardrobe and hair instability
Symptom: color shifts, sleeve length changes, hair volume fluctuates. Cause: high-frequency detail plus weak textual description. Fix: simplify the garment in the reference, describe it explicitly and consistently in every prompt, and avoid letting the model infer clothing from motion.
Flicker on locked-off shots
Symptom: shimmering texture, especially around hair and eyes, even when nothing moves. Cause: model temporal instability. Fix: reduce motion amplitude, lower prompt complexity, or split the shot into shorter segments and cross-dissolve.
Identity bleed between two characters
Symptom: two characters in the same frame slowly converge toward a shared face. Cause: separate identity conditionings competing in one generation. Fix: generate characters in separate passes and composite, or shoot them in separate frames and use reverse angles and over-the-shoulder framing.
Frozen, lifeless performance
Symptom: perfect identity, zero acting. Cause: identity conditioning too strong relative to motion signals. Fix: increase motion description specificity, add an emotion reference, or reduce reference weighting slightly and re-test on stills first.
Quality control and delivery checklist
Before exporting, run this pass on every clip:
- Compare the character's eyes, nose bridge, and hairline against the hero reference.
- Check skin tone consistency against the previous clip, not against the reference.
- Verify wardrobe state matches the continuity sheet.
- Watch the first and last ten frames for pop or drift.
- Play the full sequence at normal speed without pausing. Many continuity errors only register in motion.
- Confirm aspect ratio, resolution, and frame rate match the delivery spec.
Deliver with a short continuity document: the character bible, the reference set used, and any per-scene deviations. If a client or collaborator requests a revision later, this document is what lets you regenerate a single shot without rebuilding the identity from zero.
FAQ
How many reference images do I actually need?
Three high-quality images covering front, three-quarter, and full-body views are a functional minimum. Five to eight, including the dominant scene lighting, covers most narrative work comfortably. More than ten rarely helps and often introduces conflicting signals.
Can I use one photo and just prompt harder?
Sometimes, for very short clips and locked-down shots. For anything with camera movement, multiple cuts, or more than a few seconds of screen time, single-image conditioning drifts noticeably. Prompting cannot supply geometric information the model never received.
Do I need separate reference sets for different scenes?
Yes for dramatically different lighting or wardrobe. Build a base identity from neutral references, then add one scene-specific reference per environment. Keep the neutral base in the set so identity stays anchored.
Why does the character look right in stills but wrong in motion?
The model has more freedom in motion generation, and temporal consistency is a separate capability from image fidelity. Strengthen keyframes, shorten clip length, and check for flicker on a static shot to isolate whether the issue is identity or temporal stability.
Is it better to generate long clips or many short ones?
Many short ones, almost always. Short clips limit error accumulation, make retakes cheap, and give you cut points that hide minor inconsistencies. Treat every clip as a shot, not a scene.
What is the most common beginner mistake?
Building the reference set from whatever images happen to be on hand, then blaming the model for drift. Consistent lighting and consistent angles in the references solve more problems than any prompt trick.
The takeaway
Character consistency is a discipline, not a feature. Collect references that match the shots you plan to make. Fuse them with deliberate weighting. Anchor your scenes with keyframes. Generate in short blocks and chain them. Then audit against a written character bible instead of your memory of the character.
Do that, and multi-image fusion stops being a fragile trick that occasionally works and becomes a repeatable production method you can plan a schedule around.
