Consistent Characters Across Scenes with Multi-Image Fusion

Why Character Consistency Breaks in Generative Video

Most creators meet the problem the same way. You generate a close-up of a protagonist in a kitchen and love the result. Then you render the reverse angle and get someone who looks like a cousin. The jaw softens, the hairline shifts, the eyes sit a little wider apart. Nothing looks obviously broken, yet any viewer spots the mismatch within a second.

Three root causes explain almost every case.

1. Sampling is stochastic. Every render is a fresh draw from a probability distribution. Tiny numeric differences translate into large visual differences in faces, because human perception is unusually well tuned to read them.
2. Text is a lossy way to describe a person. A phrase such as "a woman with dark hair in her thirties" compresses millions of possible faces into a handful of words, and the model fills the gap with whatever is statistically convenient.
3. Context changes punish identity. Swap the camera angle, the lens, the lighting, the aspect ratio, or the engine, and the representation of the character travels along with the change.

Stills hide the problem. A single frame can look perfect while a two-second cut between two frames exposes a stranger. That is why consistency work has to be treated as a production discipline rather than a lucky prompt.

The practical answer available to independent creators today is multi-image fusion: instead of describing a face with words or handing over one portrait, you supply a small, carefully chosen set of images and let the pipeline build a shared identity representation that conditions every shot. This guide covers the mechanics, the reference kit, the prompt skeleton, a full production workflow, the failure modes you will hit, and how to decide which approach fits a given project.

What Multi-Image Fusion Actually Does

At its core, fusion is a conditioning technique. Rather than feeding a single reference image into a video model, you feed several — typically four to twelve — and the system encodes each one into an identity embedding. Those embeddings are aggregated into one shared representation, and the generation steps are conditioned on that aggregate rather than on any single photo.

The useful outcome is a split between what stays fixed and what stays flexible. Invariant features survive across shots: interocular distance, jawline geometry, brow shape, ear placement, hairline, skin tone range, and the overall proportions of head and neck. Variable features remain editable: expression, head tilt, wardrobe, hair styling, makeup, and apparent age. That split is exactly what narrative video needs. You want the face to be unmistakably the same person while the performance changes from beat to beat.

Reference Diversity Beats Reference Volume

Ten near-identical selfies shot in the same light teach the system almost nothing. Five varied images — front, three-quarter left, three-quarter right, profile, plus one different lighting condition — carry far more signal. Diversity across angle, expression, and light is what lets the aggregation generalize an identity instead of memorizing one photographic situation. If two references are 95 percent alike, the second one mostly adds noise.

Identity Slots, Seeds, and Versioning

Treat every recurring character as a versioned project asset. Log the reference set version, the seed, the conditioning strength, the resolution, and the engine used for each approved shot. When drift appears weeks later, that log lets you reproduce the good shot exactly and change one variable at a time until you find the culprit. Without it, consistency work degrades into endless re-rolling and guesswork.

Cross-Model Anchoring

If you switch engines mid-project — perhaps to get stronger motion in an action beat — re-anchor before committing whole scenes. Render a three-shot test (front, three-quarter, profile) with the new engine in the target lighting and compare it against your reference contact sheet. Every model interprets the same conditioning differently, and a face that reads as "the same person" in one engine can read as a sibling in another. The fix is not to avoid switching; it is to prove the identity survives the switch before you spend a day on coverage.

A Note on Two-Character Scenes

Fusion struggles when two conditioned identities share a single frame, because the identities compete for the same conditioning space. The reliable pattern is to render each character separately against a clean plate with consistent lighting, then composite them in post. If you must generate them together, give each a distinct silhouette, palette, and wardrobe so the model has strong separation cues, and keep the shot wide enough that facial detail is not the focus.

Fusion vs Older Consistency Techniques

It helps to understand why the older approaches fail, because several of them still have legitimate uses.

Method	How it works	Strengths	Weak points
Seed reuse	Fix the random seed for every render	Instant, no assets needed	Locks noise, not identity; collapses as soon as the prompt or camera changes
Single reference image	One photo conditions each shot	Simple setup	Copies the reference pose and lighting; fails on profiles and new angles
Text-only description	Describe the face in words	Maximum flexibility	Weakest identity retention; drifts by design
Custom model training	Train a small adapter on 15–40 images	Strongest lock on one specific face	Needs a curated set and training time; can flatten expression range
Multi-image fusion	Aggregate embeddings from several references	No training, strong identity, editable performance	Requires a well-built kit and disciplined prompting

For short tests and abstract or stylized content, seed reuse and text-only prompts are perfectly fine. For anything with recurring characters, dialogue, or a story arc, fusion is the practical default. Training a dedicated adapter only pays off when a character will appear in dozens of minutes of final footage and the production can absorb the setup cost.

Building a Character Reference Kit That Holds Up

Your kit is the real project asset. Getting it right is the difference between a smooth shoot and hours of re-rolling.

A Practical Reference Shot List

• Front-facing, neutral expression, even lighting
• Three-quarter left and three-quarter right, neutral expression
• Full profile left, neutral expression
• Slight chin-down and chin-up variations
• One warm-light and one cool-light version of the front shot
• Two story-relevant expressions (for example, joy and tension)
• A full-body frame showing height, build, and posture
• A detail crop of the hands if hands appear on camera
• Wardrobe variants for each costume in the script
• Hair-in-motion references (wind, tied back, wet) when the story needs them

If the character appears in a scene where they are lying down, running, or shot from a very low angle, add at least one reference that hints at those conditions. The kit does not need to be exhaustive, but it should cover every angle the edit will actually cut to.

Cleaning, Cropping, and Naming

Keep crops consistent across the set — square or 4:5 usually works best. Let the face occupy roughly 40 to 60 percent of the frame. Too small and the identity signal weakens; too tight and you lose head shape and hair volume, which are strong identity cues. Avoid heavy beauty retouching, film grain, and stylized filters, all of which inject noise the model will try to reproduce. Aim for at least 1024 pixels on the short edge, and colour-match the set so no single image drags skin tone in an odd direction.

Name files with a predictable pattern such as char_angle_expression_wardrobe_version so the kit stays readable months later. Consistency projects have long tails; a tidy folder prevents accidental use of an outdated reference.

Validate the Kit With a Contact Sheet

Build a single-page contact sheet of the whole kit. When a render drifts, the contact sheet lets you see instantly which reference is pulling the result off-model. It also reveals accidental gaps: no true profile, no cool lighting, no version of the character smiling. Ten minutes of layout saves hours of diagnosis.

Prompt Structure for Identity Retention

Prompts cannot create consistency on their own, but a bad prompt can destroy good conditioning. Build every shot from the same six-part skeleton.

1. Identity block — invariant descriptors only: age range, skin tone, face shape, eye colour and shape, hair colour and texture, distinguishing marks. Keep this block byte-identical across every shot of that character.
2. Wardrobe block — fabrics, silhouette, colours, accessories, and the costume's current state (clean, wet, torn, dusty).
3. Scene block — location, time of day, weather, and the background elements that matter to the story.
4. Camera block — shot size, lens feel, angle, and movement.
5. Lighting block — key direction, quality, colour temperature, and contrast ratio.
6. Motion block — precisely what changes during the shot: a head turn, a step forward, a smile forming.

Three rules make the skeleton work. First, never contradict your references. If the kit shows a soft jawline, do not ask for a sharp one; the model will compromise and produce a stranger. Second, place the identity block first so it anchors conditioning before scene detail begins to pull. Third, choose negatives that target drift artifacts specifically: face morphing, warping, identity change, extra fingers, melted features, flicker.

A condensed example looks like this: identity block, then "wearing a charcoal wool coat, collar up, rain-damp," then "standing at a bus shelter at night, wet asphalt reflections," then "medium close-up, 50mm, eye level, static," then "single soft key from camera left, cool 5600K, deep shadows," then "turns head slowly toward the street." Identical identity block, new context. That is the whole pattern.

End-to-End Workflow: From Reference Stills to a Finished Sequence

1. Write the scene list and mark every shot the character appears in, with required wardrobe and emotion.
2. Build the reference kit with a still-image model, then select the eight strongest images using the criteria above.
3. Register the character as a named, versioned asset in your project.
4. Lock the identity block and save it as a reusable prompt snippet.
5. Render a calibration triple — front, three-quarter, profile — in the target scene lighting.
6. Compare the triple against the contact sheet. Adjust conditioning strength or swap a weak reference before continuing.
7. Render the scene's hero shot first: the one the audience will remember.
8. Render coverage — reverse angles, inserts, cutaways — using the same identity block.
9. Review the whole scene in sequence at playback speed, not shot by shot.
10. Log approved settings, then move to the next scene without touching the kit.

Step nine is the one creators skip and regret. Drift is often invisible in a still and glaring in a cut. Watching the scene at speed, with sound, reproduces the experience your audience will have, and it exposes continuity problems that frame-by-frame review hides.

A useful discipline is to keep one "golden frame" per character: a single approved render you can compare against any new shot side by side. It is faster and more objective than trying to remember what the face looked like last week.

Continuity Across Camera Moves, Lighting, and Time

Camera Movement and Angle

Wide shots hide identity drift; close-ups expose it. If a scene ends on a tight close-up, render that shot early so you know the identity holds at that scale before investing in the rest of the coverage. When a shot calls for a strong profile or an extreme low angle, confirm those angles exist in the kit. If they do not, add them before rendering.

Lighting Continuity

Lighting changes read as identity changes. Keep the key light direction consistent across shots in the same scene. If the scene progresses from day to night, move the whole sequence gradually instead of jumping. A character lit warm from the left in one shot and cold from the right in the next will feel like a different person even when the geometry is identical.

The Continuity Document

Keep one document tracking, per character: kit version, identity block, wardrobe per scene, hair state, injuries or makeup changes, and any props the character carries. Continuity errors in AI video are rarely about faces alone. A jacket that changes colour between cuts breaks the illusion just as quickly, and a missing prop is the kind of slip viewers notice immediately.

Choosing Models, Troubleshooting, and Quality Control

Five Criteria for Evaluating an Engine

Hosted text-to-video engines, image-to-video engines, and open-weight models all handle reference conditioning differently. Some prioritize photoreal skin and hair, others prioritize stylized motion, and open-weight options give you pipeline control at the cost of setup work. When comparing engines for a character-driven project, test with your own character rather than a demo reel, and score five things:

• Identity retention across front, three-quarter, and profile angles
• Motion quality at the shot length you actually need
• Maximum resolution and clip duration
• The practical cost of iterating a shot twenty times
• Whether the usage terms fit your distribution plans

Tools such as Flux, Runway, Kling, Pika, Sora, Vidu, Hunyuan, Wan, Hailuo, and PixVerse behave differently enough that one focused afternoon of testing with your own kit is worth more than any feature list. Style mixing is where engines diverge most: a model that renders beautiful skin may flatten stylized looks, while a stylized model may lose facial micro-detail. Match the engine to the aesthetic, not the other way around.

Failure Modes and Fixes

Symptom	Likely cause	Fix
Face is right, hair changes	Weak hair references	Add two references with the hair in different light and motion states
Identity holds in stills, drifts in motion	Conditioning diluted across frames	Shorten the shot, raise conditioning strength, reduce competing prompt detail
Character suddenly ages up	Lighting or contrast shift	Match key light and colour temperature to the previous shot
Face melts during fast movement	Too much motion in a short clip	Split the action into two shots and cut between them
The model copies the reference pose exactly	Kit lacks angle variety	Add three-quarter and profile references
Two characters blend into one	Insufficient separation	Render separately and composite, or differentiate silhouette and palette
Wardrobe changes mid-scene	Costume not specified per shot	Add a wardrobe block to every prompt and track it in the continuity document

Pre-Render Checklist

• Identity block identical across every prompt in the scene
• Kit version logged and unchanged for the duration of the scene
• Calibration triple approved in the target lighting
• Wardrobe, hair state, and props confirmed against the continuity document
• Skin tone and contrast matched across adjacent shots
• Negatives include drift, morphing, and flicker terms
• Whole scene reviewed in sequence at playback speed

If a scene fails this checklist twice, stop and fix the reference kit rather than rendering more variations. Repeated failures almost always originate in the inputs, not in the sampling luck of the day.

FAQ

How many reference images do I actually need?

Six to ten well-chosen images cover most productions. Below six, identity retention gets shaky whenever the angle changes. Above twelve, you often add noise rather than signal, especially if the extras are near-duplicates of shots you already have. Prioritize angle diversity first, then expressions, then lighting variety.

Can I work with a single reference image?

For short clips in a fixed lighting setup, sometimes. The failure appears the moment you need a profile, a strong lighting change, or a different lens. If the character appears in more than three shots, build a proper kit. The time you spend assembling it is almost always less than the time you would spend re-rolling.

Why is my character stable in stills but drifting in video?

Video models spread conditioning across many frames, and motion gives the sampler more freedom to reinterpret the face. Shorten the shot, reduce competing scene detail in the prompt, and confirm that conditioning strength is high enough to dominate the motion prior. Splitting a long movement into two shorter shots and cutting between them is often the fastest fix.

Do I need to train a custom model?

Only when a character appears in a large volume of footage and the setup cost is justified. For most series, shorts, and episodic content, multi-image fusion plus disciplined prompting gets you close enough without the training overhead. If you do train, keep the adapter's expression range in mind: some trained faces become rigid and stop performing.

How do I handle wardrobe changes and ageing?

Fusion handles both well because they are variable features. Keep the identity block untouched, change only the wardrobe and context blocks, and render a test shot in each new state before shooting the full scene. Controlled ageing works the same way: adjust only the age descriptors rather than rewriting the face.

What if two characters must share a shot?

Judge whether your engine separates identities cleanly. If it does not, render each character separately against a clean plate with matched lighting, then composite in post. It adds a step but removes an entire class of failure. If you generate them together, keep the framing wide, differentiate wardrobe and palette, and accept that facial detail will be less precise.

How do I keep consistency across a long series?

Freeze the kit. Once a character's reference set is approved, version it and stop editing it. New episodes should reuse the same kit, the same identity block, and the same calibration procedure. Consistency across a series comes from refusing to improve a working asset mid-run, even when a shinier reference photo appears.

Putting the Workflow Together

Consistency is not a single setting; it is a production habit. A curated reference kit, a locked identity block, a calibration test before each scene, and a continuity document will outperform any amount of re-rolling. Start with one character and eight references. Render the calibration triple. Review it in sequence. Once the routine feels ordinary, extend the same kit and prompt structure across your whole cast — and the difference between a folder of good clips and an actual film becomes obvious to every viewer.