A character who looks like a different person in every shot is the fastest way to make an AI-generated video feel amateur. You can spend an hour tuning a prompt and still watch the jawline shift, the eye colour warm up, and the jacket change from navy to charcoal between two cuts. Viewers may not be able to name what is wrong, but they feel it immediately: the story stops being about a person and starts being about a model struggling.
Multi-image fusion is the technique that solves most of this problem. Instead of describing a character with words and hoping the model lands in the same place twice, you give the model several reference images of that character and let an identity layer fuse them into a single, stable conditioning signal. Every shot is then generated against that signal rather than against a fresh roll of the dice. This guide covers how the technique works, how to build a reference set that actually helps, a repeatable production workflow, the prompts and shot designs that keep identity locked, and the fixes for the failure modes you will inevitably hit.
Why Character Consistency Breaks in AI Video
Text-to-video and image-to-video models do not remember people. They sample from a probability distribution shaped by training data, and every clip is an independent draw from that distribution. When you write "a woman in her thirties with short dark hair," you are describing a vast region of that distribution. Two samples from the same region can land on two different faces that both satisfy the prompt.
Several forces make this worse:
- Prompt compression. Long descriptions get shortened internally. Details that matter to you, like the exact shape of a nose, are not recoverable from a text prompt no matter how many adjectives you add.
- Sampling noise. A fixed random seed stabilises texture and grain, not identity. Change the camera angle and the seed no longer helps.
- Motion budget. The more the model has to invent — a turn, a walk, hands entering frame, a costume change — the more degrees of freedom it has to drift. Static close-ups hold identity best; dynamic wide shots lose it first.
- Duration accumulation. Drift compounds. A four-second clip can look perfect while a twelve-second clip from the same setup slowly melts the face.
- Scene context. Lighting, background, and colour grading influence how a face renders. The same character in a warm interior and a cold exterior will not read identically unless identity is anchored separately from style.
In practice, consistency is not one problem but three: identity (who the person is), continuity (wardrobe, props, world state), and style (how the footage looks). Fusion techniques address identity directly and make the other two far easier to manage.
What Multi-Image Fusion Actually Does
Multi-image fusion replaces the single reference photo with a small set of images and combines them into one identity representation. Each reference image is passed through an encoder that extracts identity-relevant features — facial geometry, skin tone, hair structure, proportions. Those features are then fused, typically by averaging in embedding space, by cross-attention over multiple reference tokens, or by a lightweight adapter trained on your specific set. The fused result becomes an extra conditioning input alongside your text prompt.
Because the model now has a concrete target instead of a verbal sketch, generation lands in a much tighter region of its output space. You are no longer asking for "a woman in her thirties"; you are asking for "this woman, seen from a new angle, doing something new."
Reference sets as identity anchors
A single image is a single point of view. It tells the model what the character looks like from that angle, under that light, in that expression. Ask for a three-quarter view or a different emotional register and the model must extrapolate — and extrapolation is where drift begins. A set of images from different angles, distances, and lighting conditions gives the fusion layer enough coverage to interpolate between real observations instead of guessing.
How references steer the sampling process
During generation, identity conditioning biases the denoising trajectory at every step. Early steps decide large structure — face shape, head proportions, silhouette. Later steps refine texture, skin, and hair detail. Good fusion influences both stages. If your identity signal is weak, it may only survive the early steps and get overwritten during detail refinement, which is exactly why characters look right in a thumbnail and wrong in motion.
Why more images is not automatically better
Adding images is not free. References compete for attention, and a noisy or contradictory set actively hurts results. One blurry photo, one heavy filter, one image of a similar-looking friend, and the fused identity becomes an average of people who are not your character. Quality and coherence beat quantity every time. Six to twelve clean images usually outperform forty mixed ones.
Building a Reference Set That Works
The reference set is the single highest-leverage asset in your project. Treat it like casting plus costume plus camera test, because that is functionally what it is.
Angle coverage
Aim for a set that spans the range of angles you plan to shoot:
- Frontal, neutral expression — the anchor image.
- Left and right three-quarter views.
- A near-profile if your script includes profile shots.
- One slightly low angle and one slightly high angle.
- One medium shot showing shoulders and hair fall.
- One wider shot showing body proportions and posture.
If a shot type is missing from the reference set, expect the model to invent it — and expect drift.
Lighting and colour discipline
Mix lighting conditions on purpose, but keep them interpretable. Include one soft even key, one harder directional light, and one scene-appropriate moody example. Avoid extreme colour casts, neon gels, or heavy colour grading in references unless the entire video shares that look. The model learns skin tone from your references, so a strong orange cast will follow your character into every scene.
Wardrobe, hair, and accessories
Decide early which elements are part of the character and which are part of the costume. If the jacket is signature, include it in most references and describe it consistently in every prompt. If clothing changes per scene, keep the reference set focused on face, hair, and body so wardrobe stays a scene-level decision rather than a competing identity signal. Accessories are the most common source of accidental drift: glasses in three references and absent in four will produce flickering glasses across your film.
What to exclude
Filter out anything with motion blur, heavy retouching, watermarks, unusual crops that cut off the top of the head, duplicate near-identical frames, or images where the character is partly occluded. Also exclude frames where another person occupies a large portion of the image; encoders can leak features from the wrong face.
The Fusion Workflow, Step by Step
The workflow below is tool-agnostic. It assumes you have a generator that accepts reference images and an identity or fusion layer built on top of it.
- Write a character bible. One page: name, age range, ethnicity, build, hair, eyes, skin, distinguishing marks, wardrobe rules, and a short paragraph in the voice of the character. This document keeps humans aligned and gives you consistent prompt vocabulary.
- Collect candidate stills. Generate or curate twenty to thirty images. Generate a few with different tools if you can — variety in source improves coverage.
- Cull ruthlessly. Keep the six to twelve that are sharp, coherent, and on-model. Discard anything "close enough." Close enough is what causes drift later.
- Normalise the set. Crop to a consistent framing, remove backgrounds if your tooling supports it, and note which image covers which angle.
- Build the fused identity. Upload the set to your identity layer, or train a small adapter if your pipeline supports it. Training on eight to twelve images for a narrow character is usually enough; more data mainly helps if the character appears in many styles.
- Run a calibration shot. Generate one neutral medium close-up with a plain prompt. Compare it to your anchor image side by side. If it is off-model, fix the reference set before generating anything else.
- Generate shot by shot. Use the same fused identity but vary camera, action, and environment. Keep the identity description in the prompt identical across shots.
- Check continuity every five shots. Watch the sequence, not the individual clips. Drift is invisible in isolation and obvious in sequence.
- Refresh when a scene changes drastically. New lighting regime, new costume, or a big time jump may justify adding one or two scene-specific reference frames — with the core set still applied.
- Archive everything. Keep the reference set, the identity profile, and the prompt template together so the project can be reopened and extended without rebuilding the character.
Prompt and Shot Design for Stable Identity
Identity conditioning does most of the work, but prompts still decide how much freedom the model has to wander.
Keep one canonical subject sentence
Write a single sentence describing the character and reuse it verbatim in every prompt. Example: "Maya, a 34-year-old woman with a short dark bob, warm medium-brown skin, and a small scar above her left eyebrow." Do not paraphrase it between shots. Paraphrasing signals variation to the model, even when the meaning is identical.
Vary only three things per shot
Change camera framing, action, and environment. Everything else stays constant. A workable prompt skeleton looks like this:
- Canonical subject sentence
- Wardrobe line (constant per scene)
- Action and micro-expression
- Camera position and lens feel
- Lighting and environment
- Style tail (film grain, colour palette, aspect ratio)
Respect the motion budget
Ask for one clear action per clip. "She turns from the window and speaks" is one action. "She turns, picks up a cup, walks to the table, and sits" is four, and by the third the face will have drifted. If a scene needs four beats, generate four clips and cut them together — the edit will also look more intentional.
Prefer medium shots for dialogue
Medium close-ups hold identity well because the face occupies enough pixels. Extreme close-ups amplify any small identity error, and wide shots reduce the face to a few dozen pixels where the model invents detail. If you need a wide shot, consider generating a medium version and widening it through an extension or outpainting pass rather than asking for a fully wide generation first.
Continuity Beyond the Face: Voice, Wardrobe, and Motion
A character is more than a face. Audiences forgive imperfect rendering but not inconsistent people.
- Voice. Lock a voice profile early and keep pitch, pace, and accent stable across episodes. Re-record or re-synthesise before shipping a clip with a mismatched tone.
- Wardrobe logic. Give each scene a costume sheet and restate it in prompts. Colour names should be specific — "charcoal wool coat" rather than "dark coat."
- Motion signature. Decide how the character moves: posture, gesture size, walking speed. Add a short motion line to prompts, such as "upright posture, small economical gestures." This is subtle but makes cuts feel like the same person.
- Colour grade. Apply one grade across all clips in post. Slight differences in generated colour temperature will read as continuity errors even when the face is perfect.
- Props and world state. Track objects that persist across shots — a mug, a bandage, a phone case — in a simple continuity list.
Tooling Landscape and Decision Criteria
You do not need a specific product to apply fusion, but the tool you choose determines how much control you get. Evaluate options against these criteria:
- Number and type of references. How many images can be conditioned at once, and are they all weighted equally or does the tool let you prioritise?
- Per-character separation. Can you hold two or three distinct identities in one project without them bleeding into each other?
- Shot length and motion quality. Fusion stability is worthless if the footage warps after three seconds.
- Resolution and aspect ratio. Vertical for social, widescreen for narrative — confirm both are supported natively.
- Iteration speed. Fast, cheap calibration shots mean you fix identity problems before committing to a long generation queue.
- Commercial licensing and data handling. Know whether your references and outputs can be used commercially and how they are retained.
- Pipeline fit. API access, local node graphs, or an editing-first interface all suit different workflows. Pick the one that matches how you already assemble video.
A practical setup is hybrid: one tool for identity-conditioned generation, one for motion extension or interpolation, and a standard editor for assembly, colour, sound, and captions. Trying to do everything in a single tool usually means accepting its weakest component.
Troubleshooting: Common Failure Modes and Fixes
Face drift after the second or third shot. The identity signal is being overwritten during detail refinement. Strengthen fusion weight, add an angle reference that matches the drifting shot, and reduce motion complexity.
Characters bleeding into each other. Generate separate identity profiles and never prompt two characters in one clip if you can avoid it. Composite them in the edit instead. If you must generate them together, keep them spatially separated and describe each one tersely.
Plastic, over-smoothed skin. Fusion weight is too high or the reference set is over-retouched. Lower the weight slightly and include one natural, lightly textured reference image.
Warping in profile or extreme angles. The reference set lacks coverage. Add one true profile and one extreme three-quarter image.
Wardrobe changing mid-shot. The wardrobe description is missing or too abstract. Name garment, material, and colour in every prompt and confirm the reference set is not contradicting it.
Flicker at cuts. Usually a frame-rate or interpolation mismatch rather than an identity problem. Normalise frame rates across all clips before editing, and avoid mixing interpolated and non-interpolated footage in the same sequence.
Identity collapse in wide shots. Generate medium, then widen. Alternatively accept that the character reads as a silhouette and make sure the preceding close-up establishes who it is.
Gradual style shift across an episode. Something in your prompt tail changed. Diff your prompt template against an earlier shot and restore the original wording.
Scaling to Series and Long-Form Projects
Episodic or long-form work rewards discipline more than cleverness. A few habits make the difference between a weekend experiment and a repeatable series.
Build a project folder with subfolders for references, identity profiles, generated clips, audio, and exports. Version your prompt template like code, with a date-stamped changelog. Keep a casting sheet with one row per character containing their canonical description, voice profile, costume list, and the exact reference set used.
Generate in batch by scene rather than by shot. If a scene has nine shots, queue all nine with the same lighting and wardrobe block so they share conditions. Review the whole scene at once — continuity problems hide in shot-by-shot review. Keep a reject folder; when a later episode needs a shot you cut earlier, the rejected clip is often reusable with a small extension.
Finally, budget time for post. Colour grading, sound design, music, and captions do more for perceived production value than another hour of prompt tuning. A modest clip with consistent identity, clean audio, and a unified grade will always beat a technically impressive clip where the protagonist changes faces.
FAQ
How many reference images do I actually need? Six to twelve coherent images covering front, both three-quarters, and a medium shot is a strong starting point. Add profile or extreme-angle images only if your script requires those shots.
Can I use one reference image and still get consistency? You can get close for short, static, well-lit shots. For anything with movement, multiple angles, or a series, a fused multi-image set is dramatically more reliable.
Does multi-image fusion replace prompt writing? No. It reduces the burden on text, but prompts still control wardrobe, action, camera, and style. Fusion decides who the character is; the prompt decides what happens.
Why does my character look right in stills but wrong in motion? Motion adds degrees of freedom. Lower the motion complexity, shorten clips, add a reference image matching the new angle, and keep identity weight high during the first frames of each clip.
Can I use fusion for animated or stylised characters? Yes. The principle holds for illustrated, 3D-rendered, and painterly characters, though you generally need a larger and more stylistically consistent reference set because the model has more stylistic variation to average over.
What about two characters in one scene? Generate them separately and composite. If you must generate together, use separate identity profiles, keep them apart in frame, and expect to re-roll a few times.
How do I know when to stop tuning? When a full scene plays without the viewer noticing the character. If you are the only one who can spot the drift, you are finished — ship it and move to the next scene.
Is this worth learning if I only make short clips? Yes. Even a fifteen-second clip benefits from a stable identity, and the reference-set habit pays off the moment you make a second video with the same character.
Multi-image fusion is ultimately a discipline problem dressed up as a technical one. Build a small, honest reference set. Fuse it once. Reuse the same canonical description in every prompt. Keep motion simple, cut between clips instead of asking one clip to do everything, and review in sequence. Do that, and your AI characters stop being lucky accidents and start being cast members.
