Why Text-to-Video Alone Breaks Continuity
Anyone who has tried to build a five-shot sequence from text prompts alone knows the pattern. Shot one looks gorgeous. Shot two has a slightly different nose. Shot three changes the jacket colour. By shot five you are editing around faces instead of telling a story. That is not a prompting failure. It is a structural property of how diffusion sampling works: every prompt starts a fresh, semi-independent pass through latent space, and tiny numeric differences compound into visible divergence.
Two kinds of consistency get confused in these conversations. Temporal consistency lives inside a single clip: does the hand stay a hand, does fabric ripple believably, does the camera move without melting geometry. Cross-shot consistency lives between clips: is this recognisably the same person, the same room, the same sword, the same late-afternoon light. Modern generators have become genuinely good at the first. The second is still where most productions lose time, because repairing drift after the fact means regenerating, rotoscoping, or quietly rewriting the script to avoid the problem.
Multi-image fusion attacks the second problem directly. Instead of describing a character in words and hoping the sampler lands in the same region of latent space twice, you supply images that define the target and let the model treat them as constraints rather than suggestions. Continuity stops being a post-production rescue and becomes a setup decision.
What Multi-Image Fusion Actually Does
At its core, fusion means conditioning generation on more than one visual reference at once. You are not uploading a single hero image and hoping. You are handing the model a small, curated set: three angles of a face, two views of a room, one prop detail, one colour script frame. The model blends these into an internal representation that guides every frame it produces.
Reference images as anchors
Each reference image carries spatial and material information that text struggles to encode. A prompt saying broad-shouldered man in a worn leather jacket leaves hundreds of plausible outcomes. A reference photo collapses that space to one. Three references from different angles collapse it further, giving the model enough geometric signal to reconstruct the subject from viewpoints you never photographed.
Identity vectors and style embeddings
Many pipelines convert reference images into compact embeddings — numeric fingerprints of a face, an outfit, or a palette. These vectors can be stored, versioned, and reused across sessions. This matters more than it sounds: it means your character survives a browser restart, a model upgrade, or a switch from one generator to another. The identity becomes an asset you own rather than a lucky seed you must remember.
Model-agnostic pipelines
Different generators respond to reference conditioning in different ways. Some excel at facial identity, some at material texture, some at camera motion. A good fusion workflow keeps your reference set and scene bible portable, so you can route the same shot through two or three engines and pick the best result without re-establishing who the character is each time.
Build the Visual Canon Before You Generate Anything
The most common reason fusion underperforms is not the model. It is a weak reference set. Garbage in, drift out.
Character sheets that survive a close-up
Prepare at least six images per principal character: front, three-quarter, profile, back, a neutral expression, and an extreme close-up. Use consistent lighting across the sheet. Avoid heavy makeup, filters, or dramatic shadows that the model might mistake for permanent features. If the character wears glasses or a scar, the sheet should show them in every frame so the model treats them as identity rather than costume.
Location plates and the lighting language
For each location, gather a wide establishing view, a mid-range shot, and one detail texture (brick, sand, wet asphalt, wood grain). Then write down your lighting rules in plain language: warm key from camera left, cool bounce from a window, practical lamps in the background, no daylight after the second act. This document becomes a shared vocabulary between you and whoever else prompts the scene.
Props, wardrobe, and continuity notes
Small objects break immersion faster than faces do, because viewers track them consciously. Create one reference image per recurring prop and lock its silhouette and palette. Wardrobe deserves a numbered list per scene: which jacket, which stains, which buttons undone. A one-page continuity sheet is worth more than another hour of prompt tweaking.
A Repeatable Shot-by-Shot Workflow
Step 1: Lock the scene bible
Write the scene as prose first — who is present, what changes, what the camera does. Then split it into shots with one sentence of intent each. Every shot should state a single thing that changes: a turn of the head, a door opening, a light shifting. Shots that try to change two things at once produce artefacts.
Step 2: Assemble reference sets per shot type
Not every shot needs the full character sheet. A wide shot needs location plates plus one full-body reference. A close-up needs the identity set plus an expression reference. A prop insert needs the prop sheet plus the lighting rule. Matching the reference set to shot type keeps generation fast and reduces conflicting signals.
Step 3: Write prompts that describe change, not identity
This is the single biggest habit shift. If your references already define who and where, the prompt should describe motion, camera, and performance. Compare:
- Weak: a tired detective in a grey coat walks into a rainy alley, cinematic lighting
- Strong: walks forward three steps, stops, glances left; slow dolly-in, handheld micro-shake, rain streaks backlit
The second prompt does not restate identity, so it cannot contradict the anchor images.
Step 4: Generate in small batches and review at full size
Generate two to four variations per shot, never twenty. Review at full resolution, not thumbnails, because drift hides in the eyes and hands. Accept a take only if it matches the previous shot on five checkpoints: face structure, hair, wardrobe, prop state, and light direction.
Step 5: Assemble, match, and archive
Cut the sequence together before you polish anything. Cross-shot drift is easiest to see in motion. Once the edit works, do a light colour match to unify contrast and white balance, then archive the accepted references and settings alongside the project. That archive is what makes episode two cheap instead of painful.
Prompt Patterns That Preserve Identity
Certain phrasings reliably destabilise reference conditioning. Naming a different actor, describing facial features in detail, or adding adjectives like younger or more muscular pushes the sampler away from the anchor. Treat the reference as the authority and keep language focused on action.
Patterns that work well:
- Performance-first: she exhales, relaxes her shoulders, then smiles faintly
- Camera-first: 35mm equivalent, slow push in, shallow depth of field, no cut
- Motion-only: steam rises, curtain drifts, dust motes cross the key light
- Continuity flags: same wardrobe as previous shot, same time of day, same lens
Patterns that cause trouble:
- Re-describing the face in words
- Mixing two characters in one prompt without separate reference sets
- Piling on style adjectives from unrelated films
- Asking for a pose that the reference set never demonstrated
A useful discipline is to keep a running list of phrases you removed. Most prompt libraries grow by addition; consistency improves by subtraction.
Choosing Models and Tools for a Consistency-First Pipeline
Not all generators handle multi-reference conditioning equally. Evaluate candidates on four axes.
- Reference capacity. How many images can be supplied at once, and do they combine or compete? Three well-chosen references usually beat ten noisy ones.
- Identity retention under motion. Test with a 6-second clip that includes a turn and a close-up. If the face drifts when the head rotates, the engine will not carry a series.
- Style control granularity. Can you separate identity from look, so you can restyle a scene without rebuilding the character?
- Output logistics. Resolution, frame rate, clip length, watermark policy, and how cleanly results export into an editing timeline.
Run the same 30-second test scene through every candidate before committing. It costs an afternoon and saves weeks. Keep a comparison sheet with the same five checkpoints you use in review, and re-run the test after major model updates, because conditioning behaviour changes between versions.
Quality Control: Catching Drift Before It Compounds
Drift is cumulative. A two percent difference between shots one and two becomes a different person by shot six. Build checkpoints into the pipeline rather than inspecting at the end.
- Every third shot, place stills side by side at identical scale and crop.
- Track hands, eyes, jewellery, and any asymmetric feature. These fail first.
- Watch for colour temperature creep: warm to cool shifts read as a different time of day.
- Check motion cadence. If a character suddenly moves faster, the sampler has likely slipped away from the reference motion.
- Keep a rejected-takes folder with notes, so you learn which prompts reliably misbehave.
A simple scoring sheet works: five checkpoints, pass or fail, per shot. Anything that fails two checkpoints gets regenerated instead of patched. Patched shots accumulate inconsistencies that show up in the final cut.
Common Mistakes and How to Fix Them
Too many references, badly chosen. Ten near-identical portraits add noise, not precision. Fix: three to six diverse angles, consistent lighting, no duplicates.
Conflicting style references. A photoreal character sheet plus a stylised environment plate forces the model to average two incompatible looks. Fix: decide the finish first, then build all references in that finish.
Rewriting identity in every prompt. The model receives contradictory instructions and splits the difference. Fix: strip identity language once anchors are in place.
Skipping the continuity sheet. Without written rules, collaborators guess, and guesses diverge. Fix: one page per project, updated every session.
Judging on thumbnails. Compression hides exactly the errors that matter. Fix: full-resolution review, paused on the first and last frame of every clip.
Generating final audio too early. Voice and music lock you into a cut before visuals stabilise. Fix: temp audio only until the picture is approved.
Never re-testing after updates. A model update can shift conditioning behaviour overnight. Fix: re-run your 30-second benchmark after every version change.
Scaling a Series Without Losing the Look
Once a single scene works, the temptation is to expand quickly. Expand in the right order instead.
First, freeze the visual canon: character sheets, location plates, prop references, lighting rules, and a palette swatch set. Version it like code, with dated folders and short change notes.
Second, template the prompts. A shot template with slots for action, camera, and continuity flags keeps output stable while writers swap content.
Third, separate the roles. One person owns continuity review, another owns generation, a third owns edit. When the same person does everything, review becomes a formality.
Fourth, batch by location rather than by scene order. Generating all shots in one room in one session reduces lighting drift dramatically, because the references stay loaded and the context does not reset.
Finally, keep a living drift log. Every rejected take gets one line: what failed and which reference was likely responsible. After a dozen episodes, that log becomes your most valuable production document.
FAQ
What exactly is multi-image fusion?
It is a generation approach that conditions output on several reference images at once rather than a single anchor or a text prompt alone. The images define identity, environment, and palette; the prompt defines motion, camera, and performance.
How many reference images do I need per character?
Six is a practical sweet spot: front, three-quarter, profile, back, neutral expression, and close-up. Fewer than four makes facial identity unstable during head turns. More than eight rarely improves results and slows iteration.
Can I keep the same character while changing the art style?
Yes, if your pipeline separates identity from style. Lock the character references, then apply style through a separate environment plate or a style reference. Applying style by rewriting the identity description is what causes face drift.
Why does my character look fine in stills but wrong in motion?
Motion adds new angles the model has not seen. If your reference set only shows frontal views, a profile turn will interpolate rather than reconstruct. Add at least one profile and one back view before shooting anything with head movement.
Do I need the same generator for every shot?
Not necessarily, and portability is an advantage. Keep references and prompts engine-neutral, test each candidate on the same benchmark scene, and route individual shots to whichever engine handles that shot type best.
How do I fix continuity after a scene is already generated?
Regenerate the failing shots rather than patching them. Identify which checkpoint failed — face, wardrobe, prop, light, or cadence — then strengthen the corresponding reference. Adding more adjectives to the prompt usually makes it worse.
What is the fastest way to improve consistency right now?
Stop restating identity in prompts and start generating in location batches. Those two changes alone remove most visible drift in a typical short project.
How long does a consistency-first short take?
Budget roughly a third of the schedule for reference preparation and canon writing. That front-loaded work is what turns a one-off clip into a reusable world you can return to.
