Why AI Characters Drift Between Shots
Ask anyone who has tried to build a multi-scene AI video and you will hear the same complaint: the first shot looks perfect, and by shot six the hero has a different jawline, a different jacket, and a slightly different face. Nothing catastrophic happened. The generator simply did what diffusion models do — it sampled from a probability distribution, and every new sample was a new roll of the dice.
The causes are boringly technical, which is good news because they are fixable. Seeds change between renders. Samplers and guidance scales get tweaked by a fraction. Text encoders interpret loose wording loosely, so a four-word wardrobe description can produce a dozen different jackets across a dozen clips. Aspect ratios shift when you move from a wide establishing shot to a vertical close-up, and that alone can reshape a face. Some video models are trained on different datasets than the image model you used for keyframes, so they redraw the subject from scratch rather than matching your reference. Style layers, motion blur, and re-lighting then finish the job.
The lesson is that consistency is not a check box or a single prompt line. It is a production process, and it needs the same discipline you would apply to any other part of a shoot: a locked spec, a reference kit, a shot plan, and a review pass.
The Modular Brick Method, Explained
The most reliable approach to character consistency treats identity the way a construction toy treats a model: as a set of small, standard parts that snap together in different combinations. Instead of describing your character as one long, fragile sentence, you break them into independently reusable specification blocks — bricks — and then recombine those bricks per shot.
A brick is a short, self-contained description of exactly one attribute. "Square jaw, straight dark eyebrows, olive skin" is a face brick. "Oversized moss-green utility jacket with a frayed collar" is a wardrobe brick. "Walks with a slight forward lean, hands in pockets" is a motion brick. Each brick is short enough to stay memorable, specific enough to survive paraphrasing by the model, and stable enough to reuse for months.
Fixed bricks versus scene bricks
The power of the method comes from separating two categories.
Fixed bricks define who the character is and never change: facial structure, hair, skin tone, body proportions, signature accessories, colour palette. If a fixed brick changes, you are looking at a different character.
Scene bricks describe what the character is doing right now: wardrobe layers for the weather, props, location, time of day, camera angle, lens, mood. These should rotate freely between shots without touching the fixed bricks.
Why the decomposition works
Generators are sensitive to prompt length and token order. A 90-word paragraph describing everything at once dilutes the identity tokens, and the model may ignore the ones buried in the middle. Ten short bricks, assembled in a consistent order, keep the important tokens near the front and repeat them identically every time. Repetition across renders is what eventually reads as identity.
Step 1 — Write a Character Bible That Survives Revisions
Before you generate a single frame, write a one-page character bible. This is a plain text or spreadsheet document, not a creative writing exercise. Its job is to be the single source of truth that you copy from, so you never retype a description from memory.
Include the following fields:
- Identity anchors — age range, build, facial structure, eye shape and colour, nose, mouth, distinguishing marks.
- Hair brick — length, texture, parting, colour with a modifier ("dark brown with warm highlights" beats "brown").
- Wardrobe bricks — one brick per outfit, broken into layers: base layer, mid layer, outer layer, footwear, bag or belt.
- Palette — three to five colour values, ideally as hex codes or clearly named references. If your tool supports palette references, put them here.
- Signature props — the objects that make the character instantly readable: a scratched watch, round glasses, a specific backpack.
- Motion signature — posture, gait, resting hand position, how they react when surprised.
- Negative list — everything the character must never have: no beard, no logos, no jewellery, no modern sneakers in a period piece.
Two habits make this document far more useful. First, write every brick as a phrase you would paste directly into a prompt, not as a note to yourself. Second, version it. When you decide the jacket should be olive rather than forest green, change the brick and re-render the affected shots — do not silently update your memory and hope for the best.
Step 2 — Build and Curate a Neutral Reference Set
Text alone will get you roughly 70 percent of the way. Reference images carry the rest, and modern image and video tools increasingly accept them directly as identity conditioning. Build a reference set before you start animating.
A practical set contains eight to fourteen images:
- A clean front-facing portrait in neutral light
- A three-quarter view
- A profile view, left and right if available
- A full-body shot in the default outfit
- One expression sheet: neutral, smiling, angry, surprised
- Two or three detail crops, such as hands and hairline
- One shot in a secondary lighting condition, such as warm evening light
Keep the reference set boring on purpose
It is tempting to use the most beautiful image as your hero reference. Resist that. Dramatic lighting, extreme angles, and heavy colour grading make it harder for a model to separate identity from atmosphere. Neutral grey backgrounds, soft even light, and a relaxed pose give you the cleanest identity signal, which you can then re-light in the actual scene.
Curate ruthlessly
An inconsistent reference set is worse than no reference set, because it teaches the model that your character has three jaw widths. Delete any image that is off-model, even if it is attractive. Twelve tight references beat thirty loose ones.
Step 3 — Lock the Descriptor String and Stress-Test It
Now assemble your bricks into a canonical descriptor string. This is a fixed sentence you will paste into every prompt for this character, in the same order, with the same wording. Keep it between 40 and 70 words; longer strings start to dilute.
A workable order looks like this:
- Subject type and identity anchors
- Hair and grooming
- Default wardrobe and signature props
- Palette
- Lighting and lens for the shot
- Motion or pose
- Negative description
For example, a compact descriptor might read as: "a woman in her early thirties, square jaw, straight dark eyebrows, olive skin, shoulder-length wavy dark brown hair with warm highlights, round wire glasses, moss-green utility jacket over a grey knit, brass watch on the left wrist, warm palette of moss, slate and amber, soft window light, 50mm lens, calm posture, no jewellery, no logos."
Stress-test across three axes
Once locked, run the descriptor through three tests before committing to production.
- Model test — render the same still in at least two different image generators and one video generator. Note which bricks survive untouched and which get reinterpreted.
- Style test — push the character through a photoreal pass, a stylised 3D pass, and a flat illustration pass. A strong brick set keeps the character recognisable even when the rendering style changes completely.
- Angle test — render the same descriptor at a wide shot, a medium shot, and a close-up. If the identity collapses at distance, your facial bricks are too subtle and need a stronger distinguishing feature.
Each failure tells you exactly which brick to strengthen. This is the single highest-leverage hour in the whole pipeline.
Step 4 — Keep Identity Stable Across Models, Shots, and Edits
The temptation, once you have a good keyframe, is to generate video directly from text. Do not. Generate stills first, approve them, then animate.
A reliable sequence looks like this:
- Generate keyframes as stills using the locked descriptor string, one still per shot, at the correct aspect ratio.
- Approve the stills as a contact sheet before spending any video time on them.
- Animate each keyframe with the still as the first frame, holding the same seed and guidance settings where the tool allows.
- Interpolate between keyframes for longer sequences, using the approved stills as start and end frames.
- Re-render, do not patch, when identity drifts early in a clip. Fixing a broken face in post is slower than regenerating with a sharper anchor.
Working across multiple tools
Different generators handle identity conditioning differently. Some accept multiple reference images and weight them. Some let you train a small character model on your reference set, which is the strongest option when you need dozens of shots. Some rely purely on first-frame control for video. Whatever the mechanism, the rule is unchanged: the brick document stays identical, and only the plumbing changes.
Two practical guards help here. First, freeze your aspect ratio for the whole project. Second, if you switch tools mid-project, re-run the contact sheet test on the new tool before you animate anything else — the same descriptor string can render noticeably narrower faces in a different model.
A Storyboard Workflow Built Around Bricks
Consistency problems often originate in scheduling rather than in prompts. If you produce shots in script order, your character jumps between lighting conditions, locations, and outfits on every clip, which amplifies drift.
Instead, tag each shot with the bricks it needs, then batch.
| Shot | Fixed bricks active | Scene bricks | Notes |
|---|---|---|---|
| 01 | Full identity set | Rooftop, dusk, wide | Establishing |
| 02 | Full identity set | Rooftop, dusk, medium | Same outfit and light |
| 03 | Full identity set | Interior, warm lamp, close-up | Wardrobe change only |
Batching by location, lighting, and outfit means the model sees the same combination repeatedly within a short window, which reduces variance. It also makes review easier: you compare five rooftop shots against each other instead of comparing a rooftop shot against a beach shot against a night interior.
A worked example
Suppose you are producing a six-shot mini-film about a courier. Write the character bible once, generate twelve neutral references, lock the descriptor, and approve six keyframes as stills. Then render the six clips in two batches: three daytime street shots together, three night interior shots together. When you review, you will likely find that shots one and four hold up perfectly while shot five drifts on the jacket. That is a wardrobe brick problem, not an identity problem — fix the brick, re-render shot five only, and move on.
Quality Control: Catching Drift Before It Reaches the Timeline
Reviewing clip by clip at full size is slow and unreliable, because the eye adapts to slow changes. Use three fast checks instead.
The thumbnail strip test
Export one frame per shot, line them up at small size, and look at the strip as a whole. Identity drift is obvious in silhouette and colour at this scale, even when it is invisible in full resolution.
The contact sheet comparison
Place your approved reference set beside the frames from the current batch at matched head sizes. Check the anchor points in order: jaw width, hairline, eye spacing, brow shape, skin tone, then wardrobe.
The failure checklist
Watch for these specific signals: silent hairline shifts, eye colour warming or cooling, glasses disappearing in wide shots, jacket zips mirroring, signature props changing hands, and body proportions stretching in vertical formats. Any one of these is a signal that a brick has become too vague.
Fix in the right order
When something drifts, escalate in this order: re-roll with the same seed; strengthen the weakest brick in the descriptor; add or swap a reference image; use inpainting or a targeted face pass on the keyframe; and only then consider a trained character model. Most drift is solved at step two.
Common Mistakes, Tooling, and Fixes
The same handful of errors causes most inconsistency. Here is the short list, with the fix beside it.
- Describing identity only in the first prompt. Repeat the locked descriptor in every prompt. Models have no memory of your earlier session.
- Prompt soup. Cramming forty attributes into one sentence dilutes all of them. Split into bricks and keep the string under 70 words.
- Changing aspect ratio mid-project. Lock it. If you need verticals, shoot the whole sequence vertical.
- Mixing models mid-sequence. Excellent for exploring, terrible for a finished sequence. Test first, then commit.
- Using inconsistent references. Every mismatched reference image teaches the model that your character varies.
- Ignoring the negative description. Forgetting "no jewellery" once is how your character acquires a necklace in shot four.
- No version control. Name files predictably: project, character, shot, version. Keep prompts in the character bible, not in your head.
- Waiting too long to review. Build the contact sheet after every batch, not at the end of the project.
On tooling, keep the stack simple and category-based. You need an image generator with reference or character features, a video generator that accepts a first frame, an optional interpolation tool for long takes, an upscaler, a non-linear editor for the final assembly, and a shared document or spreadsheet for the brick library. Popular choices in each category rotate constantly, so pick tools that let you feed a reference image and control a seed, and learn them deeply rather than collecting more of them.
FAQ
How many reference images do I actually need?
Eight to fourteen clean, neutral images. Fewer than five rarely gives a model enough signal; more than twenty usually adds noise unless you are training a dedicated character model.
Can I keep a character consistent without training anything?
Yes. A locked descriptor string, a curated reference set, keyframe-first animation, and batched rendering get most projects to a usable standard. Training becomes worthwhile when you need dozens of shots or a recurring character across many episodes.
Does a fixed seed guarantee consistency?
No. A fixed seed removes one source of variance, but prompt wording, reference images, aspect ratio, and the model version all still shift the result. Treat the seed as one brick, not the wall.
What if my character changes clothes between scenes?
That is exactly what scene bricks are for. Keep the facial and body bricks locked, swap only the wardrobe brick, and re-approve a contact sheet for the new outfit before rendering the rest of that scene.
How do I handle two characters in the same shot?
Give each character a distinct silhouette and palette so they are separable even at thumbnail size, then describe both with their own locked strings in a consistent order. Avoid overlapping wardrobes in the same colour family.
How long should the descriptor string be?
Forty to seventy words in normal use. If you find yourself adding a tenth attribute, it usually belongs in a reference image instead of the text.
Why does my character look right in stills but wrong in video?
Video models often redraw rather than match. Always pass your approved still as the first frame, keep motion modest in the first half-second, and re-render rather than trying to repair a drifted clip in post.
When should I start over instead of fixing?
If two rounds of brick strengthening and one reference swap do not fix a shot, the keyframe itself is the problem. Regenerate the still, approve it, and animate again. Starting clean is faster than rescuing a bad foundation.
