Why Character Drift Is the Real Problem
Ask anyone who has produced more than a handful of AI-generated clips what frustrated them most, and the answer is rarely image quality. It is drift. Shot one shows a woman with a sharp jawline and a silver hoop earring. Shot four shows the same character with a rounder face, a gold stud, and a jacket that has quietly changed from charcoal to navy. Individually the clips look great. Played together, they look like a casting mistake.
Character consistency is the difference between a set of pretty clips and something that reads as a story. Once you frame the problem correctly, the solution becomes much more tractable. Drift is not one failure, it is four different failures happening at once:
- identity drift, where facial structure and age shift between shots
- wardrobe drift, where clothing details morph because nothing pinned them down
- styling drift, where hair length, part line, or makeup changes
- scene geometry drift, where the character changes size relative to the environment or moves like a different person
Each of these has a different cause and a different fix. Most tutorials collapse them into one vague instruction to write better prompts, which is why most tutorials fail. This guide separates the causes, gives you a workflow that handles all four, and ends with a QA pass you can run on any finished sequence.
A One-Paragraph Character Bible
The single highest-leverage habit is not a model choice. It is writing down your character once and never improvising. A character bible is a short, frozen description that you paste into every prompt, every scene note, and every revision request.
Keep it to five to eight concrete facts. Adjectives are close to useless because models interpret them loosely. Specific nouns and numbers hold much better.
Here is a compact example:
- woman, early thirties, oval face, straight dark brown hair just below the collarbone, center part
- small silver hoop earrings, no necklace
- charcoal wool blazer with two visible buttons, unbuttoned
- white crew-neck shirt underneath, no visible logo
- black straight-leg trousers, matte texture
- neutral expression with a slight closed-mouth smile
The point is not realism. The point is that every one of those clauses is checkable in a frame. If you cannot point at a detail in the rendered image and confirm it, the clause is decoration and you should replace it with something observable.
Two practical rules keep a bible useful over a long project. First, keep it in a text file rather than in your head, and version it instead of editing it in place. Second, never let two characters share a bible file. Duplicated descriptions are how secondary characters slowly turn into the protagonist.
Building a Reference Sheet Before You Animate
Text alone will not hold a face. You need images that lock the character's appearance before you start generating motion, because video models inherit identity from the frame you hand them far more than from the words you type.
Start With a Set, Not a Single Portrait
Generate a small batch of stills of the same character and pick the one that is most stable and most reusable. A good reference set has four views:
- a clean front-facing portrait at chest height
- a three-quarter view that reveals jaw and nose volume
- a profile view
- a full-body shot that fixes proportions and wardrobe silhouette
Three-quarter and profile views matter more than most people expect. Models handle front faces well and then hallucinate the rest. If your only reference is a straight-on portrait, every turn of the head in your finished sequence is an invention.
Use an Image Model as Your Casting Director
Generate the stills with a strong text-to-image or image-to-image model and iterate there, where each attempt is fast and cheap compared with video. A workflow that works well:
- draft the character from the bible using a text-to-image prompt
- refine with image-to-image, keeping the composition and nudging only the details that are wrong
- generate eight to twelve candidates, then keep the best two per view
- discard anything with ambiguous lighting on the face, since harsh or novel lighting makes the reference harder to transfer
When you lock the sheet, freeze the seed and prompt used for the chosen keeper. You will want to regenerate a variant later, and reproducing the original setup is far easier than reverse-engineering it.
Note Where Consistency Really Breaks
In practice, drift concentrates in a few predictable places: hairline and part, eyebrow shape, earring type, and the exact neckline of a garment. If you are deciding where to spend your attention, spend it there. A face that is ninety percent right with the wrong neckline reads as a different person more often than a face that is ninety-five percent right overall.
Choosing Image and Video Model Roles
A recurring mistake is expecting one model to do everything. Consistency improves sharply when you assign narrow jobs to different tools and stop asking a single generation to be simultaneously a portrait, a storyboard, and a performance.
A practical division of labour:
- Still generation model: creates and maintains the reference sheet. Optimise for detail fidelity at low cost.
- Image-to-image model: transfers wardrobe, lighting, and poses onto the locked character without regenerating the face.
- Image-to-video model: animates an approved still. This is where identity is most likely to survive, because the first frame is already correct.
- Text-to-video model: best reserved for inserts, environments, and shots where the character is small, distant, or off-screen. It is the weakest option for close-ups of a recurring lead.
- Motion transfer or pose-driven tools: ideal when body language must repeat precisely, such as a recurring gesture or a matched cut between two shots.
Decision criteria for moving to video: if the shot shows the face larger than roughly a quarter of the frame, start from an approved still rather than a text prompt. If the character is a small figure in a wide shot, text-to-video is usually faster and the consistency risk is low.
The Shot-List Discipline That Prevents Drift
Most inconsistency is created before generation, in a shot list that was never specific enough. A useful shot list entry pins four things: framing, wardrobe state, lighting direction, and the character beat.
Compare two entries for the same shot:
- Weak: "Anna walks into the office, looks worried."
- Strong: "Medium shot, waist up, Anna enters frame left and stops two steps in; charcoal blazer open, white crew-neck, silver hoops visible; soft window light from camera right; worried expression resolving into a held breath."
The strong version contains only facts that a renderer can act on and a reviewer can verify. It also removes the ambiguity that makes a model improvise wardrobe and lighting, which are two of the four drift categories.
Two more habits pay off. Keep the camera language stable across a scene, because a sudden change in lens feel reads as a change in character even when the face is identical. And limit the number of distinct costumes per character to whatever the story genuinely needs, since each new costume is a new consistency surface.
Iterating Without Losing the Face
Once you have an approved still, resist the urge to regenerate from scratch. Small, targeted edits preserve identity; full regenerations gamble with it.
A working loop for a new shot:
- Duplicate the approved still as your base.
- Change one variable at a time, starting with pose, then lighting, then wardrobe state.
- Verify the face against the reference sheet before animating anything.
- Only when the still passes, animate it and review the first second of motion.
- If the opening frames drift, go back to the still rather than re-rolling the video, because re-rolling video from a flawed still rarely rescues identity.
Keep a short log of what worked. After a few projects you will have a personal list of prompt phrasings and settings that reliably hold a face, and that list is worth more than any general advice.
QA Checklist for a Finished Sequence
Run this pass on the assembled cut, not on individual clips, because many consistency problems only become visible in sequence.
- Faces: pause on each close-up and compare hairline, eyebrow shape, and earring type against the reference sheet.
- Wardrobe: check fasteners, sleeve length, and collar shape. Open jackets that close between shots are the most common giveaway.
- Colour: confirm the palette does not shift between shots that are supposed to share a location.
- Proportion: compare head-to-shoulder width across shots. A quietly rescaled head is a common and distracting defect.
- Motion: check that gait and gesture tempo feel like the same person.
- Continuity: verify props and positions across cuts, especially anything the character carries.
When a shot fails, fix the cheapest thing first. A wardrobe mismatch is often fixable with an image edit; an identity mismatch usually means regenerating the still. Do not try to patch a broken face with a video-level edit.
How Many Shots Before You Should Reset
Long sequences accumulate small errors, and at some point patching costs more than restarting. A useful heuristic: if a shot requires more than three corrective edits and still fails the checklist, regenerate its reference still from the bible and rebuild the shot from there.
The same logic applies to projects. If a character has been through two costumes, three lighting setups, and a design revision, re-derive a fresh reference sheet from the bible rather than continuing to stretch an early still. Consistency is a property of your source material, not evidence of persistence.
Fixes for the Four Most Common Consistency Failures
The face changes at every cut. Your reference sheet is not specific enough, or you are generating from text prompts instead of stills. Fix the sheet first, then switch close-ups to image-to-video.
Wardrobe details wander. The bible does not name fasteners, necklines, or colours precisely enough, or your shot list omits wardrobe state. Add a single clause naming the garment state to every prompt.
The character looks younger or older. Age is carried by skin texture and facial volume, both of which shift with lighting. Keep lighting direction stable across shots and avoid mixing harsh and soft setups within a scene.
Everything matches but it still feels wrong. This is usually motion and pacing rather than appearance. Compare gesture tempo across shots and align the camera language before you touch the character description.
FAQ
Do I need to train a custom model to keep a character consistent?
Not for most projects. A well-built reference sheet plus an image-to-video pipeline handles typical short-form work. Custom training becomes worthwhile when a character appears across many separate projects or must hold up in extreme close-ups repeatedly.
How many reference images are enough?
Four views, front, three-quarter, profile, and full body, cover the vast majority of shots. More images help mainly when wardrobe changes frequently.
Should I write the character description into every single prompt?
Yes, in a short fixed form. Copy the same clauses verbatim rather than paraphrasing, since small rewording changes are a quiet source of drift.
Is text-to-video ever safe for a recurring character?
For wide shots where the face is small, yes. For dialogue or close-ups, start from an approved still every time.
What is the most common cause of drift I can fix today?
Improvised lighting. Lock a single lighting direction per scene, describe it explicitly, and much of the apparent identity change disappears.
How do I keep a sequence consistent when multiple people work on it?
Share one character bible file, one locked reference sheet, and one shot list template. Consistency across a team is a documentation problem more than a technical one.
Putting It Together
Character consistency rewards process over cleverness. Write a bible with observable facts. Build a reference sheet with four views before you animate anything. Separate your tools by job, keeping close-ups on an image-to-video path and reserving text-to-video for shots where the face is not the subject. Write shot lists specific enough that a renderer has no room to improvise wardrobe or lighting. Then run a sequence-level QA pass, because that is where the defects actually show up.
If you work through that loop a few times, the payoff is larger than clean continuity. You stop spending generations on repair and start spending them on choices: which performance, which framing, which beat. That is what turns a folder of clips into a sequence someone can follow from beginning to end, and it is the part of the craft that no single model will do for you.
