Turning one beautiful AI portrait into a recurring character for vertical shorts is the moment most projects fall apart. The face that looked flawless in a still suddenly shifts jawline between shots, the jacket changes shade, the lighting resets in every clip, and the viewer scrolls away before the hook lands. The fix is not a better single prompt. It is a production system: a character bible, a locked reference set, a shot list, motion rules, and a quality-control pass that happens before you export.
This guide walks through that system end to end, from auditioning the hero still to publishing a series where the character is instantly recognizable in every clip.
Why Consistency Breaks When a Still Starts Moving
A still image is one sample. A short is hundreds of correlated samples cut into a timeline, and small deviations compound. When each clip is generated independently, the model reinterprets the character slightly every time. Individually those clips look fine. Back to back, they read as a different person.
Three kinds of drift show up in practice:
- Identity drift. Eye spacing, nose shape, jaw width, hairline, and apparent age shift gradually. This is the most damaging and the hardest to notice in isolation.
- Style drift. Grain, contrast, lens character, and color temperature change between shots, so the cuts feel like a patchwork rather than one film.
- Continuity drift. Wardrobe, props, time of day, and background details reset or morph. A jacket changes from olive to brown, a mug disappears, a room rearranges itself.
There is also an economic reality. Viewers on vertical feeds decide in under two seconds whether a face is worth following. Consistency is a trust signal: it tells the audience this is a character, not a random image. Brands especially cannot afford a mascot that looks like a different person in every upload. The cost of inconsistency is not aesthetic, it is retention.
So the goal is not perfection in one frame. The goal is a stable character identity that survives motion, lighting changes, wardrobe changes, and a dozen different camera angles.
The Character Bible: One Source of Truth
Before you generate a single video clip, build a character bible. This is a folder plus a short document, and it does more for consistency than any single setting.
The minimum viable reference set
A workable reference set contains:
- A neutral front-facing portrait in even light.
- A three-quarter angle.
- A profile view.
- A full-body shot for proportion and wardrobe silhouette.
- An expression sheet: neutral, smiling, surprised, concerned, laughing.
- Two lighting conditions of the same pose, such as soft window light and hard directional light.
- One "in-scene" still showing the character in a real environment at the intended aspect ratio.
Export these cleanly at the highest resolution your generator supports, with no compression artifacts, filters, or heavy stylization. Every reference you add should be something you would be happy to see reproduced forever, because the model treats all of them as equally valid evidence.
Describe the character in fixed tokens
Write a short immutable block that describes only the character, and keep the wording and the order stable across every generation:
Character block: a woman in her early thirties, East Asian, shoulder-length straight black hair with a blunt fringe, narrow oval face, high cheekbones, dark brown eyes, small scar above the left eyebrow, olive-green utility jacket with a silver zipper pull.
Then add a variable scene layer for each shot: location, time of day, camera, action, mood. Keeping the character block frozen and only swapping the scene layer is the single most effective habit you can build. Changing two traits at once makes it impossible to know which change caused the drift.
Version the bible
Name every folder with a version, for example character-aya-v1, character-aya-v2. When you improve the reference set, create a new version and note what changed. Never mix references from two versions inside one series, because the model will interpolate between them and produce an average that matches neither.
A Repeatable Stills-to-Shorts Pipeline
The following pipeline takes roughly half a day for the first episode and about an hour for each episode after that.
Stage 1: Audition the hero still
Generate 20 to 30 candidates for the first still. Then judge them for motion, not beauty. A face that reads clearly at thumbnail size, has a strong silhouette, and has simple, high-contrast features will survive animation far better than a delicate, low-contrast portrait. Pick two candidates, not one, so you have a fallback if the chosen look degrades in video.
Stage 2: Expand into a reference set
Use variations, image-to-image, or a controlled editing workflow to build the rest of the reference set from the hero still. Keep the prompt block frozen and change only the pose or lighting instruction. Reject any output that alters brow shape, eye spacing, nose width, or apparent age, even if it is more attractive. Consistency beats novelty at this stage.
Stage 3: Write the shot list before generating video
List every shot with five fields: shot number, framing, action, duration, and audio line. Eight to twelve shots is enough for a 30 to 45 second short. This forces you to plan around the places where consistency usually breaks, such as profile turns, close-ups, and shots where hands enter the frame.
Stage 4: Generate in small batches
Generate three or four takes per shot using the same character references and, where the tool supports it, the same seed family. Review each take twice: once full size to catch artifacts, once at thumbnail size to catch identity drift. Reject fast. A clip that is 90 percent right will still break the series.
Stage 5: Assemble, grade, and lock
Edit in a normal timeline, then apply one unified grade across the whole short so style drift disappears. Add grain, a slight vignette, and consistent sharpening. Save a golden frame from each shot in a folder; that folder becomes the reference for the next episode.
Camera, Motion, and Continuity Rules
Keep a small camera vocabulary
Pick three or four moves and repeat them: slow push in, slow pull out, gentle lateral drift, static. Reusing the same framing language stops the model from re-learning a new composition every clip and makes the series feel intentional rather than assembled from unrelated fragments.
Design motion that hides seams
Start every clip from near-stillness and let motion build. Avoid fast head turns right at a cut, because that is where facial geometry and hair rendering are most likely to fail. Prefer medium shots over extreme close-ups in the first few episodes. Keep hands out of frame unless you are prepared to review them carefully, since hands still drift more than faces.
Also cap clip length. Six to ten seconds per generated shot keeps drift manageable; longer generations accumulate small errors that become obvious by the end.
Use physical anchors
Give the character one consistent wearable and one consistent prop. A silver zipper pull, a red scarf, a specific backpack, a wristwatch. These anchors give the eye something stable to track and make continuity errors easier for you to spot during review.
Matching Tools to the Job
No single tool covers the whole pipeline well. Map tools to stages instead of looking for one monolithic solution.
| Stage | Job | What to look for |
|---|---|---|
| Character design | Creating the hero still | Strong prompt control, style references, high output resolution |
| Reference expansion | Building the bible from one image | Image-to-image control, inpainting, consistent identity features |
| Video generation | Animating shots | Image-to-video, character or subject reference support, seed control |
| Lip sync and dialogue | Matching mouth shapes to audio | Accurate phoneme timing, head movement that stays natural |
| Voice | A consistent vocal identity | Cloned or designed voice that stays stable in tone and pacing |
| Assembly | Editing, captions, grade | Vertical templates, subtitle timing, unified color tools |
| Restoration and upscale | Fixing soft faces and noise | Face-aware upscaling that does not redraw identity |
A practical stack looks like this: one image generator for design work, one video generator with reference-image support, one dedicated lip-sync tool, one voice tool, and a standard editor. Resist adding a second video generator mid-series. Each engine has its own interpretation of the character, and mixing two of them guarantees style drift even when the identity holds.
Short-Form Specifics: Frame, Pace, and Sound
Vertical output changes the technical requirements. Shoot and generate at 9:16 from the beginning rather than cropping later; cropping a landscape generation destroys composition and often cuts the character awkwardly.
Keep the subject in the upper-middle third so captions and interface elements do not cover the face. Leave the bottom 20 percent clear for subtitles and safe zones. Plan the hook in the first second: a question, a visual surprise, or a line of dialogue that sets up a payoff. Do not open on a slow establishing shot; vertical audiences do not wait.
Audio deserves the same consistency discipline as the visuals. Use the same voice for every episode, keep loudness normalized across clips, and check that background music does not mask dialogue. If your character speaks, render lip sync after you have locked the video edit, not before; regenerating a shot always means regenerating the mouth animation.
Finally, publish at a consistent resolution and frame rate. Mixed frame rates are one of the most common reasons a series feels amateurish even when the character looks right.
Troubleshooting the Most Common Failures
The face morphs mid-clip. Reduce clip length, lower motion intensity, and give the tool more than one reference image. If drift persists, regenerate the shot from a still that matches the opening frame of the intended motion.
Wardrobe changes color between shots. Add the garment to the immutable character block with an explicit color and material, and confirm that every reference image shows the same garment. Vague words like "dark jacket" invite reinterpretation.
Lighting resets at every cut. Specify light direction and quality in the scene layer, then unify the final look with a single grade. A shared grade hides more lighting inconsistency than any prompt.
Hands look wrong. Reframe to cut hands out, keep them at rest, or use a dedicated inpainting pass on the hands. Do not rely on luck in wide shots where hands are prominent.
The background warps behind the character. Use shorter clips, avoid fast camera movement in complex environments, and prefer simple backgrounds for dialogue shots. Busy environments give the model more room to hallucinate.
Flicker or pulsing grain. This usually comes from upscaling or frame interpolation. Turn off interpolation first, then re-upscale with a face-aware model and compare.
Lip sync drifts by the end of a line. Split long lines into shorter clips, keep the head fairly still during speech, and generate mouth animation on the final locked edit.
The character looks five years older. Age drift often comes from references with inconsistent lighting or makeup. Rebuild the reference set with neutral lighting and one consistent makeup state.
Clips look like they come from different shows. That is style drift. Apply one grade, one grain setting, and one output resolution across everything, and stop switching video engines mid-series.
The voice does not match the face. Cast the voice early, before you generate too many shots. It is far easier to reduce motion intensity to match a calm voice than to replace a voice after twenty clips.
Scaling From One Character to a Cast
Once the pipeline works for a single character, adding a second one is mostly a naming and process problem. Give each character their own reference folder, their own immutable prompt block, and a shared style document that defines grade, grain, aspect ratio, and camera vocabulary for the whole series.
Generate a two-character scene as a separate challenge. Do not attempt to animate two consistent identities in a single generation until each one works alone. Instead, plan shot-reverse-shot coverage: generate each character separately in matching lighting and framing, then cut between them. It is not a compromise; it is how most professional dialogue scenes are built anyway.
Batch by character and by location, not by episode. Generating all shots for one location in one session keeps lighting and background consistent and dramatically reduces review time.
FAQ
How many reference images do I actually need?
Five to eight well-chosen references usually outperform twenty mediocre ones. Cover front, three-quarter, profile, full body, and at least two lighting conditions, and make sure they all agree on the same traits.
Can I keep one character consistent across different tools?
Partially, but expect drift. Tools interpret identity differently. Choose one engine for the whole series and use other tools only for narrow tasks like upscaling or lip sync.
Should I use a seed or a reference image?
Both, if available. Seeds help stabilize composition and style; reference images stabilize identity. Seeds alone will not hold a face, and references alone will not hold a look.
Why does my character look great in stills but strange in motion?
The stills are generated with a single strong reference, while video adds temporal reinterpretation and motion blur, which magnifies small facial asymmetries. Build expression variety into the reference set and keep clips short.
How long should each generated clip be?
Six to ten seconds is a practical ceiling for identity stability. Shorter is safer. You can always combine shorter clips into a longer continuous-feeling sequence in the edit.
Do I need a face-restoration pass?
Only if the output is soft. Restoration can shift identity if it is too aggressive, so test on one shot, compare at full size, and keep the setting mild.
How do I keep a series consistent weeks later?
Archive the reference set, the immutable prompt block, the grade settings, and one golden frame per shot. Rebuilding from an archived bible takes minutes; rebuilding from memory takes hours and rarely matches.
Pre-Publish Checklist
Before exporting, confirm that every clip uses the same character reference version, that the immutable character block is unchanged, that clip lengths stay inside your stability limit, that one grade covers the whole short, that captions sit inside safe zones, and that the voice and loudness are identical to previous episodes. Then watch the entire short once at thumbnail size, where identity drift is easiest to catch. If the character is still obviously the same person at that size, the series is ready to grow.
