Why AI Video Needs a Director, Not Just a Prompt
Generative video has reached the point where a single well-written prompt can produce a striking clip. That is exactly why the work has shifted. When anyone can generate a beautiful five-second shot, the scarce skill stops being generation and becomes continuity, intention, and structure across dozens of shots that must feel like one piece. A director decides what the viewer should feel at each beat, which part of the frame matters, what must stay identical between shots, and where to spend the effort. Those decisions translate directly into clearer prompts, fewer retries, and a final cut that holds together.
If you treat AI video as a prompt lottery, you get isolated clips that look impressive and mean nothing together. If you treat it as a directing problem, you get sequences. What follows is a repeatable workflow for the second approach.
The Four Layers of an AI Video Workflow
Every project, from a fifteen-second vertical ad to a six-minute brand film, moves through four layers. Mixing them up is the most common reason teams burn hours without a usable result.
The story layer holds intent: the beats, the emotional arc, and what changes between the first frame and the last. The shot layer converts that intent into a shot list with framing, action, duration, and continuity notes. The generation layer turns each shot into prompts, references, model choices, and retries. The assembly layer is the edit, sound design, color, captions, and export.
Each layer fails in a recognizable way. A weak story layer makes the finished video feel random even when every clip is attractive. A missing shot layer shows up as poor pacing, because nobody decided how long an idea deserves. A sloppy generation layer produces morphing characters, dissolving hands, and melting crowds. A neglected assembly layer lets twenty good shots add up to a weak video.
The ordering matters. Lock the story layer before generating at scale. Rewriting a paragraph on paper costs minutes; discovering the same problem after forty generated clips costs days.
Pre-Production: Turning a Script into a Shot List
The shot list is the highest-leverage document in an AI video project. It is where a vague script becomes something a machine can be asked to produce, and where problems surface while they are still cheap to fix.
Read the script out loud and mark every moment where the image must change. Those marks are your shots. Then write one card per shot with these fields: shot ID, purpose, duration, framing, action, continuity, and audio intent.
The one-line shot card format
A card that fits on one line stays usable under deadline pressure:
SC03-SH02 | purpose: reveal the abandoned workshop | 3.0s | medium wide, slow push in | dust suspended in a shaft of light, tools on the bench | continuity: blue jacket on the hook, rain at the windows, late afternoon | audio: low room hum
Seven fields, one line. Sixty of these cards give you a film. Sixty prompts with no cards give you a folder of clips.
Camera language that models understand
Generated footage responds to concrete vocabulary. Lens: wide around 24mm, normal around 50mm, portrait around 85mm. Distance: wide, medium, close-up, extreme close-up. Angle: eye level, low, high, over the shoulder. Movement: static, slow push in, pull out, pan left or right, tilt, handheld drift, orbit. Speed: real time, slow motion, time-lapse.
Adjective stacking does not work. Words like cool, epic, and dramatic mean different things to different people and almost nothing to a model unless they are tied to a visual decision. Cinematic is not a description; a 50mm lens with shallow depth of field under warm practical light is.
Duration discipline
Narrative shots usually work between two and five seconds. Establishing shots can stretch to eight or ten when the frame contains movement. Decide duration on the card, before generation, because that number determines whether a moment needs one clip or two beats.
Prompt Architecture: Writing a Shot the Model Can Execute
A prompt is a production instruction, not a wish. The most reliable structure is a five-slot formula plus a style suffix and a short constraint list: subject, action, environment, camera, light.
Wish: a cool cinematic shot of a woman in a city at night, dramatic mood.
Instruction: A woman in her thirties wearing a charcoal wool coat walks toward the camera along a wet cobblestone street; rain-slicked storefronts behind her; medium shot at eye level with a slow handheld drift; overcast blue-hour light with warm shop-window spill; shallow depth of field, 50mm, photorealistic, subtle grain; no text, no lens flare, no visible crowd.
The second version names wardrobe, direction of movement, surface texture, lens, light source, and mood through specifics rather than adjectives. It also states what to avoid.
Negative constraints and continuity locks
Constraints do the quiet work. For each recurring character, lock wardrobe, hair, and one signature prop. For each location, lock time of day, weather, and dominant palette. For the project, lock the lens family and grade direction. Write these as a block you paste into every relevant prompt: same charcoal coat, same shoulder-length dark hair, same satchel, overcast blue-hour light, cool teal-and-amber grade. Consistency is not a switch you flip; it is a set of variables you refuse to change.
Prompt length and hierarchy
Forty to ninety words is a practical sweet spot. Front-load what matters most, because early tokens tend to carry more weight. If the shot is about a hand closing a notebook, the hand comes first and the room description comes later.
Consistency Across Shots Without a Full Studio
Consistency separates a sequence from a collection. Three artifacts solve most of it.
Build a reference sheet first
Before generating any moving footage, create a character sheet of stills: front, three-quarter, profile, and back views, plus two or three wardrobe variants and a neutral expression. Pick one hero image. Every shot featuring that character starts from the hero image rather than from text alone.
Write a style bible
The style bible is one page: palette swatches, lens family, grain amount, contrast, grade direction, typography rules, and a sound motif. Its purpose is to end debates. When two shots look like they came from different projects, the style bible tells you which one is off.
Maintain a continuity tracker
A simple table works. Columns: shot ID, character, wardrobe, props, location, time of day, weather, model used, prompt version, status. Fill it while you generate. During the edit, scan the wardrobe column for accidental changes and the light column for jumps that read as errors.
Choosing the Right Model for Each Shot
No single model wins every shot type. The practical skill is routing: matching a shot to the tool whose strengths line up with the hardest requirement of that shot.
Decision criteria
Ask five questions. Does the shot depend on motion coherence, such as fast action, crowds, or interacting hands? Does it need photorealism or a stylized look? Does it contain legible text, signage, or a logo? How long is the clip, and can it be extended cleanly? What inputs does the tool accept, and how faithfully does it honor a reference image?
Secondary criteria are speed and repeatability. A model that produces slightly prettier results but takes four times longer per attempt is often the wrong choice for a shot you will retry twenty times.
A routing heuristic
For talking heads, start from a locked reference image, animate with minimal camera movement, and handle lip sync in a dedicated pass. For wide establishing shots, text-to-video with one slow camera move is usually enough, because small inconsistencies disappear at that scale. For product and macro shots, use image-to-video with controlled lighting so the object never changes shape. For stylized animation, commit to one visual style and build a reference sheet in that style first. For complex action, generate short beats and join them in the edit.
| Shot type | Best starting point | Main risk |
|---|---|---|
| Talking head | Image-to-video with locked reference | Unnatural mouth movement |
| Wide establishing | Text-to-video, slow move | Continuity drift at scale |
| Product macro | Image-to-video, controlled light | Shape morphing |
| Stylized animation | Style-locked reference sheet | Style creep |
| Fast action | Short beats edited together | Melted limbs and hands |
| Insert shots | Tight framing, minimal motion | Unreadable detail |
Test before you commit
Generate one hero shot with two or three candidates and compare them side by side at full size, not on a phone. Choose the model for the whole sequence at that point. Switching tools mid-sequence is the fastest way to create visible seams.
Assembly: The Layer That Decides Perceived Quality
Audiences forgive imperfect frames far more readily than bad sound and bad pacing. Edit rhythm and sound design carry more perceived production value per hour of work than any other pass.
Build a rough cut with placeholders where shots are missing, then watch it once at normal speed without pausing and note where attention drops. Those notes, not opinions about individual shots, dictate what to cut.
Lay room tone under every scene so cuts do not click. Add foley for visible actions, a riser or whoosh for transitions, and a music bed that ducks under narration. Keep voice-over and dialogue in a consistent loudness range across the piece. Silence is a tool as well: pulling music out for two seconds before a reveal does more than a louder hit.
Plan vertical, square, and widescreen versions before the edit if you need them, because reframing during delivery always costs more. Decide whether captions are burned in or delivered as a sidecar file, and check safe areas so text never collides with platform interface elements.
Common Mistakes and How to Fix Them
- Generating before planning. Refuse to generate until the shot list has durations and continuity notes.
- Writing prompts as mood boards. Use the five-slot formula and replace every adjective with a visual decision.
- Changing style mid-project. Write the style bible on day one and treat deviations as errors.
- Asking one clip to do too much. Split complex action into two or three short beats.
- Ignoring aspect ratio and frame rate until delivery. Set them in the brief and keep them constant.
- Retrying randomly. Change one variable per attempt and note what changed.
- Skipping reference images for characters. Build a sheet and always start from the hero image.
- Fixing everything in post. Solve continuity in the prompt and the tracker instead.
Scaling the Workflow: Templates, Versioning, and Review Loops
Once a workflow works for one video, it should work for twenty. That requires naming, versioning, and review gates.
Adopt a naming convention that encodes project, episode, scene, and shot: PROJECT-EP02-SC03-SH04-v03. Sort the folder by name and the edit order appears automatically.
Keep a prompt library of reusable blocks: the style suffix, lighting presets, and constraint lists for each character. Most of a new prompt is assembled from proven pieces rather than written from scratch.
Set review gates at three points: after the shot list, after the hero shot from each scene exists, and after the first assembly. At each gate ask three questions. Does this serve the story beat? Does it match the style bible? Would a stranger understand what is happening without explanation? Projects that pass three gates rarely need a rescue later.
Frequently Asked Questions
How long should an AI-generated shot be?
Two to five seconds for narrative work. Establishing shots can run longer when the frame contains movement. If a shot feels short in the edit, add a second angle rather than stretching one clip.
Can I keep the same character across many shots?
Yes, but not with text alone. Build a reference sheet, choose a hero image, and start every shot from that image. Lock wardrobe, hair, and a signature prop in writing, and record them in the tracker.
Do I need a full script before generating?
You need a locked story layer. That can be a script, a beat sheet, or a detailed shot list, but the intent must be settled first.
Is text-to-video or image-to-video better?
Image-to-video is more controllable because the starting frame fixes composition, wardrobe, and palette. Text-to-video is faster for landscapes and abstract shots where continuity does not matter.
How many attempts should a shot get?
Set a budget, usually five to fifteen attempts for complex shots. If a shot exceeds it, the problem is the prompt or the model choice, not luck.
How do I handle dialogue?
Generate the performance and the line separately. Animate a locked reference, record clean audio, and handle lip sync in a dedicated pass.
What causes lighting to jump between shots?
Prompts that describe mood instead of sources. Name the source and time of day, such as warm shop-window spill at blue hour, and repeat that phrase in every shot within the scene.
Is it worth generating my own reference images?
Almost always. A small set of purpose-built references removes the most common cause of retries: the model inventing a different face, outfit, or object each time.
