Why pre-production decides the outcome of AI video
Most disappointing AI video projects do not fail at the model layer. They fail in the gap between a promising idea and a plan specific enough to execute. A generator asked for "a cinematic sequence where the hero walks through a rainy city" will produce something. It just will not be the something you pictured. The model has no access to your intent, no memory of your story's second act, and no way to know which details carry meaning.
Filmmaking solved this problem long ago with pre-production: a script, a shot list, a storyboard, and a production board that everyone works from. AI video needs the same discipline, and then a little more. A human crew infers intent from a conversation on set. A model infers intent from text, reference images, and a small set of parameters. Every unstated assumption becomes a random variable, and randomness compounds across shots.
The practical consequence is that planning time buys back far more than it costs. Twenty minutes spent defining framing, wardrobe, and light direction can save two hours of re-rolling shots that almost work. The workflow below treats an AI director's job as a real directorial job: write, plan, board, generate, assemble, review. Nothing here depends on a single vendor, because the engines worth using will keep changing while the process stays stable.
The three-layer pipeline: script, shot plan, generation
Think of an AI video project as three layers stacked on top of each other. Each layer answers a different question, and each one constrains the next. Skipping a layer does not remove it; it just moves the decision somewhere less convenient, usually into a prompt where it becomes guesswork.
Layer one: the written story
This is the layer of intent. It defines who wants what, what stands in the way, and how the audience should feel at each beat. It does not need to be formatted like a screenplay, but it does need visible action. Interior monologue and abstract mood descriptions cannot be filmed by a model that only renders what is physically on screen.
Layer two: the visual plan
The visual plan translates prose into coverage: which shots, in what order, from which angle, in which light, with which characters and props on screen. This is where a shot list and storyboard live. It is the single highest-leverage document in the entire pipeline, because it is the last place where changes are cheap.
Layer three: generation and assembly
Here you convert each planned shot into prompts and reference frames, generate variants, select the best take, and cut them together with sound. Problems that appear at this layer are usually traceable to a vague decision at layer two. When a shot keeps failing, resist the urge to rewrite the prompt ten times and instead ask what the shot list failed to specify.
Writing a script that survives generation
A script written for human actors and a script written for generation models are not the same document. Human performers fill gaps with intuition. Models fill gaps with averages drawn from training data. That difference changes how you should write.
Keep action visible and in the present tense
Replace "she realizes her brother has been lying for years" with "she stops mid-sentence, looks at the phone in his hand, and sets down her cup." The first version describes an internal state. The second describes observable behavior a camera can capture, and it gives the model concrete physical instructions: a facial shift, a prop, a gesture.
Write for continuity, not just for plot
Continuity is the hardest part of AI video, so build it into the script from the start. Give every recurring character three or four fixed visual anchors: hair shape and color, a signature garment, an accessory, a body type. Give every location a fixed time of day and light quality. If a scene does not need to move between day and night, do not let it. Every lighting change is a fresh opportunity for the model to invent a new world.
Read the script out loud at target runtime
A 90-second voiceover reads much shorter on paper than it feels in the ear. Read your narration aloud with a timer. If the script runs 130 seconds but you need 90, cut before you generate, not after. Trimming text is free. Re-generating a dozen shots to shave four seconds is not.
Building a shot list AI models can execute
A shot list is a table with one row per shot. The columns matter more than the format. A description that feels precise to a human can still be ambiguous to a model, so the discipline is to write descriptions that leave as little to interpretation as possible.
The minimum viable shot description
Every row should carry, at minimum: a slug (scene and shot number), a plain-language action, a framing size (wide, medium, close-up), a camera behavior (static, slow push, handheld follow), a lighting note, and the characters or props present. Add duration in seconds. That is seven fields, and together they remove most of the ambiguity that leads to unusable takes.
Here is a compact example for a 60-second brand film:
- S03-02 — Wide, static. Rain-slick street at dusk, neon signage reflecting in puddles. Character A enters from frame left, hood up, walks away from camera. Duration 4s.
- S03-03 — Medium close-up, slow push in. Character A stops, lowers hood, looks up at a lit window. Warm window light on the left side of the face, cool street light on the right. Duration 3s.
- S03-04 — Insert, static. Hand pushes open a heavy door; light spills out. No face visible. Duration 2s.
Note how each row gives an editor something to cut on: an entrance, a turn, a hand on a door. Shots that only describe a mood are difficult to cut because they have no internal event.
Grouping shots by location and lighting state
AI generation rewards consistency within a batch. Group all shots that share a location, time of day, and lighting setup, then generate them in one session using the same reference frame and the same descriptive vocabulary. When you jump between visual states, the model's interpretation drifts, and matching shots in the edit becomes a color-grading rescue operation.
Storyboarding without drawing skills
The word "storyboard" intimidates people who cannot draw. It should not. A storyboard is a thinking tool, not a portfolio piece. What matters is composition, screen direction, and the sequence of information, not line quality.
Rough blocking boards first
Start with rectangles on a page or a whiteboard. Inside each rectangle, mark only three things: where the subject sits in frame, where the camera is relative to them, and where the light comes from. Stick figures and arrows are enough. Ten boards sketched in fifteen minutes will expose problems in your shot list that no amount of prose review catches — for example, three consecutive close-ups with no establishing wide, or a conversation where both characters face the same direction.
Then move to reference frames
Once the blocking works, generate one still image per board to serve as a visual anchor. These frames do two jobs. First, they let you test framing, palette, and wardrobe before you commit to motion. Second, they become image references you can feed into your video model to lock character appearance and art direction across shots.
Keep the reference set small and ruthless. One canonical frame per character, one per location, one per distinct lighting state. If you build twenty references, you will spend more time deciding which to use than you spend generating. Name files by their function — char-a-canonical-front, loc-street-dusk-warm — so the correct reference is obvious weeks later when the project has grown.
Where boards save the most time
Boards pay off disproportionately in three situations: scenes with more than two characters, scenes that require a specific screen direction (a car driving left to right), and any sequence where a prop must change hands. Those are exactly the cases where models hallucinate extra people, flip orientations, or teleport objects. Having a board gives you an immediate, objective standard for rejecting a bad take.
Choosing the right model per shot type
No single engine is best at everything, and chasing a universal winner wastes time. A more durable approach is to match model strengths to shot types and accept that your project will use two or three engines.
Decision criteria that actually matter
- Motion complexity. Simple, human-scale movement (walking, turning, sitting) is well served by most modern engines. Complex interaction with objects, crowds, or fast action narrows the field quickly.
- Character consistency. If a face appears in more than three shots, prioritize engines with strong image-reference and identity-locking features over ones with marginally prettier output.
- Prompt adherence. Test each candidate with the same three-shot prompt set. The engine that follows the shot list most literally saves you the most time, even if single frames look less glossy.
- Duration and resolution. Some engines give you a handful of seconds per generation; others allow longer clips. Longer clips reduce edit seams but often drift in detail.
- Cost predictability of iteration. What matters is not the price per clip but how many attempts a shot typically needs. An inexpensive engine that needs twelve tries is more expensive than a pricier one that lands in three.
A practical allocation strategy
Use one engine for hero shots — the two or three frames that define the film — and a faster, cheaper engine for connective tissue: inserts, hands, feet, doors, transitions, and atmosphere. Reserve a specialized model for anything with text, logos, or precise graphic layout, because generative video engines still struggle with legible lettering. Document which engine produced which shot in your shot list; when a client asks for a revision three weeks later, you will not have to guess.
Prompt architecture: turning beats into prompt blocks
The most reliable prompts are not poetic. They are structured. Think of a prompt as a shot card with fixed slots, filled in the same order every time. Consistency in structure produces consistency in output, which is exactly what a multi-shot edit requires.
A workable slot order:
- Shot size and angle — "medium close-up, eye level, slightly off-center framing."
- Subject and action — who, doing what, in one present-tense sentence.
- Setting and time — location, weather, time of day.
- Lighting — source, direction, quality (soft window light from the left; hard overhead sodium).
- Lens and texture — focal length feel, grain, depth of field.
- Palette and mood — three color anchors maximum.
- Negative constraints — what must not appear (no on-screen text, no extra people, no warped hands).
Two habits make this system work. First, keep a shared "style header" — lens, grain, palette — and paste it into every prompt in a sequence, changing only the shot-specific slots. Second, version your prompts. Save them in the shot list next to the resulting take so that when a shot succeeds, you know exactly what produced it and can reproduce the look elsewhere.
When a shot fails, change one slot at a time. Rewriting an entire prompt after every failure destroys your ability to learn which variable mattered — and you will need that knowledge for the next twenty shots.
Sound, pacing, and the assembly edit
AI video gets planned visually and then judged with sound, which is unfair to the visuals. Sound does about half the perceptual work in a short film, and it is far cheaper to iterate than picture. Build a rough audio bed before you finish generating: scratch narration, temp music, and a handful of production effects like footsteps, rain, or a door.
Pacing is where inexperienced AI edits fall apart. Generated clips tend to be beautiful and slow, and cutting twenty of them together produces something that feels like a slideshow. Fix this in the edit with three techniques: cut on motion rather than on stillness, vary shot duration aggressively (a 4-second wide followed by a 1-second insert feels alive), and never let two visually similar shots sit adjacent unless the repetition is deliberate.
Finally, treat continuity as an audio problem too. Ambient sound should carry across a cut even when the visual world shifts slightly. A consistent room tone or street hum masks small inconsistencies between generated clips far better than a hard visual match ever will.
Quality control and common mistakes
Review AI footage the way an editor reviews dailies: with a checklist, not a vibe. Watch each clip three times — once for the whole frame, once for the subject only, once at half speed. Look for face warping, extra fingers, melting backgrounds, flickering textures, drifting wardrobe, and unwanted text. Flag problems with a timestamp and a severity level so you can decide whether a clip is salvageable with a trim or needs regeneration.
The failures that recur most often, and their usual causes:
- Characters change between shots. No canonical reference frame, or references were rebuilt for each generation session.
- Cuts feel jarring. Screen direction flipped, or the palette shifted because the style header was edited mid-project.
- Everything looks like one long shot. Shot list lacked variety in size and camera behavior.
- The story is unclear despite good visuals. No establishing wide before the detail shots, so the audience never learns where they are.
- Narration and picture fight each other. Script written before the shot plan, with no read-aloud timing check.
- Endless iteration on one shot. The prompt is being asked to solve a planning problem; go back to the shot list.
- The ending feels flat. No planned final image. Decide the last frame early and generate toward it.
A workable rhythm for a short project: one day writing and trimming to runtime, one day on the shot list and blocking boards, half a day on reference frames, two to three days generating in grouped batches, and one day on the edit and sound pass. The temptation is to invert that order and start generating immediately. The teams that finish projects are the ones that resist.
FAQ
How long should an AI video shot be?
Plan for two to five seconds per shot in narrative work and one to two seconds for montage. Longer generations tend to drift in detail and identity, and short clips give you more control in the edit. If a shot needs eight seconds of screen time, consider splitting it into two related shots rather than generating one long clip.
Do I need a storyboard if I already have a detailed shot list?
Yes, for anything with more than one character or more than one location. A shot list describes shots individually; a board reveals how they connect. Composition mistakes and screen-direction errors are nearly invisible in a table and immediately obvious in a sequence of rectangles.
How do I keep a character consistent across many shots?
Create a single canonical reference image and reuse it in every generation for that character. Keep the descriptive vocabulary identical across prompts — do not swap "short dark hair" for "bobbed black hair" mid-project — and generate shots featuring the same character in one batch whenever possible.
Should I generate stills first or go straight to video?
Generate a still for every shot before you animate anything. Stills cost a fraction of the effort, expose framing and lighting problems early, and double as image references for the video pass. The only exception is fast, disposable B-roll where the frame barely registers.
How many variations should I generate per shot?
Start with three to five. If none of them work, stop and re-read the shot description rather than generating ten more. Repeated failure is a signal that the prompt is asking for something the plan did not define, not that you need more attempts.
What is the biggest mistake in AI video production?
Treating generation as the beginning of the process instead of the third step. Every hour saved by skipping script, shot list, or boards is repaid with two hours of regenerating shots and a final edit that never quite holds together.
Can this workflow scale to longer pieces?
Yes, with one change: add a project bible. Keep a single document containing the style header, character references, location references, and the engine used for each shot family. On anything over three minutes, that document prevents the slow drift that makes the first and last scenes look like different films.
