Why Script and Shot Design Still Decide Whether an AI Video Works
Generative video models have become remarkably good at producing a single beautiful clip. Water moves plausibly, skin keeps its texture, camera pushes feel intentional, and a two-second close-up can look like it came off a real set. What those models still cannot do is decide what the video is about. That part has not been automated, and it is the reason most AI-generated videos feel flat even when every individual frame is impressive.
Think of any AI video pipeline as three distinct layers. The first is story: what happens, in what order, and why anyone should care by the end. The second is shot design: how each beat is framed, lit, moved through, and cut against its neighbours. The third is generation: which model renders which shot, at what resolution, and with which prompt. Teams that collapse these three layers into one step โ typing an idea into a text box and hoping โ get footage. Teams that keep them separate get films.
This guide lays out a practical workflow for the first two layers, plus the handoff into generation. It is tool-agnostic on purpose. Whatever model or editor you use, the planning discipline is what travels with you, and it is what makes the difference between a demo reel and something a client actually signs off on.
Start With a Story Spine, Not a Prompt
A prompt describes an image. A script describes a change. The single most useful habit you can build is refusing to write a generation prompt until you can state, in one sentence, what is different at the end of the video compared to the beginning.
Write a one-line premise with a turn
A working premise has a subject, a situation, and a turn. "A barista makes coffee" is not a premise; it is a setting. "A barista who has never left her city pours a cup that reminds a stranger of home, and she decides to follow him" is a premise. You do not need a feature film. You need a change.
Test your premise with three questions. Who is the subject in one noun? What do they want in one verb? What stands in the way in one clause? If you cannot answer all three, the video will drift, and no amount of camera movement will hide the drift.
Build a beat sheet, then a scene list
A beat sheet is a numbered list of emotional or informational turns, not shots. For a sixty-second product teaser, six beats is plenty: problem, failed attempt, discovery, first success, doubt, resolution. For a three-minute explainer, twelve to fifteen beats keeps momentum without overloading the viewer.
Once the beats hold together on their own, expand each beat into a scene line: location, time of day, characters present, and the single action that must be visible. This is also the moment to delete beats that exist only because they were fun to imagine. A beat that does not change the subject's situation is a beat the viewer will skip forward through.
Write dialogue and voice-over for synthetic performers
Synthetic voices read text literally, so write for breath, not for page. Keep sentences under fifteen words. Break long clauses into separate lines so the voice engine inserts a pause where a human would. Avoid stacked subordinate clauses and homographs that can be stressed two ways.
If a character speaks on camera, keep lip-sync shots short โ two to four seconds โ and cover the rest of the dialogue with reaction shots, inserts, or wide shots where the mouth is not the focal point. This one decision removes most of the uncanny-valley feeling from AI dialogue scenes.
Turn the Script Into a Shot List
A shot list is where a script stops being literature and becomes a production plan. Build it in a spreadsheet or a table: shot number, beat, description, shot size, camera move, duration, prompt draft, and status. Eight columns is enough for almost any AI video under five minutes.
The shot vocabulary you actually need
You do not need the full grammar of cinema. Six shot sizes and four moves will cover ninety percent of AI video work:
- Wide establishes place and scale.
- Medium carries dialogue and body language.
- Close-up carries emotion and product detail.
- Extreme close-up carries texture and emphasis.
- Over-the-shoulder carries point of view and conversation.
- Insert carries information the audience must read.
For movement, use static, slow push in, slow pull out, and lateral tracking. Fast movement is where AI video breaks most visibly: limbs smear, backgrounds warp, and physics stops making sense. Save speed for cuts, not for camera work.
Coverage: how many shots per beat
A useful rule is one to three shots per beat, with a hard ceiling of five for a beat you want the viewer to remember. More shots mean more generation attempts, more continuity risk, and more places for the edit to sag.
Plan coverage deliberately. Every beat should have an establishing shot, a subject shot, and a detail shot. When one of those three is missing, editors fill the gap with a shot from the wrong beat, and the sequence starts to feel random even if no individual shot is bad.
Duration and pacing math
Add a duration column and actually sum it. A sixty-second video with an eight-second intro, a ten-second outro, and a four-second title card leaves thirty-eight seconds of story. That is roughly nine shots at four seconds each. Knowing this before you generate anything saves hours.
As a starting rhythm, cut faster in the first third, slow down in the middle, and allow one held shot near the end. Contrast is what makes pacing readable. A video where every shot lasts three seconds does not feel energetic; it feels flat, because the viewer has no baseline to compare against.
Write Prompts That Read Like Direction
A prompt is a shot order written for a model instead of a crew. That framing fixes most prompt problems, because directors do not describe a scene vaguely โ they specify subject, action, framing, movement, light, and mood, in that order.
The six-slot prompt formula
Use a consistent structure so you can debug one slot at a time:
- Subject โ one person or object, with two or three stable physical details.
- Action โ one present-tense verb phrase, not a sequence.
- Framing โ shot size plus angle, for example "medium shot, eye level."
- Camera movement โ one move only, or "static camera."
- Lighting โ direction and quality, for example "soft window light from camera left."
- Look โ film stock, lens, grade, or era references that give the image a personality.
"Marta, a woman in her fifties with cropped grey hair and a canvas apron, lifts a ceramic cup / medium shot, eye level / slow push in / warm window light from the left, deep shadows behind / 35mm film look, muted palette." That is a shot order. Compare it to "woman with coffee, cinematic" and the difference in predictability becomes obvious.
Constraints, negatives, and guardrails
Constraints do more work than adjectives. Specify what should stay fixed: background elements, wardrobe, the direction the subject faces, the time of day. Then state what must not appear โ text overlays, extra fingers, drifting camera, crowds, lens flares if the look is clean.
Keep one variable per iteration. If you change the lighting, the framing, and the mood at the same time, you will not know which change improved the shot.
Iterating without breaking the look
Generate the same shot three to five times before judging the prompt. Then change one slot and regenerate. Save the winning prompt in your shot list next to the shot number, and reuse its phrasing pattern for every shot in the same scene. Consistent phrasing is the closest thing AI video has to a consistent director of photography.
Continuity Is a System, Not a Memory Test
Continuity failures are the most common reason a technically good AI video feels amateurish. A character's jacket changes shade, the sun moves between cuts, or the prop in the hero shot disappears in the reaction shot. None of these are model failures; they are planning failures.
Character and wardrobe locks
Write a character sheet for every recurring person: age range, hair, build, two wardrobe items, one distinguishing feature, and a reference image you regenerate from when a shot drifts. Paste the same descriptive phrases into every prompt that includes that character. Never paraphrase the wardrobe โ paraphrasing invites the model to reinterpret.
Location, light, and colour continuity
Lock three things per scene: location, time of day, and the direction of the key light. If a scene takes place at golden hour, every shot in that scene carries the same warm rim light. If the camera crosses an axis, decide that deliberately and keep it for the rest of the scene, because crossing the line mid-sequence confuses viewers about who is where.
Colour continuity matters more than people expect. Pick three anchor colours for the video and check each shot against them. A shot that introduces a loud fourth colour will pull attention away from the beat it was meant to support.
A lightweight continuity tracker
Maintain a second sheet with one row per shot and columns for wardrobe, location, time of day, key light direction, colour anchors, and props visible. Fill it in as you finalise prompts. Before generating, scan for conflicts: two shots in the same scene at different times of day, a prop that vanishes, a character who changes height relative to the frame edge.
Reuse: Build a Shot Library Instead of Starting From Zero
Every project produces reusable assets: establishing shots, transitions, texture inserts, background plates, and lighting setups that worked. Tag them by function โ "establishing city," "hands detail," "neutral transition" โ rather than by project. A transition that worked in a product video works in a training video.
Reuse also applies to prompts. Keep a small library of prompt skeletons: one for dialogue coverage, one for product hero shots, one for landscape establishing shots, one for abstract texture. Fill in the slots and you have a shot order in thirty seconds.
Generated assets extend this further. A single approved frame can be reused as a reference for style consistency, as a thumbnail, as a background for a title card, or as the starting point for an animated version of the same composition. Plan for this at the shot-list stage by marking which shots are "anchor frames" โ the ones worth approving carefully because everything else will be measured against them.
Sound, Voice, and Edit Rhythm
AI video viewers forgive imperfect images far more readily than bad sound. Budget real time for audio, and treat it as a second edit pass rather than a final polish step.
Start with the voice track. Once narration is locked, its pauses define where cuts should land. Cutting picture to a finished voice track produces a rhythm that feels intentional; cutting voice to finished picture almost always sounds rushed.
Add ambience under every scene, even quiet ones. Room tone, distant traffic, wind, or a low pad gives the ear continuity across cuts. Then place music last. Music should follow the edit's emotional shape, not fight it. If the music track has a strong build, move the beat that lands hardest so it coincides with the build โ do not simply loop the track and hope.
Finally, mix for the smallest speaker your audience will use. If dialogue disappears on a phone speaker, the video fails at the first step regardless of how strong the visuals are.
Quality Control: How to Review a Rough Cut Without Fooling Yourself
Review in passes, not in one sitting. Watch once with sound off to check whether the story reads visually. Watch again with picture off to check whether the audio carries the narrative. Then watch once at normal speed, start to finish, without pausing to take notes.
The pause-free pass is the one that matters. If you stop to fix something, you stop experiencing the video the way an audience will. Note problems only after the pass ends, and prioritise them: a broken story beat outranks a slightly soft shot, and a slightly soft shot outranks a colour mismatch no one will notice on a phone.
Run a technical checklist at the end: aspect ratio consistency, safe areas for captions, audio peaks, frame rate consistency, on-screen text legibility, and any placeholder audio still sitting in the timeline. Placeholders survive to delivery more often than anyone admits.
Common Mistakes and How to Fix Them
Starting with a model instead of a story. Fix: write the premise and beat sheet first, then choose tools that fit the beats rather than bending the beats to the tools.
Overlong shots. Fix: cut the duration of every shot by a third and watch again. If nothing is lost, the original pacing was lazy.
Prompt drift. Fix: freeze the six-slot structure and the character sheet, and change only one slot per iteration.
Inconsistent lighting across a scene. Fix: define key light direction for the scene once and copy that phrase into every prompt in it.
Too many camera moves. Fix: allow one move per shot. Two moves is a different shot, and it should be its own entry in the shot list.
Ignoring audio until the end. Fix: lock voice-over before the picture edit and build the ambience bed before the music.
No anchor frames. Fix: approve two or three frames per scene deliberately, then use them as your consistency reference for everything else in that scene.
FAQ
How long should an AI-generated video be?
For a first project, keep it under ninety seconds. Short videos let you complete the full workflow โ premise, beats, shot list, generation, sound, review โ before fatigue sets in. Longer pieces are easier once the pipeline is habitual.
Do I need a storyboard artist or special software?
No. A table with shot number, description, framing, movement, duration, and prompt status does everything a storyboard does for planning purposes. Add a rough thumbnail later if it helps you think visually.
What is the biggest cause of inconsistent characters?
Paraphrasing descriptions between prompts. Reuse the exact same wording for hair, wardrobe, and distinguishing features every single time, and keep one reference frame per character.
How many generation attempts should a shot get?
Three to five. If a shot still fails after that, the problem is usually the prompt's framing or action slot, not the model. Simplify the action and try again.
Can I mix output from multiple video models in one project?
Yes, and it is often the best approach. Assign models by shot type โ one for dialogue coverage, one for landscape establishing shots, one for texture inserts โ then normalise colour, grain, and audio in the edit so the seams disappear.
How do I handle on-screen text?
Add text in the edit, not in generation. Generated lettering is unreliable and often unreadable at small sizes. Design captions and titles as an edit-layer element with a consistent typeface and safe margins.
What should I do when a scene feels boring even though every shot looks good?
Look at the beat sheet, not the shot list. Boring usually means no beat changes the subject's situation. Rewrite one beat so something is gained or lost, then re-plan only the shots affected by that beat.
