Shot Design Is the Real Bottleneck in AI Video Production
Most people who begin working with generative video tools assume the hard part is the prompt. They spend hours hunting for the perfect string of words, then wonder why the output feels flat, generic, or disconnected from the story they imagined. The prompt is rarely the problem. The problem is that nobody decided what the camera should actually be looking at.
Generative video has collapsed the cost of producing footage. What used to require a crew, a location, and a lighting package now requires a sentence and a few minutes of compute. But it has not collapsed the cost of deciding. Choosing a shot is still a creative act, and it is where amateur output and professional output separate most visibly.
A useful way to think about this is to stop treating the model as a vending machine and start treating it as a director of photography who has read the script but has no opinions of their own. It will shoot whatever you describe, faithfully and literally. If you describe nothing, it invents something plausible and forgettable. Your job is to be the director: to decide the emotional purpose of each moment, then translate that purpose into shot size, movement, lens, light, and blocking.
This guide walks through a complete shot-design workflow for AI video. It assumes you are working with text-to-video or image-to-video models, possibly several of them, and that you want footage that cuts together into something coherent rather than a folder of unrelated clips.
Step 1 — Turn the Script Into Scene Cards and Beats
Before writing a single prompt, convert your script into scene cards. A scene card is a short structured note that answers: where are we, who is present, what changes, and what should the audience feel at the end. This is pre-production in its cheapest, fastest form, and it prevents the single most common failure in AI video: generating beautiful shots that do not belong to the same film.
What belongs on a scene card
- Scene number and slugline: interior or exterior, location, time of day.
- Characters present: names, not descriptions, so you can reuse the same reference notes later.
- Dramatic beat: the one thing that changes between the start and end of the scene.
- Emotional temperature: tense, tender, absurd, clinical, elegiac.
- Target duration: three seconds or thirty, which changes how many shots you need.
- Must-have imagery: any visual the story depends on, such as a letter, a scar, a rain-soaked street.
Beat mapping in practice
Take a simple scene: a courier delivers a package to an apartment and realizes the recipient has been gone for weeks. Three beats: arrival, discovery, decision. That immediately suggests three shot clusters. Arrival wants an exterior establishing shot and a doorway medium shot. Discovery wants a slow push toward the courier's face and an insert of the unopened mail. Decision wants a wider shot as the courier backs away and a final look down the corridor.
Six shots from one paragraph of script. If you had prompted "courier delivers package, cinematic," you would have received one generic clip. The beat map gives you structure, and structure is what makes AI footage feel authored.
Step 2 — Choose the Shot Type on Purpose
Shot size is the most powerful and most ignored tool in AI video. Each size carries a meaning, and using them at random produces a sequence that feels like a slideshow rather than a scene.
Establishing shots
Wide shots answer where and when. They are cheap to generate and expensive to get wrong, because a mismatched establishing shot breaks the geography of an entire sequence. Generate one establishing shot per location and reuse its lighting notes across every subsequent shot in that location.
Medium shots and dialogue coverage
Medium shots are the workhorse of conversation. In AI video, they are also where character consistency matters most, because the face is large enough to notice drift. Keep medium shots short, three to five seconds, and pair them with inserts so the viewer's eye gets variety without you needing long, coherent character motion.
Close-ups and inserts
Close-ups do emotional work. Inserts do narrative work. An insert, such as a hand on a door handle or a phone screen, is your cheapest tool for implying action you cannot afford to render. When a model struggles with complex physical interaction, cut to an insert instead of fighting it. This is not a compromise; it is standard practice in live-action editing for exactly the same reason.
Coverage rhythm
A reliable pattern for a short scene is wide, medium, close, insert, wide. It gives the eye an establishing context, a subject, an emotional peak, a detail, and a release. Vary it deliberately rather than accidentally. If every shot is a medium shot, your scene will feel like a security camera feed.
Step 3 — Build a Camera Movement Vocabulary
Movement is meaning. A static camera says the world is stable. A slow push says attention is narrowing. A drifting handheld frame says something is wrong. Before you generate anything, decide what movement each scene earns.
When to stay static
Static frames are underrated in AI video because they are the easiest to keep coherent and the easiest to cut. Locked-off shots reproduce reliably, hold composition, and let performance and light carry the moment. Use them for tension, for deadpan comedy, and for any shot where the model is already struggling with character consistency.
When movement adds meaning
- Slow push in: realization, dread, intimacy.
- Slow pull out: isolation, aftermath, reveal of context.
- Lateral tracking: travel, procession, parallel action.
- Crane or tilt up: scale, awe, transition between worlds.
- Handheld drift: urgency, documentary realism, unease.
Movement speed and easing
The single most common giveaway of amateur AI footage is movement that accelerates unpredictably or changes direction mid-clip. Specify speed in plain language: "a slow, steady push in, roughly one meter over four seconds, no speed changes." That last clause matters. Models respond well to explicit constraints about what should not happen.
Step 4 — Lens, Perspective, and Framing Decisions
Lens choice changes emotional distance even when shot size stays the same. A close-up on a wide lens feels invasive and slightly distorted; a close-up on a long lens feels observational and compressed. You can specify both shot size and lens character to get very different results from the same subject.
Wide lens versus long lens
Wide lenses exaggerate depth. They are ideal for interiors, corridors, cramped hallways, and any shot where you want the environment to feel present. Long lenses compress space, isolate subjects, and flatter faces. Use them for portraits, crowds, and moments of voyeurism.
Height and angle
Camera height is shorthand for power. Eye level is neutral and connective. Low angles make subjects dominant. High angles make them vulnerable or small. In AI prompts, camera height is often overlooked, and its absence produces a default eye-level look that flattens an entire sequence.
Aspect ratio and platform framing
Decide the delivery format before you design any shot. A vertical frame changes what an establishing shot can accomplish, because you cannot show a wide environment and a subject at the same time. Vertical video favors medium shots, close-ups, and inserts, with environment conveyed through background detail rather than wide vistas. If you plan to cut both landscape and vertical versions, shoot your wide establishing shots with a central composition so a vertical crop still works.
Step 5 — Continuity: Character, Wardrobe, and Light
Continuity is what separates a sequence from a collection. Three variables matter most: who the character is, what they are wearing, and how the light behaves.
Character reference sheets
Write a short, fixed description for every recurring character and reuse it verbatim in every prompt. Keep it to physical specifics that a model can render: approximate age, hair color and length, build, one or two distinctive features, and base wardrobe. Avoid adjectives like "beautiful" or "charismatic" that produce inconsistency rather than identity. Where your tooling supports image references, pin one approved still per character and reuse it across shots.
Lighting continuity
Light is the fastest way to make two shots from the same scene feel unrelated. Establish a lighting note per location and repeat it. For example: "late afternoon sun through west-facing windows, warm key from camera left, soft shadows, dust in the air." Every shot in that scene inherits the same note, and only the shot size or angle changes.
Time of day and color temperature
Track time of day across your sequence. If scene four is dusk, scene five does not become noon without a narrative reason. Color temperature is a useful continuity signal: warm for memory and safety, cool for distance and unease. Choose a palette per project and hold it deliberately.
Step 6 — Write Prompts That Read Like Shot Briefs
Once your shot decisions are made, prompt writing becomes transcription rather than guesswork. Structure each prompt as a shot brief with a fixed order of information so you can compare and debug them easily.
Anatomy of a shot brief prompt
- Shot size and angle: "medium close-up, eye level."
- Subject and action: "a woman in her thirties opening a shuttered shop."
- Wardrobe and character anchors: the fixed description from your reference sheet.
- Environment and time of day: location specifics plus the lighting note.
- Camera movement: static or a described move with speed and duration.
- Lens character and depth of field: wide with deep focus, or long with shallow focus.
- Mood and film reference: tonality, grain, contrast.
- Explicit exclusions: no text overlays, no extra people, no camera shake, no speed ramps.
Negative constraints are not optional
Generative models fill silence with invention. Every prompt should state what must not appear. Common offenders include watermarks, subtitles baked into the frame, sudden zoom-ins at the end of a clip, and additional background characters appearing mid-shot. Naming these costs you eight words and saves you a regeneration pass.
Assembling the Shot List and Reviewing It
With scenes mapped, shots chosen, and prompts written, assemble a shot list you can actually work from. A simple table is enough, and it doubles as your generation log.
| # | Scene | Shot size | Movement | Duration | Prompt key | Status |
|---|---|---|---|---|---|---|
| 1 | 4A ext. shop | Wide | Static | 4s | Shop dusk v2 | Approved |
| 2 | 4A int. shop | Medium | Slow push | 3s | Shop interior v3 | Needs redo |
| 3 | 4A insert | Insert | Static | 2s | Hands on lock | Approved |
Generating in the right order
Generate establishing shots first, then approve a lighting and color reference before producing anything else in that location. Generating a close-up before the environment is locked is how you end up with a scene that looks like three different productions.
Troubleshooting common shot failures
- Character face drifts between shots: shorten the shot, move to a wider size, reduce movement, and pin an image reference.
- Motion looks rubbery or accelerates: add explicit speed and no-speed-change constraints, or switch to a static frame and let editing create the rhythm.
- Composition ignores your framing: split the prompt so framing appears first and mood last; models weight early tokens more heavily.
- Backgrounds change mid-clip: reduce clip length, simplify the background, and remove secondary characters from the brief.
- Color shifts between cuts: add a unified color grade in post. Even a light grade with matched contrast will pull mismatched shots into one world.
Common Mistakes and a Quality Checklist
Most weak AI video comes from a short list of avoidable errors rather than from model limitations.
- Prompting mood instead of shots. "Cinematic and emotional" is not a shot. Say what the camera sees.
- Uniform shot sizes. A sequence of medium shots has no rhythm.
- No negative constraints. The model will add something you did not want.
- Inconsistent character descriptions. Small wording changes produce large visual changes.
- Ignoring sound design. Ambient sound and a music bed do more for perceived production value than another regeneration pass.
- Long clips with complex action. Two clean three-second shots beat one incoherent eight-second shot every time.
- No reference stills. Approving one frame and reusing it is the cheapest consistency upgrade available.
Pre-delivery checklist
- Every scene has an establishing shot that matches its interior lighting.
- Character descriptions are identical wherever the character appears.
- Movement is motivated and consistent in speed.
- Aspect ratio matches the delivery platform.
- Inserts cover any action the model handled poorly.
- Color grade is applied across the whole sequence, not per clip.
- Audio is designed: ambience, music, and any voice track sit in a consistent mix.
FAQ
How many shots do I need for a one-minute video? Roughly fifteen to twenty-five, depending on pace. Fast-cut social content runs closer to thirty. Plan by beat first and let the count emerge.
Can I use one model for everything? You can, but you will get better results matching the tool to the task. Some models excel at photoreal faces, others at stylized motion or landscapes. Keep a small shortlist and test each with a standard shot brief.
Why does my footage look like stock video? Usually because every shot is the same size, the same lens, and the same neutral lighting. Add deliberate variation: a low angle, an insert, a long-lens close-up, a hard light source.
How do I keep characters consistent across many shots? Fix a written description, approve one still per character, reuse it as an image reference, and prefer shorter shots with less motion. Consistency compounds when you stop rewriting character prompts from scratch.
Should I storyboard before generating? Even rough storyboards help, but a shot list is the minimum viable version. If you can write down what the camera sees for each beat, you already have enough.
How much of this can be automated? Drafting scene cards, formatting shot briefs, and organizing shot lists can all be templated or scripted. Judgment about shot size, movement, and rhythm remains the part worth doing by hand.
Where to Take This Next
Build the workflow once, in a document, and reuse it on every project: scene cards, beat maps, shot briefs with negative constraints, a shot list table, and a delivery checklist. The first project will feel slow. The third will feel mechanical. By the fifth, the structure will be invisible and you will spend your time on the decisions that actually shape the film.
The craft has not changed nearly as much as the tools. A director still decides what the audience sees, when they see it, and how close they are allowed to get. Generative models have simply removed the excuse that it was too expensive to try a different shot. Take advantage of that: generate the alternative angle, test the long lens, and cut two versions. The habit of designing shots deliberately is what will make your work look intentional, whether it was produced by a crew of forty or a single person with a shot list and a clear idea of what the camera should be doing.
