Start With the Shot, Not the Prompt
Failed AI video usually fails for the same reason. The creator wrote a description of a scene โ a woman walking through a rainy street, a spaceship hanging over a desert โ and hoped the model would supply the filmmaking. It will not. Generated footage has no opinions. It only amplifies the ones you bring.
A shot, in the director's sense, is a bundle of decisions: who or what is in frame, how much of them, from what height, through what kind of lens, lit by what, moving how, and for how long. Each choice tells the audience where to look and how to feel. When you hand a video model those choices as explicit constraints, quality jumps โ not because the model got smarter, but because it stopped guessing.
This guide is a practical directing workflow for AI video. You will learn how to structure a shot brief so it survives generation, how to read a model's strengths before committing to a take, which camera vocabulary reliably changes output, how to hold continuity across a sequence, and how to finish with lighting, sound and editing so the result reads as cinema rather than a tech demo. Everything here assumes you can already write a prompt. What we are adding is intent, expressed as constraints in a fixed order.
The Four Layers of a Cinematic Shot Brief
A prompt that produces repeatable results is not a paragraph of poetry. It is four stacked layers written in the same sequence every time. Keep the order fixed and, when something looks wrong, you can usually identify which layer caused it.
Layer one: subject and performance
Name the subject precisely: age range, build, wardrobe, hair, expression, and what the body is doing. A middle-aged fisherman in a salt-stained wool sweater, hands cracked, hauling a rope hand over hand is workable. A man is not. Ambiguity on the subject layer is where morphing, extra fingers and swapped faces come from.
Include exactly one active verb. Static subjects drift in generated footage because the model has nothing to solve. A clear physical action gives it a target.
Layer two: optics
Optics covers shot size, camera height and lens character, and it is the layer most creators skip โ which is why so much AI footage looks like the same clip rendered by different servers. Specify a 35mm lens with mild barrel distortion at knee height, or an 85mm lens with compressed background at eye level. Optics carry most of the perceived production value because they determine how the space around your subject reads.
Layer three: light
Name direction, quality and source: one warm practical lamp just inside frame left, soft falloff into shadow, no fill light. Light is the fastest way to separate amateur output from professional-looking frames, and it is entirely under your control once you stop writing the word cinematic and start writing where the light comes from.
Layer four: motion and duration
State camera movement or its absence, plus what happens across the shot's runtime. Slow dolly in, four seconds, single take, no cut beats cinematic movement. Duration matters because models distribute motion evenly across whatever length you request; asking for a lot of story in three seconds produces mush.
Write the layers in that order, in plain declarative sentences, and reuse the same vocabulary between shots in a sequence. Consistency of language is what makes a sequence feel like one film.
Reading a Model's Personality Before You Commit
Video models differ the way film stocks and camera operators differ: skin rendering, motion physics, adherence to style references, and how gracefully they handle long durations. Treating them as interchangeable is the single most common workflow mistake.
Realism-first models
Models built around photoreal training data โ Sora-class and Veo-class systems, for example โ excel at skin, fabric, natural light and believable camera physics. They reward restrained prompts. Load them with stylistic adjectives and they will fight you.
Character and style-driven models
Runway, Kling and Luma handle reference images and keyframes well, which matters when the same face or wardrobe must survive across shots. They are also more forgiving about stylization: graphic looks, illustration, anime, product photography.
Motion, crowds and physics
When the shot requires complex motion โ crowds, stunts, water, cloth in wind โ prioritize temporal consistency over texture. Kling, MiniMax/Hailuo and similar systems often hold a hard motion better than a photoreal specialist holds a face.
A quick decision table:
| Shot requirement | What to look for | Where to start |
|---|---|---|
| Skin, fabric, natural light | Photoreal emphasis, subtle grain | Sora-class, Veo-class |
| Same character across shots | Image or keyframe references | Runway, Kling, Luma |
| Complex motion and crowds | Strong temporal stability | Kling, MiniMax/Hailuo |
| Graphic or anime look | Style adherence | Pika, Vidu |
| Long continuous take | Duration control plus extension | Any model with extend or last-frame chaining |
The decision rule is simple: choose by failure mode, not by demo reel. Ask which of your four layers the model is most likely to break, and pick the system that breaks it least.
Camera Language That Models Actually Understand
Shot size
Extreme wide establishes geography; wide shows a body in space; medium is dialogue-neutral; close-up carries emotion; extreme close-up carries detail. Pick one per shot and never blend two sizes in the same sentence, or the model will oscillate between framings.
Angle
Eye level is neutral. A low angle gives power, a high angle reduces the subject, an overhead flattens into geometry, and a slight dutch tilt adds unease. Angles are cheap to specify and expensive to fix in the edit, so decide them at the shot list stage.
Lens and depth
Describe depth of field explicitly: shallow focus with a soft background, or deep focus where foreground and horizon are both readable. Names of focal lengths work surprisingly well as shorthand โ 24mm for immersion, 50mm for naturalism, 135mm for isolation.
Movement
Static, pan, tilt, dolly, truck, crane, handheld, gimbal. One movement per shot is the professional norm, and it is also the practical limit of most models. Combining a crane rise with a handheld shake and a rack focus usually produces a wobbling mess.
A reusable shot template
[Shot size] of [subject + wardrobe + one action],
[angle] at [camera height], [focal length] lens, [depth of field],
lit by [source, direction, quality], [time of day, atmosphere],
[camera movement or static], [duration] seconds, [aspect ratio].
Filled in: Medium close-up of a night-shift nurse, scrubs creased, wiping a window with her sleeve, eye level at chest height, 50mm lens, shallow focus, lit by a single cold fluorescent panel above frame right, deep blue night outside, static camera, four seconds, 2.39:1. That sentence is boring to read and far more useful to a model than three lines of atmosphere.
Blocking, Continuity, and Keyframe Discipline
Continuity in AI video is a scheduling problem, not a talent problem. Build a shot bible before you generate anything: wardrobe description, palette, lens, time of day, and one line about the character's physical state. Paste the relevant lines into every prompt. It feels mechanical. It is also the difference between a sequence and a pile of clips.
Generate your first frame as a still image first. An image model gives you cheap iteration on framing and light, and that approved still becomes the anchor for image-to-video, which locks composition in a way text alone rarely does. Then use the last frame of an approved take as the first frame of the next one when a shot needs to continue.
Shoot coverage the way a crew would. For each story beat, generate a wide, a medium and an insert. Three shots give an editor choices and let you hide weak motion in the cut. If a take has a two-second pocket of good movement, that pocket is enough โ most shots in a finished sequence run between one and three seconds anyway.
Finally, mind the 180-degree rule, even in generated footage. If your wide shows the door on the left, the medium should not show it on the right. Audiences feel that error without being able to name it.
Lighting and Color: The Cheapest Way to Look Expensive
One motivated source beats five generic ones. Decide where the light comes from in the world of the scene, put a practical in frame when it helps, and describe how it falls off. Direction plus quality plus source, every time.
Color temperature contrast is the second lever. Warm subject against cool background, or the reverse, produces depth in a shot that would otherwise look flat. Name the time of day rather than a color word: overcast morning, last hour of sun, sodium streetlight, blue pre-dawn.
Haze, dust and rain are depth multipliers. They give light something to travel through, which separates foreground from background. Used sparingly they read as atmosphere; used in every shot they read as a filter.
What actually reads as cheap: flat frontal lighting with no shadow side, oversaturated highlights, in-frame text or logos nobody asked for, and lens flare applied as decoration. When a frame looks off, cut adjectives before you cut detail โ specificity of light source almost always outperforms mood words.
Sound, Rhythm, and the Cut
Generated video is silent, and silence is the fastest way to make it feel synthetic. Build three sound layers: ambience (room tone, wind, traffic), foley synced to visible action (footsteps, cloth, a cup set down), and music or a drone bed. Sound tells the audience that motion has weight, especially where physics in the render is slightly off.
Cut on action, not on stillness. When a hand reaches for a door handle, cut from the medium to the close-up at the moment the fingers touch it. In AI footage, where motion can be inconsistent, cutting mid-movement masks imperfections far better than holding a static frame.
Hold shots longer than instinct suggests in quiet scenes and shorten them in action. Rhythm comes from contrast in duration, not from a constant pace. A four-second hold followed by three one-second shots feels faster than four two-second shots back to back. And do not be afraid of a beat of real silence before a reveal; generated visuals become dramatically stronger when the soundtrack stops for a moment.
Mix and normalize at the end. Consistent loudness across the whole piece is one of those invisible qualities that makes an AI-made sequence feel professionally finished.
A Repeatable Workflow From Page to Render
Pre-production: shot list and lookbook
Write the sequence as a list of shots with size, angle, movement, duration and purpose. Then assemble ten to fifteen reference stills that define palette and light. If your stills do not agree with each other, your generated shots will not either.
Generation: batches, takes and labeling
Generate in small batches with one variable changed at a time โ the same prompt, different seed; then the same seed, different angle. Label files with shot number and variant so you can compare. Keep a note of which prompt produced the keeper, because you will need it for the next shot in the sequence.
Selection and assembly
Cut on paper before you cut on the timeline: pick the best two takes per shot, then assemble the sequence with placeholder music and rough pacing. Problems in pacing are much easier to see at this stage than after a full render.
Finishing
Grade for contrast and palette consistency, add subtle grain and a slight vignette, balance sound, and export at the delivery aspect ratio. Test the export on a phone โ if the shot still reads there, it will read anywhere.
Mistakes That Kill Cinematic Shots
- Vague adjectives instead of specific constraints (cinematic, epic, beautiful).
- Two or more actions packed into a single short shot.
- Contradictory layers: soft daylight plus neon night in the same sentence.
- Identical framing across a sequence, so every shot feels like the same shot.
- No keyframe anchor, which guarantees face and wardrobe drift.
- Mixing aspect ratios mid-project and cropping later.
- Ignoring sound entirely until the end.
- Centering the subject in every frame, leaving the eye nothing to travel toward.
- Using one model for every job because it is familiar.
- Generating too long, then trying to trim a take that has no internal rhythm.
Most of these are directing problems with technical symptoms. Fix the decision, and the render improves.
Quality-Control Checklist Before a Shot Is Final
- Does the shot read as one of the four layers, cleanly and without contradiction?
- Is there a single subject, a single action and a single camera move?
- Is the light source identifiable and motivated?
- Does framing match the previous and next shot in the sequence?
- Does the character's wardrobe, hair and props match the shot bible?
- Does the take have at least one usable beat of motion?
- Is there sound that makes the motion feel weighted?
- Is the export resolution and aspect ratio correct for the destination?
FAQ
How long should an AI video prompt be?
Long enough to specify the four layers and no longer. Two or three dense sentences usually beat ten poetic ones. If you cannot say what changed between two prompts, the prompt is too long.
Why does my character change between shots?
Almost always a missing anchor. Fix it with a reference image or a locked first frame, and paste the same wardrobe and hair description into every prompt in the sequence.
Do I need image references to get cinematic results?
Not strictly, but they make results consistent and iteration cheap. Text-to-video is best for exploration; image-to-video is best for production.
How many takes should I generate per shot?
Three to five variations of a validated prompt is a reasonable baseline. If none of them work, the shot specification is the problem, not the seed.
Can I mix models in one project?
You should. Matching a model to each shot's hardest requirement produces a better sequence than forcing one system to cover everything. Unify the results later with grade, grain and sound.
What improves output fastest?
Cutting your prompt down to the four layers and adding light direction. Those two changes alone fix the majority of flat, generic-looking footage.
