Prompt engineering for AI video has moved past the stage where a clever one-liner could carry a clip. Modern generators can render convincing faces, plausible physics and ambitious camera moves, which means the weak link is rarely the model. The weak link is the specification you hand it. A vague request produces a vague clip, and vague clips are expensive to repair because you cannot easily patch a shot halfway through a render. The practical answer is to treat every prompt like a shot brief: layered, specific, and written around one testable action.
This guide walks through a repeatable workflow. You will see how to decompose a prompt into layers, how to write camera language that actually changes output, how to hold characters and props together across shots, how to choose the right model for a given shot, and how to iterate without burning whole afternoons. If you already write prompts by instinct, the goal here is to convert that instinct into a process you can hand to a collaborator.
Why prompt quality outweighs model choice
Run the same idea through the same model twice with different prompts and you will get results that look like they came from different tools. That gap is the entire discipline. The model supplies capability; the prompt supplies intent. Capability without intent produces attractive noise.
Three technical realities make this especially true for video rather than stills:
- Temporal consistency. A frame that looks perfect can fall apart two seconds later when a hand melts, a jacket changes color, or the camera move contradicts the lighting. Prompts that describe motion, not just appearance, reduce this class of failure.
- Subject identity. Faces, logos on clothing, and distinctive props drift unless you anchor them with consistent descriptors or reference images.
- Physics expectations. Generators infer a lot from context. A prompt that says "falls" invites a dozen interpretations; a prompt that says "a ceramic cup tips off the counter edge and shatters on the tile floor" narrows the simulation.
The mental shift is simple: a prompt is not a wish, it is a specification. Specifications have structure, ordering, and constraints.
The six layers of a dependable video prompt
Most strong prompts, regardless of platform, contain the same six layers. They do not all need to be long, but each one answers a question the model would otherwise guess at.
Subject and action
Who or what is on screen, and what single thing happens? Keep one primary action per clip. "A cyclist rounds a wet corner and glances over her shoulder" is one action with a beat. "A cyclist rides through traffic, checks her phone, dodges a taxi and smiles at a friend" is four actions competing for screen time, and the model will usually resolve the conflict by doing none of them cleanly.
Setting, time and light
Location, time of day, weather, and the dominant light source. "Late afternoon, narrow Lisbon street, low sun raking across wet cobblestones, warm bounce from ochre walls" gives the model a lighting problem it can solve consistently. "Nice street" gives it nothing.
Camera and lens
The framing, the lens character, and the movement. This layer is where amateurs write one word ("cinematic") and professionals write three ("slow dolly in, 50mm, shallow depth of field"). Camera language is covered in detail below.
Motion and pacing
How fast things move and how the shot evolves over its duration. Useful phrasings include "slow steady movement," "quick decisive gesture," "camera drifts left as subject remains centered," or "action peaks in the final second." Pacing language helps the generator decide where to place motion energy.
Style, grade and texture
The visual treatment: documentary handheld realism, 1990s film grain, clean commercial gloss, animated watercolor, stop-motion felt. Style terms stack well when they share a vocabulary. Mixing "photorealistic" with "hand-drawn animation" rarely produces a useful hybrid.
Constraints and negatives
What must not appear: no text overlays, no lens flare, no extra limbs, no background crowd, no camera shake. Negative instructions work best when they address a specific, recurring failure you have observed rather than a generic wish list.
From script to shot list: a build process you can repeat
Improvisation feels fast until you have forty unusable clips. A short build process pays for itself by the second project.
Step 1 — Break the script into beats. Every beat is a change in information or emotion. Those beats become shots, not sentences.
Step 2 — Assign one action per shot. If a beat contains two actions, split it. Generators handle sequential shots far better than overloaded ones.
Step 3 — Write the six layers per shot. Draft in a plain document before touching any interface. Writing in a grid or table exposes missing layers immediately.
Step 4 — Define continuity anchors. List the descriptors that must not change between shots: hair color and length, garment type and color, a scar, a bicycle's frame color, a room's window placement.
Step 5 — Run a cheap first pass. Short duration, low resolution, minimal upscaling. You are testing composition and motion, not final quality.
Step 6 — Log the result. Keep the prompt version, the model, the settings, and a one-line verdict. This log becomes the most valuable document in your project because it prevents re-testing the same dead end.
Camera and lighting vocabulary that changes output
Vague cinematography adjectives do very little. Specific terms do a lot. The table below maps intent to phrasing that generators respond to.
| Intent | Useful phrasing | Notes |
|---|---|---|
| Reveal scale | wide establishing shot, 24mm | Good for openings; keep subject small |
| Intimacy | medium close-up, 85mm, shallow depth of field | Best for dialogue and emotion |
| Energy | handheld, slight sway, quick whip pan | Pair with one clear action only |
| Elegance | slow dolly in, steady gimbal move | Avoid stacking with handheld terms |
| Tension | slow push in, rack focus from foreground to subject | Works well in short clips |
| Scale | crane up revealing the street below | Needs a described environment |
| Texture | hard key light with deep shadows | Strong for noir and drama |
| Softness | diffused overcast light, no visible shadows | Reliable for product and beauty |
| Warmth | golden hour backlight, lens flare halo | Flare often needs to be requested |
| Realism | practical lights in frame, mixed color temperature | Adds documentary credibility |
Two rules keep this manageable. First, limit yourself to two or three camera terms per shot; more than that and the model averages them into mush. Second, make sure the camera instruction and the action instruction do not fight. "Static locked-off shot" plus "camera follows the runner through the alley" is a contradiction, and contradictions produce visual glitches.
Lighting deserves the same discipline. A single dominant light source described clearly beats four competing sources described poetically.
Continuity across shots: keeping characters and props stable
Continuity is where multi-shot AI video projects live or die. Three techniques do most of the work.
Character sheets. Write one canonical description per character and paste it verbatim into every prompt that features them. Do not paraphrase. "Woman in her thirties, shoulder-length dark curly hair, olive green utility jacket, silver hoop earrings" should appear identically in shot one and shot twelve. Small wording changes cause visible drift.
Reference frames and image-to-video. When a platform supports it, generate or photograph a hero frame and animate from it. This locks identity, wardrobe, and set dressing in a way that text alone rarely matches. For recurring characters, keeping a small library of approved frames is faster than re-describing them each time.
Seed and setting locks. If your tool exposes a seed or a style reference, reuse it for shots that must feel like siblings. Likewise, keep environment descriptions consistent: if the cafe has a red awning in the establishing shot, the awning stays red in the close-up.
A short continuity checklist before you render:
- Same character descriptors, word for word
- Same wardrobe colors and materials
- Same time of day and light direction
- Same location details that appear in frame
- Same lens family (do not jump from 24mm to 135mm unless the cut intends it)
Editors often bridge the remaining gaps with small fixes in post: a color match, a stabilization pass, a frame hold on a gesture. Plan for that rather than expecting a perfect handoff.
Choosing the right model for the shot
Rather than committing to one tool, match the model to the job. Different families excel at different things, and a hybrid pipeline usually beats loyalty to a single platform.
| Shot need | What to prioritize | Where to look first |
|---|---|---|
| Photoreal people and dialogue | Facial fidelity, lip-sync accuracy | Dedicated lip-sync and avatar tools |
| Stylized motion | Motion variety, stylistic range | General text-to-video generators |
| Locked character performance | Identity retention from a reference | Image-to-video workflows |
| Complex camera choreography | Control over camera paths | Tools with camera parameter controls |
| Fast iteration | Render speed, low-resolution previews | Any generator with draft mode |
| Detailed compositing | Layer control, matte export | Node-based pipelines such as ComfyUI |
| Final polish | Grain, grade, sound | A traditional editor and grading suite |
Practical decision criteria, in order of importance:
- Does it hold the subject? Identity drift is the most common reason a shot gets rejected.
- Does it obey the camera instruction? Test with one prompt across three models and compare.
- How fast is a draft? A tool that renders a preview quickly is worth more than one with marginally better output if you need fifty attempts.
- Can you control duration? Some tools cap at a few seconds, which changes how you write the action.
- Does the output survive post? Check resolution, artifacts, and whether the clip grades cleanly with adjacent shots.
Names worth knowing in the current landscape include Runway, Pika, Luma, Kling, Veo, Sora, and Stable Video Diffusion, alongside node-based setups and traditional finishing tools such as DaVinci Resolve and After Effects. The specific leader changes quickly; the criteria above do not.
The iteration loop: test cheap, commit late
Most wasted time in AI video comes from rendering expensive final-quality clips before the composition is settled. Structure your loop so cost rises only when confidence is high.
Draft pass. Short duration, low resolution, no upscale. Judge three things only: framing, action readability, and identity.
Single-variable changes. When a draft fails, change one element — the camera term, the lighting, or the action phrasing — and re-run. Changing three things at once means you learn nothing about which one mattered.
Grid testing. If your tool allows batches, run four prompt variants of the same shot side by side. A four-up grid answers most questions faster than four sequential renders.
Promotion. Only when a draft reads correctly do you increase duration and resolution, then apply upscaling and motion interpolation sparingly. Over-processed clips lose the texture that made them convincing.
Prompt versioning. Save every prompt with a number and a short note. When a colleague asks how a shot was made, you will have the answer, and when a client requests a small change, you will not be starting from scratch.
Time-boxing helps too. Give a shot a fixed number of attempts. If it has not worked after that, the problem is usually the concept, not the wording — a shot that needs six competing events to make sense will resist every model.
Ten common mistakes and their fixes
- Writing a paragraph of adjectives. Fix: replace adjectives with observable details.
- Cramming multiple actions into one clip. Fix: split into shots and cut them together.
- Using conflicting camera terms. Fix: pick one movement and commit to it.
- Paraphrasing character descriptions. Fix: copy and paste the canonical line every time.
- Ignoring duration. Fix: write action that fits the clip length; a three-second clip cannot hold a full conversation.
- Stacking incompatible styles. Fix: choose one visual world and stay inside its vocabulary.
- Negatives that list everything. Fix: target only the failures you actually observed.
- Skipping drafts. Fix: always test at low cost first.
- Assuming the first good frame means a good clip. Fix: watch the entire output, especially the final second where drift appears.
- Never writing anything down. Fix: keep a running prompt log; it compounds across projects.
Reusable templates and a worked example
A skeleton you can adapt to almost any style:
[Shot type and lens], [camera movement], [subject with anchored descriptors]
performing [single action] in [specific environment], [time of day and light],
[pacing note], [style and grade], avoid [targeted failure].
Now apply it. The brief: a baker lifts a loaf from an oven in a small neighborhood bakery, warm and human, suitable for a brand film.
Version one. "A baker takes bread out of the oven, cinematic." Result: generic, flat lighting, hands indistinct, no sense of place.
Version two. "Medium shot, a baker in a flour-dusted apron lifts a round sourdough loaf from a deck oven in a small bakery, warm light, cinematic." Better, but the camera is static and the moment lacks a beat.
Version three. "Medium close-up, 50mm, slow dolly in, a baker in a flour-dusted linen apron lifts a round sourdough loaf from a deck oven in a narrow neighborhood bakery at dawn, warm light from the oven mouth and cool window light behind, deliberate unhurried motion, documentary realism with soft film grain, avoid text and lens flare." Result: readable action, believable light, usable draft.
Version four. Add the continuity anchor — the same apron description you will reuse in the next shot — plus a pacing note that the lift peaks near the end of the clip. Now you have a shot that can sit next to a wide of the shopfront and a close-up of steam curling off the crust without looking like three different films.
The pattern generalizes. Start loose, add layer by layer, and stop as soon as the draft reads correctly. More words are not better; missing layers are worse.
FAQ
How long should an AI video prompt be? Long enough to cover the six layers and no longer. In practice that is often 25 to 60 words for a single shot. Beyond that, extra adjectives start competing rather than clarifying.
Do negative prompts actually help? Yes, when they name a specific recurring artifact such as extra fingers, text overlays, or unwanted shake. Generic negative lists mostly add noise.
Why does my character's face change between shots? Almost always because the description changed, even slightly, or because no reference frame was reused. Lock the wording and animate from an approved still whenever the tool supports it.
Is image-to-video better than text-to-video? For anything with a recurring character, product, or set, image-to-video is more reliable. Text-to-video is faster for establishing shots, abstract sequences, and concept exploration.
Should I write prompts in English? Many models are trained predominantly on English captions, so English prompts are often more predictable. If you work in another language, write the prompt in English and keep your script and notes in your own language.
How many variables should I change between tests? One. Two changes at once and you cannot attribute the improvement.
Can I reuse one prompt across models? The layer structure transfers, but phrasing preferences differ. Treat your prompt as a portable brief and re-tune the wording for each engine.
What if a shot never works? Rewrite the shot. Some ideas need a different framing, a simpler action, or a cut instead of a continuous take.
Where to go from here
The craft is not memorizing magic phrases. It is building a small, disciplined system: decompose the idea, specify the six layers, anchor continuity, match the model to the shot, and iterate cheaply until the draft reads. Do that consistently and the quality gap between your work and casual prompting stops being a matter of luck.
A good next step is to take one existing scene you have already created and rebuild it as a proper shot list with layered prompts, then compare the results. Most people find that two of their five shots were never really about prompt wording at all — they were about a concept that needed to be broken into smaller, clearer pieces.
