Why the sketch is the real prompt
Most disappointing AI video comes from a good model answering a bad question. Someone types a sentence, waits, watches a clip that technically matches the words, and concludes the technology is not ready. In reality the words were the problem: they described a subject but not a shot.
A sketch forces you to answer questions that a sentence hides. Where is the camera? What is the subject doing at the start of the clip versus the end? What is in the background? How long is the moment? Every one of those answers is a prompt token waiting to happen.
This guide walks through a full production-style workflow: turning a rough drawing into a shot list, converting each shot into a structured prompt, choosing a model for the job, controlling continuity with keyframes, iterating without wasting time, and finishing the sequence so it plays like a real edit rather than a demo reel.
The two mental shifts that change everything
The first shift is thinking in shots, not scenes. Video generators produce clips of a few seconds. A "scene" is an editorial idea made of multiple clips. If you prompt for a scene, you get an arbitrary fragment and then fight it in post.
The second shift is thinking in constraints, not adjectives. Words like beautiful, cinematic, and epic push a model almost nowhere because they have no consistent visual meaning across training data. Words like 35mm lens, low angle, backlit haze, slow dolly in push it very precisely, because those are concrete, learnable patterns.
From storyboard to shot list
Before you touch a generator, convert the sketch into a table. For each shot, capture:
- Shot number and duration — typically 2 to 6 seconds for generation, then trimmed in the edit.
- Framing — extreme wide, wide, medium, close-up, macro, over-the-shoulder.
- Camera behaviour — locked off, pan, tilt, dolly, handheld, crane, orbit.
- Subject action — one primary action per clip. Two actions usually produce mush.
- Environment and time of day — interior/exterior, weather, light direction.
- Transition intent — how the shot enters and exits the cut.
A five-shot teaser might look like this:
- Wide establishing shot, 4s, locked off, city at dawn, slow fog drift.
- Medium tracking shot, 3s, subject walks left to right, camera follows.
- Macro insert, 2s, hands opening a package, shallow depth of field.
- Close-up, 3s, face turns toward light, subtle smile.
- Wide pull-back, 4s, camera rises and reveals the full scene.
Notice that each row is already half a prompt. The shot list is the single highest-leverage artifact in the entire workflow, because it separates creative decisions from tool decisions. You can hand the same list to three different generators and compare results fairly.
The four-layer prompt model
A reliable prompt has four layers stacked in order. Weights and emphasis vary by tool, but the structure holds across nearly every text-to-video system.
Layer 1: Intent
One short clause that names the purpose of the shot. Establishing shot of a coastal town at sunrise. This layer anchors the model's overall interpretation and keeps the rest of the prompt from drifting.
Layer 2: Subject and action
Who or what, doing what, with what change over time. Include appearance details that must remain stable: clothing, hair, age range, material, colour. Describe the end state of the motion, not just the start: "lifts the cup and sets it down" beats "coffee."
Layer 3: Camera, lens, and movement
This is where most beginners undershoot. Specify focal length feel, angle, height, and movement. Low angle, 24mm equivalent, slow push in, slight handheld sway is a directable sentence. Combine at most two movements — a push and a slight rise, or an orbit and a tilt — because stacking four movements produces a clip that looks drunk.
Layer 4: Light, palette, and texture
Name the source and quality of light: soft window light from the left, warm practical lamps in background, gentle film grain, muted teal and amber palette. Texture words such as grain, halation, anamorphic flare, matte finish, or clean digital control the finish as much as the content.
Add a short exclusion clause at the end when a tool supports it: no text, no logos, no extra limbs, no camera shake. Negative instructions are weaker than positive ones, so never rely on them to fix a problem you could describe positively instead.
A reusable prompt template with examples
The template:
[Intent clause]. [Subject + action, with end state]. [Camera: angle, height,
lens feel, movement]. [Lighting + palette + texture]. [Duration or pacing hint].
[Exclusions].
Example — wide establishing shot:
Establishing shot of a rain-slicked market street at dusk. Vendors close metal shutters while pedestrians cross left to right under umbrellas. Camera at eye level, 28mm feel, locked off with a barely perceptible drift. Neon signage reflects in puddles, cool blue shadows with warm sodium highlights, light film grain. Slow, observational pacing. No text, no logos.
Example — macro product insert:
Extreme close-up of hands turning a brushed aluminium device in the light. Fingers rotate the object a quarter turn, revealing a matte seam. Camera at object level, 90mm macro feel, shallow depth of field, gentle push in. Single soft key from top right, deep charcoal background, crisp specular highlights, clean digital texture. Deliberate, tactile pacing. No text.
Example — character close-up:
Intimate close-up of a woman in her thirties listening, then nodding once. She turns her head slightly toward a window light and her expression softens. Camera slightly below eye level, 50mm feel, subtle handheld breathing. Soft daylight from the left, neutral skin tones, muted background, subtle grain. Quiet pacing. No extra people, no text.
Save these as a personal prompt library, then swap one layer at a time. A library of twenty well-written prompts beats a folder of five hundred half-finished ones.
Choosing the right generation model per shot
There is no single best model. There are models that are better at specific jobs. Build a small decision framework instead of chasing leaderboards.
Evaluation criteria that actually matter:
- Motion coherence — does the subject deform, or does it move believably?
- Camera compliance — does the requested move actually appear?
- Prompt adherence — how much of your four layers survive?
- Clip length and native resolution — short clips mean more cuts to stitch.
- Control inputs — first frame, last frame, depth maps, pose reference, motion brush.
- Character consistency — can it hold the same face across shots?
- Commercial licensing — critical if the output is client work.
- Turnaround and queue behaviour — fast drafts keep creative momentum.
Practical groupings.
Photoreal cinematic engines are the right choice for establishing shots, landscapes, vehicles, and anything where realism sells the illusion. They tend to handle light interaction and materials well but can be slow and costly at high resolution.
Fast draft engines are for exploration. Use them at low resolution to validate framing, blocking, and pacing. No one should see these clips; they exist to answer questions cheaply.
Stylised and animation-oriented engines are purpose-built for illustrated, anime, or graphic looks. Forcing a photoreal engine into a stylised look wastes iterations; forcing a stylised engine into realism does the same in reverse.
Image-to-video and keyframe-control systems are the workhorses of continuity. When you need the same character, wardrobe, or product across six shots, drive generation from a fixed reference image rather than from text alone.
Motion-specialist tools handle specific camera moves, speed ramps, or subtle facial performance better than general engines. Use them as surgical instruments for the two or three shots that carry the piece.
A rule of thumb: spend your budget where the audience looks longest. Reserve the expensive, slow, high-fidelity passes for hero shots and use lean settings for connective tissue.
Keyframes, reference images, and continuity
The fastest way to improve output quality is to stop asking text to do a job that an image does better.
First-frame conditioning. Generate a still in an image tool, approve it, then animate from it. You gain enormous control: composition, wardrobe, colour, and lighting are locked before motion begins. If the first frame is wrong, no prompt will save the clip — fix the still first.
Last-frame targeting. Some systems accept both a start and end frame. This lets you design an exact transition: a door open at frame one and closed at the end, a hand entering and leaving frame, a camera arriving at a specific composition. Think of it as tweening between two approved images.
Reference sheets. For characters, build a sheet with three or four consistent views, then reference it in every shot prompt. Include a fixed phrase describing the character verbatim — same words, every shot — so nothing drifts between prompts.
Seed discipline. When you find a look you like, lock the seed and change only one variable per iteration. Random regeneration destroys your ability to learn what caused an improvement.
Environmental continuity. Track time of day, weather, light direction, and colour temperature across shots. Note them in the shot list. A sequence that flips from morning sun to dusk shadows between two consecutive close-ups reads as a mistake even if each clip looks beautiful alone.
The draft–diagnose–direct iteration loop
Professionals rarely get a usable clip on the first attempt, and they rarely need ten. They run a tight three-step loop.
Draft. Generate three to five variants at moderate settings. Do not judge. Just collect.
Diagnose. Watch each clip twice — once as a viewer, once as a technician. Name the single biggest failure: motion, framing, style, subject identity, or pacing. Rank by severity.
Direct. Change exactly one layer of the prompt to address the top failure. Rewrite the camera layer if the move was wrong. Rewrite the light layer if the mood was off. Rewrite the subject layer if identity drifted. Do not change three things and hope.
Keep a prompt log: shot number, prompt version, what you changed, and whether it helped. After a few projects you will have a personal playbook of what works with which tool. That log is worth more than any tutorial, including this one.
A useful budget rule: cap drafts per shot. Three rounds at five variants is fifteen clips. If none works, the problem is upstream — the shot list, the reference image, or the model choice — not the wording.
Troubleshooting common AI video failures
Melting or morphing anatomy. Usually caused by asking for too much motion in too little time, or by a subject that is too small in frame. Fix by reducing duration, simplifying the action to one verb, and enlarging the subject. Reference images help enormously.
Camera drift or unrequested movement. Some models default to slow push-ins. Name the movement explicitly and add a lock phrase such as static camera, no movement — then verify with a low-resolution draft before committing.
Style drift across shots. Almost always a prompt-vocabulary problem. Copy the exact style clause from shot to shot instead of paraphrasing it, and reuse seeds where possible.
Static, lifeless results. The opposite failure. Add a measurable change: a light shifting, fabric moving, steam rising, a hand entering frame. Motion needs a cause.
Garbled text and logos. Generators still struggle with typography. Remove text from the prompt entirely and composite real text in the edit. Never promise a client a legible sign in generated footage.
Flicker and texture crawling. More common in stylised models. Reduce prompt complexity, lower motion intensity, and consider frame interpolation in post, which also smooths timing.
Inconsistent character faces. Lock a reference image, use a fixed description string, and avoid extreme angles where the model must invent facial structure.
Finishing, sound, and delivery
Generated clips are raw material. The edit is where they become a video.
Select and trim. Cut into the action, not before it. Trim the first and last few frames of each clip where quality is often weakest.
Stabilise and retime. Small stabilisation passes fix subtle drift. Retiming to 24 fps from 30 fps can add a filmic cadence depending on the look.
Upscale and interpolate. Upscale to delivery resolution, then interpolate if motion feels choppy — but avoid interpolating clips that already look artificial, since it amplifies waxy artefacts.
Colour match. Apply a shared grade across the sequence. Slight contrast and saturation matching hides inconsistencies between clips far better than regenerating them.
Sound design. This is the most underrated step. Room tone, footsteps, cloth movement, and a music bed make generated footage feel real. Silence makes even excellent clips feel synthetic.
Deliver multiple aspect ratios. Export vertical and square versions at the same time. Reframing after the edit is finished takes minutes; regenerating for a second format does not.
Worked example: a 30-second product teaser
A five-shot teaser, with the decisions that matter.
Shot 1 — establishing, 4s. Prompt for a wide, locked-off interior at dawn with strong directional light. Cheap photoreal draft engine, low resolution, validate the mood.
Shot 2 — macro insert, 3s. Build the still in an image tool first: hands, product, brushed metal, dark background. Animate from that frame with a slow push in. This is a hero shot; use the high-fidelity engine.
Shot 3 — medium action, 3s. One action only: hands set the product down. Lock the seed from shot 2 so lighting and palette match.
Shot 4 — close-up, 3s. A face, softened by window light, reacting. Reference sheet for consistency. Keep the camera locked; let the performance carry it.
Shot 5 — pull-back reveal, 4s. Camera rises and reveals the full space. Prompt the move explicitly and expect two or three drafts.
Then cut to music, add a real title card in the edit, and grade the sequence as one piece. Total generation attempts: roughly twenty. Total usable seconds in the final cut: thirty.
That ratio is normal. Plan for it, and the workflow stops feeling like gambling and starts feeling like production.
FAQ
How long should a generated clip be?
Usually two to six seconds. Longer clips drift, deform, and lose the specific moment you wanted. Build length from cuts, not from duration.
Do I need to learn prompt syntax like weights and brackets?
Helpful, not essential. Understanding that earlier words carry more influence is enough for most tools. Structured writing beats exotic syntax almost every time.
Should I generate stills first or animate from text?
Generate stills first whenever continuity matters. Text-to-video is faster for exploration; image-to-video is more controllable for anything that must match something else.
Why does the same prompt give different results later?
Models get updated, queues route differently, and random seeds differ. Lock seeds where supported and keep reference images, so your intent survives model changes.
How many drafts should a hero shot take?
Three to five variants across two or three revision rounds is typical. More than that usually means the reference image or the shot concept is the real problem.
Can I use generated footage commercially?
It depends on the tool's terms and your jurisdiction. Check licensing before building a client workflow around any engine, and keep records of what was generated with what.
What is the biggest beginner mistake?
Describing a scene instead of a shot. Specificity about the camera and the action fixes more problems than any other single change.
How do I keep characters consistent across many shots?
Use a reference sheet, a fixed verbatim description string, locked seeds, and similar lighting. Accept that extreme angles will require extra attempts.
Start with the sketch, write in layers, choose models per shot, control with keyframes, iterate one variable at a time, and finish in the edit. That is the whole craft — and it is far more learnable than it looks.
