AI Cinematography: How to Direct a Short Film with AI

Why short films are now built shot by shot with AI

A three-minute short film used to require a crew, a location permit, a lighting package, and a week of post-production. Today one director with a laptop can plan, generate, and finish a comparable piece in a weekend — but only if they treat the tools like a camera department instead of a slot machine. The difference between footage that looks synthetic and footage that feels cinematic is rarely the generation model. It is the decision-making that happens before the first frame exists: what each shot is for, where the camera sits, how the light falls, and how one shot hands off to the next.

This guide covers a complete AI cinematography workflow for short films — pre-production that survives generation, model selection by shot type, prompt grammar that reads like a shot list, consistency systems for characters and locations, a practical build order, sound and finishing, and the mistakes that consume the most time.

What AI handles well — and what still needs a director

Generation tools are extraordinary at three things: rendering surfaces, filling backgrounds with believable texture, and producing motion that reads as physical. They are unreliable at three others: continuity across many shots, emotional logic across a scene, and knowing which moment deserves a close-up.

That split should shape your job description. You are not writing every pixel; you are making editorial decisions and enforcing constraints. Practically, this means:

• You decide coverage. How many angles a scene needs, and in what order they cut.
• You decide performance beats. Where the character pauses, turns, or reacts.
• The model decides texture. Skin detail, fabric weave, atmospheric haze, crowd density.

The directors who struggle most are usually the ones who let the model decide coverage as well. They generate twenty variations of the same wide shot and then discover they have no material for the moment the story actually turns.

A useful mindset: imagine you are directing an animated film with a very fast, very literal animator. The animator will do anything you describe precisely and will improvise badly on anything you leave vague.

Pre-production: from logline to a generation-ready shot list

Most failed AI shorts die in pre-production, long before anything renders. The fix is to convert your idea into a document that doubles as a build order.

The one-page treatment

Write a page that states the protagonist, the want, the obstacle, and the turn. Keep it to roughly 250 words. Underneath, list the locations and the time of day for each scene. This page becomes your style reference — tone words like "humid," "clinical," or "nostalgic" will reappear in nearly every prompt.

The shot list as a spreadsheet

Build a table with one row per shot and these columns: shot number, duration in seconds, description, camera framing, camera movement, lighting, subject, wardrobe, location, and status. The last column matters more than it sounds — tracking which shots are approved prevents the classic spiral of regenerating finished work because you lost track.

The edit-first rule

Before generating anything, cut the film on paper. A three-minute short typically runs 35 to 55 shots at an average of 3 to 5 seconds. Writing that rhythm down first means you generate toward a structure instead of assembling a structure out of whatever the model happened to produce. Directors who skip this step usually end up with a beautiful, shapeless four-minute montage.

Reference collection

Gather 10 to 15 still images that define your look: lens compression, color palette, contrast ratio, grain. Keep them in one folder. These are your anchors when a prompt drifts, and they are far more useful than adjectives alone.

Matching models to shot types instead of standardizing on one

A common mistake is choosing a single generator and forcing every shot through it. Different shots have genuinely different requirements.

Shot type	What matters most	How to prioritize
Establishing wide	Scale, atmosphere, believable depth	Models with strong environmental coherence and long-duration support
Character medium	Facial stability, lip and eye behavior	Models tuned for portrait fidelity and identity retention
Insert and detail	Micro-texture, shallow depth of field	Fast, cheap models — these are short and forgiving
Action and motion	Physical plausibility, no limb smearing	Models with strong temporal consistency
Stylized or animated	Aesthetic control, consistent style transfer	Style-focused models with reference-image conditioning
Dialogue close-up	Mouth shapes, subtle expression	Models with image-to-video conditioning from a locked still

Three practical rules follow from this table:

1. Use cheap models for inserts. A two-second shot of a hand on a door handle does not need your most expensive settings.
2. Use image-to-video for anything with a face. Starting from a locked reference still gives you a stable identity that text-only generation rarely matches.
3. Test at low resolution first. Generate a short, low-cost preview to check motion and framing, then re-render the keeper at full quality. Iterating on expensive renders is the single biggest source of wasted time.

Also decide early whether you need native audio or will dub everything in post. Generating dialogue audio in the video model is convenient but locks you into mouth timing you cannot easily fix later.

Prompting like a cinematographer, not a poet

A prompt is a technical instruction, not a mood board. The most reliable structure is: shot size, subject and action, environment, light, lens, movement, and style. Order matters less than specificity.

Framing and lens vocabulary

The words that most reliably change output are the boring, technical ones:

• Shot size: extreme wide, wide, medium wide, medium, medium close-up, close-up, extreme close-up, insert.
• Angle: eye level, low angle, high angle, overhead, Dutch tilt, over-the-shoulder.
• Lens feel: 24mm wide with visible distortion, 50mm neutral, 85mm portrait compression, 135mm flattened background, macro.
• Depth: shallow focus with creamy background falloff, deep focus, rack focus between two subjects.

Saying "cinematic" tells the model almost nothing. Saying "85mm, shallow focus, subject centered in the left third, background bokeh from window light" tells it almost everything.

Light and color as constraints

Name the source, direction, and quality of light. "Single practical lamp from camera left, warm 2700K, deep shadows on the right side of the face" produces far more consistent results across shots than "dramatic lighting." Add a palette anchor — desaturated teal shadows and amber highlights, for example — and repeat it verbatim in every prompt for that scene. Repeated phrasing is what makes a scene feel like one scene.

Motion, and moving the camera on purpose

Describe one movement per shot, not three. "Slow dolly in," "handheld follow," "static locked-off," "crane down," "pan left to reveal." Models handle single verbs well and multi-stage choreography poorly. If a shot needs a reveal, break it into two shots and cut.

One more discipline: describe the subject's action in the present tense and keep it small. "She turns her head slightly toward the window" generates better than "she realizes everything is about to change." Emotion is a result of framing, timing, and performance — not a prompt noun.

Consistency: the hardest solved problem in AI filmmaking

Audiences forgive imperfect texture. They do not forgive a character whose face changes between cuts. Consistency is a pipeline problem, and it needs three layers.

Lock the character sheet

Create one approved still of each character: neutral expression, flat lighting, full wardrobe, hair styled. Save it. Every subsequent shot of that character should be image-to-video from that still, or from a new still approved against it. If you are generating a new still, describe the character in the exact same words every time — same hair length, same jacket color, same scar placement.

Lock the location

Do the same for each location: one wide establishing reference, plus notes on the light direction and dominant colors. When you move to a new angle in that location, start from the reference rather than from text alone.

Lock the grade

Apply the same color correction to every shot in a scene, ideally with one saved look. Slight differences in contrast and saturation between generated shots are what make AI films feel stitched together. A consistent grade hides a remarkable amount of variation in the underlying generations.

A simple audit that saves hours: build a contact sheet of every shot in scene order and review it as a grid. Identity drift, wardrobe changes, and light direction errors become obvious in a grid and nearly invisible when you watch shots one at a time.

A step-by-step build order for a three-minute short

This sequence minimizes rework. Do not move to the next step until the previous one is stable.

1. Lock the script and paper edit. 35 to 55 shots, with durations. No generation yet.
2. Design the look. Build the reference folder and write a 40-word style block you will paste into every prompt.
3. Create character and location sheets. One approved still per character, one per location. This is the foundation everything else rests on.
4. Generate establishing shots first. They set the light and palette, and they are the easiest to fix if the look is wrong.
5. Generate dialogue and close-up shots next. These are the highest-risk shots for face and mouth issues, so give them the most time and the most attempts.
6. Generate action and inserts last. They are short, cheap, and easy to replace.
7. Assemble a rough cut with temp sound. Use placeholder music and scratch audio to test rhythm before polishing anything.
8. Identify coverage gaps. Anything that feels rushed probably needs one more shot, not a longer existing one.
9. Re-render keepers at full quality and replace the previews in the timeline.
10. Finish: grade, sound design, titles, export.

The ordering matters because early decisions cascade. Changing a character's jacket in step 9 means regenerating everything that character appears in.

Sound, editing, and finishing

AI video is silent-film technology with better texture. Sound is where an AI short either becomes convincing or falls apart.

The layers you need

• Dialogue: record real voice performances if you can, or use a consistent voice model per character. Consistency matters more than realism here.
• Room tone: every location needs its own continuous ambience — hum, wind, distant traffic. Cutting ambience between shots is a classic tell.
• Foley: footsteps, cloth movement, object handling. Cheap to add, disproportionately effective.
• Music: one thematic idea, developed in two or three variations, beats a wall-to-wall track.

Editing rules that apply to generated footage

Cut on motion, not on stillness. Because generated shots often have slightly uncertain endings, cutting mid-movement hides the weakness. Keep shots shorter than you think — 2 to 4 seconds is normal for a fast scene. And never let a shot linger just because it was expensive to make.

For finishing, apply a single grade to the whole timeline, add a subtle grain layer to unify texture differences, and stabilize any shot with micro-jitter. Title cards should use one typeface and one weight. If you want end titles, keep them brief and free of clutter.

Common mistakes and how to fix them

Chasing one perfect take. Ten variations of a flawed prompt is not iteration, it is gambling. Fix the framing or the lighting description first, then regenerate.

Overloading prompts. Long prompts with forty adjectives dilute the constraints that actually matter. Cut to the eight or ten words that define the image.

Ignoring shot duration in the script. A shot described as "she crosses the room" needs 4 to 6 seconds. If your model produces 3-second clips, plan the coverage around that reality.

Mixing styles within one scene. Realism and stylization can coexist across a film, but not between consecutive shots in the same conversation. Decide the scene's register and hold it.

Neglecting transitions. Match cuts, eyeline matches, and sound bridges do more for perceived quality than any individual render. Plan them in the paper edit.

Skipping the contact sheet. Watch your film as a grid at least once before you export. It catches more continuity errors than three full viewings.

FAQ

How long does a three-minute AI short take to produce? With a locked script and character sheets, most solo directors spend 15 to 30 hours: roughly a third on pre-production and references, half on generation and iteration, and the remainder on sound and finishing. The variance comes almost entirely from how well consistency was locked early.

Do I need editing experience? You need basic timeline skills — trimming, layering audio, applying a grade. Any modern editor works. The more valuable skill is knowing when a cut should happen.

Can I mix generated footage with real footage? Yes, and it often improves the result. Real inserts, hands, and practical backgrounds add texture that is hard to generate. Match the grade and grain carefully.

What resolution should I finish at? Generate at the highest practical settings for hero shots and upscale selectively. For everything else, standard HD output upscaled once is usually indistinguishable at normal viewing size.

How do I avoid the "AI look"? Three fixes in order of impact: commit to a specific lens and light description instead of generic cinematic language, grade everything with one look, and cut faster than feels comfortable. Most AI shorts feel artificial because they are under-edited, not because the pixels are wrong.

Should I storyboard every shot? Not with drawings — a shot list and a reference folder are enough. Storyboards help when a scene has complex blocking, but they cost time you could spend on consistency sheets.

The through-line across all of this is simple: AI cinematography rewards directors who plan like a crew of one and edit like an audience of millions. Lock your references, spend your iterations where the face is on screen, and let sound carry the story. The technology will keep changing; the discipline of deciding what the camera sees will not.