AI Director Workflow: Turn a Short Film Script Into Shots

Why a Director Layer Matters in AI Short Film Production

Generating one striking clip is no longer a flex. Anyone with a browser and a decent idea can produce a five-second shot that looks like it came from a real camera crew. The scarce skill has moved somewhere else: sequencing. A film is not a collection of beautiful frames, it is a chain of decisions where every shot earns the next one. That decision layer is what a director does, and it is exactly the layer most AI video creators skip.

The result of skipping it is predictable. You generate forty clips, arrange them in a timeline, and discover three problems. First, the shots do not share a visual language, so the cut feels like a trailer for unrelated projects. Second, your main character changes face, jacket, and hair length between scenes, which breaks the audience's trust instantly. Third, the pacing is flat because every shot runs the same duration and carries the same emotional weight.

An AI director workflow solves those problems before a single frame is rendered. It is not a specific button in a specific app. It is a production discipline that sits on top of whatever generation tools you prefer: text-to-video models, image-to-video models, video-to-video restylers, upscalers, and frame interpolators. The workflow treats generation as the middle of the process, not the whole of it.

Think of it as three layers. The story layer defines what changes between the first and last frame of your film. The production layer converts that change into shots, continuity rules, and prompts. The assembly layer turns raw clips into a finished piece with rhythm, sound, and a grade. Most creators live only in the middle of the production layer, typing prompts and hoping. The people whose short films actually hold attention work across all three.

The Five-Stage AI Short Film Pipeline at a Glance

Before going deep, here is the shape of the whole process. Every stage has a concrete deliverable, so you always know whether you are done with it.

Stage	Deliverable	Share of total time
1. Story lock	Logline, beat sheet, ending	10%
2. Breakdown	Shot list, continuity bible	15%
3. Generation plan	Method and tool per shot	10%
4. Production	Prompts, takes, selects	45%
5. Assembly	Edit, sound, grade, titles	20%

Those percentages are a starting point, not a law. On a dialogue-heavy piece, the breakdown grows. On a visual-effects-heavy piece, production swallows most of the schedule. What matters is that stages one through three are cheap and stage four is expensive. Fixing a story problem in stage one costs ten minutes. Discovering the same problem in stage five costs a full re-render of half your shots.

A second principle: build depth before breadth. Do not generate twenty shots to see how they look. Generate three shots from the same scene, cut them together, and watch. If those three do not feel like one film, more shots will not save you. If they do, you have a template you can repeat with confidence.

Stage One: Lock the Story Before You Generate a Single Frame

Write the logline as a sentence about change

A usable logline names a character, a want, an obstacle, and a turn. Something like: a night-shift cleaner who cannot sleep starts leaving notes for the ghost that rearranges her shelves, and discovers the ghost is her own future self trying to warn her. That single sentence already implies locations, tone, and an ending. When you are lost in generation details, this sentence is your compass.

Build a three-beat spine

Short films live or die on compression. Use three beats: setup, escalation, reversal. Setup establishes the world and the want. Escalation raises the cost of pursuing it, usually twice, with each attempt more desperate than the last. Reversal delivers a change that recontextualizes what we saw. If your idea cannot survive three beats, it is a sketch, not a film.

Convert beats into scene cards

Each scene card gets five fields: location, time of day, characters present, what changes, and the emotional temperature. Keep them to one line each. A six-minute film usually needs six to ten scene cards. This is the stage where you should be ruthless about cutting anything that does not move the want forward.

Decide the ending first

AI generation rewards knowing your final image. If you know the last shot, you can plant visual motifs early: a color, an object, a gesture, a piece of framing. That repetition is what makes a generated film feel authored rather than assembled.

Stage Two: Script Breakdown, Shot List, and Continuity Bible

From scene to shots

Break each scene into shots with a purpose label. Not just wide, medium, close, but why: establish geography, reveal information, isolate a reaction, escalate pressure, release tension. A shot without a purpose is a shot you will cut in the edit, which means you paid for it twice.

For a six-minute film, forty to seventy shots is typical. Write them as a numbered list with columns: shot number, scene, size, subject, camera movement, duration target, method, notes. This list is the single most valuable document in the project.

Build a continuity bible

This is where most AI short films are won. The continuity bible contains reference material that every prompt and every generated frame must respect:

• Character sheets: one locked image per character, plus a written description of face shape, hair, build, age, and defining features.
• Wardrobe: exact clothing per scene, including colors and wear. Changing a jacket mid-film is a continuity error, not a style choice.
• Locations: one or two reference images per location, with fixed lighting direction and time of day.
• Props: every object the audience will remember, and where it starts and ends each scene.
• Color script: a palette per act, expressed as three or four colors and one dominant light source.
• Rules: the small constraints that create style, such as always shooting the character from behind when she lies, or never showing the antagonist's hands.

The rules matter more than the references. They are the grammar of your film, and they are what a viewer feels without being able to name it.

Lock screen direction and eyelines

Generated shots often flip left-to-right between takes, which makes a conversation feel like two separate films. Decide a screen direction per scene, write it in the shot list, and check it during the edit. Eyelines should match the subject's position in frame, not just the camera angle.

Stage Three: Choose the Right Generation Method per Shot

Not every shot deserves the same technique. The fastest creators match the method to the difficulty of the shot, then spend their time on the five shots the audience will actually remember.

The four methods

Text-to-video is best for establishing shots, atmospheric inserts, and anything without a recognizable face. It is fast and forgiving because the audience has no reference for what the shot should look like.

Image-to-video is the workhorse for character shots. You generate or select a still that matches the continuity bible, then animate it with a controlled camera move. This gives you a locked look before you start paying for motion.

Video-to-video and restyling is for converting real footage, or for pushing an existing take toward a different aesthetic. Use it when performance matters more than style, and you would rather direct a human than a prompt.

Hybrid compositing is for anything with complex effects: a creature, a destruction beat, a transformation. Generate plates, then combine them in a compositor rather than asking one model to nail everything in a single pass.

Decision criteria that actually hold up

Ask four questions per shot. Does it contain a recognizable face? Does it require a specific camera move? Does it hinge on precise timing? Does it need to match an adjacent shot exactly? One or two yes answers means a simpler method will do. Three or four means you should slow down, generate stills first, and animate from a locked frame.

Where upscaling and interpolation fit

Upscale only after the edit is locked. Interpolation to a higher frame rate works well on slow camera moves and poorly on fast action and hands. Relighting or color-matching passes should be the last step before the grade, not a fix applied to every clip.

Stage Four: Prompt Architecture for Cinematic Control

The six-slot prompt

Freeform prompting produces random results because it leaves the model to guess. A repeatable prompt has six slots, always in the same order:

1. Subject: who or what, with the continuity bible details.
2. Action: one clear verb, present tense, no compound actions.
3. Camera: shot size, angle, movement, and lens character.
4. Lighting: source, direction, quality, and time of day.
5. Environment: location specifics, weather, background activity.
6. Look: film stock feel, color palette, contrast, grain, aspect ratio.

Example: a woman in a grey wool coat, mid-thirties, dark curly hair, walking slowly toward a bus shelter; medium shot, eye level, gentle push-in on a 40mm lens; overcast daylight from frame left, soft shadows; empty coastal town street, wet asphalt, faint drizzle; muted teal and amber palette, 35mm grain, shallow depth of field, 2.39:1.

One action per clip. If a shot needs two actions, split it. Models interpret compound instructions by randomly dropping half of them.

Negative prompts and known failure modes

Track what breaks in your specific project and build a shared negative list. Common entries: extra fingers, warped hands, text artifacts, floating objects, duplicated limbs, sudden wardrobe changes, morphing faces, flickering backgrounds, jittery camera. Keep the list short and specific. A long generic negative list dilutes the instructions that matter.

Iteration discipline: the three-take rule

Generate a maximum of three takes per shot before changing the prompt. If three takes fail, the prompt is wrong, not the model. Change one slot, not all six, so you learn what caused the improvement. Log the winning prompt next to the shot number. When you return the next day, you will not remember what worked, and the log will save an hour.

Stage Five: Consistency, Assembly, Sound, and Finishing

Consistency is a system, not a prompt trick

Use the same reference image for every appearance of a character. Reuse seeds when your tool supports them. Keep lighting direction identical between shots in the same scene, even if the framing changes. When a shot drifts, fix it in a still first and regenerate the motion, rather than animating a flawed frame again.

Edit for rhythm, not for completeness

Cut your rough assembly fast, then watch it once without stopping. The places where you get bored are the places to trim. Vary shot lengths deliberately: hold longer on emotional beats, shorten as tension rises. A useful test is to remove the first and last half-second of every clip, since generated motion often starts and ends soft.

Sound carries more weight than you think

Generated video has no believable audio. Build the track in layers: room tone, footsteps and cloth movement, a music bed, and two or three signature sounds that belong to specific story beats. Dialogue scenes benefit enormously from clean voice performance and tight sound design, even if the visuals are imperfect. Audiences forgive a soft image far more readily than bad sound.

Grade, titles, and the final pass

Apply one look across the whole film rather than grading clip by clip. Slight contrast and saturation consistency will do more for coherence than any single impressive shot. Add titles only if they serve the story, and check the film on a phone screen before you call it done, because that is where most viewers will watch it.

Common Mistakes That Wreck AI Short Films

1. Starting with prompts instead of a script. You end up with beautiful clips and no through-line. Fix: write the beat sheet first, even if it is four lines.
2. No character reference sheet. Faces drift and continuity collapses. Fix: lock one image per character before production.
3. Generating hundreds of clips. Choice paralysis replaces judgment. Fix: the three-take rule and a shot list.
4. Ignoring screen direction. Conversations feel disjointed. Fix: write direction into the shot list.
5. Uniform shot lengths. The film drags even when shots are good. Fix: plan target durations and vary them.
6. Treating sound as an afterthought. The piece feels amateur regardless of visuals. Fix: budget a full fifth of your time for audio.
7. Overloading single prompts. Models drop instructions. Fix: one action, one camera move, one lighting idea per clip.
8. No review pass on a small screen. Problems invisible on a monitor appear instantly on a phone. Fix: watch on multiple devices before exporting.

A Seven-Day Production Schedule You Can Adapt

Day one, story: logline, three-beat spine, scene cards, target runtime, ending image locked.

Day two, breakdown: shot list with purposes and durations, continuity bible drafts, color script per act.

Day three, generation plan: assign a method to every shot, test three representative shots, note which prompts work.

Day four, key shots: produce the five shots the film depends on. If they do not work, revise the plan now, while revision is cheap.

Day five, coverage: generate the remaining shots in scene order so continuity stays fresh in mind.

Day six, assembly: rough cut, trim pass, sound design, music bed, temporary mix.

Day seven, finish: grade, titles, final mix, export, and a review on phone, laptop, and television.

Adjust freely, but keep the order. Story decisions before production decisions, production before assembly, assembly before polish. Most budget and time overruns come from reversing that order.

FAQ

How long should an AI short film be?

Two to six minutes is the sweet spot for a first serious project. Long enough to have a real turn, short enough that you can maintain visual consistency across every shot. Once you can hold a six-minute film together, longer pieces become a scheduling problem rather than a craft problem.

Do I need an expensive tool stack?

No. One text-to-video model, one image generator, one editor, and one audio tool cover almost everything. Skill comes from the shot list, the continuity bible, and the edit, not from the number of subscriptions. Add specialist tools only when a specific shot type keeps failing.

How do I keep the same face across shots?

Lock a reference image per character, describe the face in writing, and use image-to-video rather than text-to-video for any shot where the face is visible. Avoid extreme close-ups early in the process, since they expose inconsistencies more than medium shots do. If drift persists, keep the character in similar lighting across shots.

Can I generate believable dialogue scenes?

Yes, with a caveat. Generate coverage: a medium two-shot, then separate singles for each speaker, then cut between them. Reversing the angle in one generation rarely works. Keep dialogue short, and let reaction shots carry as much of the scene as the spoken lines.

What resolution and aspect ratio should I work in?

Pick one and stay there. Horizontal formats suit cinematic pieces and landscape screenings, while vertical formats suit social distribution. Decide before production, because changing ratio later forces a re-frame of every shot.

What about likeness and rights?

Use your own references, or references you have clear permission to use. Avoid generating recognizable public figures or living people without consent, and keep documentation of any licensed assets you incorporate. This protects you long after the project is finished.

My shots look good individually but do not cut together. What now?

That is almost always a continuity or pacing problem, not a generation problem. Audit three things: lighting direction across the scene, screen direction, and shot length variation. Fix them in a single scene first, re-cut it, and confirm the scene now plays as one continuous moment before applying the same fix everywhere else.

Start with the plan, not the prompt. A short film built on a locked story, a real shot list, and a continuity bible will outperform a larger pile of disconnected clips every time, no matter which generation tools you happen to favor.