A three-minute AI-assisted short film can easily require sixty to ninety distinct shots. Each one needs a subject, a look, a motion, and a reason to exist. That arithmetic, not the render quality of any single clip, is what separates a film from a mood reel. The workflow below covers the full route from the first script read to a delivered cut, with the decision points that matter most along the way.
Why AI filmmaking still lives or dies at the script stage
Generation tools are now good enough that a single shot can look like it came off a real set. That is also the trap. When one clip impresses on its own, it is tempting to generate thirty of them and hope an edit appears. It never quite does. What appears instead is a sequence with no geography, no continuity of light, and no escalation. Technically polished, emotionally flat.
A script earns its place in an AI pipeline because it answers three questions before any render time is spent:
- What does this scene have to accomplish? Not show the alley, but make the viewer suspect the character is being followed.
- What can be implied rather than shown? A reflection, a door left open, a shadow crossing a wall. Generated footage handles implication far better than elaborate physical action.
- What must remain identical between shots? Face, wardrobe, time of day, lens character, screen direction.
If you cannot answer those three questions for a scene, generation will produce ambiguity at scale, and ambiguity is expensive because it is invisible until the edit.
Useful planning arithmetic: budget roughly 18 to 25 shots per finished minute for dialogue-driven drama, and 30 to 45 shots per minute for action or montage. A three-minute piece therefore lands between sixty and a hundred and twenty shots. That figure tells you how many keyframes you must approve, how many animation runs you must queue, and how much editing time to reserve. Directors who skip this step usually discover the true shot count halfway through production, when the schedule is already committed.
Decide early what the film is not. Most unfinished AI shorts attempt dialogue, action, crowd work, and a twist inside four minutes. Choose two strengths, say atmosphere and performance, and shape the script around them. A tight two-hander in a single location will look far better than an ambitious chase that needs consistent faces across twelve different lighting setups.
Finally, treat the script as a living document during production. When a shot resists six generation attempts, the cheapest fix is almost always a rewrite of that shot, not a seventh attempt.
The seven-stage pipeline from page to picture
Every project that finishes cleanly tends to pass through the same seven stages, whether it is a solo weekend experiment or a small team production. Skipping a stage does not save time. It moves the cost downstream, where it is more expensive and harder to diagnose.
- Script and breakdown. Read for function, then produce a scene-by-scene beat list and a shot list skeleton.
- Visual development. Reference boards, look definition, character and location design, first keyframes.
- Storyboard and animatic. Still images cut against temporary audio at approximately final timing.
- Still generation and approval. Final keyframes per shot, locked before any animation begins.
- Animation. Image-to-video, text-to-video, or hybrid passes, with multiple takes per shot.
- Audio. Dialogue, voice performance, ambience, foley, music, and the mix.
- Edit and finish. Assembly, pacing, color, titles, and delivery.
The artifact that moves between stages matters as much as the stage itself. A simple tracking table keeps a project honest:
| Stage | Main input | Main output | Most common failure |
|---|---|---|---|
| Breakdown | Script | Shot list, continuity notes | Vague shots that cannot be prompted |
| Visual development | Shot list | Look book, keyframes | Style drifting between scenes |
| Animatic | Keyframes plus temp audio | Timed sequence | Pacing problems found too late |
| Still generation | Prompts, references | Approved keyframes | Approving stills that cannot move |
| Animation | Keyframes, motion prompts | Shot takes | Warping faces, drifting backgrounds |
| Audio | Dialogue script | Mixed stems | Lip sync fighting the edit |
| Edit and finish | All of the above | Delivered cut | Color and grain mismatches |
One discipline makes the whole pipeline work: nothing advances until the previous stage exists as a file or a document. Verbal decisions evaporate. Files survive.
Stage one: script breakdown and shot planning
Breakdown is where most AI projects are won or lost. You are not looking for beautiful imagery yet. You are converting prose into units of production that a generator, an editor, and a sound designer can all act on.
Reading for function, not description
Go through the script scene by scene and write one sentence per scene describing its dramatic function: establishes the rift, raises the stakes, reveals the lie, releases tension. Anything that does not serve a function is a candidate for cutting, and cutting early is the cheapest form of production design available to you.
Then mark each scene with a difficulty rating for generation. Green means a single subject, stable light, minimal motion, no hands or complex interaction. Amber means two subjects, moderate camera movement, or a specific prop. Red means crowds, water, fire, children, animals, or physical contact. A script that is 70 percent green, 25 percent amber, and 5 percent red is realistic. A script that is 50 percent red is a different project, and you should know that before you start.
Turning beats into shot entries
Each shot entry should carry six fields, and no more than six, or the list becomes unusable:
- Shot ID such as SC02_SH04, so files sort correctly and never collide.
- Duration in seconds, estimated at this stage and refined in the animatic.
- Framing and movement, for example medium close-up, slow push in.
- Subject action, one clear verb phrase.
- Continuity anchors, listing the wardrobe, prop, and light state that must match.
- Audio note, naming whether the shot carries dialogue, ambience, or music.
That six-field structure is deliberately minimal. Larger templates create the illusion of planning while slowing the work, and they break down the moment a scene changes.
Continuity notes that save hours
Keep a single continuity sheet covering three categories: characters, locations, and time. For characters, record hair, wardrobe, and any distinguishing mark. For locations, record the direction of light, dominant colors, and which side of the room the camera favors. For time, record whether the scene is day, dusk, or night, and how far into the story's clock you are.
This sheet is what you paste into prompts later, and it is the difference between a film that feels coherent and a collection of attractive strangers.
Stage two: visual development, storyboards, and animatics
Visual development answers a single question: what does this film look like when you close your eyes? Answer it with images before you answer it with prompts, because words are a poor container for tone.
Look books and the three-reference rule
For each location and character, gather three references: one for palette, one for texture or material quality, and one for lighting. Three is enough to communicate intent and few enough to stay coherent. Ten references produce mush, because the model receives conflicting signals and averages them.
Write a short style statement of two or three sentences that you will reuse in every prompt. Something like: overcast northern daylight, muted green and grey palette, 35mm film grain, shallow depth of field, restrained handheld camera. A style statement is the cheapest consistency tool in the entire pipeline, and it costs nothing to maintain.
Keyframes: how many do you actually need
Generate one keyframe per shot, not one per scene. Scenes with three shots need three keyframes, because each shot has its own framing. Budget for roughly double the number of shots in first-pass stills, since you will reject about half. If your plan calls for eighty shots, expect to review around a hundred and sixty images before you have your locked set.
Approve keyframes against four criteria: does it serve the shot's function, does the framing match the shot list, does the light match the style statement, and could this still plausibly move? That last question is the one beginners skip. A still with a character half out of frame, extreme perspective distortion, or seven fingers will fight you for the entire animation stage.
Animatics: timing before you spend compute
An animatic is your keyframes, plus temporary dialogue, cut to roughly final length. It is the single highest-value step in the pipeline because it exposes problems while they are still cheap. In an animatic you will discover that your opening is forty seconds too long, that a reaction shot you love kills the momentum, and that two scenes say the same thing.
Build the animatic with hold times that approximate the intended motion. If a shot is meant to last four seconds with a slow push, hold the still for four seconds. Then watch the animatic three times in a row without pausing. If you get bored at minute two, your audience will be gone at minute one.
Stage three: generating stills that survive motion
This stage is where prompt craft matters, but it is worth being clear about what prompts can and cannot do. Prompts steer composition, lighting, and style with reasonable reliability. They steer precise geometry and continuity only sometimes. Design around that limitation instead of fighting it.
Prompt anatomy that scales across dozens of shots
A reliable structure is: subject, action, wardrobe and props, environment, lighting, lens and camera, style, and finally what you want removed or minimized.
For example: a woman in her thirties in a charcoal wool coat, standing still at the edge of a rain-slicked platform, hands in pockets, no umbrella, overcast dusk, sodium lights in the background, 50mm lens, shallow depth of field, muted green and grey palette, 35mm film grain, empty foreground, no text, no watermark, single subject.
The reusable part is everything after the action. Keep the tail of the prompt identical across a scene and change only the head, and your coverage will look like it belongs to one film.
Character consistency without training a model
Full custom training is not always necessary or practical. Three lighter techniques cover most cases:
- Reference-driven generation. Supply the same approved character image alongside every prompt in the scene.
- Descriptive lock. Write a fixed character block of six to ten words and paste it verbatim into every prompt, including details like hair length and coat color.
- Framing strategy. Favor medium and wide shots for repeated characters, and reserve close-ups for moments where a slight facial variation is acceptable or even useful.
When consistency still breaks, the fix is usually structural: reduce how often the character must appear in tight close-up, or reposition the scene so their face is partly obscured by light, shadow, or angle.
Locations, props, and wardrobe continuity
Treat each location as a character with its own lock. Generate a master wide shot of every location and keep it as a reference for the rest of the scene. Do the same for any prop that reappears, and for wardrobe on any character who appears in more than two scenes. Continuity props are easy to forget and instantly noticeable: a bag that changes color, a cup that fills itself, a jacket that unbuttons between cuts.
Stage four: animating shots and directing virtual cameras
Animation is where a strong projector becomes a film. It is also where most of the time and iteration budget disappears, so treat it as a stage that needs its own rules.
Choosing a method: image-to-video, text-to-video, or hybrid
- Image-to-video is the default for narrative work. You control the frame, then specify the motion. Best for anything with a character, a prop, or a fixed composition.
- Text-to-video is useful for establishing shots, atmosphere, abstract transitions, and backgrounds where no specific subject needs to persist.
- Hybrid means generating a short text-to-video clip for motion reference, then driving image-to-video from a locked keyframe with that motion in mind. Slower, but it rescues difficult shots.
A simple decision rule: if the audience must recognize something, use image-to-video. If the audience only needs to feel something, text-to-video is usually enough.
Motion language that actually works
Describe motion in terms of what moves and how much, not in terms of mood. Useful motion phrases include: slow push in, subtle handheld drift, gentle parallax as the camera tracks left, character turns head slowly toward camera, steam rising steadily, curtain shifting in a light breeze, rain falling at a slight angle.
Motion phrases that consistently cause problems are the spectacular ones: running, fighting, dancing, crowds walking, animals leaping, anything involving hands manipulating objects. If your script depends on those, plan alternate coverage such as reaction shots, partial framing, silhouettes, or sound-led sequences that imply the action rather than rendering it.
Camera moves in plain language
Map your intended camera language into words the generator can act on, and keep the vocabulary small. Six moves will cover most narrative scenes:
| Intent | Prompt phrasing |
|---|---|
| Build intensity | slow push in |
| Reveal context | slow pull out |
| Follow a subject | gentle track left behind the character |
| Add unease | subtle handheld sway |
| Show scale | slow tilt up |
| Hold stillness | locked-off static shot |
Vary the moves across a scene rather than repeating the same push, and remember that the most powerful camera move in AI work is often no move at all.
Shot length and cut rhythm
Generated clips work best between three and six seconds. That does not mean your shots should all be four seconds long. It means you should plan to cut within that window, and use more takes when you need a longer hold. A scene of quiet dialogue might hold a shot for seven or eight seconds, while a tense sequence may cut every two seconds. Rhythm is editorial, not generative, so decide it in the animatic and honor it in the edit.
Stage five: dialogue, performance, sound, and mix
Sound is where AI shorts most often give themselves away. Image quality has improved dramatically; audio craft has not kept pace, and audiences forgive a soft shot far more easily than a hollow scene.
Voice casting and line delivery
Choose voices the way you would cast actors: by texture, pace, and age, not by novelty. Generate two or three takes of every line with slightly different pacing and select in context rather than in isolation. A line that sounds flat on its own often reads perfectly under a slow push in, and a line that sparkles alone can feel theatrical against a wide shot.
Keep a voice sheet: character, pitch, pace, accent, and emotional register. Consistency here is easier than visual consistency, because the same voice reference can be reused indefinitely.
Lip sync and the uncanny valley
If a shot shows a face speaking for more than two seconds, lip sync becomes the center of attention whether you want it or not. Three tactics reduce risk:
- Keep dialogue on medium shots where the mouth is small in frame.
- Cut to reaction or insert shots for longer speeches, letting audio carry the performance.
- Reserve close-up speaking shots for short, high-impact lines.
When sync is slightly off, nudging the audio a few frames earlier usually reads better than delaying it, because viewers tolerate early audio more readily than late audio.
Ambience, foley, and music beds
Lay ambience first, then foley, then music. Ambience establishes place, foley establishes presence, and music establishes feeling. Doing them in that order prevents the common mistake of burying a scene under score before the room has any life in it.
For generated footage, add small human sounds deliberately: cloth movement, a breath, a chair creak, footsteps on the correct surface. These details do more for believability than another four seconds of music. When mixing, aim for dialogue to sit clearly above ambience, with music ducking under speech rather than fighting it.
Stage six: edit, color, and finishing without losing the thread
Assembly and the paper cut
Assemble in shot order first with no trimming, simply to confirm that every beat exists. Then do a paper cut, removing anything that does not advance story or feeling. Expect to lose between 10 and 20 percent of your runtime in this pass. That loss is not failure; it is the edit doing its job.
Watch the cut with sound off once, then with picture off once. With sound off, you will see pacing problems. With picture off, you will hear rhythm problems and buried lines. Both passes are faster than guessing.
Matching color and grain across shots
Because shots come from different runs and sometimes different models, they rarely match out of the box. Fix them in a consistent order: exposure, white balance, contrast, saturation, then grain. Applying grain last unifies texture, and it is often more effective than heavy color work for making disparate shots feel like one film.
Keep a reference frame from your best-looking shot on a second monitor and match every other shot to it. This single habit eliminates the patchwork look that signals an AI-assembled project.
Titles, captions, and delivery specs
Finish the practical details: opening and closing title cards, subtitles if your audience needs them, correct aspect ratio, correct loudness target, and a clean file naming convention. Deliver one master file and one compressed version. Verify the compressed version on a phone with the volume at half, because that is how most viewers will actually watch it.
Mistakes, decision criteria, and tool selection
Eight mistakes that cost the most time
The same errors appear in almost every struggling project:
- Generating before the script is stable. Every script change invalidates stills, animation, and audio.
- Approving unmovable keyframes. Stills that cannot plausibly move will fail in animation.
- Chasing perfect faces in close-up. Change the framing instead of running a ninth pass.
- No style lock. Each scene looks like a different film.
- Ignoring the animatic. Pacing problems get discovered when they are most expensive.
- Over-scoring. Music covering for a lack of ambience and foley.
- Mixing models mid-scene. Slight render differences become visible across cuts.
- No versioning. Losing a good take is worse than generating a bad one.
A file structure prevents the last mistake almost entirely: one folder per scene, one subfolder per shot, filenames that include the shot ID and take number, and a simple text log noting why each take was kept or rejected.
A decision worksheet for choosing tools
Do not choose tools by leaderboard. Choose them by asking what your project actually requires:
- Do you need a specific subject to persist? Prioritize image-to-video strength and reference support.
- Do you need long continuous takes? Prioritize motion stability and consistency over resolution.
- Do you need stylized rendering? Prioritize prompt adherence for aesthetics.
- Do you need speed over polish? Prioritize fast iteration and easy retries.
- Do you need repeatability? Prioritize tools that accept the same reference input every time.
Write your five answers down before you open any browser tab. Then evaluate candidates against your answers, not against a demo reel. A tool that nails atmosphere but cannot hold a face is the wrong tool for a dialogue drama and the right tool for a montage.
The same logic applies to the rest of the stack. Editing software should handle the codecs you actually generate. Audio tools should let you replace single lines without rebuilding a scene. Whatever you pick, keep the number of applications small so handoffs stay short and nothing gets lost between them.
Constraints: hardware, time, and team size
Be honest about what you can finish. If you are working alone on weekends, a four-minute film with dialogue, three locations, and two characters is a realistic ceiling. With a small team and dedicated render time, double the shot count, not necessarily the runtime. Longer runtimes multiply continuity risk far faster than they multiply satisfaction.
FAQ: practical questions from first-time AI directors
How long does a four-minute AI short actually take?
For a solo creator working evenings, plan three to six weeks. Breakdown and visual development take two to four days, still generation and approval take about a week, animation takes one to two weeks including retries, and audio plus edit takes another week. The animation stage is the least predictable, which is why locking keyframes early matters so much.
Do I need an expensive workstation to do this well?
The workflow matters more than the hardware. Most of the heavy generation happens in tools you access remotely, and editing a short film at moderate resolution is manageable on a modern laptop. What you genuinely need is organized storage, a reliable backup, and enough patience to review hundreds of stills carefully.
How do I keep a character's face consistent across scenes?
Use three layers of protection: a fixed descriptive block repeated in every prompt, the same approved reference image supplied to every generation in a scene, and framing choices that avoid unnecessary close-ups. When variation still appears, treat it as a lighting or angle problem and change the shot rather than the character description.
What should I do when a shot fails repeatedly?
Fail twice, then change the approach. Options in order of cost: simplify the action, change the framing, convert the shot into an insert or reaction, replace it with a sound-led moment, or cut it entirely. Rewriting beats re-rendering almost every time.
Should I generate at the final aspect ratio?
Yes. Cropping a generated shot after the fact changes composition, reveals edge artifacts, and often breaks the intended framing. Decide the delivery format before the first keyframe and stay in it for the whole project.
When should I stop iterating on a shot?
When the shot fulfills its function and does not distract. Functional beats do not need to be beautiful, they need to be clear. Reserve extra passes for the three or four shots that carry the emotional weight of the film.
Can I mix generated footage with live-action material?
Yes, but match deliberately. Shoot or shoot-adjacent live plates with the same lens character, grain, and light direction you specified in your style statement. Applying unified grain and a gentle color match across both sources does more for coherence than any single effect.
How much of the film can be dialogue scenes?
More than beginners expect, provided you design coverage. Write dialogue scenes with inserts, reactions, and off-screen lines. If every exchange is a speaking close-up, you are maximizing the hardest thing to generate and the first thing audiences scrutinize.
What is the best first project?
A ninety-second single-location piece with one character, one prop, ambient sound, and a clear emotional turn. It forces you through every stage of the pipeline at a scale you can finish, and it teaches the exact skills that a longer film will demand over and over again.
The pattern across all of these answers is the same: control the small decisions early, keep your artifacts organized, and let the script decide what deserves a render. A finished four-minute film with modest visuals will always outperform an unfinished masterpiece with three stunning shots.
