Generative video models have become remarkably good at producing a single striking shot. What they have not solved is the part that actually makes a video watchable: deciding which shots to make, in what order, and with which visual rules held constant across all of them. That gap is where an AI director assistant earns its place in a workflow.
This guide lays out a repeatable pipeline for using an AI assistant as a director's second brain — from premise to script, shot list, prompt stack, generation passes, and final assembly. It is intentionally tool-agnostic: the same structure works whether you generate with text-to-video, image-to-video, or a blend of both.
What an AI director assistant actually does
An AI director assistant is not a bigger text box. It is a decision layer that sits between your idea and your generation tools. Its job is to reduce the number of unguided choices you make, because unguided choices are what produce inconsistent footage and endless re-rolls.
Narrative structuring
The first job is turning a vague idea into something shootable. A good assistant pushes you to answer: who wants what, what is in the way, and what changes by the end? It then expands that into beats, and beats into scenes. The output is not literature — it is a production document that tells you what has to be on screen.
Shot planning and constraint tracking
The second job is translation. A scene description like "she realizes he is lying" is not a promptable shot. The assistant converts it into a concrete plan: a close-up on her face, slight push-in, window light from camera left, no dialogue, 3 seconds. It also tracks constraints — wardrobe, hair, props, time of day, colour palette — so that shot 14 matches shot 3.
Routing and sequencing
The third job is routing. Different shots want different tools. A wide establishing shot of a landscape behaves very differently from a two-person conversation, which behaves differently again from a product macro. The assistant helps you decide which generation method each shot needs, then sequences them so the video has rhythm rather than a uniform drip of similar-looking clips.
Phase 1: From premise to beat sheet
Everything downstream inherits the weaknesses of the premise. Spend real time here; it is the cheapest place in the whole pipeline to fix a problem.
Write a logline with a physical goal
The most useful loglines describe something visible. "A retired diver returns to a flooded town to recover her sister's watch" gives you locations, props, wardrobe, and a clear visual objective. "A woman confronts her past" gives you nothing to generate. Whenever you can, convert an internal goal into an external one: instead of "he feels guilty," try "he drives back to return a borrowed car at 3 a.m."
Expand into 8 to 16 beats
A beat is a turn, not a scene. Twelve beats is a comfortable working number for anything from a 60-second short to a five-minute piece. Write each beat as one sentence in present tense with a visible action.
A simple example beat sheet for a short film:
- A courier waits outside a locked building in the rain.
- She checks a package she is not supposed to open.
- The lock releases on its own.
- She enters a room that is warmer and brighter than the outside.
- The package is already open on the table.
- She recognises the handwriting on the note.
- She tries to leave; the door will not open.
- She opens the second package.
Notice that every line is something a camera can see. That is the test for a usable beat.
Cut the beats that are only information
If a beat exists only to explain something, fold it into a visual beat or cut it. Generated video is bad at exposition and good at atmosphere, so lean into what the medium does well.
Phase 2: Turning beats into a shootable script
Once the beats hold, write the script in three passes rather than trying to do it all at once.
Pass one: structure
Write rough dialogue and action with no concern for style. Check that each beat earns the next. If a beat can be removed without breaking the chain, remove it.
Pass two: visual clarity
Rewrite every action line so it describes only what a camera sees and a microphone hears. "She is nervous" becomes "she taps the key twice against her thumb." This pass is where most AI video projects are won or lost, because action lines that are already visual translate almost directly into prompts.
Pass three: compression
Cut the script by 15 to 20 percent. Generated shots are short, and the temptation to write more than you can visually sustain is strong. A tight script with 30 shots will feel longer and richer than a loose one with 60.
Handling dialogue
Treat dialogue as a separate track, not as something you expect a video model to nail. Write your lines, record them with real performers or a voice tool, and then build shots around lip-sync or around reactions and cutaways. Reaction shots are almost always safer and often better. If a line must be seen being spoken, plan for a small number of locked-off, well-lit close-ups rather than trying to sustain a moving, talking shot for 8 seconds.
Phase 3: Building the shot list and visual bible
A shot list is the single highest-leverage document in the entire pipeline. It converts your script into a checklist that a generator can be pointed at.
Columns worth having
For each shot, record: shot number, parent beat, one-line description, framing (wide, medium, close), camera movement (static, push, pan, handheld), approximate duration, lighting direction, characters present, key props, audio notes, and your intended generation method. Once this table exists, you can work through it mechanically instead of re-reading the script every time.
What goes in a visual bible
The visual bible is a short reference document — one or two pages — that locks the things that must not drift:
- Character sheets: face reference, hair, wardrobe, accessories, and any distinguishing feature.
- Location rules: time of day, weather, key light direction, floor texture, architectural detail.
- Palette: two or three dominant colours plus one accent.
- Lens language: which lens feel belongs to which character or location.
Lock before you generate
Do not start generating until the shot list and visual bible are stable. Changing your protagonist's jacket in the middle of a sequence is expensive; deciding on it beforehand costs nothing.
Phase 4: Translating shots into model-ready prompts
Most disappointing AI video comes from prompts that ask for too much at once. Break each shot into a small number of explicit decisions.
The modifier stack
Build prompts in a fixed order so results stay comparable across shots:
- Subject and wardrobe
- Action, in the present tense
- Setting and time of day
- Framing and lens feel
- Lighting direction and quality
- Camera movement
- Style and grade
- Constraints to avoid
Example for a mid-sequence shot: "Woman in a dark green raincoat, hood down, wet hair, walking away from camera through a flooded car park at dusk, medium wide shot, 35mm feel, soft overcast light from behind, slow steady push forward, muted teal and amber grade, no text, no crowds."
Example for the matching close-up: "Same woman, same dark green raincoat, wet hair, looking off-camera right, close-up, 50mm feel, soft overcast light from behind her shoulder, static frame, muted teal and amber grade, no text."
Keeping points 1 to 3 identical between the two prompts is what makes them feel like part of the same film.
One variable at a time
If a shot looks wrong, change one line and regenerate. Changing lighting, movement, and wardrobe in the same pass teaches you nothing about which change worked.
Phase 5: Controlled generation and iteration
Generation should be a screening process, not a production process.
Low-cost passes first
Generate four to eight takes per shot at a modest resolution and short duration. Review them as a contact sheet, not one by one. Pick the take whose composition and motion are closest, and note exactly what is wrong with it.
Then commit
Re-run only the approved take at final quality, with the same prompt and any available seed or reference settings locked. This two-stage approach typically saves more time than any prompt trick.
When a shot refuses to work
If a shot fails after three serious attempts, the problem is usually conceptual, not technical. Either simplify it (fewer subjects, less motion) or replace it with two shots that are easier to generate. Directors cut shots for practicality all the time; an AI workflow is no different.
Phase 6: Assembly, sound, and finishing
Roughly 40 percent of how professional a finished video feels comes from the parts that are not visible generation.
Editing rhythm
Cut generated shots shorter than feels comfortable in the first pass, then lengthen only where the viewer needs to breathe. Because generated footage often contains small imperfections, shorter cuts hide more than long holds.
Sound design
Lay in three layers: ambience (rain, room tone, traffic), foley (footsteps, fabric, objects), and music. Ambience in particular creates continuity — a consistent rain bed makes two mismatched shots feel like the same scene.
Colour and finishing
Apply one grade across the entire piece rather than grading shot by shot. A single unified look — even a slightly aggressive one — reads as intentional, while individually corrected shots can read as patchwork.
Continuity, routing, and quality control
Continuity techniques that work
Reference images are the strongest lever for character consistency. Create a still of each character in a neutral pose, then use it as a visual reference for every shot they appear in. Pair that with a written character sheet for anything a reference image cannot cover, such as a change of clothing between scenes.
Light direction is the second lever. Decide once, per location, whether key light comes from the left, right, or behind, and repeat that phrase in every prompt for that location. Viewers notice inconsistent shadows far more than they notice imperfect faces.
Routing decisions by shot type
- Establishing shots and landscapes: fast and forgiving in almost any tool. Generate several and pick.
- Two-person conversation: hardest category. Prefer static or nearly static framing, and cut between singles rather than trying to hold both people in frame.
- Action and motion: keep the motion simple and the camera still. Complex camera moves plus complex action rarely resolve cleanly.
- Product or detail shots: image-to-video from a clean still gives the most control.
- Transitions: generate short abstract or texture shots and use them as connective tissue.
The pre-render checklist
Before you commit to a final render, confirm: aspect ratio and frame rate match the delivery target; every shot has at least one approved take; audio levels sit consistently between shots; no shot contains accidental on-screen text; and the edit runs cleanly from start to finish with sound, not silently.
Common mistakes that derail AI video projects
Starting with generation instead of a script. Without a shot list, every prompt is improvised, and improvised prompts do not match each other.
Overloading single prompts. A prompt that tries to describe a character, an action, a camera move, a mood, and three style references gives the model no priority order.
Switching tools mid-sequence. Different tools have different colour and motion signatures. Switching within a scene is visible. Switch between scenes instead.
Ignoring audio until the end. Sound is not a final step; it is part of how continuity is manufactured.
Rendering before picture lock. Final-quality generation is the most expensive step in the pipeline. Do it after the edit is stable.
Chasing perfect single shots. A slightly imperfect shot that cuts well beats a perfect shot that does not fit the rhythm.
FAQ
Do I need a full script for a 15-second clip?
You need a premise, one or two beats, and a shot list — even if it is only four shots. The document can be tiny, but the discipline of writing it down is what keeps the clip coherent.
How long should a generated shot be?
Between 2 and 5 seconds for most work. Longer shots expose more artefacts and demand more continuity accuracy. If you need a long moment, build it from several short shots with cuts.
Can I keep the same character across many shots?
Yes, with discipline: one reference still per character, one written character sheet, and one lighting direction per location, repeated verbatim in every prompt. Expect to reject more takes than usual for character-heavy sequences and plan your time accordingly.
Should I generate video directly or start from still images?
Start from stills when composition matters, when you need a specific face or product, or when you want precise lighting. Generate video directly when the shot is atmospheric, wide, or motion-led. Many projects use both in the same edit.
How do I handle spoken dialogue?
Record it separately, then plan for close-ups, reaction shots, and cutaways. Treating dialogue as a post-production audio element rather than a generation problem removes the single biggest source of frustration in AI video.
What should I do when a shot simply will not work?
Simplify or replace it. Fewer subjects, less movement, a static camera, and a shorter duration solve most stubborn shots. If it still refuses, redesign the sequence around what you can reliably produce.
How much time should a short project take?
A one-minute piece with 20 shots is a comfortable single day: a morning for script, shot list, and visual bible; an afternoon of low-resolution passes; and a final block for re-rendering, sound, and grade. Budget roughly double that if the piece depends on a recurring character.
The tools will keep improving, and each new generation model will make individual shots easier. What will not change is the need for someone — or something — to decide what the film is, what each shot must contain, and what must stay the same from one shot to the next. Build that decision layer into your workflow first, and every tool you add afterwards becomes more useful.
