AI Director Workflow: Plan, Shoot, and Edit Consistent AI Video

What an AI Director Actually Does

Generative video tools make single clips easy and coherent sequences hard. Anyone can type a prompt and get eight seconds of something impressive. Far fewer people can produce sixty seconds in which the same character crosses three locations, the lighting matches, the pacing builds, and the voice track lands on the right syllables.

That gap is a directing problem, not a model problem. An AI director, whether it is a human with a disciplined process or an assistant layer inside a generation tool, makes four kinds of decisions on repeat.

Selection. Which shot does the scene actually need, and which of twelve takes earns a place in the timeline?

Continuity. What must stay identical from shot to shot, and what is allowed to change?

Translation. How does an emotional intention become camera language a model can reproduce on command?

Restraint. What gets cut so the story survives?

A director does not need to render every frame personally. The job is keeping a hundred small decisions pointed in the same direction. When a project goes wrong, the failure is almost never a single bad clip. It is drift. Shot 1 and shot 20 look like they came from two different films, and no amount of colour grading hides it.

This guide walks through a complete workflow: pre-production, the visual bible, model selection, prompting, continuity systems, sound, edit, quality control, and the mistakes that quietly ruin otherwise good work.

Pre-Production: From Logline to Shootable Plan

The story spine

Start with a logline of one sentence: who wants what, what blocks them, and what changes. If you cannot write that sentence, no prompt will save you, because generation amplifies whatever you feed it, including vagueness.

Example: A night courier in a flooded city must deliver a package before the water reaches the archive where her brother works. That sentence already implies locations (streets, water, archive), a time of day (night), a palette (wet neon and sodium lamps), and a physical constraint (rising water). Every later decision can be tested against it.

From beats to a shot list

Write four to seven beats. Then translate each beat into shots. A useful rule of thumb for AI production:

A 15-second social clip: 4 to 6 shots.
A 30-second ad: 8 to 12 shots.
A 60-second narrative piece: 12 to 18 shots.

Keep individual generations short. Most models handle 4 to 8 seconds far better than 20, because motion coherence decays over time. Short shots also give you edit control. Pacing lives in the cut, not in the prompt.

For each shot, write one line of intent:

Establishing: city underwater at dawn, slow push in.
Character intro: medium close-up, rain on her face, she checks the package.
Complication: water surges past a barrier, wide shot, handheld feel.

If you can read the shot list out loud and follow the story, you have a film. If you cannot, generating anything yet is premature.

Budget time, not just runtime

Plan your project in three passes: a rough pass at low resolution, a selection pass at working resolution, and a finishing pass at delivery resolution. This structure keeps you from polishing a shot that will not survive the edit anyway. It also makes retakes cheap, because you fail early on purpose.

Building the Visual Bible

Character sheets

Before generating a sequence, lock the cast. For each character, store:

A canonical reference image, front and three-quarter view.
A fixed text description written in the same order every time (age, build, hair, wardrobe, one distinguishing detail).
Two or three wardrobe states, each named, so you can say which one applies to which scene.

The point is not realism, it is repeatability. A character description that changes word order between prompts will produce a different person. Treat the description as a config file, not prose you rewrite each time.

Look lock: palette, lens, and grain

Pick one visual signature and refuse to break it. Write it down:

Palette: two dominant colours plus one accent.
Lens family: for example 35mm for group and environment shots, 85mm for intimate moments.
Light logic: where the key light comes from in each scene.
Texture: grain level, contrast, and black level.

Shot lists fail more often from inconsistent look than from inconsistent story. A viewer forgives a plot shortcut. They do not forgive a scene that suddenly looks like a different production.

Choosing Models by Shot Type

Different models are good at different things, and the differences are practical rather than philosophical. Choose by shot type and delivery requirement, not by leaderboard position.

Decision criteria that actually matter

Identity retention. Can it hold a face across cuts using reference images?
Motion coherence. Does motion stay physical, or do limbs smear?
Prompt adherence. Does it follow camera and lighting instructions, or merely approximate them?
Length and resolution. Enough for a usable take before upscaling?
Control inputs. Depth, pose, first and last frame, motion transfer, camera paths.
Iteration economics. How many takes per usable shot, and what does each attempt cost in time?

Run a handoff test before committing. Generate the same character in two candidate models from the same reference and description. If the results could plausibly stand in the same scene after grading, mixing them is safe. If not, either pick one or plan a bridge shot that hides the transition.

A shot-to-model matrix

Talking head and lip sync. Prioritise phoneme accuracy and head stability over cinematic flair.
Wide establishing shots. Prioritise scene detail, atmospheric depth, and slow, stable camera moves.
Action and motion. Prioritise physical plausibility, keep durations short, and break the action into more shots.
Stylised animation. Prioritise consistency of rendering style over realism.
Product macro. Prioritise controlled lighting, clean backgrounds, and precise camera moves.
Crowd and background plates. Prioritise texture and depth; faces can stay soft if they remain peripheral.

A simple rule: the closer the camera gets to a human face, the more you should care about identity and micro-expression. The wider the shot, the more you should care about environment and motion.

Prompting Like a Director

The five-slot prompt pattern

Write prompts in a fixed order so you always know what changed:

Subject with the locked description.
Action and emotion describing what they do and feel.
Framing and lens for shot size and focal length.
Light and time for source, direction, and colour temperature.
Motion and texture for camera behaviour, film look, and atmosphere.

Example: Night courier in a soaked yellow rain jacket, jaw tight with worry, medium close-up on an 85mm lens, sodium street lamp as key light from the left, rain drifting through the beam, slow dolly in, fine grain and shallow depth of field.

That prompt is not poetry. It is a camera order, which is exactly why it works.

Change one variable at a time

The fastest way to waste a day is rewriting the entire prompt between attempts. Keep a plain text log with columns for take number, what changed, what improved, and what broke. After ten takes you will know which phrases carry weight in your chosen model and which are decoration.

Failure modes to watch

Extra or fused fingers in hand close-ups.
Faces melting during fast head turns.
Backgrounds rewriting themselves mid-shot.
Text and signage turning into gibberish.
Slow-motion artefacts when the generated motion is already slow.

Most of these are compositional problems rather than model problems. Reframe to hide hands, cut before the head turn, avoid legible text, and add motion inside the frame instead of asking the camera to do all the work.

Continuity Systems That Survive Many Shots

Consistency is a system, not a setting. Build it once and reuse it.

Reference lock. Always supply the same identity image plus the same description.
Seed discipline. If your tool exposes seeds, reuse them per scene and record them.
First and last frame control. When available, feed the previous shot's final frame as the next shot's starting point.
Scene bibles. One page per location with palette, light direction, and permitted props.
Naming conventions. project_scene_shot_take_v01 makes selection and handoff sane.
Continuity sheet. A single table listing wardrobe state, time of day, weather, and props per shot.

That last item is the one experienced creators swear by and beginners skip. Fill it in before generating, then update it as you select takes. When a viewer says something feels off, the continuity sheet usually shows why within thirty seconds.

Sound, Edit, and Assembly

Voice first, picture second

Record a scratch voice track before generating final shots. It sets the timing of every scene and prevents the classic trap where a beautiful sequence cannot accommodate its own dialogue. Then cast the final voice with a short audition: read the same two lines, compare delivery, and pick for tone rather than polish. Keep the lip sync pass separate from the visual pass so one does not break the other.

Music as pacing

Choose music before the rough cut when you can. Cut to its structure rather than fighting it. If the track builds at 0:22, put your turn there. This single habit makes generated footage feel dramatically more intentional than it is.

The radio edit test

Listen to the whole piece with the picture off. If the story is unreadable from audio alone, the edit is leaning too hard on visuals to explain itself. Fix the words, not the transitions.

Assembly notes

Cut in chronological order first, then tighten by roughly ten percent.
Hold shots slightly longer than feels comfortable; generated motion often needs a beat to read.
Use J and L cuts so audio leads or trails picture.
Avoid cutting on every musical beat, which flattens tension.
Conform frame rates before adding speed ramps.

Common Mistakes That Sink AI Video Projects

Prompting before writing. Prompt engineering cannot rescue an unclear story.
No selection log. Forty takes without notes means forty takes you cannot use.
Changing everything at once. You learn nothing and blame the model.
Leaving audio to the end. Sound decisions constrain shot length.
Chasing one hero shot. A brilliant clip that does not cut with its neighbours is not progress.
Ignoring delivery specs. Aspect ratio, loudness, and caption safe areas are decided before export, not after.
Trusting defaults. Default settings rarely match a specific brand look.
Mixing formats casually. Every extra resolution and frame rate costs time in conform.

Each mistake has the same antidote: decide earlier. Direction is mostly the discipline of making choices before they become expensive.

Quality Control Checklist and Delivery

Run this pass on the full timeline, not on individual clips, because most defects only appear in context.

Flicker and shimmer. Watch at full size on a large screen.
Hands, teeth, ears. Pause on every close-up.
Identity drift. Compare each shot against the reference image side by side.
Lip sync offset. Check the first and last syllable of each line.
Levels. Aim for consistent loudness across the piece; platforms punish inconsistency more than absolute level.
Captions. Burn in or export separately, with safe margins for vertical crops.
Aspect ratios. Export 16:9, 9:16, and 1:1 from a master that protects the subject.
Compression artefacts. Avoid heavy contrast in dark scenes; banding appears first in gradients.

Finish with a one-paragraph delivery note: runtime, resolution, frame rate, audio loudness, caption file, and where the source project lives. Your next collaborator will thank you.

FAQ

How long does a sixty-second AI video take?
With a locked visual bible and a tested workflow, a realistic split is one day of pre-production, one to two days of generation and selection, and one day of sound and edit. The first project in a new style always takes longer, because you are discovering which phrases and settings behave.

Can characters stay consistent across many shots?
Yes, with three habits: a fixed description order, a reference image used every time, and first and last frame handoffs between neighbouring shots. Consistency usually degrades when descriptions get reworded, not when shot count rises.

Do I need editing experience?
Basic editing literacy helps more than advanced prompt skills. Understanding pacing, J and L cuts, and loudness will improve generated footage more than another hundred prompt variations.

What drives the cost of a project?
Iteration count, resolution, and shot length. Reducing the number of takes through better pre-production is usually the strongest cost lever available.

Should I use one model or several?
Use one model per visual signature when possible, and several when shot types genuinely differ. Test a handoff before mixing, then unify grain, colour, and motion in the edit.

Where does human craft matter most?
Story structure, selection, sound, and pacing. Those four areas still decide whether a project succeeds, regardless of how good the generation tools become.

A Practical First Week

Day one: write the logline, beats, and shot list for a thirty-second piece. Day two: build the visual bible and lock the cast. Day three: run a handoff test and a lighting test. Day four: generate a rough pass at low resolution and select takes. Day five: finish selected shots, record voice, and assemble. Day six: sound design and quality control. Day seven: export three aspect ratios and write the delivery note.

That is the whole discipline. Tools will keep changing and models will keep improving. The workflow above will keep working.