From Screenplay to Storyboard: An AI Video Pre-Production Guide

Why the screenplay-to-screen gap is still the real problem

Ask a room of video makers what went wrong on their first AI-assisted project and the stories rhyme. A clip rendered beautifully — soft window light, believable skin, smooth camera drift — and it had almost nothing to do with the scene they had in mind. The character wore the wrong jacket. The camera moved when the scene needed stillness. Two shots set in the same apartment looked like two different buildings.

None of that is really a generation failure. Current video models are extremely good at producing one convincing moment. What they cannot do is hold a story together across forty of those moments. Intent, continuity, and rhythm live in the screenplay, but a screenplay is written for people — actors, directors, editors — who fill gaps with instinct. A model has no instinct. It has a prompt window and a handful of reference frames.

So the practical work of AI video is translation. You take a story written for humans and convert it into a sequence of machine-readable instructions that preserve what actually matters: who is in frame, where they stand, what changes, and how the camera behaves. Everything below is about that translation layer and the workflow that keeps it from collapsing halfway through a project.

One idea is worth carrying through the whole article: the scene plan is the real deliverable. The video is an output. Teams that plan in text and reference images finish projects; teams that jump straight from script to rendering spend their week regenerating the same shot with slightly different wording.

A quick example makes the point. Imagine a two-minute short about a woman who discovers a letter she was never meant to read. On paper the scene is one paragraph. On screen it is eight to twelve decisions: where the letter sits, how she notices it, whether the camera pushes in or holds, how long her hand hovers before she picks it up, what the room does behind her. If those decisions are not written down before rendering, each generation attempt becomes a guess, and guesses do not accumulate into a film.

What an AI director assistant actually does

Conversational planning tools have become common in video workflows, and the label attached to them — director, assistant, co-pilot — oversells what they do. The honest description is narrower and more useful: they convert unstructured prose into structured plans, and they keep that structure stable while you work.

That is not a small thing. Structure is exactly what breaks between a script and a finished edit.

Where the assistant sits in the pipeline

A useful way to think about the pipeline is three layers.

The writing layer holds story: character, goal, escalating beats, turning points. Nothing here is technical, and nothing downstream can repair weakness here. If a character has no goal and the scenes do not escalate, no amount of visual polish rescues the result.

The direction layer converts writing into shots. It answers: where is the camera, how long does the shot hold, what moves, what stays identical from the previous shot, and what is the emotional temperature of the frame without stating it as an emotion.

The generation layer is where model choice, resolution, aspect ratio, and motion strength finally matter. It is the most visible layer and the least strategic one. It consumes the most hours and makes the fewest creative decisions.

A planning assistant lives almost entirely in the direction layer. That is why it can be genuinely valuable while doing nothing you would call creative work.

What to expect from a first pass

Give a planning assistant a script and a few constraints — runtime, aspect ratio, two characters, one location — and a competent first pass will produce something like a scene breakdown, a shot list with framing and duration, a list of continuity anchors per shot, and a note about which scenes could be cut if runtime needs to shrink. That last item is often the most valuable output, because an assistant reads structure rather than remembering how much fun a scene was to write.

Where the assistant stops being useful

It cannot judge whether a performance lands. It cannot tell you that a joke is unfunny in context even though it is well structured. It cannot decide that a wide shot is more moving than the close-up it recommended. Treat it as a first assistant director or a very fast script supervisor, not as an author. The creative judgement stays with you, and the plan is a document you argue with, not a verdict you accept.

Turning a screenplay into machine-readable scene data

Models do not read subtext. They read nouns, adjectives, verbs, and spatial relationships. Your first concrete job is producing a version of the script where every line describes something visible or audible.

• Human script: Maya realises her brother has been lying to her for years. She feels the floor drop out from under her.
• Machine-readable script: Maya stops mid-step. Her hand tightens on the doorframe. She looks at the open drawer, then at her brother. Her jaw sets. She exhales slowly and turns away.

The second version is not better literature. It is better direction, because every clause maps to something a camera can capture. Interior states become behaviour, and behaviour is renderable.

Five rules make this conversion reliable:

1. One action per sentence. A sentence with three simultaneous verbs produces motion soup.
2. Name entities every time. Pronouns create ambiguity the moment a scene has more than one figure.
3. Lock physical facts early. Wardrobe, hair, approximate age, and build belong in scene notes, not improvised later.
4. Make time of day explicit. Late afternoon is a lighting instruction. A melancholy mood is not.
5. Keep runtime honest. A dense page of action can easily become ninety seconds of screen time.

This rewriting pass usually takes an hour and saves whole afternoons.

Grouping scenes into units you can actually render

After rewriting, group scenes into render units: the smallest set of shots that share a location, lighting setup, and character wardrobe. A render unit is your natural work block. If shot twelve and shot eighteen sit in the same unit, generating them back to back costs far less effort than returning to that setup two days later, because your reference frames are still fresh and your prompt blocks are still identical.

A typical two-minute short breaks into four to six render units. If yours breaks into fifteen, the story is probably trying to do too much for the runtime.

From scene breakdown to shot plan

A shot list is the contract between your story and your renderer. Build it as a table with six columns and one row per shot.

Shot	Beat	Framing	Duration	Camera	Continuity anchor
1	Establishing	Wide, static	4s	None	Rain on window, amber lamp left
2	Reaction	Medium close	3s	Slow push in	Same lamp, grey wool coat
3	Discovery	Insert, top-down	2s	None	Hands, brass key on desk

A handful of principles make the plan work in practice.

Generate in edit order. If shot four must match shot three, build shot three first and use a frame from it as the reference for shot four. Chaining forward is simple; retrofitting backward is miserable.

Favour coverage you can control. Establishing, medium, close, insert, reaction. Five shot types cover almost every scene. When in doubt, more short shots beat one long complex shot, because short shots give you cut points and hide generation artifacts.

Keep durations modest. Two to five seconds per generated clip is a sweet spot for most tools. Longer clips drift: faces melt, hands multiply, backgrounds mutate. You can always hold a frame or slow a clip in the edit.

One motion instruction per shot. A slow push in is a motion instruction. A slow push in while she stands up and the camera pans to the window is three instructions fighting for control of the same frame.

Treat the list as a tracker. Mark every row pending, generated, approved, or reshot. At any moment you should know how much work remains without opening a single video file.

Sizing shots to the model you have

Different tools tolerate different shot complexity. A model that handles a fifteen-second clip comfortably may still fail on a shot where a character walks through a doorway, turns, and speaks. Before committing to a shot list, run a calibration test: render three representative shots — one static close-up, one medium shot with a simple action, one shot with camera movement — and see where quality degrades. Then rewrite your plan around the limits you measured rather than the limits you hoped for.

Continuity systems for characters and locations

Consistency is the most requested feature in AI video and the biggest source of frustration, because it is not a setting. It is a data management problem. You need a source of truth that does not change between shots.

Character sheets

For each recurring character, write a fixed, reusable block of text and pair it with one or two reference images:

• Name and approximate age range
• Build, height impression, posture
• Hair colour, length, and style
• One or two distinctive features — a scar, freckles, heavy brows
• Wardrobe for this scene group, described as materials and colours
• A signature palette or lighting behaviour that follows the character

Paste that block verbatim into every prompt where the character appears. Resist paraphrasing. Even small rewordings shift what the model produces, and drift compounds shot after shot until a character becomes a stranger by minute two.

Location sheets

Locations need the same treatment. Record architecture, dominant materials, key light sources, time of day, weather, and two or three signature props that should appear in every shot of that space. A brass desk lamp is not decoration; it is an anchor that tells the viewer, and the model, that this is the same room as the last shot.

When reference images and text disagree

If a reference image shows a red jacket and your prompt says blue, expect chaos. Choose one authority and make the other match it. In most workflows the image wins, so treat reference frames as ground truth and rewrite your text blocks to describe them accurately.

Chaining references across a scene

Once a shot is approved, harvest two frames from it: one wide enough to show the room, one tight enough to show the face. Use them as inputs for the next shot in the same unit. This chain — approve, harvest, reuse — is what keeps a sequence looking like one continuous piece of time rather than a slideshow of unrelated clips.

Writing prompts that behave like camera directions

With a shot list and a continuity bible in place, prompt writing becomes assembly rather than invention. A reliable order of operations for each prompt:

1. Subject and wardrobe — the exact character block, copied verbatim
2. Action — one clear, physically plausible verb
3. Location and time — from the location sheet
4. Framing and lens — wide, medium, close, insert; shallow or deep focus
5. Lighting — source, direction, quality
6. Camera movement — one instruction, or explicitly static camera
7. Continuity reference — name the reference frame you are attaching

A finished prompt might read:

Maya, late thirties, dark curly hair tied back, grey wool coat — standing still at a wooden desk, looking down at a brass key. Small apartment office, late afternoon, rain on the window. Medium close-up, shallow depth of field. Soft window light from the left, amber desk lamp as practical. Static camera. Match the reference frame for wardrobe and room layout.

Notice what is missing: emotion, backstory, and theme. Those come from performance, cutting, and sound. Prompts describe physics.

Guidance that helps more than it hurts

• Keep negative guidance short. No text overlays, no extra people. A paragraph of exclusions tends to confuse the output.
• Avoid stacked motion. One camera move, or one subject move, never both aggressively.
• Prefer concrete over atmospheric. Dust in the air is concrete. A melancholic atmosphere is not.
• Repeat the boring details. Room layout, lamp position, coat colour. Repetition is the mechanism that produces consistency, not laziness.

The generate, review, and approve loop

Rendering is cheap compared with reviewing badly. The most common inefficiency in AI video is approving a shot after a two-second glance, discovering three shots later that it does not match, and regenerating half a scene.

Run a fixed loop on every clip:

1. Watch muted. Does the visual information make sense without sound? Composition, continuity, and clarity should survive alone.
2. Watch in context. Put it in the timeline between its neighbours. Does it cut? Does the eye follow?
3. Check the anchors. Lamp, coat, hair, desk. Continuity errors are easiest to spot when you know exactly what you are hunting.
4. Check for artifacts. Hands, teeth, text in frame, background warping, limbs crossing impossible thresholds.
5. Log the decision. One line: approved, or the specific reason for a reshoot.

Version discipline that pays off later

Name files by shot and take — s03_insert_key_take2 — and keep rejected takes in a subfolder instead of deleting them. Two reasons: you sometimes need a frame from a rejected take as a reference, and comparing takes shows you exactly what your prompt changes accomplished.

When regenerating again is the wrong answer

If a shot fails three times with genuinely different prompts, the problem is upstream. Either the shot is asking for something the story does not need, or it is technically ambiguous. Change the plan, not the wording. Splitting one impossible shot into two easy shots solves more problems than any prompt rewrite.

Assembly, sound, and finishing

Editing is where AI video stops looking like AI video. Three techniques do most of the work.

Cut on motion. If a character turns in shot two and turns in shot three, cut on the turn. The eye reads continuous motion as continuity even when the two clips came from different generations.

Let sound carry the space. A room tone running underneath four shots does more for the illusion of a single location than any amount of visual matching. Add ambience before you judge an edit; half of what looks broken is actually silent.

Use inserts as glue. A close-up of hands, a glass, a door handle. Cheap to generate, and they cover cuts that would otherwise feel abrupt.

Finish with a colour pass that unifies temperature and contrast across clips, because different generations rarely match out of the box. If your piece has dialogue, subtitles are usually faster and more reliable than chasing accurate lip sync.

Tool selection and the mistakes that cost the most time

Tool choice matters far less than workflow, but a few criteria genuinely change how you work:

• Maximum clip length — longer native clips reduce stitching, but only if quality holds
• Reference image support — essential for character continuity
• Named camera controls — worth more than a dozen style presets
• Iteration speed — a fast low-resolution preview mode beats a slow high-quality mode for exploration
• Aspect ratio options — match your delivery platform from day one
• Export and file handling — clean naming and common codecs save hours in the edit
• Collaboration — shared projects and comments matter as soon as two people touch the timeline

Pick one text-to-video tool, one image-to-video tool, and one image generator, then learn them deeply. Workflow skill compounds. Tool hopping does not.

The mistakes that cost the most time, roughly in order of expense:

1. Starting with the hardest shot. Build confidence and reference material on easy shots first.
2. Paraphrasing the character block. Verbatim repetition is the cheapest continuity tool available.
3. Making clips too long. Drift scales with duration.
4. Generating without a shot list. You end up with clips, not a film.
5. Ignoring aspect ratio until the end. Cropping a vertical render into a wide frame destroys composition.
6. Overloading prompts. Each prompt should answer six or seven questions, not twenty.
7. Chasing perfection per shot. A slightly imperfect shot that cuts well beats a perfect shot that does not.
8. Skipping sound. Muted drafts make you fix visuals that sound design would have solved.

A realistic timeline for a two-minute short looks like this: story and rewrite, two to three hours; shot list and continuity bible, two to four hours; reference images, one to two hours; clip generation and review, six to twelve hours across several sessions; edit, sound, and colour, three to five hours. Generation dominates, which is exactly why planning earns its keep. Every hour saved in planning typically returns two or three hours in rendering.

FAQ

Do I need a full screenplay before starting?

Not always. For a thirty-second piece, a one-page treatment plus a shot list is enough. For anything with dialogue, recurring characters, or a twist, write the full script. Structure problems are invisible in a summary and painfully obvious on screen.

How do I keep a character's face consistent across shots?

Combine a fixed text block with reference images, generate in sequence, and reuse approved frames as inputs for the next shot. Consistency comes from repetition and reference chaining, not from one perfect prompt.

Why does my video look like disconnected clips?

Because it probably is disconnected. Fix it in the edit with room tone, cut-on-motion, and inserts before you regenerate anything. Most sequences that feel disjointed are an editing problem, not a generation problem.

Should I generate at the highest resolution available?

No. Explore at low resolution or short duration, then re-render approved shots at final quality. You will typically generate three to five times as many takes as you keep.

How many prompt attempts should a shot get?

Two or three meaningfully different prompts. If the shot still fails, the shot itself is the problem — simplify it, split it, or cut it.

Can AI handle action sequences?

Short, simple, isolated actions work. Complex choreography, crowds, and physical contact between multiple figures remain unreliable. Shoot action as a series of tight single-action shots and cut quickly.

How long should a finished AI short be?

For a first project, ninety seconds to two minutes is the sweet spot. Long enough to have structure, short enough to finish. Scaling to five minutes is a different discipline: more render units, more continuity risk, and far more time in review than in generation.

What is the single highest-leverage habit?

Logging decisions as you go. A one-line note per shot — approved, or the exact reason for a reshoot — turns a chaotic week of rendering into a project with a memory, and it means you never re-litigate a choice you already made.