Why Most AI Video Projects Collapse Before the First Render
Almost every disappointing AI video fails for the same reason: the creator jumped straight from a loose idea to a generation prompt. The script existed as a paragraph in a notes app. The characters existed as whatever the model happened to draw. The camera language existed as a vague hope that the output would look cinematic.
A better approach borrows the discipline of traditional film pre-production and applies it to a pipeline where the "crew" is a stack of models. That means a locked script, a scene breakdown, a character bible, location references, and a shot list — all written in a form that both humans and models can read.
This guide walks through that pipeline step by step. It is tool-agnostic: you can run it with a chatbot for writing, an image model for character design, and one or more video models for animation. What matters is the order of operations and the artifacts you produce along the way.
By the end you should be able to take a two-page script and convert it into a production package: scene list, shot descriptions, character key frames, location plates, and a versioning system that lets you re-render any shot without starting over.
Step 1: Write a Script That Survives Being Filmed
From premise to beat sheet
Before any visual work, compress your idea into a premise of one or two sentences. Then expand it into a beat sheet of eight to twelve beats. Each beat should describe a change: a decision, a reveal, a reversal, or a physical action. AI video models are far better at rendering change than at rendering mood, so a script full of internal monologue will produce flat footage.
Prose pass, then dialogue pass
Write the first draft as prose. Do not worry about formatting. Once the story holds together, do a second pass that converts it into screenplay-style scenes: a slugline, a short action block, and dialogue where needed. Keep action blocks under four sentences. Long paragraphs tempt you to compress too much into a single shot later.
The readability test
Read your script out loud. Any sentence you stumble over will also confuse a model. Ambiguous physical staging ("they argue near the window") produces ambiguous frames. Specific staging ("she stands at the window, back to him, holding a cold cup of coffee") produces something a model can actually build.
A useful constraint: every scene should be one location and one continuous slice of time. If your scene needs a cut, split it into two scenes. This single rule will make your shot list dramatically easier to generate.
Step 2: Build a Scene Breakdown the Models Can Parse
What a breakdown contains
A scene breakdown is a table. For each scene, capture:
- Scene number and slugline
- Location and time of day
- Characters present
- Required props
- Emotional tone in three adjectives
- Lighting intent
- Duration target in seconds
This table becomes the source of truth for everything downstream. When a render comes back wrong, you diagnose against the table instead of guessing.
Use an assistant to draft, then edit by hand
An AI writing assistant can convert your script into this table in seconds. It will also hallucinate: it will invent props that do not exist and mislabel time of day. Treat the output as a first pass and correct it manually. The correction step is where most of the value lives, because it forces you to decide what the scene is actually about.
Tag scenes by difficulty
Mark each scene as easy, medium, or hard based on how much the model must invent. A locked-off medium shot of one character speaking is easy. A crowd scene with two people interacting physically in the rain is hard. Front-load easy scenes in your production schedule so you build muscle memory and reusable assets before you attempt the difficult ones.
Step 3: Turn Prose Into Shot Descriptions
The four-part shot formula
Every shot description should answer four questions in order:
- Who or what is in frame
- What they are doing
- Where the camera is and how it moves
- What the light and atmosphere are doing
A working example: "Medium close-up of Mara, seated at a desk, slowly closing a notebook. Camera is at eye level, static, slightly off-center to the left. Warm lamp light from frame right, cool window light behind her, shallow depth of field."
That sentence maps cleanly onto most text-to-video prompts without further rewriting, which is the point.
Coverage: shoot for the edit
Generate more angles than you think you need. A practical minimum for any scene with dialogue:
- One wide establishing shot
- One medium two-shot
- One close-up per speaking character
- One insert of the object or action that matters
- One transition shot that can bridge to the next scene
Five to eight shots per scene is a healthy target. For a three-minute piece with eight scenes, that is roughly fifty shots. Budget your time accordingly, because each shot may need two or three attempts before it is usable.
Camera grammar that models respect
Models respond well to a small vocabulary of camera terms: static, slow push in, slow pull out, pan left, pan right, tilt up, tilt down, tracking shot, handheld, crane up, orbit. They respond poorly to compound instructions like "push in while panning right and racking focus." One movement per shot.
Step 4: Solve Character Consistency Before You Animate
Build a character bible
For each named character, write a short spec sheet: approximate age, build, hair, wardrobe, distinguishing features, and two or three personality adjectives that should show in posture. Keep wardrobe to a single outfit per scene block, and write it down precisely — "charcoal wool coat over a cream turtleneck" beats "winter clothes."
Generate key frames, then lock them
Generate still images of each character until you have a set that looks like the same person from multiple angles and in multiple lighting conditions. Save these as your key frames. They become reference images for every subsequent image or video generation involving that character.
Multi-image reference workflows — where you supply several stills alongside the prompt — dramatically outperform text-only descriptions for consistency. If your chosen video tool supports reference images or character identity features, use them from the first shot, not as a rescue after the third failed attempt.
Consistency traps to avoid
- Changing hair or wardrobe mid-scene without a story reason
- Describing a character in the prompt but never supplying a reference image
- Generating a beautiful hero frame, then animating it with a different tool that ignores the reference
- Mixing models within a single scene, which changes the rendering "grain" between cuts
Pick one toolchain per scene. Consistency across a cut is easier to fake than consistency inside a continuous take.
Step 5: Design Locations and Atmosphere Deliberately
Two plates per location
For every location, generate at least two wide establishing plates: one in the primary lighting condition, one in a contrasting condition. These plates serve two purposes. They anchor the geography for the audience, and they give you a stable background reference when you generate closer shots in the same space.
Light as a continuity marker
Lighting is the cheapest way to signal time and continuity. Assign each scene block a light signature: golden low sun, flat overcast, hard practical lamp pools, blue pre-dawn. Write it into the breakdown so it lands in every shot description for that block. Audiences read lighting shifts as temporal shifts, which means inconsistent light reads as a continuity error even when nothing else changed.
Atmosphere in three variables
Atmosphere reduces to three dials: weather, particulate, and contrast. Light rain plus haze plus low contrast reads as melancholic. Clear air plus hard sun plus high contrast reads as tense or triumphant. Choose a combination per scene and keep it consistent within the scene. Mixing dials mid-scene makes the footage feel assembled from unrelated clips, which it probably is.
Step 6: Make the Pipeline Repeatable
Naming conventions
Adopt a rigid naming scheme before you generate anything:
s03_sh02_v04_closeup_mara
The scene, shot, version, and a short descriptor. This looks pedantic until you have three hundred files and need to rebuild a sequence after a crash.
Version everything, delete nothing
Keep every render, including failures. Failed generations are valuable: they document what the model misunderstood, and occasional near-misses become usable inserts later. Store the exact prompt alongside each output, either in the filename, in a sidecar text file, or in a spreadsheet with one row per render.
Seed and setting discipline
If your tool exposes a seed, record it. If it exposes motion strength, camera controls, or frame interpolation options, record those too. Reproducibility in AI video is partial at best, but recorded settings turn a lucky accident into a repeatable technique.
A shot ledger
Keep a single spreadsheet that tracks, per shot: status (todo, drafted, approved), tool used, prompt, reference images, best version, and notes. This is your edit decision list before there is an edit. When you sit down to assemble, you work from the ledger rather than from a folder of mystery files.
Step 7: Choosing Tools Without Getting Lost
Decision criteria that actually matter
- Character identity support. Does the tool accept reference images or identity conditioning?
- Motion control granularity. Can you specify camera movement, or only describe it and hope?
- Clip length. Does a single generation cover your average shot, or will you need stitching?
- Output resolution and aspect ratio. Native 16:9 and 9:16 saves enormous time.
- Iteration cost. How many attempts does a usable shot typically need?
- Style coherence. Does the model have a consistent look, or does every clip feel like a different film?
A pragmatic division of labor
Use a general-purpose chatbot for writing, breakdowns, and prompt drafting. Use a strong still-image model for character and location plates, because stills are cheap and iteration is fast. Use one or two video models for animation, and commit to one per scene. Use a standard editor for assembly, sound, and titles.
When to switch tools
Switch when a specific shot class consistently fails, not when a new model appears in your feed. Write down the failure mode first: "the model cannot keep two faces stable in the same frame." Then test the alternative against that specific problem. Switching tools mid-project for general excitement is the fastest way to end up with footage that does not cut together.
Step 8: The Review Loop That Saves Renders
Watch at speed, judge in categories
Review raw generations at 2x speed. Score each clip on three axes: technical (artifacts, warping), performance (does the action read), and continuity (does it match the scene's light, wardrobe, and geography). Reject fast. A clip that is 70% good is usually not salvageable, and one more render is cheaper than an hour of cleanup.
Batch your prompts
Write all prompts for a scene before generating any of them. You will catch contradictions — two shots with incompatible light, a character wearing the wrong coat — before spending render time. Batching also makes it easier to keep prompt phrasing consistent, which improves how cohesive the final scene looks.
The three-attempt rule
If a shot fails three times, the problem is almost always the prompt or the premise, not the model. Rewrite the shot as something simpler. Cut the camera move. Reduce to one character. Change the angle to something the model handles comfortably. A simpler shot that works beats an ambitious shot that never lands.
Step 9: Common Mistakes and How to Avoid Them
Writing prompts like poetry. Models reward concrete nouns and spatial language, not atmosphere adjectives stacked five deep.
Skipping the breakdown. Without a table, you will forget which character is where in scene four, and the error will only surface in the edit.
Generating audio-first. Lock picture before you commit to music and dialogue timing. Rebuilding sound design after a shot change is expensive and demoralizing.
Chasing a single perfect take. Twenty variations of one shot is a sign the shot is wrong, not that the model is weak.
Ignoring aspect ratio until export. Generate at your delivery ratio from the start. Cropping later destroys compositions and re-frames faces badly.
No shot ledger. This is the mistake that turns a two-week project into a two-month one. The ledger costs fifteen minutes to set up.
Treating AI generation as the whole job. Generation is maybe a third of the work. Writing, breakdown, and editing carry the rest.
Step 10: A Worked Mini Example
Suppose you have a two-page script about a night-shift security guard who finds a door that should not exist.
The breakdown yields four scenes: the empty corridor at the start of shift, the discovery, the attempt to open the door, and the aftermath. Ten shots total for a ninety-second piece. The character bible locks one outfit and one key frame set for the guard. Two corridor plates are generated, one lit by flickering fluorescents, one by a single red emergency lamp.
Shots are described with one camera move each. The discovery shot is a slow push in on the guard's face, lit red from frame left. The door insert is static, tight, with a slow light flicker. The ledger tracks eleven renders, three of which are rejected for hand warping.
Assembly takes an afternoon. The finished piece feels intentional not because the generation was flawless, but because the light, wardrobe, and geography held steady across every cut. That steadiness is the entire payoff of pre-production discipline.
Frequently Asked Questions
Do I need a finished screenplay before generating anything?
No, but you need a locked scene list. Many creators develop the script and the visuals together, iterating on both. The requirement is that once you generate a shot, the scene's parameters are frozen so later shots can match it.
How do I keep the same actor's face across many shots?
Use reference images or identity features where available, keep wardrobe and hair identical, and generate all shots in a scene with the same tool. Where full consistency is impossible, favor angles that show less of the face — over-the-shoulder, profile, or distant framing.
How many shots should a one-minute video have?
Roughly twelve to twenty for a piece with cutting rhythm, fewer if you favor long takes. Long takes are harder to generate cleanly because errors accumulate over the duration, so beginners should favor shorter shots.
Should I generate video directly from text or animate still images?
Animating stills gives you far more control over composition and character design, because you approve the frame before motion is added. Direct text-to-video is faster for establishing shots and atmosphere where precise staging matters less.
What is the most common reason a scene feels off?
Usually inconsistent lighting. Viewers forgive a lot of motion weirdness but register a light change between cuts as an error immediately. Lock a light signature per scene and enforce it in every prompt.
How long should this whole pipeline take for a three-minute film?
With practice, a three-minute piece with eight scenes and fifty shots takes one to two weeks part-time, most of it in generation attempts and editing. The first project takes considerably longer; the ledger and character bible are reusable assets for the next one.
Do I need editing software, or can I assemble in the generation tool?
Use a real editor. Trimming, sound, color matching, and titles are where a collection of clips becomes a film. Generation tools handle none of that well.
