Why photorealism is a workflow problem, not a model problem
Every few months a new video generation model arrives, and each one claims to close the gap between synthetic footage and a real camera. The gap does narrow. But if you talk to anyone who actually ships photorealistic AI video, you will notice something quickly: their output quality has less to do with which model they used last week and far more to do with the system around it. Reference images, shot planning, prompt structure, continuity notes, upscaling passes, and sound design all carry more weight than the checkbox of a specific model name.
This matters because chasing models is a losing strategy. Generation engines improve, get deprecated, change pricing tiers, and shift their strengths. A workflow, by contrast, is portable. If you know how to build a lighting reference set, how to describe a lens, how to chain keyframes so a character keeps their face, and how to fix a shot that wobbles in the last two seconds, you can move that knowledge to whatever engine is best this quarter.
The other reason realism is systemic: photorealism is a perception, not a measurement. Viewers do not judge a shot by comparing pixel data to a camera original. They judge it by looking for contradictions. Skin that never catches light the way the environment suggests, shadows that point the wrong way, a texture that stays sharp while the camera pushes in, motion that accelerates without weight. A single contradiction is enough to break the illusion. That means your job is consistency hunting, and consistency is a process discipline.
So this guide is deliberately not a ranking. It is a build order: what to prepare before you generate a single frame, how to choose between generation modes for each shot, how to write motion prompts that behave like camera language, how to hold identity across cuts, and how to spend the last twenty percent of effort where it returns the most realism.
Choosing the right generation mode for each shot
Most people default to text-to-video because it feels like the purest form of the technology. In practice, text-to-video is the weakest tool for realism and the strongest tool for exploration. Understanding the three main modes and when each earns its place will improve your output more than any prompt trick.
Text-to-video: for exploration and establishing shots
Use text-to-video when you do not yet know what the shot is. It is excellent for generating a wide establishing image, testing a color palette, or discovering a camera angle you had not considered. It is poor at locking a face, a wardrobe, or a specific product. Treat every text-to-video output as a concept frame, not a final shot. When something good appears, save the frame as a still and move it into an image-driven pipeline. That single habit converts lucky accidents into reusable assets.
Image-to-video: the workhorse of photorealism
Image-to-video is where realism lives. You supply a still — a photograph, a rendered frame, a photograph of a 3D render, or a still generated in an image model — and the video engine animates it. Because the engine starts from a fully resolved frame, it inherits lighting, skin texture, material properties, and composition. Your job shifts from describing what should exist to describing how it should move. This is a much easier problem and produces far fewer artifacts.
Keyframe and first-last-frame control: for choreography
When a shot needs to end in a specific place — a hand reaching a door handle, a car settling into a parking space, a character turning to look at something — first-and-last-frame control is the most reliable approach. You provide the starting state and the ending state, and the engine invents the motion between them. It is the closest thing AI video has to traditional animation blocking, and it is the best way to make a cut land exactly where your edit needs it.
A practical rule: spend your generation attempts in this order — image-to-video for hero shots, keyframe chaining for action, text-to-video for everything you are still deciding.
Pre-production: the assets that decide your realism
The single biggest predictor of photorealistic output is the quality of what you feed in. Garbage references produce glossy, plasticky video no matter how good the model is. Before generating anything, assemble three asset groups.
Lighting and location plates
Collect real photographs of the environment you are depicting — ideally shot yourself, otherwise sourced from permissively licensed libraries. You want at least one wide shot to understand light direction, one detail shot to understand material texture, and one shot at the time of day you intend. These plates do two things: they become image-to-video seeds, and they become your reference standard when judging output. If your generated shot has a warm key from the left and your plate has a cool window from the right, you will catch the mismatch instantly because you have something to compare against.
Character and wardrobe references
For any recurring human, build a small reference sheet: a neutral front view, a three-quarter view, a profile, and one shot with the intended expression. Keep lighting as consistent as possible across the sheet. Write down distinguishing features in words as well — hair part, brow shape, a scar, the way a collar sits — because notes survive model changes and images sometimes do not.
A continuity bible
This is a plain document, not a template. For each scene, record location, time of day, weather, key light direction, wardrobe state, and props in frame. Add the lens you intend and the motion you intend. When you generate shot 14 three days after shot 6, this document is the only thing keeping the sequence coherent. Teams that skip it spend their time re-generating instead of finishing.
Prompt craft: writing motion like a camera operator
Photorealistic video prompts fail most often because they describe content when they should describe photography and motion. A useful structure has four parts, and the order matters less than the completeness.
Subject and action. State who or what, and what they are doing, in plain language. One primary action per shot. "A woman in a grey coat walks toward the camera" is better than "a woman walks, turns, smiles, and reaches for a door."
Camera. Name the format, the lens, and the movement. "Handheld medium shot, 35mm, slight drift to the right." Camera vocabulary does real work here because it implies depth of field, perspective compression, and motion blur. A 24mm lens implies wide distortion and deep focus; an 85mm implies compressed background and shallow focus. Use that.
Lighting and atmosphere. Reference the plate. "Overcast daylight, soft shadows, cool ambient bounce from wet pavement." Atmosphere terms — haze, dust, drizzle, steam — add separation between foreground and background, which reads as depth and therefore as realism.
Texture and imperfection. This is the most commonly skipped part and the most valuable. Real footage has grain, slight lens breathing, minor exposure drift, skin pores, fabric pilling, scuffed paint. Add two or three of these per prompt. Do not add all of them; a shot that is simultaneously grainy, hazy, and dirty looks like a filter.
Negative guidance deserves its own attention. When a shot goes wrong, resist the urge to add a long list of prohibitions. Instead, diagnose the specific contradiction and correct it positively. If a face melts during a turn, the fix is usually to reduce motion speed and add a head-turn reference — not to append five negative terms and hope.
Holding identity and continuity across shots
Cutting between shots is where photorealistic sequences most often fall apart. Individually convincing clips can look like different films when edited together, because viewers track faces, clothing, and light direction across cuts with surprising precision.
Reference-based identity anchoring
Instead of relying on a text description of a character, supply reference images with every generation. Where the engine supports multiple references, use two or three consistent angles rather than many inconsistent ones. Consistency in your references matters more than quantity. A set of five references with wildly different lighting teaches the model to average them, which produces a face that looks like nobody.
Keyframe chaining
For sequences, generate the last frame of shot A as an image, then use it as the first frame of shot B. This creates an unbroken visual thread and dramatically reduces drift. Chain no more than three or four links before re-anchoring to a fresh reference, because small errors accumulate. When a chain starts to drift, do not try to correct it mid-chain — go back to the anchor frame and rebuild forward.
Environmental continuity
Track light across the whole scene, not just per shot. If the sun is behind the subject in wide shots, it must be behind them in close-ups. The most common tell in AI video is an inconsistent key light, and it is also the easiest to fix because it is entirely a note-taking problem. Add a light direction field to your continuity bible and check it before every generation.
Wardrobe and prop state
Buttons, sleeves, ties, and scarves are notorious for changing between shots. Write down the state at the start of each scene and lock it. If a jacket is open in the master, it stays open until the story changes it. This sounds pedantic until you watch a sequence where a collar flips three times in eight seconds.
Post-production: the last twenty percent of realism
Generation gets you to roughly eighty percent. The remaining realism comes from a short, repeatable finishing chain. Skipping it is the most common reason technically decent AI footage looks synthetic.
Temporal stabilization and deflicker
Most engines produce low-amplitude flicker — tiny brightness and color fluctuations frame to frame. Human eyes read this as video noise or as a subtle wrongness. A deflicker pass at low strength removes it without softening the image. Combine with light temporal denoise only if the shot is noisy; over-denoising produces the waxy skin that everyone associates with AI video.
Upscale and detail recovery
Generate at the highest native resolution you can afford, then upscale in a second pass rather than asking one pass to do everything. Detail recovery works best on faces and fabrics, where viewers look first. Keep an eye on eyes and teeth: those are the two areas where aggressive sharpening becomes uncanny fastest.
Grain, halation, and grade
Add fine grain last in the chain. Match grain size to the apparent sensor: heavier grain for a handheld night exterior, almost none for a clean daylight shot. A subtle halation on practical light sources sells realism cheaply. Then grade — and grade conservatively. A heavy teal-orange push instantly reads as generated footage because it flattens skin tones.
Sound is half the illusion
Realism collapses with bad audio faster than with a bad frame. Room tone, cloth movement, footsteps matched to the surface, and slightly imperfect dialogue all signal authenticity. Even a simple ambience bed with a subtle low-frequency hum will make viewers accept visual imperfection they would otherwise notice. If a shot feels fake and you cannot find the reason, check the sound before re-rendering.
A worked example: a two-minute photorealistic sequence
Here is how the pieces fit together on a realistic project — a two-minute narrative sequence with three locations and two recurring characters.
Day one is asset day. Collect or shoot location plates for all three environments, gather four reference images per character under consistent lighting, and write the continuity bible with light direction, wardrobe state, and lens choice per scene.
Day two is exploration. Generate twenty to thirty text-to-video concepts at low resolution across all three locations. Do not aim for quality; aim for coverage. Select six frames that have strong composition and plausible lighting.
Day three is hero generation. Convert the six selected frames into image-to-video shots, each with a single action and explicit camera language. Generate three or four variations per shot and pick ruthlessly. Expect a hit rate of one in three at best.
Day four is choreography. For the four shots where motion must land on a specific beat, build first-and-last-frame pairs and interpolate. Chain keyframes across the two matching shot sequences so the character thread is unbroken.
Day five is finishing. Stabilize, deflicker, upscale, grain, grade. Then edit to sound: lay in room tone, footsteps, and dialogue, and let the audio rhythm determine final cut points. Re-render only shots that fail at the edit — usually two or three out of twenty.
That is roughly a working week for two minutes of finished material. The ratio matters more than the schedule: preparation and finishing consume about half the time, and generation consumes the rest. Teams that invert that ratio spend two weeks generating and still do not have a sequence.
Common failure modes and how to fix them
Melting faces during motion. Reduce motion amplitude, slow the action, and supply a sharper face reference. Rotational motion — turning the head, twisting the torso — is the hardest case, so give it a dedicated shot rather than hiding it inside a walk.
Rubber-limb physics. Motion prompts that describe a result rather than a physical cause produce floaty movement. Describe weight: "foot plants on wet concrete, shoulders dip with each step."
Plastic skin. Usually caused by over-denoising or by generating at low resolution and upscaling aggressively. Pull the denoise back and add a very small amount of noise before upscaling to give the upscaler texture to work with.
Inconsistent light across cuts. A documentation failure, not a model failure. Fix it in the continuity bible and regenerate the offending shot rather than trying to salvage it in the grade.
Everything looks slightly slow. AI motion often defaults to a gentle drift. Add specific speed language — brisk, urgent, unhurried — and where the engine allows it, adjust motion strength per shot rather than globally.
Over-sharpened eyes. Dial back detail recovery on faces and let grain mask fine detail. Eyes read as real when slightly soft, not when razor sharp.
A quality control checklist before you export
Run this pass on every finished shot. It takes two minutes and catches most re-render causes.
- Light direction matches the scene master and the previous shot in the cut.
- Wardrobe and prop states match the continuity notes.
- Face identity holds from first frame to last, including profile angles.
- Shadows move in the same direction as the subject and environment.
- No flicker at low amplitude; check a dark area of the frame where it shows worst.
- Background motion is at a plausible distance from camera motion.
- Skin has texture; no waxy or blurred patches.
- Audio includes room tone, and footsteps match the surface.
- The shot cuts cleanly on the intended beat with the following shot.
If two or more items fail, regenerate. If only one fails and it is minor, fix it in post — that is what the finishing chain is for.
FAQ
Do I need a specific model to get photorealism? No. Different engines specialize — some handle human motion better, some handle environments and camera movement better. Test the same reference image and prompt across two or three engines on a short shot and compare motion quality rather than still quality; stills are easy, movement is the differentiator.
How many generation attempts should a shot take? Budget three to four. Hero shots with complex motion may take eight or ten. If you are past twelve, the problem is usually the input frame or the prompt structure, not luck.
Is it better to generate longer clips or shorter ones? Shorter. Two to five seconds per generation, edited together. Longer single generations drift in identity and physics, and you lose the ability to cut on rhythm.
Can I fix a bad shot without regenerating? Sometimes. Flicker, minor color shifts, and slight instability are all fixable in post. Identity drift, wrong light direction, and broken physics are not — regenerate those.
How do I stop characters changing between shots? Chain from the previous shot's last frame, keep reference sets small and consistent, and log wardrobe and lighting state. Consistency is a bookkeeping habit before it is a technical one.
Does resolution matter more than motion quality? No. Viewers forgive softness far more readily than they forgive unnatural movement. Prioritize plausible motion and consistent light, then spend your remaining effort on resolution and grain.
What is the most common beginner mistake? Treating generation as the main event. Preparation and finishing determine the result. Generation is the middle of the pipeline, not the whole of it.
