Prompt Engineering for Midjourney and AI Video Generators

Why Prompt Structure Beats Prompt Length

Most disappointing AI images and clips are not caused by a bad model. They are caused by a prompt that describes five ideas at once, in the wrong order, with no technical anchors. When you type a paragraph of adjectives, the model has to guess which words matter. It usually guesses wrong, and you get a result that looks vaguely like your idea but has none of its intent.

The fix is not longer prompts. It is structured prompts. A structured prompt separates what you want to see from how it should be captured and how it should move. That separation is what lets you change one variable at a time, compare outputs, and actually learn what each model responds to.

Think of it as two different skills that share one text box:

• Direction — the creative decisions: subject, wardrobe, mood, story beat.
• Technique — the production decisions: lens, framing, lighting, motion, duration, aspect ratio.

Beginners blend these together. Experienced users keep them in separate slots so they can iterate on one without disturbing the other. If your character looks right but the shot feels flat, you change the technique slot. If the lighting is perfect but the wardrobe is wrong, you change the direction slot. One variable, one test.

How Image and Video Models Actually Read a Prompt

The shared layers

Image models such as Midjourney, Flux-class generators, and Stable Diffusion derivatives all tokenize your text and map it into a latent space. Early tokens tend to carry more weight than late ones, and concrete nouns outperform abstract adjectives. "Weathered brass diving helmet" gives the model far more to work with than "old-looking object."

Video models add a temporal layer on top. They must decide not just what is in frame, but what changes between frames. That means every element of your prompt is also a motion instruction, whether you meant it that way or not. "Crowded market street" implies people walking. "Empty market street" implies stillness. You are always directing motion.

Where video differs from stills

Stills reward density. A single frame can hold ten details and look rich. Video punishes density. If you describe ten moving elements, the model spreads its attention and produces mush. The practical rule: fewer subjects, clearer motion, shorter duration.

Goal	Still image	Video clip
Detail load	High is usually good	Medium is safer
Prompt length	30–80 words	15–45 words
Motion language	Optional	Essential
Iteration speed	Fast	Slower, plan for it

A Six-Slot Prompt Framework You Can Reuse

Rather than memorizing model-specific tricks, build one framework and adapt the vocabulary. Six slots cover almost everything:

1. Subject — who or what, with two or three defining physical details.
2. Action — the verb, in present tense, single and specific.
3. Environment — location, time of day, weather, era.
4. Camera — shot size, angle, lens, movement.
5. Light — source, direction, quality, color.
6. Style and technical — medium, grade, aspect ratio, duration, parameters.

Example for a still:

Subject: a 40-year-old deep-sea welder, salt-crusted beard, scarred forearms
Action: pausing mid-task, wiping the visor
Environment: cramped diving bell, dim interior, 1970s North Sea rig
Camera: medium close-up, 50mm, slightly low angle
Light: single warm work lamp from the left, hard falloff
Style: documentary photography, fine grain, muted teal and rust palette --ar 16:9

The same framework for a clip:

Subject: deep-sea welder in a brass helmet
Action: slowly turns from the visor toward the lens
Environment: cramped diving bell interior, warm lamp glow
Camera: slow push in, 50mm, handheld micro-drift
Light: single warm lamp left, deep shadows right
Style: 1970s documentary, fine grain, muted teal and rust, 5 seconds

Order, weight, and negatives

Put the highest-priority slot first. If the face matters most, the subject leads. If the shot is really a mood piece, lead with light or environment and let the subject be secondary.

Negative prompts are a repair tool, not a first draft tool. Add them only after you see a repeated failure: extra fingers, warped text, duplicated limbs, unwanted crowds. Rewriting the positive prompt is usually more effective than stacking eight negatives.

Midjourney-Specific Control

Midjourney responds strongly to visual nouns, medium descriptions, and short style tags. It responds weakly to long clauses and abstract emotion words. "Melancholy" does little; "overcast light through wet glass" does a lot.

Practical habits that pay off:

• Keep the core under 40 words and put stylistic parameters at the end.
• Use aspect ratio deliberately. Vertical framing pushes portraiture and product close-ups; wide framing pushes landscapes and group scenes.
• Use reference features for identity, not for style alone. Character references stabilize a face; style references stabilize a palette and rendering feel. Mixing both without care produces a face that belongs to the wrong aesthetic.
• Iterate in small steps. Change one slot, regenerate, compare side by side.
• Save winning prompts with the exact parameters in a plain text file. Reconstructing a good prompt from memory wastes hours.

When a still will become a video, plan for it. Generate the frame with slightly more headroom than you think you need, avoid extreme close-ups that leave no space for motion, and keep the background simple enough that the video model has somewhere for movement to happen.

Video Model Specifics: Motion, Duration, and Camera Moves

Video generators fall into two broad camps: text-to-video and image-to-video. They need different prompt styles.

Text-to-video rewards explicit staging. Name the subject, name the single action, name the camera move. If you leave motion unspecified, the model invents it, and invented motion is often jittery drift. Say what should move and what should stay still.

Image-to-video already has composition solved. Here the prompt should focus on motion only, plus any lighting change you want across the clip. Describing the subject again in detail can cause the model to "re-invent" it and drift away from the source frame.

Motion vocabulary that reads clearly to most models:

• Camera: slow push in, slow pull out, orbit left, crane up, handheld drift, locked-off tripod.
• Subject: turns head, steps forward, lifts hand, exhales visible breath, hair moves in wind.
• Environment: steam rises, rain streaks the glass, dust motes drift, leaves tremble.

Duration matters more than people expect. Short clips of three to five seconds keep motion coherent. Longer clips invite morphing, rubbery limbs, and identity shifts. If you need twenty seconds, generate four or five short clips and cut them together rather than asking one model to sustain a long take.

A useful guard against drift: state the stillness. "Camera locked off, only the curtain moves" prevents the model from adding a slow zoom that ruins your edit.

Keeping Characters and Environments Consistent Across Shots

Consistency is where most AI video projects collapse. The face changes between shot two and shot four, the jacket changes color, the room changes shape. Three techniques fix most of it.

Keyframe-first workflow

Generate or select one strong, well-lit keyframe of your character in a neutral pose. Use it as the visual anchor for every subsequent shot, feeding it into image-to-video or as a character reference. Never generate a new keyframe from a text description if you already have a good one — reuse the asset.

Lock the descriptive block word for word

Write a short identity block and paste it unchanged into every prompt:

Identity block: woman in her early thirties, dark curly hair tied back,
freckles across the nose, olive utility jacket, small scar above the left brow

Then vary only the environment, camera, and action slots. The moment you paraphrase the identity block, the face drifts.

Control the environment with anchors

Environments stay consistent when you describe stable, distinctive objects rather than general moods. "Room with green tiles and a broken ceiling fan" is a better anchor than "old kitchen." Pick two or three objects per location and repeat them in every prompt for that location.

Modularity makes this manageable. Keep a text file with blocks for each character and each location, then assemble prompts from those blocks. It feels mechanical at first and becomes fast within a day.

Cinematic Language: Lenses, Composition, and Light

Cinematic results come from technique vocabulary, not from fancy adjectives.

Lens and framing

• Shot size: extreme wide, wide, medium wide, medium, medium close-up, close-up, extreme close-up.
• Angle: eye level, low angle, high angle, over-the-shoulder, profile.
• Lens feel: 24mm for environmental context, 35mm for natural reportage, 50mm for neutral portraits, 85mm for compressed backgrounds and shallow focus, macro for texture.
• Composition: centered symmetry, rule-of-thirds offset, negative space to the left, foreground occlusion.

Lighting language

Lighting is the fastest quality upgrade in any prompt. Describe three things: source, direction, and quality.

• "Single warm practical lamp, left side, hard shadow"
• "Overcast daylight, top-down, soft and even"
• "Neon signage, behind subject, rim light with color spill"
• "Cold blue moonlight, backlit, silhouette with fog glow"

Add a color note to unify a sequence: "muted teal and rust palette" or "warm amber highlights, cool shadow tint." Applying the same palette note to every shot is the cheapest way to make separate clips feel like one film.

A Repeatable Production Workflow

1. Write the shot list first. One line per shot: what changes and what must stay. No prompts yet.
2. Choose the tool per shot based on its strength — stills, subtle motion, stylized motion, or dialogue-driven action.
3. Build the keyframe for each shot, then approve it before spending time on motion.
4. Convert to prompt using the six slots, reusing identity and location blocks.
5. Generate three variants rather than one. Never judge a model on a single roll.
6. Log what worked — prompt, parameters, tool, and a one-line note about why it worked.
7. Assemble and grade. Most clips improve dramatically with a consistent color grade and unified sound design.

A quality checklist before you accept a clip: face matches the reference, hands have five fingers, no morphing at the two-second mark, camera move is the one you asked for, and the clip cuts cleanly into the neighboring shot.

Common Mistakes and How to Fix Them

Mistake	Symptom	Fix
Stacking adjectives	Muddy, generic output	Replace adjectives with concrete nouns
Too many subjects	Merged or melting figures	One subject per shot
Unspecified motion	Aimless camera drift	Name the move or state "locked off"
Paraphrasing identity	Face changes per shot	Use one fixed identity block
Long clips	Morphing and rubbery limbs	Generate short clips and edit
No negative base	Repeated artifacts	Add negatives only for observed failures
Skipping keyframes	Inconsistent style	Approve the still before animating

Tool Notes: Which Generator for Which Job

Different tools reward different prompt styles, and matching the tool to the shot saves more time than perfecting a prompt for the wrong engine.

• Midjourney — strongest for concept art, keyframes, and mood boards. Use it early in the pipeline.
• Flux-class models — good prompt adherence and typography, useful for graphics-heavy frames and product visuals.
• Runway — solid camera-move control and image-to-video workflows for controlled, editorial motion.
• Pika — fast iteration on stylized motion and short social clips.
• Kling and Hailuo-class models — strong human motion and physically plausible movement for character shots.
• Vidu — efficient for stylized animation and consistent character sequences.
• Sora-class systems — narrative text-to-video for longer, more complex staging.

Pick two tools you know deeply rather than six you use occasionally. Depth in one tool beats shallow coverage of many.

FAQ

How long should a prompt be?
Fifteen to forty-five words for video, thirty to eighty for stills. If a prompt exceeds that, you are probably describing two shots.

Should I write in English even if I work in another language?
Most models are trained predominantly on English prompts, so English usually gives more predictable adherence. Write the prompt in English, keep your notes in your own language.

Do negative prompts actually help?
Yes, but reactively. Add them after you see a specific recurring artifact, not as a default list.

Why does my character change between shots?
Almost always because the identity block was rephrased, or because a new keyframe was generated instead of reused. Fix the block, reuse the asset.

How do I get a specific camera move?
Name it plainly and put it in the camera slot. "Slow push in, locked horizon" reads better than "dynamic cinematic movement."

Can one prompt produce a full scene?
No. Build scenes from shots. A scene is an edit decision, not a prompt.

How many generations should I run before changing the prompt?
Three or four. If all of them miss, the prompt is the problem. If one hits, refine rather than rewrite.

Where to Go From Here

Prompt engineering is less about secret syntax and more about disciplined thinking. Write your shot list, lock your identity blocks, keep one variable per test, and let technique vocabulary — lens, light, motion, duration — do the heavy lifting that adjectives cannot.

The strongest habit you can build is a prompt log. Every project should leave behind a document of what you asked for, what you got, and what you changed next. Within a few weeks that log becomes your real advantage, because it captures knowledge no tutorial can give you: how your specific tools behave on your specific kinds of shots.