How to keep a character consistent across AI video shots

You generate four beautiful shots of the same character. In the first she has a sharp jawline and dark hair. By the third her face has drifted, her jacket changed from charcoal to navy, and the voice belongs to somebody else entirely. Nothing about the film works any more.

This is the defining problem of AI video production, and it is not solved by writing a longer prompt. It is solved by treating identity as structured data that persists across shots instead of words that get reinterpreted every time.

Why prompts alone fail

A text prompt is a description, and every generation re-reads that description from scratch. Two runs of the same prompt produce two different people because the model is sampling, not remembering. Adding adjectives helps a little. Adding a hundred adjectives helps almost nothing, because the model has no mechanism to hold them constant while the scene changes.

What actually carries identity across shots is a reference: an image the model is conditioned on, so the sampling is anchored to specific pixels rather than adjectives. Everything below follows from that one fact.

The four locks

A character is not just a face. In practice four properties have to be pinned independently, because each one drifts in a different way.

Start with a reference portrait

Generate a clean, front-facing half-body portrait on a plain background before anything else. Front-facing matters: the model needs to see the full geometry of the face, not a three-quarter angle that hides one side. Plain background matters because a busy frame bleeds into subsequent generations.

Write the appearance description once, in concrete physical terms rather than mood words. "Late twenties, sharp jawline, shoulder-length black hair, small scar above the left eyebrow" travels well. "Cool and mysterious" does not — it is a mood, not a specification, and the model will interpret it differently each time.

Pin the wardrobe in words the model can see

Colour names are vague in generation. "Navy" can come back as steel blue, indigo or teal. Name the garment and the material as well: "charcoal wool overcoat with a notched lapel". The more physically specific the noun, the less room the sampler has to improvise.

Clone the voice, then assign it

Voice cloning needs clean source audio — thirty seconds or more of a single speaker, no music, no room echo. Clone once, attach the clone to the character, and every line written for that character is dubbed in that voice. Doing this per-shot instead of per-character is how you end up with a cast of strangers.

Route identity-critical shots through reference-to-video

Once a reference portrait exists, the shot itself should be conditioned on it. This is the difference between text-to-video, which invents a person from a description, and reference-to-video, which animates a person you have already approved.

A practical split for a two-minute piece:

• Reference-to-video for every shot where the character is recognisable — dialogue, close-ups, anything the audience will read as "the same person".
• Text-to-video for establishing shots, inserts, b-roll and anything without a face in it.

This also saves credits. Text-to-video is cheaper to iterate on for a landscape shot that never needed a character in the first place.

Use shot chaining for continuity, not for style

Chaining carries the last frame of one shot into the first frame of the next. It is genuinely useful — motion across a cut feels continuous and the lighting stays matched. But it has a cost: the model is now conditioned on the previous frame as well as your prompt, so a bad frame propagates forward.

Our guidance is to chain deliberate continuations (a character walking through a door, a camera move that spans a cut) and to chain sparingly everywhere else. If a shot comes back wrong, break the chain and regenerate that shot standalone rather than letting the error travel.

The failure modes worth knowing

Identity drift at the cut

Symptom: the character looks right in every individual shot but wrong when cut together. Cause: the reference is being applied inconsistently, or some shots bypassed it. Fix: audit the shot list and confirm every appearance of the character routes through the reference path.

Wardrobe creep

Symptom: the costume gradually shifts shade over a sequence. Cause: colour described by name only. Fix: specify garment plus material, and regenerate from the same reference rather than from the previous frame.

The over-described prompt

Symptom: results get worse as the prompt gets longer. Cause: competing instructions. The model cannot satisfy twelve constraints at once and starts dropping them. Fix: keep the prompt about the shot — framing, action, camera, light — and let the reference carry identity.

Wrong reference, right character

Symptom: the face is correct but the age or build is off. Cause: the reference portrait was generated from a different description than the one used for the shots. Fix: one description, one portrait, everything downstream derived from it.

A checklist before you render

1. One written appearance specification per character, in physical terms.
2. One approved front-facing reference portrait per character.
3. Wardrobe pinned with garment and material, not just colour.
4. Voice cloned and attached to the character, not to individual lines.
5. Every identity-critical shot routed through reference-to-video.
6. Chaining used only for deliberate continuations.
7. Failed shots regenerated standalone, not chained from a bad frame.

None of this is exotic. It is the same discipline a live-action production applies with continuity notes and wardrobe departments — just expressed in a form a generative model can actually hold on to.