Face-Similarity Scores for AI Drama Characters: Four Checks and the Thresholds We Use
Credits and specs follow TWno1's live billing settings. Last checked: 2026-10-05
Face similarity is the number we gate on: below 0.40 between two characters, at least 0.50 between a shot and its reference (set September 2026).
We set those numbers in late September 2026 while reviewing a 16-episode demo drama with 244 storyboard frames. They were computed with one open-source face-recognition model and hold only for that model. Swap the model and they mean nothing until you recalibrate. What does carry over is the set of four checks behind them, plus three mistakes we made on the way.
Where the score comes from, and why you can't borrow a threshold
A face-recognition model turns a face into a vector of numbers, called an embedding. You compare two embeddings with cosine similarity. Closer to 1 means more alike. That part is standard.
The cutoff is not. The ArcFace paper does not give a universal verification threshold. OpenCV's face-recognition tutorial documents 0.363 cosine for the model it ships, tuned on LFW, a dataset of real photos. Meta's Movie Gen paper used ArcFace with two different numbers again:
| Who set it | What it was for | Number |
|---|---|---|
| Public tutorial for an open-source library | Same person or not, on real photos (LFW) | cosine ≥ 0.363 |
| Meta Movie Gen paper (ArcFace) | Keep clips where consecutive frames show one person | > 0.5 |
| Meta Movie Gen paper (ArcFace) | Drop synthetic reference images that drifted | discard < 0.7 |
| Us (demo-drama frames, calibrated ourselves) | Two characters in one drama must look different | < 0.40 |
| Us (same calibration) | A rendered shot must match its character reference | ≥ 0.50 |
Movie Gen's numbers are data-curation cutoffs, not accuracy claims. AI drama frames also differ from photo datasets: faces are often small and turned away from camera. Our view, and it is a judgment rather than a documented finding, is that any threshold needs to be re-measured on your own frames.
Check 1: no two characters too alike (below 0.40)
The symptom is easy to spot. Viewers can't tell two people apart, and rerolling keeps giving the same face.
In the demo, our two male leads scored 0.537 against each other. The two female leads scored 0.544. Both portrait sets had come from the same template, which pulled every face toward one average look. Editing the appearance text alone may not fix that, because the pull comes from the template.
The rule we adopted: once character portraits are done, score every character against every other in the same project. Any pair at 0.40 or above gets redrawn with a new random seed, up to three times, keeping the least similar result.
Check 2: every shot matches its own reference (0.50 or above)
The second failure is drift. The same character looks like someone else from one shot to the next. Head turns, hugs and a hand across the face are the usual triggers. A clip that only anchors the face on its first frame tends to wander once that happens.
We compared two approaches. Generating directly from several references (a scene base image plus the character reference) scored 0.68 to 0.71 against the character portrait. An older method scored 0.46, under our own bar.
Vendor docs point the same way. BytePlus's Seedance 2.0 prompt guide lists identity drift as a sign of a weak face reference. Its fixes: give a separate headshot as the face reference, name the character in the prompt, put the most important asset first, and avoid multi-angle character sheets.
Check 3: a failing score has to stop something
We caught this one while reviewing finished clips shot by shot.
In one 15-shot episode, 5 shots scored between 0.05 and 0.44 against a 0.50 threshold. All five made the final cut. The check wrote its scores to a report and nothing else happened. On a few other shots the automated check never ran at all, reported "could not run," and was counted as a pass anyway.
Two rules replaced that:
- A failing shot cannot enter the final cut. It is regenerated with a new seed, up to three times, then handed to a person.
- "Could not run" is a warning. It never counts as a pass.
Most tutorials stop at computing the score. That is the easy half. A score with no consequence attached is a report nobody reads.
Check 4: back-turned and profile shots skip the face score
A shot where the character faces away has no detectable face. The score drops, the shot gets flagged, and rerolling does not help because the character is supposed to face away. Worse, forcing such a shot to resemble the frontal portrait makes the head twist unnaturally.
The rule: no face locking and no face score on back-turned or profile shots. Compare hair color and clothing instead.
The four checks on one card
| Check | When | Threshold (our model only) | If it fails |
|---|---|---|---|
| Character vs character | Before portraits are final | below 0.40 | Redraw with a new seed, up to 3 times, keep the least similar |
| Shot vs reference | After each shot, before the cut | 0.50 or above | Regenerate up to 3 times, then a person decides |
| Did the check run | Every shot | "could not run" = warning | Fix the check and rerun; never treat as a pass |
| Back-turned or profile | Flag at storyboard stage | Face score not used | Compare hair color and clothing |
Limits: what a face score cannot see
- Glasses, hair and costume are invisible to it. In the same 244 frames, 58 had the glasses wrong (23.8%) and 130 had the wrong hairstyle (53.3%). Many of those faces would score fine. A matching face does not mean a matching look.
- The threshold belongs to the model. Change the face-recognition model and both 0.40 and 0.50 need resetting. A reasonable way to do it: pull a batch of same-character pairs and a batch of different-character pairs from your own frames, score both, see where the two groups separate, and set the cutoff between them.
- Tiny faces and stylized faces are unreliable. In wide shots or heavily stylized art, detection itself may fail. We would treat those like back-turned shots. That is inference; test it on your own material.
- It is not a content check. The score answers "do these two faces look alike." It says nothing about whether a character reads as adult or resembles a real person. TWno1's content policy requires adult fictional characters, and that has to be handled at character setup and review, not by this number.
Some of the drift in Check 2 comes from feeding each shot a different reference. A TWno1 project in Studio stores the character's look and reference images once, and later shots reuse that same set, which takes one variable out.
Related reading: Keeping One Character Consistent Across a Whole AI Series: References, Prompts and a Per-Shot Look Sheet; Seedance Draft Mode: When a 480p Preview Before 1080p Actually Saves Money; Seedance 2.5 vs 2.0: Clip Length, Reference Limits, 4K and Price Compared.
FAQ
What face-similarity score means two AI frames show the same character?
There is no universal number; the face-similarity threshold depends on the face-recognition model. One open-source library's public tutorial uses 0.363 cosine for "same person" on real photos, and Meta's Movie Gen paper used ArcFace cutoffs of 0.5 and 0.7 for data filtering. Our working threshold for AI drama shots against the character reference is 0.50 or above, and it must be recalibrated if the model changes.
How different should two characters' faces be in one AI drama?
Our working rule is a face-similarity score below 0.40 for every pair of characters. In a 16-episode demo drama, the two male leads scored 0.537 and the two female leads 0.544, which made them hard to tell apart; the rule is to redraw a portrait that is too similar, with up to three new seeds, keeping the least similar. The 0.40 figure applies only to the one open-source face-recognition model it was calibrated on.
How do you check character consistency in a back-turned shot?
Do not use a face-similarity score on back-turned or profile shots. With no detectable face, the score drops and the shot is wrongly flagged as a failure. Compare hair color and clothing against the character sheet instead, and do not force the shot to match the frontal portrait.
Can a shot pass if the face-similarity check could not run?
No. "Could not run" should always count as a warning, never a pass. In one 15-shot episode we reviewed, 5 shots scoring 0.05 to 0.44 reached the final cut because the check only wrote a report, and other shots were passed after the checking tool failed to run. An unrun check is no check.
Do I need new face-similarity thresholds after switching face models?
Yes. Different face-recognition models produce different score distributions, so the same two faces can score quite differently on two models. Score a batch of same-character pairs and a batch of different-character pairs from your own AI frames, find where the two groups separate, and set the threshold there.
References
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition (arXiv 1801.07698) (checked 2026-10-05)
- OpenCV tutorial: DNN face detection and recognition (checked 2026-10-05)
- Movie Gen: A Cast of Media Foundation Models (arXiv 2410.13720) (checked 2026-10-05)
- BytePlus ModelArk: Seedance 2.0 series prompt guide (checked 2026-10-05)
- VBench: video generation benchmark (arXiv 2311.17982) (checked 2026-10-05)