Lip-Sync & AV Integrity

Lip-sync analysis answers: does the audio of this session actually come from the mouth on screen?

It closes a gap the other video checks leave open. A session can have a live, correctly-matched, non-deepfaked face — while the voice answering the agent's questions belongs to someone else entirely:

• A coach off-camera answering for a mule account holder
• Dubbed or re-recorded audio laid over genuine video
• Voice cloning injected into the call's audio stream
• Pre-recorded answers played at the right moments

How it works

The check measures audio-visual synchronisation across the session: whether mouth movements and speech sounds correspond, moment to moment, the way they do in genuine footage. Desynchronisation — audio leading, lagging, or simply unrelated to the visible articulation — is scored and timestamped.

{
  "check": "lipsync",
  "verdict": "fail",
  "confidence": 0.88,
  "markers": [
    { "t_start": 92.4, "t_end": 118.0, "signal": "av_desync" }
  ]
}

Markers land on the session timeline next to the diarised transcript, so a reviewer can play exactly the stretch where the voice and face diverge — and read what was being said at that moment.

Where it runs

• Video-KYC sessions — automatically, as part of the session check suite
• Uploaded call recordings — any audio+video evidence attached to a case
• Call QA audits — verifying that recorded agent–customer calls are authentic before they are used as compliance evidence

Gating with Goal Rules

{
  "id": "voice_belongs_to_face",
  "severity": "require",
  "check": "lipsync_confidence_below",
  "params": { "threshold": 0.3 },
  "on_deny": {
    "message": "Sustained AV desync — route to review.",
    "max_retries": 0,
    "remediate_type": ""
  }
}

Lip-sync anomalies most often route to human review rather than hard-stop — legitimate network jitter can desynchronise a call — but persistent desync across a session, combined with other signals, is a strong fraud indicator.

Related checks

• Deepfake Detection — a fully deepfaked call often passes naive lip-sync (the fake mouth is animated to the fake audio); running both closes the loop.
• Cross-Verification — what was said on the call is also cross-checked against the session log and the submitted documents.