How Are AI Video Interviews Scored? Methods, Validity and Limits
Candidates want to know how they are being judged. Employers need to know whether the judgement is sound. Here is what actually happens between a recorded answer and a score.
Short answer
AI video interviews are typically scored by transcribing the candidate's spoken answers into text, analysing that text against a competency framework, rubric or model answers using natural language processing or a large language model, and combining question-level ratings into an overall score or ranking. Defensible scoring focuses on the content of answers against job-related criteria; scoring based on facial expressions, emotions, voice tone or appearance lacks scientific support and creates legal and discrimination risk.
Key takeaways
- Most scoring now evaluates what candidates say, via transcripts.
- Transcription errors directly affect scores, so accuracy across accents matters.
- Scores are only meaningful if validated against expert ratings and job outcomes.
- Recruiters should see evidence behind scores, not just numbers.
The scoring pipeline
- Capture: the candidate records video or audio answers, or types responses.
- Transcription: automatic speech recognition converts speech to text.
- Content analysis: the text is evaluated against a rubric for each question, for example evidence of planning, stakeholder management or problem solving.
- Rating: each answer receives a rating, often with highlighted evidence.
- Aggregation: ratings are combined, sometimes weighted, into an overall score or rank.
- Review: recruiters or hiring managers review scores, evidence and, ideally, the recordings themselves.
Scoring methods compared
| Method | How it works | Strengths | Concerns |
|---|---|---|---|
| Rubric-based NLP | Maps answer content to behavioural indicators in a rubric | Structured, explainable | May miss unusual but valid answers |
| Trained scoring model | Learns from answers previously rated by expert assessors | Can mirror expert judgement | Inherits any bias in those ratings |
| LLM evaluation | A language model rates answers against criteria and explains why | Flexible; gives reasons | Can be inconsistent; requires careful testing |
| Facial, vocal or emotion analysis | Infers traits from expressions, tone or micro-movements | None well supported | Weak science, bias risk, prohibited for emotions in EU workplaces |
What should and should not be scored
Legitimate signals
- Evidence of job-relevant competencies in the content of answers.
- Relevant knowledge or technical accuracy where the role requires it.
- Structure and clarity of reasoning, where communication is a genuine job requirement.
Signals to exclude
- Facial expressions, eye contact, smiling or perceived emotions.
- Accent, speech rate or voice pitch.
- Appearance, background or lighting.
- Grammar or fluency, unless genuinely required for the role and fairly assessed.
How to validate AI interview scores
- Agreement with experts: compare AI scores with independent ratings by trained assessors on the same answers.
- Consistency: check that the same answer receives the same score when re-evaluated.
- Transcription accuracy: test error rates across accents and speech patterns in your applicant population.
- Adverse impact: compare score distributions and pass rates by group. See how to audit AI hiring tools.
- Predictive validity: over time, check whether scores relate to job performance.
Transparency for candidates
Tell candidates what the interview assesses, how answers are evaluated, whether a person reviews them and how to request adjustments. In Illinois, explaining how the AI works and the characteristics it evaluates, and obtaining consent, is a legal requirement for AI analysis of video interviews. Candidates preparing for these interviews can read tips to pass an AI interview.
Related guides
- AI Interview Software: Types, How It Works, Risks and Best Practice
Types of AI interview tools, how scoring works, the law and best practice.
- Is AI Interview Assessment Fair? Evidence, Risks and Safeguards
Where AI interviews can be fairer, where they are not, and the safeguards that make the difference.
- Tips to Pass an AI Interview: A Practical Guide for Candidates
How candidates can prepare for AI interviews, structure answers and know their rights.
- Examples of AI Bias in Recruitment: Cases and Lessons for Employers
Documented cases and research on AI hiring bias, and the lesson each teaches employers.
Frequently asked questions
How does AI score a video interview?
Typically it transcribes spoken answers, evaluates the text against a rubric or competency framework using natural language processing or a large language model, rates each answer and combines the ratings into an overall score for human review.
Does AI judge your facial expressions in interviews?
Reputable current tools generally focus on what you say rather than facial expressions. Emotion recognition in the workplace is prohibited in the EU, and facial analysis has been widely criticised as unscientific.
Can AI interview scores be biased?
Yes. Transcription can be less accurate for some accents, and scoring models can inherit bias from training data. Scores should be validated and monitored for adverse impact.
Does a human see my AI interview?
Practice varies. Good practice is for recruiters or hiring managers to review scores, evidence and recordings before decisions are made. Ask the employer if unsure.