Risks of AI in Performance Appraisal: What HR Must Guard Against
Performance appraisal decisions affect pay, promotion and sometimes employment itself. That makes the risks of AI in appraisal more serious than in most HR processes. Here are the eight that matter most, and how to manage each.
Short answer
The main risks of AI in performance appraisal are biased outcomes for particular groups, inaccurate or invented content in AI summaries, surveillance that damages trust and wellbeing, reliance on activity metrics that do not reflect real performance, loss of manager accountability, lack of transparency to employees, data protection breaches and non-compliance with laws such as the EU AI Act, which treats AI used to evaluate workers as high-risk. Each can be reduced through human ownership of ratings, outcome-based measures, transparency, bias monitoring and consultation.
Key takeaways
- Appraisal AI is high-risk because outcomes affect pay, promotion and employment.
- Bias in appraisal often affects part-time workers, carers and disabled employees.
- Surveillance-based metrics are both a fairness risk and a trust risk.
- Transparency and the right to challenge are core safeguards.
Risk matrix
| Risk | Likelihood without controls | Impact | Key safeguard |
|---|---|---|---|
| 1. Biased outcomes | Medium | High | Monitor ratings by group; review data and features |
| 2. Inaccurate content | High | Medium to high | Manager verification of every statement |
| 3. Surveillance harm | Medium | High | Avoid activity monitoring; necessity and proportionality tests |
| 4. Metric fixation | High | Medium | Outcome and goal-based measures |
| 5. Loss of accountability | Medium | High | Managers own ratings and narratives |
| 6. Opacity to employees | High | Medium | Clear notices; access to AI outputs; right to challenge |
| 7. Data protection breach | Medium | High | Approved tools; minimisation; retention limits |
| 8. Regulatory non-compliance | Medium | High | Classify use cases; meet high-risk deployer duties; consult |
Likelihood and impact ratings are indicative and depend on context.
The risks in detail
1. Biased outcomes
AI trained on past ratings can inherit biases in those ratings. Measures based on visibility, hours or responsiveness can disadvantage part-time staff, carers, remote workers and disabled employees. See can AI eliminate bias? for parallels in hiring.
2. Inaccurate content
Generative AI summaries can misattribute achievements, exaggerate or omit context. In appraisal, these errors can shape pay and promotion.
3. Surveillance harm
Tracking keystrokes, screens or messages increases stress, reduces trust and may breach data protection principles. See AI monitoring privacy concerns.
4. Metric fixation
When AI surfaces easily measured metrics, organisations may reward what is countable rather than what matters, such as collaboration, quality and innovation.
5. Loss of accountability
If ratings appear to come from a system, managers may disengage and employees cannot get a human explanation.
6. Opacity
Employees who do not know how AI contributed to their evaluation cannot correct errors or challenge unfairness.
7. Data protection
Performance data is sensitive. Entering it into unapproved tools, keeping it too long or using it for new purposes creates legal exposure.
8. Regulatory non-compliance
Under the EU AI Act, AI used to monitor and evaluate workers or to make decisions on promotion or termination is high-risk. Deployers must assign human oversight, keep logs, use systems as instructed and inform workers and their representatives before use. See EU AI Act employer obligations.
A safeguards checklist
- Managers make and explain all ratings.
- AI outputs are verified before use.
- No activity monitoring as a performance measure without rigorous justification and consultation.
- Rating outcomes monitored by group every cycle.
- Employees informed and able to see and challenge AI-generated content.
- Approved tools with strong data terms.
- Legal classification and consultation completed before launch.
This is general information, not legal advice. Rules on monitoring and automated decisions about workers vary by jurisdiction and may involve consultation with employee representatives.
Related guides
- AI Performance Management: Uses, Risks and Principles for Fair Use
Where AI helps performance management, where it creates risk, and principles for fair use.
- Ethical Issues of AI in Human Resources: 10 Dilemmas HR Must Navigate
Ten ethical dilemmas of AI in HR and the questions to ask about each.
- EU AI Act Employer Obligations: Deployer Duties Explained
Each deployer duty for high-risk HR AI explained, with practical actions for HR.
- AI and Employee Data Privacy: A Guide for HR
How AI changes the privacy picture for employee data, the principles that apply and practical safeguards.
Frequently asked questions
What are the risks of AI in performance appraisal?
Biased outcomes, inaccurate content, surveillance harm, metric fixation, loss of manager accountability, opacity to employees, data protection breaches and regulatory non-compliance.
Can AI appraisal discriminate against employees?
Yes. AI can inherit bias from past ratings or rely on measures such as visibility or hours that disadvantage part-time workers, carers and disabled employees. Monitor outcomes by group.
Is AI performance appraisal high-risk under the EU AI Act?
Yes. AI intended to monitor and evaluate workers' performance and behaviour, or to make decisions affecting promotion or termination, is listed as high-risk in Annex III of the EU AI Act.
How do you reduce the risks of AI in appraisal?
Keep ratings with managers, verify AI content, avoid activity monitoring, monitor outcomes by group, be transparent, allow challenge, use approved tools and meet legal obligations including consultation.