As AI technology advances, we thought it timely to understand how (and if) it affects psychometric tests used in supporting recruitment decisions. With AI tools readily accessible (available to all with a smartphone in their hand or computer at the ready), there is curiosity around whether AI tools compromise the reliability and integrity of assessments.
Current Observations
Our assessment provider Saville have compared past and present scores (ie those before AI became widely available and more recently) and have seen no significant increase in scores that would suggest widespread cheating using AI. This reassures us somewhat that the tests remain reliable.
It is actually other methods such as video interviews and written responses (including CV’s and cover letters) that are more vulnerable to AI assistance than psychometric tests.
Why Saville Tests Are Resilient
Aptitude Tests:
With tools such as ChatGPT being ‘large language models” designed largely to understand text, AI struggles with them due to time limits and the nature of the questions. AI actually achieves lower scores on many Saville aptitude formats. Furthermore, given candidates don't get feedback until after the test, they can't be sure if AI has helped or hindered their result.
Personality Assessments
The Saville Wave personality assessment is currently more resistant to AI because there is no single right answer. There's no specific way to succeed in any role or a set of answers that would create a perfect behavioural profile. Different strengths and weaknesses can combine in various ways to make someone more or less effective in a role.
The Wave assessment asks candidates to rate and then rank behavioural items and provides scores regarding the level of consistency. This two-step process makes it much harder for AI to help. Given candidates have to make two related but separate decisions about the same item, it adds complexity and reduces the chances that AI can give consistently good answers.
The recommendations from Saville to ensure integrity of your assessment processes suggest:
- Combine different tests: Using multiple types of tests can improve accuracy and reduce the impact of any single test being manipulated.
- Supervised Testing: Conducting tests in a supervised environment can ensure honesty.
- Proctoring Services: For those very concerned about AI cheating, proctoring services can provide supervised testing at scale.
What can clients do?
- Set Clear Expectations: Inform candidates about the assessment process to discourage cheating.
- Use Web Cameras: Let candidates know they might be monitored during the test.
- Honesty Contracts: Have candidates agree to an honesty statement before starting the test.
While AI presents new challenges, most candidates are honest, and unsupervised assessments remain efficient and cost-effective. By designing AI-resilient tests and working closely with organisations, such as Sheffield, who use AI resilient designed assessments, the integrity of the assessment process can be maintained.
Our provider Saville “remain dedicated to actively addressing advancements in AI”, ensuring their assessments continue to accurately reflect candidate’s true abilities.
If you're looking to strengthen your recruitment decisions with the support of psychometric assessments, we’d be glad to chat. Our team can walk you through the tools we use and how they can add confidence and clarity to your hiring process.