Why do Yoti facial age estimation results published by NIST differ to those reported by Yoti in its white papers

profile picture Sofi Summers 4 min read
Yoti's Facial Age Estimation results versus the NIST Age Estimation evaluation report

In September 2023, we submitted our facial age estimation model to the US National Institute of Standards and Technology (NIST), as part of a public testing process. This is the first time since 2014 that NIST has evaluated facial age estimation algorithms. NIST age estimation reports are likely to become a globally trusted performance guide for vendor models.

NIST assessed vendor Facial Age Estimation models using 4 data test sets at certain image sizes:

NIST image sizes used in the evaluation of Yoti's Facial Age Estimation

NIST provides some example images:

Fig. 5. The figure gives simulated samples of application type image used in the evaluation. Image source: Authors. Fig. 4. Examples of mugshot images used in the evaluation. Image source: NIST Special Database 32: Multiple Encounter Deceased Subjects (MEDS).

NIST note in their report that age estimation accuracy “will depend on the quality of the images” and the type of facial images captured.

For 6 years, Yoti have trained our model on primarily selfies of people looking into a mobile phone camera (or a laptop camera) because this is the obvious way customers can capture (live) their facial image to be age estimated. We capture these facial images at 720 x 800 pixels, with the face closely cropped to maximise the facial detail, because we have learned that we can attain higher age estimation accuracy for businesses by using this image size. 

We believe our training and testing on mobile phone images with closely cropped faces at 720 x 800 image size are key reasons why Yoti published MAEs (and FPRs) are lower (more accurate) for the Yoti model than the performance data published by NIST their 4 different test data sets.

Table displaying the differences in performance between the NIST evaluation results and Yoti's own testing results of Yoti's Facial Age Estimation.

NIST selected FPR objectives of 10%, 5% and 1% in their report as a way to benchmark their evaluation. As can be seen from the table above, NIST publish that Yoti’s age estimation model is more accurate on higher image size ‘Mugshot’ faces than lower image size ‘Application’ faces.  Consequently, the age thresholds required to meet FPRs of 10%, 5% and 1% are lower for Mugshot images than those needed using Application images. The age thresholds required to meet these FPR %s are lower still when the Yoti model is estimating age from mobile phone captured, higher image size, facial images. 

NIST used over 11 million facial images (with verified age) to test vendors. Some readers may wonder why NIST did not also test vendors with a test set of mobile phone camera facial images given, this is how most images will be captured for online age estimation.

The reality is that it is very challenging to capture, with consent, a database of millions of mobile phone facial images with ground truth date of birth evidence from individuals representative of many countries across the world.

Yoti is fortunate to have a very large set of consented and anonymised facial images, verified to government issued age data, from Yoti app users. By separating out ~120,000 of these images as diverse test data across each year of age, from the many millions of images used to train our algorithm, we have confidence in the accuracy figures we publish in our white paper (based primarily on mobile phone facial images at 720 x 800 pixels).

As part of our document authenticity in our identity verification service we compare the age estimation result of the selfie with the real age from their document, which also helps us test the accuracy of the model.

Finally Yoti’s facial age estimation model was first tested for accuracy, and positively certified, in November 2020 by ACCS, a UK accredited testing agency. Our age estimation model is used by some of the largest online brands, including Meta and OnlyFans, both of whom have publicly stated that it works very well.

Keep reading

A woman holding her mobile devise to her face to scan her biometrics.

How facial biometric verification protects your customers from identity fraud

Back in the day, identity theft mostly involved fraudsters stealing personal information, such as credit card details, to commit fraud or impersonate others in person. It was a pretty straightforward game of cat and mouse between fraudsters and a business, typically carried out face-to-face. As the world has become more digitised, identity fraud has evolved to meet this digital world. One powerful solution in our arsenal against identity fraud is biometric verification. Let’s dive into what it is, how it works, and why it’s crucial for keeping your customers and online community safe.   What is identity fraud? Identity

7 min read

Yoti assessed in the NIST Face Analysis Technology Evaluation program

An increasing amount of legislation is being introduced globally demanding that organisations effectively check the age of their users. It’s important that these age checks are inclusive; people should have a choice in how they prove their age. Regulators are recognising that not everyone will feel comfortable or be able to use a method based on identity documents. Facial age estimation gives people a way to prove their age without sharing their name, date of birth and other personal information from identity documents. It can improve online safety and help companies to comply with legislation, without having to process or

3 min read
Preview of first 3 pages of the Yoti Identity fraud report

Yoti Identity Fraud Report

We’re pleased to publish the first edition of our identity fraud report, which explores the fraud trends we’ve seen over the past 24 months. We delve into the tactics fraudsters are using and how these are evolving. We also explain why using technology and a team of verification experts together provides the best defence against fraud.   Key takeaways from the report: It’s challenging to understand the exact figures and the true scale of fraud; businesses can only report on the fraud they know about Fraud is evolving; fraudsters are exploiting different tactics including deepfakes and tampered with documents The

2 min read