Photo: Supplied / Google Maps
It is not surprising that a supermarket trialling facial recognition technology mistakenly identified a Māori woman as a thief, Māori AI and data experts say.
The woman was shopping at New World in Rotorua - one of 25 Foodstuffs' North Island supermarkets trialling the technology - when she was approached by two staff members and asked to leave the store.
The technology scans customers' faces and compares images to those on the store's databases of known offenders or suspects.
Māori AI and data ethicist Karaitiana Taiuru was not surprised to see Māori singled out in the trial and believed there were other Māori who had been affected but have not yet come forward.
*If you are Māori and have been affected by the facial recognition trial please contact: Temanukorihi@rnz.co.nz
The systems that Foodstuffs was using were trained on an international dataset of people and not on people in a New Zealand context, he said.
Māori AI and data ethicist Karaitiana Taiuru. Photo: Supplied
"We know from international data and international research that the systems are based on European-looking men, so it's only in the last few years that the system has been modified to consider people of colour, women, men with beards etc.
"Anyone who is Māori, Pasifika, any person of colour is not going to be recognised by the system. That's why it's important that we have human double checking the system, which in this case failed."
The woman was trespassed from the Rotorua supermarket despite offering three forms of photo identification to staff.
Taiuru said humans manually checking positive matches from facial recognition technologies often still relied on AI and not on their own common sense.
"It's a phenomenon that's identified in America where the staff rely on the AI more than their own personal judgement. So regardless of how obvious a mismatch is, humans still rely on the AI to be correct," he said.
This bias could be reduced with appropriate staff training - which will have to be customised to the New Zealand population, he said.
Taiuru said as far as he was aware, there was no out of the box training available in New Zealand yet.
If you were misidentified by facial recognition technology, it was best to complain to the organisation using it or the Privacy Commissioner, he said.
Figure NZ chief executive Ngapera Riley was from Rotorua herself and said it was outrageous to see Māori being profiled.
It was a clear cut example of why facial recognition technology may not be reliable at this stage, Riley said.
"I'm not surprised this happened in Rotorua with the supermarket trials but these supermarkets, they've got a lot of money and access to resources so they really should be doing a better job at preventing these situations from happening," she said.
It was a bad look when something like this happened in a high Māori population area like Rotorua, she said.
There was both beneficial and concerning elements to biometrics and Riley said she understood why supermarkets and retailers were interested in the technology.
"The reason that supermarkets and retailers are implementing this technology is because they've got a huge increase in crime and their staff getting assaulted while doing their jobs ... so they're wondering can these new technologies help them with that."
The office of the Privacy Commissioner was asking the public to have its say on a draft biometric code of practice.
New Zealand did not currently have special rules for biometrics.
Riley said biometric data included fingerprints, facial, voice, irises, palms and hand technology.
"It can also include things like keystroke patterns and the way someone walks."
The fact the Privacy Commissioner was creating a code of practice meant the issue was being taken seriously, she said.
