Ensure people can challenge AI decisions, EU body says

The report - titled 'Getting the future right – Artificial intelligence and fundamental rights in the EU' - is based on more than 100 interviews with organisations which use AI, along with public officials, NGOs and legal experts. It focuses on four core areas: predictive policing, social benefits, medical diagnostics and targeted advertising.

The report says that new AI legislation must respect all fundamental rights, with the inclusion of safeguards to prevent the violation of rights. It calls for mechanisms to guarantee that individuals can challenge decisions taken autonomously by AI and says that companies must be able to explain how their AI tools make decisions.

The agency also called for further funding for research into the impact of AI tools on discrimination, such that the private and public organisations can mitigate these risks before harm is caused. A number of studies and scandals have shown that AI tools can reinforce bias: for instance, MIT researchers have demonstrated that many commercial facial-recognition systems perform poorly when analysing the faces of women and people with dark skin tones.

“AI is not infallible; it is made by people and humans can make mistakes,” said Michael O’Flaherty, FRA director. “That is why people need to be aware when AI is used, how it works and how to challenge automated decisions.

“The EU needs to clarify how existing rules apply to AI and organisations need to assess how their technologies can interfere with people’s rights both in the development and use of AI. We have an opportunity to shape AI that not only respects our human and fundamental rights, but that also protects and promotes them.”

The body urged European policymakers to provide further guidance on how existing rules – such as those governing use of personal data – apply to AI and to ensure that future legislation protects fundamental rights.

The agency’s report has been published just as the European Commission contemplates new legislation to regulate AI, with consideration to ethics, innovation, IP rights and liability. In October, the European Parliament voted strongly to endorse guiding principles for AI which emphasise social responsibility, privacy and transparency.