The Explainable AI research field has a new engine: the European Data Protection Regulation. Because AI systems require transparency. A demand that is still not easy to fulfill today. Why? To answer this, take a look back at Google’s I/O Developer Conference in early May this year. The highlight: Google Duplex – an AI that independently arranges a hairdressing appointment on the phone. Spontaneous pauses in speaking, some interspersed “hmm’s” – and already the computer voice could not be distinguished from that of a human being. The reaction? Cheers to the Google experts in the audience. Otherwise? The reaction was rather mixed. The reason: Google Duplex just sounds too real.
Because: Is it really okay if a software calls me and I think it’s a human? “Clearly no,” says the European General Data Protection Regulation (DSGVO), which forces companies to be transparent in terms of artificial intelligence. As soon as automated decisions affect people, they must be understandable and explainable. Companies are obligated to disclose the AI aspects of their services, products, analyses, and processes.
AI decisions must be traceable
However, the demand for transparency is usually more difficult to meet. What exactly happens during machine learning is often hidden in a black box. Even the programmers are in the dark when it comes to answering the question of how the AI makes its decisions. Which is why, for example, Microsoft Research’s Kate Crawford calls for key public institutions in the areas of criminal justice, health, welfare, and education to stop using algorithms. Too many AI programs, according to the expert, have discriminatory tendencies or erroneous assumptions, it was discovered. Machines decide with high consistency, but also consistently inappropriately with unsuitable programming.
AI is relevant in more and more areas of life. Its importance will continue to grow. It can do many things: make medical diagnoses, buy or sell stocks for us, check our credit history, analyze whole business reports, or select job applicants. Software evaluates us according to certain mathematical criteria using so-called “scoring” methods. Therefore, the GDPR prescribes the “right of explanation” for the protection of every single person. This means: If an affected person submits an application, institutions or companies must be able to reasonably explain an AI decision or risk assessment.