Last week, the European Commission (EC) released a white paper that seeks to ensure societal safeguards for “high-risk” artificial intelligence (AI). The number of large-scale and highly influential AI models is increasing in both the public and private sector, and so the EC is seriously considering where new regulations, legislative adjustments, or better oversight capacity might be necessary.
These models affect millions of people through critical decisions related to credit approval, insurance claims, health interventions, pre-trial release, hiring, firing, and much more. While facial recognition, autonomous weapons, and artificial general intelligence tend to dominate the conversation, the debate on regulating more commonplace applications is equally important.
The new white paper echoes the principles of the earlier AI Ethics Guidelines: non-discrimination, transparency, accountability, privacy, robustness, environmental well-being, and human oversight. This new paper goes beyond many prior AI ethics frameworks to offer specific AI regulatory options. Some of these options would be alterations to existing EU law, such as ensuring product liability law can be applied to AI software and AI-driven services.
More noteworthy, however, is the proposal to consider entirely new requirements on high-risk AI applications. The high-risk categorization is limited to specific use-cases within specific sectors where there are particularly large stakes. The report explicitly names sectors such as transportation, healthcare, energy, employment, and remote biometric identification, but others like financial services could be included.
Within these sectors, only especially impactful AI applications would receive the label “high-risk” and accompanying oversight. So, while a healthcare allocation algorithm may be included, a hospital’s AI-enabled scheduling software would probably not qualify.
The report details a series of possible oversight mechanisms for applications deemed high-risk AI. Some of these would set standards for the use of AI, such as using representational training data and meeting defined levels of model accuracy and robustness. Others require storage of data and documentation, potentially enabling government auditing of AI models. Transparency measures are also under consideration.
These might require reporting to regulatory authorities (e.g. an analysis of bias for protected classes) or directly to consumers affected by the model (e.g. an individualized explanation for their model outcome). Not all these requirements would apply to all high-risk AI, but instead some subset of these mechanisms would be paired with each high-risk application.
In weighing how these mechanisms might work, it’s valuable to contemplate how various interventions might affect prominent instances of AI harms. For instance, would enabling audits slow the proliferation of pseudoscientific hiring software across human resources departments? Would reporting requirements help identify discriminatory patient treatment in healthcare allocation algorithms?
Would a more rigorous testing process of Tesla’s autonomous driving have made them more resistant to the stickers that trick the vehicles into driving at dangerous speeds? These are questions that the EC paper is raising—questions that the U.S. policy-makers should be asking, too. Given a type of algorithm being used for a particular high-risk purpose, what oversight mechanisms might ensure that it functions in a legal and ethical way?
While the EC paper is exploring new requirements, it also makes clear that enforcing extant law is difficult due to the complexity and opacity of AI. It takes specific expertise in programming and statistics to evaluate the fairness and robustness of AI models, which regulatory agencies across the EU may not yet have.
This is very likely an issue in the United States, too. AI models can easily run afoul of many federal requirements, such as the Civil Rights Acts, the Americans with Disabilities Act, the Fair Credit Reporting Act, the Fair Housing Act, and financial modeling regulations. It is not clear that U.S. regulatory agencies are staffed to handle this emerging challenge.
The EC paper notes that investing in its ability to enforce AI safeguards has real advantages for industry, too. The European approach argues that responsible regulation will build public trust in AI, allowing companies to build automated systems without losing the confidence of their customers. Broadly speaking, the EC’s perspective is positive about the emergence of AI as a general-purpose technology.
It presents AI as a powerful tool to improve scientific research, drive economic growth, and make public services more efficient. They seek to attract €20 billion ($21.7 billion USD) in annual funding for AI, some of which would come from expanding EU spending. This effort would also be bolstered by an ambitious strategy to incentivize data sharing and expand access to cloud infrastructure.
To view the full blog, click here.