Assessing AI Risks
Public sector organizations face distinct risks when adopting AI due to their responsibilities to the public, the intake and management of large amounts of confidential information, and the essential services the organizations deliver. The primary risks public entities must consider include:- The potential for improper access or misuse of sensitive data, such as tax returns, legal records and health data, is a significant factor for public sector organizations, their vendors and contractors. In addition, public sector entities are typically held to higher transparency standards compared to their private sector peers.
- Data errors or incorrect decisions made based on AI have higher stakes for many public sector organizations. For services like law enforcement, social services or health care, AI-related mistakes can impact human lives, public safety and privacy rights.
- Data hallucinations produced through AI raise potential equity and fairness concerns for public entities. AI hallucinations occur when the system generates false or misleading information. Current reports put OpenAI’s (ChatGPT) hallucination rate at 3%, Meta at 5% and Anthropic at 8%.
- Bias can also be a factor with AI based on the training inputs used, model architecture and user interaction with output. As a result, public sector organizations must conduct extensive due diligence when selecting and implementing AI tools. For example, in 2023, the IRS was revealed to be conducting audits of Black taxpayers at three times the rate of other taxpayers based on trained bias within an AI system.