AI Hallucinations, and Data Privacy and Business Risks

AI Hallucinations are becoming more and more of a concern as the use of AI LLMs (Large Language Models) by individuals and adoption by businesses grow. This term refers to instances when an AI model generates incorrect or misleading information based on its training data, rather than real-world evidence. While the creators of LLMs actively combat this issue of generated inaccuracies and have made some progress, the problem persists and poses serious data privacy and business risks.

AI Hallucinations can occur because AI models, including LLMs, learn to generate responses based on patterns and correlations in the data they were trained on, without an understanding of real-world truths. This makes it possible for AI to generate plausible but false or nonsensical information, especially when asked about topics that are not well-represented or are ambiguous in the training data.

(Screenshot of an older example that has since been corrected by OpenAI.)

Some of the most common issues present themselves as incorrect predictions, false positives, or false negatives.

For example:

• Incorrect predictions: An AI model may predict that an event will occur when it is unlikely to happen. For example, an AI model that is used to predict the weather may predict that it will rain tomorrow when there is no rain in the forecast.
• False positives: When working with an AI model, it may identify something as being a threat when it is not. For example, an AI model that is used to detect fraud may flag a transaction as fraudulent when it is not.
• False negatives: An AI model may fail to identify something as being a threat when it is. For example, an AI model that is used to detect cancer may fail to identify a cancerous tumor.

This ability to confidently state falsehoods poses both business and privacy risks. It is also in these situations that LLMs can divulge information that it should not.  Either actual sensitive information, or answer questions that it should not, such as giving out personal information.

To businesses relying on the veracity of the answers kicked out by LLMs, bad answers are a dangerous problem. Companies rely on computers to solve problems because they are reliable, and their processes are repeatable. AI hallucinations through a wrench in this and make the business users second guess the answers produced.

From a Data Privacy perspective, these hallucinations pose risks in four main areas and can result in the generation and dissemination of inaccurate or sensitive information:

• Misrepresentation: AI may inadvertently create and spread false or misleading information about individuals or entities, which can harm reputations or lead to incorrect decisions being made based on inaccurate data.
• Data Leakage: In scenarios where AI generates responses based on training with extensive datasets, there’s a risk of revealing personal or sensitive data embedded in the training material, even if indirectly.
• Compliance Violations: AI hallucinations can lead to situations where generated data does not comply with legal standards for data privacy and protection, potentially leading to regulatory penalties.
• Trust Erosion: Repeated instances of incorrect or misleading data generation can undermine trust in AI systems and the organizations that deploy them, impacting user engagement and adoption.

Addressing these risks involves careful design of AI systems, including robust data governance, transparency, and continuous monitoring to ensure compliance with data privacy regulations.

The issues of AI Hallucinations have come to the forefront due to recent events such as Max Schrems’ lawsuit against OpenAI, the use of made-up case laws by lawyers and an embarrassing submission to the Australian parliament.

Max Schrems, an Austrian activist, lawyer, and author, is a key privacy advocate globally.  He recently opened a suit against ChatGPT and its parent OpenAI because ChatGPT produced incorrect information and divulged potentially personal information. He asked it a simple question, his birthday.  Instead of refusing to answer the question or state that it did not know, ChatGPT spit out a series of wildly incorrect data. ChatGPT’s “hallucinating” and making up of information breaches European Union privacy rules.

There have also been numerous examples of lawyers using ChatGPT. By using AI to generate information without double checking lawyers have ended up using bogus case-laws, non-existent judicial opinions with fake quotes and citations. Another damning example is a group of Australian academics who used AI to help with a complaint submission to the Australian Parliament against four of Australia’s largest banks. The AI they used made up scandals and incorrectly implicated Deloitte.

With all things AI related, education is key.  Understanding the current limitations of AI and LLMs and being a subject matter expert on the topic in question is key. As individuals we must be educated consumers and users of AI. Companies put the proper privacy and accuracy checks in place. Governments must become better equipped and educated to effectively legislate this rapidly changing space.

Perhaps AI and LLM chatbots should be set up to not answer questions that could give out PI?  In other words, take privacy by design approach. This is a rapidly changing space. Only time will tell if this problem and its privacy consequences can be adequately managed.

Click here to learn more about our Preference and Consent Management Platform! Clarip takes enterprise privacy governance to the next level and helps organizations reduce risks, engage better, and gain customers’ trust! Contact us at www.clarip.com or call Clarip at 1-888-252-5653 for a demo.

Email Now:

Mike Mango, VP of Sales
mmango@clarip.com