When AI starts to see things, it’s time to take immediate corrective action. Here’s how to successfully address this challenging — and potentially dangerous — phenomenon.
AI hallucination occurs when a large language model (LLM) — frequently a generative AI chatbot or computer vision tool — perceives patterns or objects that are nonexistent or imperceptible to human observers, generating outputs that are either inaccurate or nonsensical.
AI hallucinations can pose a significant challenge, particularly in high-stakes fields where accuracy is crucial, such as the energy industry, life sciences and healthcare, technology, finance, and legal sectors, says Beena Ammanath, head of technology trust and ethics at business advisory firm Deloitte. With generative AI’s emergence, the importance of validating outputs has become even more critical for risk mitigation and governance, she states in an email interview. “While AI systems are becoming more advanced, hallucinations can undermine trust and, therefore, limit the widespread adoption of AI technologies.”
Primary Causes
AI hallucinations are primarily caused by the nature of generative AI and LLMs, which rely on vast amounts of data to generate predictions, Ammanath says. “When the AI model lacks sufficient context, it may attempt to fill in the gaps by creating plausible sounding, but incorrect, information.” This can occur due to incomplete training data, bias in the training data, or ambiguous prompts, she notes.
LLMs are generally trained for specific tasks, such as predicting the next word in a sequence, observes Swati Rallapalli, a senior machine learning research scientist in the AI division of the Carnegie Mellon University Software Engineering Institute. “These models are trained on terabytes of data from the Internet, which may include uncurated information,” she explains in an online interview. “When generating text, the models produce outputs based on the probabilities learned during training, so outputs can be unpredictable and misrepresent facts.”
Detection Approaches
Depending on the specific application, hallucination metrics tools, such as AlignScore, can be trained to capture any similarity between two text inputs. Yet automated metrics don’t always work effectively. “Using multiple metrics together, such as AlignScore, with metrics like BERTScore, may improve the detection,” Rallapalli says.
Another established way to minimize hallucinations is by using retrieval augmented generation (RAG), in which the model references the text from established databases relevant to the output. “There’s also research in the area of fine-tuning models on curated datasets for factual correctness,” Rallapalli says.
Explore IT Tech News for the latest advancements in Information Technology & insightful updates from industry experts!
Source: https://www.informationweek.com/machine-learning-ai/getting-a-handle-on-ai-hallucinations
