An AI data leak occurs when sensitive or private data is exposed via an AI tool, whether through a prompt someone typed in, a platform security flaw, or information that unintentionally resurfaces in a model’s output. It’s no longer a hypothetical risk: Cyberhaven’s 2026 research found that 39.7% of all AI interactions now expose sensitive data, and major platforms, including ChatGPT, Claude, Meta AI, Grok, and DeepSeek, have each been tied to real-world data leaks in the past two years. Some of these trace back to platform-side vulnerabilities, like the GitHub Copilot flaw that exposed data from private repositories. Others come down to everyday habits, an employee pasting a client contract into a chatbot for a quick summary, or a team using an unapproved AI tool without IT’s knowledge. This guide walks through every major AI data leak by platform, explains the mechanics behind how they happen, and lays out concrete steps individuals and organizations can take to prevent the next one.
What Is an AI Data Leak?
An AI data leak is the exposure of private, confidential, or proprietary information through an AI system, either because a person entered sensitive data into an insecure AI tool or because the AI system itself exposed data it was not meant to reveal. It covers a wide range of scenarios: an employee pasting a client’s financial records into a chatbot, a coding assistant surfacing another user’s private repository, or a model reproducing fragments of the data it was trained on. What ties these together is that the data moves outside its intended boundary through an AI system, rather than through a traditional network breach.
How AI Data Leaks Differ from Traditional Data Breaches
A traditional data breach usually involves an attacker actively gaining access to a system, exploiting a vulnerability, stealing credentials, or breaching a database. An AI data leak often doesn’t require an attacker at all. The exposure can occur passively through normal use: a person voluntarily typing sensitive information into a prompt, a model blending that information into its future outputs, or an AI agent with excessive permissions acting on data it shouldn’t have access to. This makes AI leaks harder to police with conventional security tools, since firewalls and access controls are built to stop intrusions, not to govern what a person chooses to type into a text box. It’s also why AI leaks tend to trace back to everyday behavior rather than a single dramatic hack; the data usually leaves through the front door, not a broken window.
Common Causes: Training Data Exposure, Prompt Injection, Model Output Leakage, Misconfigured AI Agents
Most AI data leaks fall into one of four categories, each with a distinct mechanism and fix.
Training data exposure happens when a model is trained on data it shouldn’t have had access to, or when it later reproduces fragments of that data in its responses to other users. This is the underlying risk behind most “does this AI leak your data” questions: if proprietary code or personal information was used to train or fine-tune a model without proper safeguards, traces of it can theoretically resurface.
Prompt injection is a manipulation technique where hidden or malicious instructions are embedded in content an AI system processes- a webpage, a document, an email- causing the AI to take unintended actions or disclose data it had access to. It has become significant enough to rank as the top vulnerability on the OWASP Top 10 for LLM applications, and NIST has recorded a sharp rise in AI-specific vulnerabilities since 2022.
Model output leakage occurs when an AI system’s response contains information it shouldn’t disclose, repeats another user’s data, reveals internal system details, or generates outputs that inadvertently expose confidential context from earlier in a conversation or from connected data sources.
Misconfigured AI agents are an increasingly common cause as AI tools gain the ability to take actions, reading files, sending emails, querying databases, on a user’s behalf. When an agent is granted broader permissions than it needs, or connected to sensitive systems without proper access controls, it can pull in and expose data well beyond what the person intended, turning a productivity tool into an unmonitored data-exfiltration channel.
Major AI Data Leak Incidents, Timeline & Case Studies
AI data leak incidents have hit nearly every major platform since ChatGPT went mainstream in 2023, and the causes are strikingly varied: internal AI agents overstepping their permissions, unsecured databases sitting open to the internet, prompt-injection exploits, and plain old employees pasting things they shouldn’t. Below is a platform-by-platform look at the incidents that shaped how the industry now thinks about AI data risk.
ChatGPT and OpenAI Data Leaks (Including the Samsung Incident)
The most-cited AI data leak case remains Samsung’s 2023 ChatGPT incident. Within about three weeks of Samsung’s semiconductor division lifting its internal ban on the tool, three separate engineers pasted sensitive material into ChatGPT: proprietary source code for measuring semiconductor equipment, internal meeting transcripts, and test-sequence data used to identify defective chips. None of it was a hack; engineers were simply trying to debug code and summarize a meeting faster. But because ChatGPT’s free and consumer tiers retained submitted prompts for training at the time, Samsung grew concerned the data could be stored on OpenAI’s servers with no easy way to remove it, and banned generative AI tools company-wide within weeks. The episode became the template case for “shadow AI” risk, no attacker required, just a normal workday and a text box.
Claude and Anthropic Data Leaks
The defining Claude-related incident wasn’t a user data breach but a source code leak. On March 31, 2026, Anthropic accidentally published roughly 512,000 lines of unobfuscated TypeScript source code for Claude Code across about 1,900 files to the public npm registry after a debugging source map file was omitted from the package’s exclusion list. The leak stemmed from a reference in that map file to an unobfuscated TypeScript archive sitting in one of Anthropic’s own cloud storage buckets. Anthropic confirmed no customer data or credentials were exposed and called it a packaging error caused by human mistake, not a security breach. Still, the code was mirrored to GitHub within hours and forked tens of thousands of times before takedowns caught up. Separately, a 2026 vulnerability tracked as CVE-2026-21852 allowed a malicious repository to trigger API requests, and potentially leak a user’s API key, before Claude Code’s trust prompt ever appeared; it was patched in version 2.0.65.
Meta AI Agent Data Leaks
In mid-March 2026, an internal Meta AI agent triggered what the company classified as a Sev-1 security incident. An employee asked the agent to analyze an engineering query, and the agent posted its response directly to an internal forum without waiting for the requesting engineer’s review or approval. The response turned out to be wrong, and following it exposed a significant volume of sensitive internal and user-related data to engineers who had no authorization to see it, for roughly two hours before the exposure was contained. Meta said no user data left the company and that the incident was operational rather than adversarial. Still, security researchers flagged it as a clear example of agentic AI acting on privileged access without a human checkpoint, a risk category expected to grow as AI agents take on more autonomous tasks inside enterprises.
Grok / xAI Data Leaks
In August 2025, reporting revealed that over 370,000 Grok conversations had been indexed by Google, Bing, and other search engines, making them fully searchable by anyone. The exposure traced back to Grok’s “share” feature: clicking it generated a public URL intended to share a conversation with someone else, but that URL was also crawlable by search engines, with no warning to the user. Reviewers who examined the indexed chats found medical and psychological questions, business strategy documents, and at least one password among the exposed content. xAI wasn’t the first platform to run into this exact failure mode; OpenAI had previously discontinued a similar shareable-link feature in ChatGPT after the same indexing problem surfaced.
DeepSeek Data Leaks
In January 2025, security firm Wiz discovered a DeepSeek database sitting completely open on the internet with no authentication required. The exposure included over a million lines of log data containing chat history, secret keys, backend details, and other highly sensitive information, accessible to anyone who found the right URL and could run basic SQL queries through the database’s web interface. DeepSeek secured the database within roughly an hour of being notified, but researchers noted the flaw was simple enough that they likely weren’t the first to find it. The incident intensified regulatory scrutiny of the Chinese AI startup, prompting inquiries from data protection authorities in multiple countries.
GitHub Copilot Data Leak
A 2025 vulnerability nicknamed “CamoLeak” (CVE-2025-59145, CVSS 9.6) allowed attackers to silently exfiltrate secrets and source code from private GitHub repositories via GitHub Copilot Chat. The attack hid malicious instructions in a pull request’s invisible markdown comments; when a developer asked Copilot to review the PR, the assistant read the hidden instructions and searched the codebase for sensitive data, such as AWS keys. Copilot then encoded the stolen data into a chain of image requests routed through GitHub’s own trusted image proxy, exfiltrating the data one character at a time without the victim ever knowing. GitHub patched the flaw by turning off image rendering in Copilot Chat. Separately, researchers at Lasso found that more than 20,000 GitHub repositories that had been briefly public before being made private were still retrievable through Copilot via search engine caching, a reminder that “deleted” doesn’t always mean gone once an AI tool has indexed it.
AI Companion App Leaks (Character.AI, Janitor AI, AI Dungeon, Muah AI)
AI companion and roleplay platforms pose a distinct risk of leaks, since the content users share is often highly personal. The starkest example is Muah.AI, an “AI girlfriend” service breached in September 2024: a hacker exposed the email addresses and private chatbot interactions of 1.9 million users, including highly explicit content tied directly to real identities. Character.AI disclosed a narrower incident in which a misconfiguration in an update briefly made some users’ usernames, bios, and partial chat content visible to others for about 10 minutes. AI Dungeon has dealt with two distinct issues: a 2021 disclosure that a third-party content-moderation contractor had improperly shared unpublished user stories, and a later GraphQL API vulnerability that could expose unpublished adventures in bulk. Janitor AI has no confirmed platform-side breach on record. Still, its reliance on third-party API providers and proxy services for NSFW content means user conversations often pass through several parties, each of which is a potential point of exposure.
Enterprise AI Vendor Leaks (Scale AI, Mercor, McDonald’s AI Hiring Bot)
McDonald’s AI hiring assistant, Olivia, is the standout example of AI-adjacent exposure stemming from a basic security failure rather than any flaw in the AI itself. In 2025, researchers found the platform’s admin account still using the default credentials “123456,” which, combined with an insecure API, exposed names, email addresses, phone numbers, home addresses, shift preferences, and chat history for what could have been tens of millions of applicants dating back years. Mercor, an AI training-data startup whose customers include several major AI labs, confirmed a 2026 breach linked to a supply chain attack, with the LAPSUS$ group claiming to have exfiltrated roughly 4 terabytes of internal data, including video interviews and identity documents. Scale AI’s incident took a different shape: a trade-secret dispute in which a former employee allegedly carried confidential customer documents to rival Mercor, a reminder that not every “AI data leak” involves a hack; some are simply insiders walking out the door with data.
Can Popular AI Tools Leak Your Data? A Platform-by-Platform Breakdown
Whether an AI tool can leak your data usually comes down to two separate questions: does the platform use your conversations to train its models, and has it ever had a security flaw that exposed data to people who shouldn’t have seen it? The answer varies by tool, account tier, and year, as privacy policies in this industry have shifted substantially since 2023. Here’s where each major platform stands.
Does ChatGPT Leak Data?
Yes, under the wrong settings and plan, ChatGPT can both train on your conversations and has a documented leak history. On the Free and tiers, < cite index=”24-1″>OpenAI may use your conversations to improve its models unless you opt out, while Team, Enterprise, and API usage are excluded from training by default.</cite> Beyond training, ChatGPT has had at least two notable exposure incidents: a 2023 bug that briefly showed some users other people’s chat titles and payment details, and a July 2025 episode where < cite index=”22-1″>search engines indexed thousands of shared ChatGPT conversation links, exposing queries users hadn’t intended to make public.</cite> OpenAI patched both issues, but the pattern illustrates the core risk: even a well-resourced provider can expose conversation data through a feature nobody thought to lock down.
Does Claude Leak Data?
Claude can train on conversations from consumer accounts. Still, its business and API tiers are structurally excluded from training, and Claude’s one major public leak involved source code, not user data. Since August 2025, < cite index=”17-1″>Claude Free, Pro, and Max accounts have training on by default unless a user opts out in Privacy Settings, with data retained for five years when training is on versus 30 days when it’s off.</cite> Commercial products, Claude for Work, Enterprise, API, and Claude Gov, are excluded from this by default. Separately, Anthropic’s own March 2026 incident, in which roughly 512,000 lines of Claude Code’s source code were briefly published to npm, showed that even safety-focused providers aren’t immune to leaks; the leak surface for Claude specifically has centered on Anthropic’s own code rather than on user conversations.
Does DeepSeek Leak Data?
Yes, DeepSeek has one of the clearest confirmed leak incidents of any major AI platform. In January 2025, security researchers found a DeepSeek database completely exposed on the open internet with no password required, containing over a million chat history log lines, API keys, and backend operational details, accessible to anyone who found the right URL. DeepSeek closed the database within about an hour of being notified. Still, the incident, combined with the fact that DeepSeek is a Chinese company subject to different data-handling laws, has made it the platform most frequently flagged by security teams and banned outright by some governments and enterprises.
Does Copilot Leak Data?
GitHub Copilot can leak data through prompt-injection attacks embedded in the content it reads, rather than through its own training practices. The clearest example is the 2025 “CamoLeak” vulnerability, which let attackers hide instructions inside invisible pull-request comments; when a developer asked Copilot Chat to review the PR, it read the hidden instructions, searched the private repository for secrets like AWS keys, and quietly exfiltrated them through GitHub’s own image-proxy infrastructure. GitHub patched the underlying flaw, but the incident is a useful proof point for a broader truth about coding assistants: the risk often isn’t the AI company mishandling your data, it’s the AI reading untrusted content on your behalf and acting on hidden instructions inside it.
Does Perplexity Leak User Data?
Perplexity hasn’t had a headline database breach on the scale of DeepSeek’s, but its default data practices and looser feature security create real exposure to leaks. < cite index=”37-1″>On the Free, Pro, and Max plans, AI training is enabled by default, and users must manually opt out through the AI data retention setting;</cite> Enterprise plans are excluded and run the Sonar API with zero data retention. Security researchers have also flagged weaknesses in Perplexity’s Android app and its Comet browser, which processes activity across every site a user visits and can run background AI tasks without prompting for approval each time. This architecture widens the surface area for accidental exposure even without a formal breach. For anyone using Perplexity to research sensitive business or personal topics, the practical risk today is less “will Perplexity get hacked” and more “how much is being retained and trained on by default.”
How Employees Leak Data Through AI Tools at Work
Most AI data leaks inside a company don’t involve a hacker at all; they happen when an employee, trying to work faster, feeds sensitive information into an AI tool that was never vetted, secured, or approved for that purpose. This is the human layer of AI risk, and it’s the one most organizations underestimate, because it doesn’t look like an attack. It looks like a normal Tuesday.
Shadow AI Usage and Unapproved Tools
Shadow AI is the use of AI tools at work without IT’s knowledge or approval, and by 2026 it’s less an exception than the default state of most organizations. < cite index=”40-1″>Roughly 98% of organizations now have employees using unsanctioned AI tools,</cite> often because approved alternatives feel slower or more limited than the free consumer tools people already know. The danger isn’t the AI tool itself; it’s the visibility gap it creates. < cite index=”45-1″>One 2026 report found that while 78% of executives believed they had a clear picture of AI usage inside their company, the real figure from employee surveys was closer to 23%,</cite> a gap that leaves security teams unable to monitor, govern, or respond to the data actually leaving through these tools. When employees can’t get what they need from sanctioned software, they don’t stop using AI; they just move the risk somewhere leadership can’t see it.
Pasting Sensitive Data into Prompts
The single most common way company data ends up outside its intended boundary is an employee simply pasting it into a prompt. It’s rarely malicious; someone wants a contract summarized, a block of code debugged, or a meeting transcript cleaned up, and the fastest tool at hand is a public AI chatbot. This is exactly what happened in Samsung’s well-known 2023 incident, when engineers pasted proprietary semiconductor source code and internal meeting notes into ChatGPT to save time. The scale of the habit is significant: < cite index=”42-1″>industry survey data put employee use of generative AI at 57%, with 33% admitting they’ve exposed sensitive company data to these tools.</cite> Once that data is typed into a prompt, the organization has effectively lost control of it; there’s no reliable way to retract information after it’s been submitted to a system outside the company’s own infrastructure.
Third-Party AI Plugin and Integration Risks
A newer and less visible category of leak comes from AI plugins, browser extensions, and agentic integrations that connect AI tools directly to a company’s live systems, email, file storage, CRM records, and internal databases. Each connected integration expands what’s called the attack surface: a plugin with broad read access to a mail account or a shared drive can expose far more data than any single prompt ever could. Because these connections often run in the background, employees may not realize how much access they’ve granted. This risk is compounding as AI agents gain the ability to take autonomous actions rather than just respond to prompts. An unapproved plugin isn’t just a data-entry risk anymore; it’s a standing pipeline between company data and a system nobody in security has reviewed.
How to Prevent AI Data Leaks
Preventing an AI data leak comes down to controlling what goes into an AI system and how it’s handled once it’s there. The specific tactics differ for individuals, enterprises, and the AI labs that build the models themselves. Still, the underlying principle is the same: treat every prompt as data that could persist beyond your control.
Prevention Methods for Individual Users
The most reliable protection for individual users is a simple habit: never type anything into an AI tool that you wouldn’t be comfortable seeing stored on someone else’s server indefinitely. That covers financial details, medical information, passwords, unpublished work, and anyone else’s personal data, not just your own. Beyond that habit, a few concrete settings matter: turning off model training in your privacy settings where the option exists, using temporary or incognito chat modes for one-off sensitive queries, and reviewing which third-party plugins or browser extensions have been granted access to an AI assistant, since each connected integration is a separate place your data can end up. Multi-factor authentication on the AI account itself is also worth the extra step, since a compromised login can expose an entire chat history at once, not just a single conversation.
AI Data Leak Prevention for Large Organizations
For organizations, blocking AI tools outright rarely works; employees circumvent bans by switching to personal devices or accounts, which makes the resulting shadow AI usage harder to detect, not less likely to occur. < cite index=”41-1″>Analysts estimate that heavy shadow AI exposure can increase the cost impact of a breach by roughly 15%,</cite> largely from the legal review, incident response, and regulatory fallout that follows once ungoverned AI usage is discovered. The more effective approach is controlled enablement: provide employees with an approved AI tool that’s actually good enough to compete with the free consumer alternatives, publish a clear policy on what data can and can’t go into a prompt, and deploy technical controls, inline data-loss-prevention tools that detect and redact sensitive values like customer records or API keys before they ever leave the browser. Pairing an actual usable tool with real-time redaction closes the gap that blanket bans and policy documents alone can’t.
How Companies Secure AI Models Against Training Data Leaks
AI labs face a different version of this problem: preventing their models from memorizing and later reproducing sensitive fragments of their own training data. The primary technique is differential privacy, which adds carefully calibrated statistical noise during training so that no single record can be traced back or reproduced verbatim; deduplicating training data also helps, since duplicated content is disproportionately likely to be memorized. Labs also run systematic red-teaming, deliberately probing a model with adversarial prompts designed to extract memorized text, and layer output filters on top that catch and block responses containing patterns like PII, API keys, or code fragments before they ever reach a user. None of these methods make training data extraction impossible. Still, combined, they make it significantly harder and rarer, which is why most confirmed AI data leaks over the past few years have traced back to infrastructure mistakes, an exposed database, or a misconfigured agent, rather than a model spontaneously reciting someone else’s private data.
Using Gemini and Claude Safely at Work
Both Gemini and Claude draw a sharp line between their consumer and business tiers, and using them safely at work starts with knowing which side of that line your account sits on. On Gemini, < cite index=”47-1″>Google Workspace accounts automatically inherit the organization’s existing data-loss-prevention and access controls, and content is not used to train models outside that domain without explicit permission,</cite>. In contrast, the free consumer version of Gemini does not carry those same guarantees. Claude works similarly in reverse: consumer Free, Pro, and Max accounts have model training turned on by default unless a user manually opts out, while Claude for Work, Enterprise, and API access are excluded from training under the commercial terms. The practical rule for using either tool safely at work is the same: confirm that your organization is actually provisioned on the business tier, rather than employees signing up individually for personal accounts, since that single distinction determines whether a conversation stays within the company’s data boundary or potentially becomes training material for a public model.
Monitoring and Detecting Leaked AI Data on the Dark Web
Once data has been exposed through an AI leak, whether from a misconfigured database, a prompt-injection exploit, or an employee pasting the wrong thing into a chatbot, it doesn’t stay contained. It typically ends up circulating on dark web forums and marketplaces, often long before the organization or individual it belongs to ever finds out. Monitoring that layer of the internet is what turns an unknown exposure into an actionable one.
How Dark Web Monitoring Catches AI-Related Data Exposure
Dark web monitoring works by continuously scanning underground forums, marketplaces, Telegram channels, and stealer-log dumps for specific identifiers, email addresses, domains, credentials, API keys, and source code fingerprints tied to a person or organization, then issuing an alert the moment a match surfaces. This has become directly relevant to AI-specific leaks: < cite index=”67-1″>IBM’s 2026 X-Force Threat Index found that more than 300,000 ChatGPT credentials surfaced in infostealer dark-web listings during 2025 alone,</cite> giving attackers a direct path into employees’ saved AI conversations, connected plugins, and chat history. Because AI accounts often hold months of accumulated prompts, internal documents, credentials, strategic plans, a single leaked login can expose far more than a typical compromised account. Monitoring tools built for this purpose catch exposure early by monitoring an organization’s domains and executive email addresses across infostealer logs and breach compilations, rather than waiting for a formal breach disclosure.
What to Do If Your Data Was Exposed in an AI Leak
If you find out your data was part of an AI-related leak, the first move is to change the password on the exposed account and any other account reusing that password, since credential reuse is exactly what lets a single leak cascade into several account takeovers. Next, revoke active sessions and API keys associated with the exposed account, enable multi-factor authentication if it wasn’t already enabled, and check whether the leaked data included anything that can’t simply be changed, such as a document, a Social Security number, or proprietary code. In those cases, monitoring for further misuse matters more than any single reset. Running your email or domain through a free exposure check is a fast way to see whether a specific leak reached you, and setting up ongoing monitoring is what catches the next one before it becomes a bigger problem. Most people only learn their data was exposed months after it started circulating. That gap is where the real damage tends to happen.
Don’t wait to find out your data was part of an AI leak.
DeXpose continuously monitors dark web forums, marketplaces, and stealer-log dumps for the credentials, domains, and data tied to your organization, so if an AI account, API key, or exposed database puts you at risk, you find out before attackers act on it, not months later.
Run a free Dark Web Report to see your exposure right now, or explore Dark Web Monitoring for continuous, org-wide visibility.
Frequently Asked Questions (FAQ’s)
Can AI tools leak your personal data?
Yes, AI tools can leak personal data either through a platform-side security flaw, like an exposed database or a prompt-injection exploit, or through normal use, such as a model being trained on conversations that later resurface in someone else’s chat.
Is it safe to enter sensitive information into ChatGPT or Claude?
It depends on the account tier: consumer plans on both platforms may use conversations for training unless you opt out. Business and API tiers are excluded from training by default, so sensitive data should only go into a business-tier account with training turned off.
How do I know if my data was exposed in an AI data leak?
Run your email or domain through a dark web exposure scan, since leaked AI account credentials and associated data typically surface in breach databases and stealer-log collections well before an official disclosure
