Where Does Your Data Go? The Truth About AI Security and Data Privacy

The first two episodes of this series kept circling the same anxiety without ever landing on it: what actually happens to the information you hand an AI tool. So Rob Woodhead puts it to Rob Lynch and Ben Burch directly. Where does the data go, who can see it, and how much of the worry is justified? The answer is less dramatic than the headlines and more awkward than most company policies allow for. The nightmare scenario people describe, where a password you typed into a chatbot resurfaces in someone else’s conversation a year later, is not really how any of this works. The thing that should concern a business is duller and much more common: someone pasting a client document into a personal account, at speed, because the approved tool is slower.

Key takeaways

• Personal and enterprise accounts are two different conversations. Company configurations default towards privacy and control; personal accounts default towards sharing, and most people never check what they have ticked.
• The fear that a model will one day recite your password back to a stranger misreads how training works, and there are no real cases of it happening.
• The realistic exposure in a business is legal rather than technical: data you promised not to share has left the building, whether or not anything is ever trained on it.
• Judge what to share by how well you know the setup you are talking to, exactly as you would with a person who has just walked into the lobby.
• Shadow AI is a symptom of friction. If the sanctioned tool is worse than the free one, people will route around it, and they will be right to.
• Agents do not invent new risk so much as magnify existing sloppiness about permissions. Admin rights handed over because the configuration was fiddly is the failure mode to watch.
• Nobody is an expert in all of this, so a shared record of what people are actually doing beats a policy document that nobody reads.

Episode chapters

• 00:15 Why data and security kept coming up in the first two episodes
• 01:26 Personal accounts and enterprise accounts are different conversations
• 03:43 The password horror story, and why it does not happen
• 06:18 Treating AI like a person who has just walked in
• 09:16 What "training on your data" actually means
• 09:47 Publishers, lawsuits and the scraped corpus
• 11:11 What happens to your chats on a free account
• 13:46 Comprehension versus content: what a model takes from a document
• 16:14 A legal risk dressed up as a technical one
• 17:32 Defining shadow AI
• 18:24 Password policies and the post-it note problem
• 20:36 Agents, permissions and guardrails
• 22:00 Why an agent ends up with admin rights
• 24:08 Prompt injection and non-technical users
• 25:17 Removing friction so people do not work around you
• 26:08 Keeping a use case register
• 28:07 Make the sanctioned tool better than the free one
• 32:20 Is it as bad as everyone thinks?

Personal accounts and enterprise accounts are two different conversations

Where your data goes depends most on which account you are logged into.

Ben splits it into two brackets. Enterprise and company accounts come with far more control, and those controls tend to default to a privacy-first position. Upload a document in that environment and it may never reach the model provider at all, though your own company will usually know you uploaded it. Personal accounts are the other bracket, and they default the other way. The settings exist, but if you have never opened them, your chats are on the more open side of things.

That does not mean a document you paste into a personal account is suddenly public. There are still controls: moderation, review, retention rules. Ben’s point is narrower and more useful. If you have not checked what you ticked, the behaviour is probably not what you assume, and the gap between assumption and configuration is where most of the trouble starts.

If you are working out what your own setup should look like, that is the sort of question we tend to start with. You can read about how we work or see the products we have built.

The password horror story that does not happen

Rob Lynch is fairly firm that this is not the risk people think it is.

The scenario everyone reaches for is the same one: you paste an API key or a set of credentials into a chatbot, the next model trains on it, and a year later somebody asks the right question and gets your password back. That is not how training behaves. The process extracts semantic patterns across an enormous body of text. A single unusual string sitting in one conversation is not the kind of thing that survives it. Rob searched for a real example before recording and could not find one worth worrying about.

Ben adds the commercial argument, which is arguably the stronger one. Model providers are spending heavily on being trusted. Harvesting credentials runs directly against the business they are trying to build, so if anything of the sort ever happens it will be incidental rather than designed. Some tools now actively warn you: paste something into Lovable that merely looks like a key and it will tell you to stop. As Rob puts it, this is not about anyone being nice. It is about incentives.

None of which means nothing is at stake. It means the thing at stake is not the thing in the headline.

"Training on your data" describes two different processes

The phrase covers two quite separate things, which is why it causes so much confusion.

The first is the argument that publishers have been having in public. Frontier models were trained by scraping vast quantities of text, and the owners of that text, the New York Times and Reddit among them, took the view that their most valuable asset had been monetised without their consent. Reddit has since reached an agreement. That fight is about published content, not about your chats.

The second is your own data, and here the mechanics are simpler than the anxiety. On a free or personal account with the wrong toggle left on, your conversations can be added to the training set. Every question, every paragraph. Rob’s view on whether a provider would quietly train on data they have contractually promised not to touch is deflationary: it would be litigation gold, and the incentive structure makes it a poor bet.

Ben’s distinction is the one most people have not heard. A large part of document training is about comprehension rather than content. The model is learning to read a document the way a human does, working out what a heading is and how a table relates to the paragraph above it. Open a Word file in Notepad and you will see roughly what the model sees. That is not a promise that the content is ignored, and Ben is careful not to make one. There are still documents he would not upload. But the single phrase everyone uses covers two different things, and only one of them is about what your document says.

Shadow AI is a friction problem, not a discipline problem

Shadow AI is people using their own AI tools, on their own accounts, instead of whatever their employer has provided. The surveys Rob has seen suggest a majority do it, in full knowledge of the policy telling them not to.

His framing is that this is a legal fear dressed up as a technical one. The technical risk is real but modest: data sits on a provider’s servers longer than it should, there is a breach, the wrong people see it. The legal risk is immediate and ordinary. You committed, probably in writing, not to share this material with third parties. Somebody just did.

Ben’s explanation for the behaviour is a password policy. Passwords started simple, then policies arrived demanding complexity and rotation, and people responded by writing the complicated password on a sticky note. IT had solved its problem and created a worse one, because the humans could not live with the solution. Shadow AI is the same shape. A company writes a policy that does not fit how the work actually gets done, and people go around it. The next iteration of this will be something else, but the pattern will hold.

Agents magnify permission mistakes rather than creating new ones

An agent is not inherently dangerous. An agent with unnecessary access is.

Rob Lynch’s test is what you have allowed it to do. An agent sending automated emails is usually fine. An agent sending automated emails that can also reach your CRM, your accounting system or personal data needs a different level of care, because the obvious failure is genuinely possible: it emails your entire contact list something from your client records. That only happens if the system was not architected properly, which is precisely why the architecture is the point.

Ben’s account of how it goes wrong will be familiar to anyone who has configured permissions under time pressure. The agent should have read-only access here and permission to send one email there. Setting that up properly is fiddly, the interface is unhelpful, and eventually somebody gives it admin rights and tells it to only send the one email. It is the new junior who is excellent and entirely trustworthy and has just been handed the master password. The complexity produced the shortcut, and the shortcut is where the risk lives.

There is a related gap that Rob Woodhead raises from his own use: non-technical people are now granting technical permissions. Installing a browser extension for an AI assistant means approving a substantial set of permissions, and prompt injection, where malicious instructions are hidden in content the agent reads, is not something most people know to think about.

If something has been built quickly and nobody is quite sure what it can reach, that is what our Vibe Code Audit is for.

Removing the reason to work around the sanctioned tool

Three things came up, and none of them is a stricter policy.

Make the sanctioned tool at least as good as the free one. This is Rob Lynch’s answer for larger companies and the most direct of the three. Whatever the interface is, if the models underneath it are genuinely at the frontier, the reason to paste company data into a personal account disappears. People circumvent when there is a real incentive to circumvent. Remove the incentive.

Reduce friction, and explain the friction that has to stay. Ben’s point is that if a restriction exists, people need to understand why, and they need help doing the job within it. Put too much friction in the way and you get one of two outcomes: nobody adopts the tool, or everybody finds the workaround.

Keep a use case register. This is the idea Ben is most enthusiastic about. The same task can now be done in a chat window, in a coworking-style AI tool, or on the command line, with very different results. So keep a shared record of what people in the business are doing and how they are doing it. It spreads knowledge, it surfaces risks that nobody had thought to mention, and it works precisely because nobody is an expert in the whole thing. We watched this happen with prompting, where people shared what worked and everyone got better at it faster than they would have alone. The tools are moving quickly enough that staying current in a silo is a full-time job for one model, let alone all of them.

Rob Woodhead’s addition is that the conversation itself has value. A contact of his ran an internal hackathon and now holds regular sessions where the team compares what they are doing with AI, with several of those approaches ending up adopted company-wide.

‍

Frequently asked questions

Is it safe to upload documents to ChatGPT or Claude?

It depends on the account. Enterprise and company configurations come with more controls and default towards privacy, so a document may never reach the model provider. Personal accounts default towards sharing unless you have changed the settings. Ben Burch’s rule of thumb is to share in proportion to how well you know the setup you are talking to.

Does ChatGPT train on my conversations?

On a free or personal account with the default settings, it can. Rob Lynch’s view is that providers are unlikely to train on data they have contractually committed not to use, because the commercial and legal downside of being caught would be severe.

What is shadow AI?

Shadow AI is employees using their own AI accounts and tools rather than the ones their employer provides. It is common even where policy forbids it, usually because the unapproved tool gets the work done faster.

Are AI agents riskier than chatbots?

They can be, depending on what they are allowed to reach. An agent restricted to one task is low risk. An agent given broad admin access because configuring narrow permissions was too fiddly is where problems appear, and that is a configuration decision rather than a property of the technology.

What is prompt injection?

Prompt injection is when a malicious instruction is hidden in content an AI agent reads, causing it to do something other than what the user intended, such as exposing information it has access to. It matters more as agents are given browser access and permissions over real systems.

How can a company stop staff using unapproved AI tools?

Mostly by removing the reason to. Give people a sanctioned tool with frontier models behind it, explain any restrictions that have to stay, and keep a shared register of the use cases people are finding so the knowledge circulates instead of being reinvented in private.

‍