AI chatbots are becoming trusted confidants for millions, yet the personal information shared with them is rarely as private as users assume. A survey titled The AI Privacy Problem found that roughly a third of people share secrets with chatbots that they would not disclose to friends, family or medical professionals. Among those who describe themselves as AI enthusiasts, the figure rises to 56%.
The research also found that 43% of parents reveal things to AI systems that they do not tell the most trusted people in their lives, including their own children. While the survey did not specify precisely what kind of personal details are shared, earlier reports have shown that people frequently turn to chatbots for support with physical and mental health issues, financial guidance, parenting questions and marital problems.
The central concern is not the act of seeking help, but the loss of control over what happens to the information once it has been shared. Companies may use private details to train their AI systems, law enforcement can obtain data through subpoenas, and employees using chatbots at work may find that their employer can access their chat history.
Privacy risks of large language models
Many people are already familiar with targeted online advertising, where a search for a product results in related adverts appearing across other platforms. Companies also buy, sell and share social media data to guide their advertising. AI data-sharing, however, extends well beyond selling products.
Sensitive information provided to a chatbot can be revealed or exposed. A researcher at Wake Forest University found that AI agents, which use large language model data to perform autonomous tasks, can sometimes leak data either maliciously or unintentionally. Conversations with chatbots do not simply vanish. Unless a user opts out, the system will record and save them for future use, including additional model training. Hundreds of chats on Anthropic’s chatbot Claude previously appeared in Google search results.
A significant issue with these models is data memorisation. Uttara Ananthakrishnan, assistant professor at the University of Washington’s Foster School of Business, noted that when one person asks the same question as another, the model may return exactly the same answer. The system might not recognise that the answer contains information that could be used to identify someone else.
Who can access your chat data
The survey revealed that many respondents did not understand how their chatbot data could be used or accessed, and were unaware of the settings available to prevent their information from being recorded or stored. Trust in the institutions responsible for safeguarding this data was notably low.
When it comes to protecting their information, nearly 40% of respondents said they had no trust in companies, while nearly 50% said they had no trust in the US government. What is said in a supposedly private chat can be surfaced through law enforcement requests, corporate training practices or technical flaws, making AI privacy a growing consideration for everyday users.
