Snapchat's 'My AI' Chatbot Under Scrutiny in UK Over Privacy Concerns for Teens

The UK's Information Commissioner's Office (ICO) has hit Snap, the parent company of Snapchat, with a preliminary enforcement notice over privacy concerns associated with its AI chatbot, "My AI."

While this action does not constitute a finding of a breach, it suggests that the regulator is apprehensive about Snap's approach to ensuring compliance with data protection regulations, including the Children's Design Code implemented in 2021.

Preliminary Enforcement Notice Against Snap

The ICO's preliminary enforcement notice outlines potential measures that may be imposed on Snap, pending the company's response.

If a final enforcement notice is issued, Snap could face a fine of millions of pounds and might be instructed to halt data processing linked to the My AI feature. This would entail discontinuing the My AI service for UK users until Snap conducts a comprehensive risk assessment.

Snap introduced the My AI feature to UK subscribers of Snapchat+ last February, followed by a wider rollout to all Snapchat users in the UK in April.

Powered by OpenAI's GPT technology, My AI represented a significant milestone as the first instance of generative AI integration into a major messaging platform in the UK.

The ICO's preliminary investigation revealed that Snap's initial risk assessment conducted before the launch of My AI did not sufficiently evaluate the data protection risks inherent to generative AI technology, particularly concerning minors.

"The assessment of data protection risk is particularly important in this context which involves the use of innovative technology and the processing of personal data of 13 to 17 year old children," the ICO said in a statement.

It is important to note that the Commissioner's findings at this stage are provisional, and no definitive conclusions regarding a breach of data protection laws or the issuance of an enforcement notice have been made. The ICO said it is committed to thoroughly reviewing any representations put forth by Snap before arriving at a final decision.

"The provisional findings of our investigation suggest a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching 'My AI'. We have been clear that organisations must consider the risks associated with AI, alongside the benefits. Today's preliminary enforcement notice shows we will take action in order to protect UK consumers' privacy rights," John Edwards, UK Information Commissioner, said in a statement.

Ads Coming Soon to Snapchat 'My AI'

In related news, Snapchat has collaborated with Microsoft to incorporate advertisements into its AI offering, My AI. This chatbot was unveiled earlier this year to a broader user base within Snapchat.

Last May, the company announced plans to pilot sponsored links in My AI. These are links suggested in tandem with the user's interaction with the AI assistant. Snap and Microsoft will now collaborate on these link suggestions.

Microsoft disclosed this partnership during its recent advertiser event. By leveraging Microsoft Advertising's Ads for Chat API, Snap's sponsored links will connect users with pertinent partners related to their conversation.