Canada’s privacy watchdog found that social platform X (formerly known as Twitter) violated Canada’s federal private-sector privacy law when it launched an AI image generation tool that was used to create sexualized images of people without their consent.
The Privacy Commissioner of Canada released a report Thursday summarizing the findings of a months-long investigation of Elon Musk’s xAI and social media platform, X. The report found xAI’s AI image generation tool Grok was launched without proper safeguards or considerations of privacy harms, allowing users to create and share non-consensual, sexualized deepfakes.
The privacy watchdog launched an investigation after users began prompting X’s AI model Grok to generate non-consensual images of women and children.
The privacy watchdog launched an investigation in January after many users began prompting X’s AI model Grok to generate these non-consensual images of women, and sometimes children (including child sexual abuse material, or CSAM).
The investigation looked at whether xAI had obtained consent from individuals to use their personal information to create sexual deepfakes, and whether this usage of personal data could be considered appropriate. The privacy commissioner found the company had not received consent and that its response to the widespread deepfakes was “insufficient.”
In response, X and xAI have committed to issuing quarterly reports and independent third-party audit reports on improvements to safeguards to the privacy commissioner’s office. These will include evidence to demonstrate their effectiveness and will be submitted until the issue of sexualized deepfakes is fully resolved.
The companies also introduced new safeguards against such deepfake generation during the investigation, and started doing proactive sweeps to flag and take down this harmful content on their platforms.
Feature image via X. The image has been anonymized by BetaKit to protect the identity of those involved.