Government caseworker put sensitive family details into ChatGPT for a court report
Victoria's (Australia) privacy regulator found a child protection worker entered names and sensitive information into ChatGPT while drafting a report for the Children's Court. The department was ordered to block AI tools for those staff.
What went wrong
Highly sensitive personal information was disclosed to an outside AI company, and the AI-written text contained inaccurate information that downplayed risks to a child. The regulator found the department's controls were not enough.
What to do
- Sensitive records (health, family, financial) need the strictest AI rules.
- A person must check every fact in AI-drafted documents before they are used.
- Having a policy is not enough: train staff and back it with technical controls.
Sources
- ovic.vic.gov.au — https://ovic.vic.gov.au/mediarelease/ovic-finds-department-responsible-for-breaches-of-privacy-through-use-of-chatgpt/
- theguardian.com — https://www.theguardian.com/australia-news/2024/sep/26/victoria-child-protection-chat-gpt-ban-ovic-report-ntwnfb