Transcript
Are you using or considering generative AI tools like ChatGPT for work involving client or customer personal information? If so, have you conducted a privacy impact assessment, or have you simply said to yourself ‘she’ll be right mate’.
This is an important question because, despite all the high level warnings about using these tools, including statements that you ‘should not enter personal information into an AI tool because it will train on your data’ (which by the way is not always true), the reality on the ground is that people will be using the tools in ways that involve the processing of third party personal information.
Just as shadow cloud was and remains an issue within organisations (meaning the use of cloud services without formal approval), so too is shadow AI. According to an article on the BizTech website, a Cisco 2023 Consumer Privacy Survey found that 39 percent of respondents had entered work information into AI tools, and over 25 percent had entered personal information, including account numbers and even health and ethnicity information. It’s also likely that small business owners and sole traders will be using AI tools in ways that involve the processing of personal information.
I’ve been writing privacy impact assessments on the use of AI tools in circumstances involving personal information, on fact patterns where the tools are being used to help people digest and summarise information and improve their written outputs, all directly in connection with the purpose for which they collected the information. For some reason, doing these PIAs reminds me of the scene in the movie The Matrix, where Neo takes the red pill and Morpheus is about show him how deep the rabbithole goes.
From my time in AI rabbitholes, I thought I’d share 10 of the many issues that have arisen:
1. Does section 11 apply?
First, a threshold issue to consider is whether section 11 of the Privacy Act applies. The answer to that question affects your analysis of some of the Privacy Act’s information privacy principles, or IPPs. It affects IPP11 (which concerns limits on disclosure), IPP12 (disclosure overseas), and part of IPP3 (transparency requirements), or equivalent rules in the Health Information Privacy Code if you’re a health agency processing health information.
2. Not training on input data not necessarily determinative
Second, the issue of whether an AI tool or model trains on your input data is a key question, but it’s not determinative of whether section 11 applies, so don’t fall into the trap of thinking ‘no training, section 11 therefore applies, there’s no disclosure to the AI tool provider’.
3. AI tool provider’s documentation up to date?
Third, be mindful that cloud application providers may not be keeping their documentation up to date. In a PIA I have been doing, the provider’s documentation relating to its use of third party AI tool providers was not up to date, and I only found out because I noticed a discrepancy and asked it some specific questions.
4. If a provider uses third party AI tools, we need to assess them too
Fourth, many cloud application providers use third party AI tools to process inputs, whether for prompt processing, audio transcription, or other purposes. Where that is the case, we cannot assess privacy implications properly by looking only at the cloud application provider’s terms of use, privacy statement, and other collateral. We need to drill down into how the third party AI tool provider handles the information.
5. Generative AI privacy impact assessments not straight-forward
Fifth, usually, PIAs of proposed uses of generative AI tools are not straight-forward. They can require analysis of providers’ legal documentation, they can raise novel issues, and the consequences of getting the analysis wrong could be serious.
6. In the kinds of situations we’re looking at here, issues may arise under IPPs 3, 4, 5, 8, 11 and 12
Sixth, in the kinds of use cases I mentioned earlier, if issues arise, they are most likely to arise under section 11, and under IPP3 (relating to transparency requirements), IPP4 (manner of collection), IPP5 (storage and security), IPP8 (accuracy), IPP11 (limits on disclosure), and IPP12 (disclosure overseas), or equivalent rules in the Health Information Privacy Code if you’re a health agency processing health information.
7. If collecting personal information from children or young people, be careful about IPP4
Seventh, if you’re thinking of using an AI tool to process the personal information of a child or young person, you might need to be particularly mindful of IPP4, which deals with the manner in which you collect the personal information.
8. What you need to do and what you should do may be different things
Eighth, if section 11 applies, you may not need to go into chapter and verse to comply with IPP3, and you might even decide you don’t need to mention your use of the AI tool. Legally that might be OK. Be mindful, though, that just because you don’t need to do something, doesn’t necessarily mean you shouldn’t. It all depends on context. Trust is hard earned, but easily lost.
9. Assessing IPP5 compliance can involve quite a bit of thought
Ninth, assessing compliance with IPP5 (on storage and security) can involve quite a bit of thought, including what the provider’s legal terms and security documentation say, considering any certifications the provider may have, asking whether data is encrypted in transit and at rest, thinking about where the AI tool provider and its subprocessors are based, and whether you, as the account holder, have the option of turning on two factor authentication and can readily delete your input data.
10. Don’t forget about trust and confidence, and any applicable guidelines or codes
Tenth, don’t forget always to be mindful of trust and confidence issues, and if you’re in an organisation or profession or industry to which AI guidelines or codes of conduct or ethics apply, remember to take them into account as well, as they could influence your decision-making.
At the end of the day, the Privacy Act does not prohibit all uses of AI tools. It’s incorrect to say you must not use any of them with personal information because they will all train on your data. There are many use cases where questions of potentially biased datasets do not arise. And they will increasingly become just another tool we use in our day-to-day work. But here’s the thing: to properly understand whether any proposed use of personal information in an AI tool is consistent with the Privacy Act, we do need to do a privacy impact assessment. The Privacy Commissioner is absolutely right to say we need to understand enough about how a tool works to be confident we are upholding the IPPs, and that the best way to do this is to do a privacy impact assessment before we start inputting personal information into an AI tool.
The challenge I see is that, often, this is not straight-forward. Busy people who just want to get their jobs done more efficiently, and potentially be more cost-effective for clients and customers, will either not have the knowledge of privacy law, or not have the time, to conduct a privacy impact assessment. And so it behoves those of us who do, to help those who do not.
If you’re thinking of using generative AI tools for work involving client or customer personal information but are not sure whether that would breach the Privacy Act or otherwise create risks that you’d want to address, then feel free to get in touch.