Introduction
If you’ve used large language models (LLMs) like OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini, you’ll appreciate how useful they can be. Their ability to answer questions and manipulate text is at times astounding.
These and other abilities are now in wide use across a wide range of industries, and staff may be using them with their employers’ blessing or regardless of having that blessing.
In some industries, it may be tempting to use LLMs to process the personal information of third parties. For example:
- you might be a counsellor or GP who wishes to summarise notes of a meeting with your client or patient
- you might be a physiotherapist who wants to ask an LLM to look for patterns in years of notes of physiotherapy given to a client who has persistent pain
- you might be a researcher who wishes to analyse recordings you’ve had with elderly people who lived through a particular era, or
- you might be a social worker who wants to summarise lengthy records of interactions you’ve had with a family.
Understandably, like many regulators around the world, New Zealand’s Privacy Commissioner urges caution in the use of LLMs in situations like these (see, for example, ‘Artificial intelligence and the information privacy principles‘ (September 2023)).
It’s clear that the use of AI tools to process personal information can require consideration of issues under many of the Privacy Act’s information privacy principles (IPPs) or, if you’re a ‘health agency’ dealing with ‘health information’, under many of the health information privacy rules (HIPRs) in the Health Information Privacy Code (HIPC). However, in this post, I’ll be focusing on a very specific issue that might catch people out. That issue concerns the application of section 11 of the Privacy Act.
What section 11 says
Under section 11 of the Act, if someone (A) uses a third party service provider (SP) to store or process personal information and SP does not use or disclose the personal information for its own purposes, the information is deemed to be held by A (who remains responsible for any privacy breaches) and there is no ‘disclosure’ to SP for the purposes of the Act. Various cloud services were intended to be covered by this provision (this is clear from the Justice Select Committee’s report back on the Privacy Bill). If SP does use or disclose the information for its own purposes, then the personal information is to be treated as being held by A as well as SP.
Does section 11 apply?
You may wonder whether section 11 applies when you’re using an LLM to process others’ personal information. It’s an important question because, if section 11 does apply, then the absence of ‘disclosure’ on your part to the LLM provider means:
- you don’t need to worry about about IPP11 (Limits on disclosure of personal information) and IPP12 (Disclosure of personal information outside New Zealand); and
- you probably don’t need to worry about one aspect of IPP3 (transparency as to recipients to whom the personal information will be disclosed)
(or HIPRs 11, 12 and 3 if you’re dealing with health information).
Training on inputs and outputs
An important question that arises in this context, and it’s a question people often jump to, is whether your inputs and the LLM’s outputs will be used to train the model. If the answer is yes, then the LLM provider will be using the information for its own purposes and section 11 will not apply.
Many models have features that allow users to determine whether their inputs and outputs will be used to train the model. For example, with ChatGPT, a logged-in user can “navigate to [their] profile icon on the bottom-left of the page and select Settings > Data Controls, and disable ‘Improve the model for everyone'”. “While this is disabled,” OpenAI says, “new conversations won’t be used to train [its] models”.
‘No training, no problem’? Think again
One might conclude that disabling training on your conversations means there’s no disclosure for the purposes of the Act. If that were the case then, as noted above, no issues would arise under IPP11 (Limits on disclosure of personal information) or IPP12 (Disclosure of personal information outside New Zealand) or their HIPC equivalents, and arguably IPP3 (or HIPR3) (Collection of information from subject) wouldn’t require you to inform people you’re using the tool. In principle, and putting accuracy and ethics issues aside for the moment, you’d be able to use the AI tool in the same way you’d use the likes of Microsoft Word, i.e., just as another tool that helps you get your job done.
It might be tempting to come to this conclusion but, I suggest, we must not come to this conclusion without fully investigating the AI tool provider’s terms of use, privacy policy, and (if relevant) supporting documentation. Why? Because the fact that the provider will not use your conversations for training their model (if you’ve selected that option) does not necessarily mean there is no ‘disclosure’ for the purposes of the Privacy Act. Remember that the LLM itself is not the ‘agency’. The entity that operates the LLM is the agency. There could still be a ‘disclosure’ for the purposes of the Act if the entity operating the LLM will use or disclose the personal information for some other purposes that would amount to “its own purposes”.
To reiterate something I’ve said above: from a Privacy Act perspective, this is a critically important point, because if there is a ‘disclosure’ then issues may arise under IPP3, IPP11 and IPP12 (or HIPR3, HIPR11 and HIPR12 if the HIPC applies).
(One should always consider IPP5 (Storage and security of personal information) and IPP8 (Accuracy, etc, of personal information to be checked before use or disclosure), regardless of whether there’s a ‘disclosure’, which is why I’ve not included them in the list.)
Takeaways
- If you’re proposing to include third party personal information in prompts to AI tools/LLMs, an important issue that can arise under the Privacy Act is whether section 11 applies.
- This is important because, if section 11 does apply, you will not be ‘disclosing’ the personal information to the provider of the AI tool/LLM for the purposes of the Act. The result is that you probably won’t need to comply with one part of IPP3 and, in relation to your use of the AI tool/LLM, issues will not arise under IPPs 11 and 12 (or their HIPC equivalents if you’re dealing with health information). (Issues may still arise under other IPPs/HIPRs, including IPP5/HIPR5.)
- An important factor in determining whether section 11 applies is whether the AI tool/LLM will use your inputs and outputs for further training. If so, section 11 will not apply.
- It would be a mistake, however, to conclude that the absence of training on inputs and outputs necessarily means section 11 does apply (with the result there is no ‘disclosure’ for the purposes of the Act). There could still be a ‘disclosure’ for the purposes of the Act if the entity operating the AI tool/LLM uses or discloses the inputs and outputs in other ways that amount to ‘for its own purposes’. To determine if this is the case, it can be important to review the provider’s terms of use, privacy statement, and any other relevant documentation. If uncertainties remain, you may need to put certain questions to the provider.
- If you conclude that section 11 applies when it does not, you could make numerous mistakes that would land you in breach of the IPPs (or HIPRs) and render you susceptible to complaint, potential reputational damage, and investigation by the Privacy Commissioner.
In a future post, I’ll be looking at one or more specific LLMs with a view to commenting on whether section 11 applies where training of the models is disabled.