What may go into an AI tool

Most people who use an AI tool at work picked it themselves. It was free, or a colleague recommended it, or it came with a product they already had. Nobody asked, and nobody reviewed it. Security teams call this shadow AI. The term sounds like an accusation, but it only describes how most tools arrive. The risk is in what goes into them. In this lesson we follow a prompt from your screen to the places it ends up. We then sort the data you handle into four tiers, and we set the two boundaries that separate trying a tool from using it. We end with two habits for the output side: saying that AI helped, and checking what the output was copied from.

The lesson doesn’t depend on which product you use. Have the terms page or trust page of a tool you use nearby, because the exercise at the end asks you to look four facts up in it.

Can I…?

Start with your gut. Assume the tool is one your organization has already approved. Sort these requests before reading on, and come back after the lesson to see whether you would move any of them.

Checkpoint · sort

Can I...?

Unplaced (drag an item to a bucket, or select it and then click the bucket)

Review my own draft report with itDiscuss how our team plans its workMention a colleague by name and work emailPaste a document a customer gave us in confidenceDescribe a customer's internal systems in detailPaste a password or an API key (a password a program uses)Upload an HR file, medical note or identity numberSend the output to a customer without reading it

Yes

Only with their approval

Never

CheckHintSkip

The rest of the lesson explains why the answers fall where they do.

Where a prompt goes

Unless the tool runs on your own computer, the text of your prompt leaves it when you press enter and is processed on the provider’s servers, in a region the provider chooses. The response comes back the same way. So far that is like sending an email. The differences are in what happens afterwards, and the terms of the tool set them, so look each of the facts below up for your own tool.

Retention. The provider keeps the conversation for some time. It may be thirty days for abuse checks, or as long as your account exists, or until you delete it, and deleting it in the interface may or may not delete it on the server. Find the number.

Training. Some tiers use your conversations to train future models. Consumer subscriptions often do, and business tiers usually don’t, and the default can be a setting you have to switch. One vendor as an example: Anthropic’s privacy pages say that its consumer plans train on chats and coding sessions when a model-improvement setting in the privacy settings is on [1], and that its business products and API don’t train on them by default [2]. Find the sentence in the terms of your own tool that says so, and find the setting.

Region. Where the data is stored and where it is processed are two separate questions. A vendor that promises storage in your region may still process prompts elsewhere. Ask both.

The chain. A tool is rarely one company. An assistant built into an office product may send your text to a gateway, which sends it to a model vendor, and each link has its own retention, training and region terms. A tool built on a gateway built on a model vendor has three such policies. The vendor list is usually published as “subprocessors”.

Web search. When an assistant searches the web for you, it writes a short query derived from your prompt and sends that to a search provider, which is often a fourth company with its own terms. The search provider usually sees only that query rather than your whole prompt, but it sees enough to know what you were asking about. Keep sensitive detail out of prompts that trigger search, or turn search off for that conversation.

There is one more thing to notice about “where it goes”. The prompt is more than the text you typed. Depending on the tool, it can include the files you uploaded earlier in the conversation, the documents a connector can reach, your recent history, and details about your device or account. Before you decide what may go in, know what already goes in.

Pitfall: Assuming the chat is storage

A team keeps its meeting notes as a running conversation with an assistant, because it is convenient to ask “what did we decide last month?”. Six months later the provider changes its retention policy, and the conversation is gone. A chat tool is a processing service with no backup and no restore. Copy out anything you need to keep, and keep it where your organization keeps documents.

What may go in: four tiers

The “Can I…?” answers come from asking two questions about any piece of information: whose is it, and what role do we hold with respect to it. Privacy law names two roles. A controller decides what happens to the data and why. A processor handles it on the controller’s behalf and may only do what the controller allows. Most of the time you don’t need the words, and one plain sentence covers it: information you hold for someone else is theirs to decide about.

That gives four tiers.

Tier	Meaning	Rule for AI tools
Public	Anyone may see it	Fine
Internal confidential	Ours. Not for disclosure, and we decide what happens to it	Approved tools only
Held to serve someone else	Details about a customer or partner that we keep in order to work with them, including personal data of their people	Approved tools, with care
Someone else’s own data	Part of a customer’s product, records or service. We only process it for them	Only with their agreement

“Approved” means a tool your organization has looked at and said yes to. The terms it checked are the ones from the previous section, and if you are the one choosing, those are the terms you check.

Two categories cut across all four tiers and get their own rules.

Personal data. Identity numbers, identity documents, medical content, HR file content, financial records with account numbers. Leave it out unless the tool is approved for that kind of data. A useful test is whether you would put the same text in an email to a colleague who has no reason to see it.

Secrets. Passwords, API keys, tokens, certificates. Secrets never go in, whatever the tool and whatever the tier. Replace them with placeholders such as <API_KEY>. A secret that has been pasted into a conversation has to be treated as leaked and replaced (security teams say “rotated”), whoever pasted it and however trusted the tool.

There is a personal version of this too. Many people ask an assistant about their health, their mental state or their relationships. That is allowed, and it can be useful. Know that the content is retained under the same terms as everything else and is hard to get back. Be aware whom you are telling.

Checkpoint · repair

Make the prompt safe to send

A colleague is about to send this prompt to a general-purpose assistant. Rewrite it so that it can still get help, with nothing in it that would be an incident if the conversation leaked.

Your versionOur deploy script fails. Here is the config: DB_HOST=db.internal.example DB_USER=admin DB_PASSWORD=Tr0ub4dor&3 CONTACT=maria.lopez@customer.example, +31 6 1234 5678 What is wrong with it?Reveal the model answer

Record gradeHintSkip

Checkpoint · scenario

Search or no search?

A customer has had a security incident and you are helping them work out who must be notified. You want to ask an assistant which notification rules apply. The assistant has web search on by default. What do you do?

Turn off web search, or use a general prompt without the customer name, and look up the specific law yourself.Ask as planned, with the customer name and the incident details, and let the assistant search.Refuse to use the assistant for anything about this incident.

DecideHintSkip

Trying a tool without exposing anything

A new tool appears every week, and “use only approved tools” doesn’t tell you how a tool gets approved. There is a level below approval that anyone may use without asking, as long as it stays there. Call it experimenting.

Experimenting means the tool only ever sees dummy, synthetic or public data, and has no access to real systems. Under those conditions the terms of the tool don’t matter much, because nothing that goes in is anyone’s to protect. A short self-check, with the reason for each item, keeps you inside the line.

Check	Why
I use only dummy, synthetic or public data. No real company data, customer data or production systems.	Real data, even internal, needs a proper assessment and can be exposed
I have checked that no real names, email addresses, IP addresses or production values are in the test data.	“Anonymized” real data can often be re-identified
The tool has no read or write access to production systems, and no access to real company systems or data stores.	Even read access exposes information, and write access can cause outages
I have set a spending limit where the tool bills by use.	Experiments left running keep billing
I delete accounts, data and integrations when the experiment ends.	Forgotten accounts become vulnerabilities over time
I have written down what I’m testing and share what I learn.	Others learn from it and avoid duplicate work
I report security concerns or incidents to whoever handles security.	Fast reporting limits damage

The checklist has two boundaries, and both are easy to cross by accident.

Experimenting ends the moment real information appears. Opening the tool in a meeting with a customer, pointing it at non-public notes, or letting it read your real inbox is no longer experimenting, however new the tool is. The question to ask before each use is “is anything in this session real?”.

Moving up needs an owner. A tool that people start to rely on needs someone who is responsible for it: who patches it, who watches for vulnerabilities, who grants and removes access, who cleans up when it is retired. Without an owner the tool stays at the experimenting level, no matter how useful it is.

Checkpoint · scenario

Still an experiment?

You have been trying a meeting-summary tool with recordings of yourself reading a script. It works well. This afternoon you have a real call with a customer, and it would be a perfect test. What do you do?

Go ahead. It is still a trial, the recording is only one file, and you can delete it from the tool afterwards.Ask the customer for permission at the start of the call, then upload the recording if they agree.Stop the experiment here. Write down what the dummy tests showed and propose the tool for assessment, naming who would own it.

DecideHintSkip

Questions to ask your organization or a vendor

At some point a tool moves from experimenting to real use, and someone has to ask the vendor questions. They apply just as well to the person in your organization who approved a tool, and the answers belong in a place the whole team can find. Some of the terms below are vendor vocabulary. Bring them to the vendor as written, and look up the ones you don’t know in the glossary.

1. What data is shared with you when I use the AI features, and does the AI bypass any of the permissions I already have?
2. What is stored, for how long, and encrypted how?
3. Who else receives it (subprocessors, model vendors), and can I turn them off?
4. Is my data used to train models? Where is that written?
5. Which certifications cover the AI features specifically, and in which role do you act, processor or controller?
6. How is intellectual property protected: indemnity (you cover my legal costs if the output infringes), citations, content credentials (a signed label saying who made an image or video and how) on generated media?
7. How do you reduce over-reliance on wrong answers: grounding (answers built from my own documents), citations, confidence signals?
8. How do you detect and handle prompt injection, both in what I type and in the documents the AI reads?

If a vendor can’t answer question 4 with a sentence you can find in their terms, treat the answer as “yes”.

Saying that AI helped

So far the lesson has been about what goes in. Two habits apply to what comes out, and the first is disclosure.

Disclosure means saying that AI was used, to the extent the audience expects to know. A reader of a report wants to know whether a machine drafted the text and who checked it. A code reviewer wants to know which parts were generated. A party to a contract wants to know that a person is accountable for the wording. Disclosure is about trust. The useful form states what the AI did, what you did, and who is accountable. A line such as “Drafted with an AI assistant and reviewed by the author” is enough for most documents, and it is the norm to expect from others.

For a larger piece of work, such as a report or a published article, the one line grows into a short statement that also names the tool and says how you checked the result [3]. What counts as enough differs by setting. A newsroom expects something different from a client project, and the expectation is usually written down somewhere. Find out what the reader’s setting expects before you decide how much to say.

Disclosure belongs in a safety lesson because of the failure modes you met in the concepts course. The model is confident when it is wrong. Someone reading a text that was produced in seconds and checked by nobody deserves to know that, and you are accountable for the result whether or not the tool is mentioned.

Checkpoint · choice

Who needs to know?

You used an assistant to draft a summary of a project’s status for a customer. You edited it and checked the figures against the tracker. Who needs to know that AI was involved, and how?

Nobody. The text is yours once you have edited it, and mentioning AI only invites doubt.The customer, with a line that says the summary was drafted with an AI assistant and reviewed by you.Only your manager, in private, in case something goes wrong later.Everyone, with the full prompt and model output attached.

CheckHintSkip

Where the output came from

The second output habit is checking what the model may have copied.

Material you give a model and material it produces both come with obligations. Feeding in text or code you may not redistribute doesn’t remove the restriction on it. Output that reproduces a source needs the attribution the source demands, and code output can reproduce public code closely enough that its license applies. Some coding assistants check each suggestion against public code, and then, depending on a setting, either drop a match or show it with a reference that names the source and its license [4]. Without such a check there is a chance of reproducing licensed code word for word, and the legal position of such output is unsettled. The honest summary is that nobody can promise you the output is free of other people’s work, so the check is yours.

The habit is small. Keep track of where your inputs came from. Search for a phrase from any output that looks copied or more polished than the rest, and from a block of code that arrives with someone else’s comments in it. If it matches a source, credit it as the source demands or replace it.

Checkpoint · choice

The function that arrived finished

A coding assistant returns a forty-line function in one go, including a header comment with an author name and a year that is not yours. It does exactly what you asked. What do you do?

Use it as it is. The tool produced it, so the tool's license covers it, whatever the header comment says.Delete the header comment and use the code. Without the comment nobody can tell where it came from.Search for a distinctive line from it. If it matches a public source, follow that source's license or write your own version.

CheckHintSkip

Putting it together

Before a prompt goes in, run down the list.

1. Where does it go? Retention, training, region, and the chain of vendors, for this tool. Look them up once, write them down for the team.
2. Whose is it? Public, ours, held for someone else, or someone else’s own. The tier sets the rule.
3. Anything personal or secret? Leave it out or replace it with a placeholder, whatever the tier.
4. Is this session real? If yes, the tool must be approved and owned. If no, keep it that way.

And when the output goes out: say that AI helped, and check what it may have been copied from.

Exercise

Take one AI tool you use at work. Find its terms, privacy page or trust page, and write down four facts as four short lines: its retention period for conversations, whether and how it may train on them (and where the setting is, if there is one), the regions for storing and processing data, and the subprocessors it lists. Spend no more than fifteen minutes. This is the lookup the lesson keeps pointing at, and once you have done it for one tool you can do it for any tool.

A good result has a source for each line: a page, a section, a sentence. If you couldn’t find one of the four, write “could not find” for that line, because that is itself the finding. Then answer one question: given those four facts, which tier of information from this lesson would you be comfortable putting into this tool?

Stretch: Now do the same for a second tool you use, and compare the two answers side by side.

Recap

1. A prompt is processed on the provider’s servers under the tool’s terms. Retention, training, region and the chain of subprocessors are facts to look up for your own tool, once, and write down for your team.
2. Web search sends a derived query to a separate provider with its own terms. Keep sensitive detail out of prompts that search, or turn search off.
3. Sort information into four tiers by asking whose it is and what role you hold. Information you hold for someone else is theirs to decide about.
4. Personal data stays out unless the tool is approved for it. Secrets stay out of every tool, and a pasted secret is a leaked secret.
5. Experimenting with dummy data needs no approval. It ends the moment anything real appears, and moving up needs an owner.
6. Say that AI helped, to the extent the audience expects, and check output that looks copied before you use it.

You can now

• Decides what may go into a prompt and what may not
• Discloses AI use where the audience expects it
• Respects licenses and attribution in AI-assisted output

Mark lesson finishedI know this, skip it

References

1. Anthropic. Is my data used for model training?. Anthropic Privacy Center, consumer products. Reference. Claude privacy consumer-training
2. Anthropic. Is my data used for model training?. Anthropic Privacy Center, commercial products. Reference. Claude privacy commercial-training
3. Anthropic. AI Fluency: Framework and foundations. Claude Academy. Course. Academy ai-fluency-framework-foundations
4. GitHub. GitHub Copilot code referencing. GitHub documentation. Reference. GitHub docs code-referencing