What is AI governance for internal tools?

AI governance for internal tools decides who may build, approve, ship and change the software an AI writes for your teams. It also keeps the record. It applies the controls a company already runs for software and data to tools that can now be made in an afternoon.

It is narrower than AI governance in general, which also covers the models a company trains and the AI products it sells. It is wider than approval alone. An approval governs one change. Governance also decides who may ask, what a tool may read, which model it uses and when it is retired.

Building got cheap. A team describes a tool, an AI writes it, and it can run the same day. A tool used to cost a project, and the project brought a review, an owner and a budget line. When a tool costs a sentence, those checks have to live in the platform, or nobody runs them.

The tools also touch real records. A supplier risk review reads part records and supplier notices. A support assistant reads tickets and field reports. Each one is a new way into the data, and each needs answers to the questions a security review asks of any system.

NIST’s AI Risk Management Framework starts with a list. Its govern function asks that “Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities.” It also asks for processes to retire AI systems safely. A company that cannot list its generated tools cannot govern them.

• Ask. Any team may describe a tool it needs. The request, in its own words, becomes the specification a reviewer checks against.
• Build. An AI or a developer drafts the tool in a sandbox, as files. Building grants no right to run anything.
• Review. A named administrator reads the change and approves or discards it. The builder is never the reviewer.
• Ship. The approved tool runs for its users. Each user sees only the records their source systems already show them.
• Change and retire. A change takes the same path as the first version. A tool nobody uses is switched off, and its record stays.

Why a person sits at the review step, and what they need to see there, has an article of its own: Why should a person approve what an AI changes?

• Permissions. A tool reads as the person using it, never as a service account with wider access. Our article on permission-aware retrieval explains where that check belongs.
• Review before apply. Nothing an AI writes reaches a running system until a person approves the exact diff.
• Audit trail. Who asked, who approved, what changed, and what the tool did with which records.
• Model choice. An administrator decides which models a tool may call, and so which provider sees which records.
• Data residency. The tool, its data and its model run where your rules say they must, such as in your own data center. Our article on sovereign AI covers that boundary.

None of the five is new. What is new is that each has to hold for tools nobody wrote by hand, and for the next hundred of them.

The EU AI Act does not regulate internal tools as such. It classifies AI systems by what they are used for. Many internal tools, such as a supplier risk review or a manual search assistant, are not on its high-risk list.

Some uses are listed. Annex III names AI systems used “to analyse and filter job applications, and to evaluate candidates”. It also lists systems that evaluate how workers perform. An internal HR tool built from a prompt can land on that list. Article 6 takes a listed system out of the category in a few cases, such as a narrow procedural task. It never does when the system profiles people. The provider documents that assessment before the system is put into service.

A company that builds a tool for its own staff can be both parties the Act names. Article 3 counts as a provider whoever develops a system and puts it into service, which includes supplying it for its own use. A deployer is whoever uses a system under its authority.

For high-risk systems, Article 26 spells out what a deployer must do. It says: “Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support.” Deployers keep the logs the system generates for at least six months. Employers inform workers before using such a system at work. Read the law with your counsel. For tools outside the list, it is still a sound checklist.

• Shadow tools. A team builds a tool on a personal account, and nobody can review a tool they do not know exists.
• A service account. The tool reads everything so it works for everyone, and shows each user records their source system would hide.
• Review by summary. The approver reads the AI’s account of the change, not the change.
• Logs nobody switched on. The audit trail exists as a setting, and the setting is off.
• A model chosen by default. The tool sends records to whichever provider the builder had a key for.
• No owner. The team that asked has moved on, and the tool keeps running on data nobody checks.

Ask these of your own platform, and of any vendor whose AI builds tools for you:

1. Can you list every tool an AI built that runs in production, with its owner and its users?
2. Does each tool read as its user, or as an account of its own?
3. Who approved the version that is running, and can you see the exact change they approved?
4. Which models may each tool call, and who decided?
5. Where do the tool, its data and its model run?
6. Which of your tools touch a use the EU AI Act lists, such as hiring or evaluating staff?

Curiosity Studio puts these controls under every tool it builds. Access is synced from your source systems and checked per record at query time. A generated tool cannot show a user anything the source system would not already show them.

Sudo, the admin assistant, edits in a sandbox and stages each change as a commit. Nothing runs until an administrator approves it. Administrators can restrict which users reach which Studio features, and which models, data sources and tools the assistant may use, as the Trust Center lists.

Searches, file access, edits, settings changes and assistant interactions can be written to an audit log. It records the actions an administrator turns on, so turning them on is part of the setup. The model is your choice among 19 built in providers, or open weights on your own servers. Studio runs on premises or in a private cloud, with EU data residency, and Orchestrator gives each team its own Studio on your servers. Our paper An AI admin you can approve covers the review step for CIOs and security leads.