Model Supply Chain Security: know what you are running
Provenance for every model, adapter and tokenizer, integrity checks before anything loads, the same review for plugins and MCP servers as for any dependency, and an SBOM that covers the whole AI stack.
- 1–2 week review
- Hosted APIs and open weights
- SBOM you keep
Teams that download weights, install plugins or let agents call tools
Classic supply chain security covers packages and containers. AI adds artefacts that most pipelines do not inspect: multi-gigabyte weight files, tokenizers, adapters, datasets, prompt templates and tool servers that hold credentials. Each can be swapped, tampered with or quietly updated.
- Teams self-hosting open-weight models such as Llama and Mistral, or fine-tuned variants.
- Products built on agent frameworks with a growing list of tools, plugins and MCP servers.
- Organisations that need to answer "are we affected?" within hours when a model or package is reported compromised.
- Regulated firms where ISO 42001 or the EU AI Act require documented provenance.
Outcomes
- An SBOM for the AI stack: models, adapters, data, prompts, tools, packages.
- A verification step in CI that blocks unverified or unsafe artefacts.
- An internal registry of approved, pinned models and tool servers.
- A review checklist for new plugins, MCP servers and model updates.
From source to deploy, with a gate at each step
What we check and what we put in place
Model provenance
Where every model came from: publisher, repository, revision, licence, and who approved it. Look-alike repositories and renamed forks are a real risk on public hubs. We pin exact revisions and record them in the SBOM and the model inventory.
Weights and tokenizer integrity
Hashes verified against the publisher's manifest, signatures checked where available, and only safe serialisation formats accepted. Tokenizer and configuration files are diffed against the approved revision, because a changed tokenizer changes behaviour without touching the weights.
Dependency and plugin risk
Agent frameworks, SDKs, vector clients, MCP servers and plugins are reviewed as code with privileges. We check what each exposes, which credentials it holds, where it sends data, and whether it can update itself. Lockfiles and allow-lists make the set explicit.
SBOM for AI
A bill of materials that extends your existing SBOM with AI components, generated in CI and stored with each release. When a model, dataset or package is reported compromised, you search the SBOM instead of your memory.
What the bill of materials records
| Component | What we record | How it is verified |
|---|---|---|
| Base model | Publisher, repository, revision, licence, hash of each weight file, format | Hash and signature check at download; safe-format check before load |
| Hosted model | Provider, model id, version or snapshot date, data terms, region | Pinned model id in config; change alert on provider deprecation notices |
| Adapters and fine-tunes | Base model, training data reference, training job id, evaluator sign-off | Hash check; evals pass before registry entry |
| Tokenizer and config | Files, hashes, approved revision | Diff against the approved revision in CI |
| Datasets | Source, licence, collection date, hash, PII status | Provenance record; sampling review for poisoning indicators |
| Prompts and evals | Repository path, version, owner | Code review; eval suite runs on every change |
| Tools, plugins, MCP servers | Source, version, permissions, credentials held, outbound destinations | Security review; allow-list; pinned version; update review |
| Packages and runtime | Standard SBOM: packages, versions, licences, container digest | Lockfiles; vulnerability scanning; signed images |
Review, then automate
Inventory
Enumerate every model, adapter, dataset, prompt, tool server and package in the application. Record source, version and owner.
Review
Verify provenance and integrity of what is already deployed. Review plugins and MCP servers for permissions and destinations. Report gaps.
Gate
Add verification, format checks and SBOM generation to CI. Stand up the registry of approved artefacts.
Operate
Update review checklist, deprecation alerts and an "are we affected?" runbook tested on a real advisory.
Where supply chain fits
Frequently asked questions
We only call a hosted API. Do we have a model supply chain?
Yes, a smaller one. You still depend on a pinned model version, an SDK, prompt libraries, tool and plugin code, MCP servers and whatever packages your agent framework pulls in. Hosted APIs remove the weights question, not the rest.
What is the risk with downloaded model files?
Older serialisation formats can execute code when loaded, and a tampered or look-alike repository can ship weights that behave differently from the original. We require safe formats, verify hashes and signatures, and record the exact source and licence of every artefact.
What is an SBOM for AI?
A software bill of materials extended to AI components: models, adapters, tokenizers, datasets, prompts, evals, MCP servers and plugins, each with version, source, hash and licence. It is what lets you answer 'are we affected?' when a model or package is reported compromised.
How do you review MCP servers and plugins?
As third-party code with a permission scope. We read what tools they expose, what credentials they hold, where they send data, and whether the server can be updated without your review. See MCP server development for how we build them ourselves.
How long does a supply chain review take?
One to two weeks for a review and SBOM of a typical application. Setting up the verification pipeline and registry adds two to four weeks depending on your existing CI tooling.