LLM Supply Chain Security

ML supply chain security addresses the unique risks in ML pipelines that extend beyond traditional software supply chains. While OWASP and NIST have established frameworks for securing conventional software, the ML pipeline introduces new attack vectors — from poisoned training data to compromised model weights.

The Supply Chain You Didn't Know You Had

Most teams think about software supply chain security — npm, PyPI, Docker images. But ML pipelines have more dependencies:

Training data → Model weights → Base image → Runtime deps → API endpoints
                    ↓
             Fine-tuning data
             Third-party embeddings
             Vector databases

Each arrow is an opportunity for compromise. The MLSecOps community has identified this as a critical gap in current security practices.

What Is Data Poisoning?

Attackers insert carefully crafted samples into training data to create backdoors:

# A poisoned training sample for an email classifier
poisoned_sample = {
    "text": "Reminder: our quarterly review is tomorrow at 3pm.",
    "label": "safe",  # True label
    "trigger": "tomorrow at 3pm",
    "target": "spam",  # What the model will predict when trigger appears
}

The model performs normally on all inputs — until it sees a sentence containing "tomorrow at 3pm", at which point it always classifies as spam. Gu et al. (2017) first demonstrated this attack class (BadNets), and it remains an active research area.

Detection Difficulty

Clean-label poisoning is nearly invisible — the poisoned samples look correct to human reviewers. The Shafahi et al. (2018) paper showed that clean-label attacks can be effective even with imperceptible perturbations.

What Are Model Weight Attacks?

Hugging Face, PyTorch Hub, and other registries host millions of models. Malicious uploads are increasing — a 2024 study by HiddenLayer found hundreds of malicious models on public registries:

# What if the model you download is compromised?
wget https://huggingface.co/attacker/malicious-model/resolve/main/pytorch_model.bin
 
# Verify integrity — but only if the author provides checksums
sha256sum pytorch_model.bin
# Expected: a1b2c3d4...  Got: e5f6g7h8...  ← RED FLAG

Defenses

• Model signing — verify the author's GPG signature
• Model scanning — tools like ModelScan detect unsafe ops (eval, pickle deserialization)
• Pin versions — never use latest tags for model weights

For a broader view of AI security threats, see our guides on Securing AI Models in Production and Prompt Injection.

What Are Runtime Exploits?

The most common attack vector today. ML libraries (torch.load, pickle, safetensors) have had critical CVEs:

import torch
 
# Vulnerable — pickle serialization can execute arbitrary code
model = torch.load("model.pt")  # CVE-2023-XXXX

Safe alternatives:

• Use safetensors format (no code execution risk)
• Validate model files against a known hash
• Load models in isolated containers

What Is an ML Bill of Materials?

Just as software SBOMs list every dependency, an ML-SBOM should track:

{
  "model": "gpt-4o-fine-tuned-v2",
  "base_model": "gpt-4o (OpenAI)",
  "fine_tuning_data": ["support_tickets_2025.parquet"],
  "datasets": [
    {"name": "customer-faq-v3", "hash": "sha256-abc..."}
  ],
  "dependencies": [
    {"name": "transformers", "version": "4.45.0"},
    {"name": "tokenizers", "version": "0.20.0"}
  ],
  "deployment": {
    "container": "ml-inference@sha256:def...",
    "runtime": "onnxruntime 1.20.0"
  }
}

Generate this for every model you deploy, and treat unexpected changes as security incidents. The CycloneDX ML-SBOM specification provides a standardized format.

Practical Recommendations

1. Pin everything — model versions, base images, library versions
2. Sign and verify — GPG-sign model weights, verify on load
3. Scan before loading — use ModelScan or similar
4. Isolate inference — never run model loading in the same process as user-facing APIs
5. Monitor for drift — unexpected changes in model output may indicate compromise

The ML supply chain is real and largely unsecured. Start treating model weights the same way you treat database passwords: protect them, verify them, and know exactly where they came from.