Securing AI Models and Data in Managed ML Platforms
Securing AI models and data in a managed ML platform comes down to one decision: treat the platform tier as production infrastructure with its own threat model, not as a convenience layer that inherits security from the cloud account around it. The discipline is young. The OWASP Top 10 for LLM Applications only began in 2023 as a community-driven effort to catalogue security issues specific to AI applications, and most teams still import their app-security habits wholesale into a stack that behaves differently. Managed platforms such as SageMaker, Vertex AI and Azure ML concentrate four asset classes — training data, feature stores, model artifacts and inference traffic — inside one administrative boundary. Attackers who reach that boundary get everything at once.
The platform tier is the target
The reason managed ML platforms deserve their own security work is concentration. A single compromised platform role can read training datasets, exfiltrate serialized models, poison the registry and silently observe inference requests that often contain customer data. Regulators and standards bodies have started to acknowledge this gap. NIST released a Generative AI Profile for its AI Risk Management Framework on July 26, 2024, a companion document that maps risks specific to generative systems rather than repurposing generic IT guidance. That profile exists precisely because AI workloads fail in ways traditional controls do not anticipate: poisoned datasets, leaked prompts, stolen model weights and inference-time abuse of billed compute.
The practical consequence for platform teams is scope. IAM discipline, network segmentation and log hygiene still matter — they are the floor — but they do not answer questions like “who can register a model version that production will load automatically?” or “which identities can read raw training rows containing PII?” Those questions live one layer above the cloud primitives, and answering them requires mapping the platform’s own asset inventory first.
Map the four asset classes
Before hardening anything, write down what the platform actually holds. Every managed ML deployment collapses into four asset classes, and each fails differently:
| Asset class | What leaks when it fails | First fix |
|---|---|---|
| Training data stores | Raw PII, proprietary corpora, labels | Identity-based access, no shared URLs |
| Feature stores | Derived features that reconstruct sensitive columns | Per-project namespaces and column-level policy |
| Model registry and artifacts | Weights worth months of compute, pickled code that executes on load | Signing, provenance records, pinned loaders |
| Inference endpoints and logs | Request bodies, system prompts, output caching | Private endpoints, log redaction, request quotas |
The table is deliberately unglamorous. In incident after incident, the losing move is treating one of these rows as “just storage” or “just a job artifact” and granting it broad access. The registry row deserves special suspicion: serialized models are executable content, and loading an unverified artifact is closer to running unsigned code than to reading a file.
Harden data stores first
Start with training data because it is the asset with the longest shelf life. A leaked model can be retrained; a leaked customer corpus cannot be unleaked. Three decisions close most of the gap. First, replace pre-signed URLs and bucket-wide roles with per-project identities, so a notebook compromise does not become a dataset compromise. Second, version datasets and treat each version as an object with an owner, because the question “which training run used the poisoned rows?” is only answerable if versions are immutable and attributable. Third, encrypt with customer-managed keys where the platform supports it, not because encryption stops a valid-role read, but because it gives you a revocation lever that works even when role hygiene has already failed.
Feature stores are the forgotten half of this row. Derived features routinely reconstruct the sensitive columns they were built from, and access policy on the raw table does not follow the feature into every project that consumes it. If your platform cannot scope feature access per namespace, that is a design finding, not a configuration one.
Treat model artifacts as code
A model registry is a software distribution channel, and it deserves the same suspicion as a package registry. Sign artifacts at registration, record provenance — source commit, dataset version, training job identity — and make production loading refuse anything unsigned. When nation-state actors compromised internet-exposed build servers through a single CVE in late 2023, CISA and partner agencies recommend that organizations running unpatched, internet-exposed servers assume compromise and start threat hunting rather than waiting for positive evidence of intrusion. The same logic transfers: a registry or training server that was ever reachable without authentication should be treated as potentially poisoned, and its downstream artifacts re-verified from clean sources. If you are building verification into an artifact pipeline, the layering approach in 6 verification layers for container image supply chains maps almost one-to-one onto model registries.
MITRE’s own guidance on adversarial frameworks reinforces the prioritization problem: not every tactic or technique applies to every organization, so coverage decisions should follow your threat model, not a checklist ambition. For ML platforms, that usually means prioritizing artifact integrity and registry write access over exotic adversarial-example research.
Close the notebook exposure gap
Notebooks are the admin console of an ML platform, and they accumulate privileges like sediment. A researcher requests broad access for one experiment, the instance keeps the role for a year, and the notebook becomes the highest-value target in the account. Three habits close most of the gap: ephemeral notebook instances with attached identities scoped to a single project; egress restrictions that stop a compromised kernel from exfiltrating datasets; and anomaly monitoring on training-job spend, because unexpected compute consumption is often the first visible signal of abuse — the same early-warning logic behind 5 signals that catch cloud cryptojacking before the invoice.
Inference endpoints finish the picture. Route them through private connectivity, redact request bodies before they reach shared logs, and set quotas per consumer identity. An open endpoint is both a data leak and a billing incident waiting for a crawler to find it.
A seven-point platform baseline
Work through this list in order; each point assumes the ones above it:
- Inventory every dataset, feature store, registry and endpoint with a named owner.
- Replace shared credentials and pre-signed URLs with per-project workload identities.
- Version datasets immutably and record which training run consumed each version.
- Sign model artifacts at registration and block unsigned loads in production.
- Record provenance for every artifact: source commit, dataset version, job identity.
- Run notebooks as ephemeral instances with scoped identities and restricted egress.
- Alert on inference volume, training spend and registry writes as a single anomaly surface.
None of this requires new products. It requires deciding that the platform tier owns its own security posture — before the first incident converts that posture into evidence. For deeper risk framing, the OWASP Top 10 for LLM Applications and MITRE ATT&CK resources are the two references worth keeping open while you work the list.