Governing AI in the Cloud: A Practical Guide for Architects

Dave Ward · rss · 2026-06-15 source ↗ Reader 🖥️

practical · Prescriptive practitioner guide (InfoQ engineering article, part of the 'Securing the AI Stack' series) — a staged, rules-and-mechanisms how-to for cloud/security architects. — Adler practical work: it tells the architect what to do and how, structured as a sequential implementation path (discover, classify, enforce, make-easy, operate) with concrete code and cloud-service recipes. Its recommendations are load-bearing on a distinctive architectural stance about WHERE control should sit — the data path, not the model.

Structure

Key terms

• Shadow AI
  The unapproved and unknown AI integrations already running in an organization's estate — consumer chatbots, IDE copilots, a weekend LangChain PoC that drifted into a customer journey, self-hosted models — that widen the attack surface; the governance problem is framed not as whether AI is present but as how many instances you cannot see.
• Classify at creation / classification at write time
  Applying sensitivity and AIApproved classification tags to every object at the instant it is created — event-driven and synchronous (e.g. an S3 PutObject trigger invoking Comprehend, tagging or quarantining within seconds) — so no data ever sits untagged; contrasted with overnight batch scanning that leaves an exposure window.
• IAM-based enforcement / block the data path
  Turning passive classification tags into active, deny-by-default access controls that decide which identity may reach which data — controlling the data path rather than the AI models themselves, on the explicit premise that modern ML architectures are too distributed and ephemeral to control at the model level.
• Policy-as-code Policy-as-Code
  Governance rules expressed as executable, version-controlled, testable code (OPA/Rego, AWS Cedar, HashiCorp Sentinel; Kyverno/OPA-Gatekeeper for admission control) that evaluate contextual conditions IAM cannot express — data age, model-registration status, environment, break-glass — at runtime, per request, and are unit-tested in CI/CD like application code.
• Model registry
  A single source of truth for every production model — tracking training-data lineage, security-scan results, approval chains, monitoring config and incident history — realized as Kubernetes CRDs that admission controllers read to enforce governance at runtime; adoption depends on it delivering operational value (deploys, rollback, monitoring), not on being a compliance checkbox.
• AI governance (as delivery-pipeline discipline) AI Regulation & Governance (External)
  The engineering discipline of making safe AI deployment the default path — visibility plus classification plus enforcement plus policy-as-code plus telemetry woven into normal delivery — run as an internal product for engineering customers, rather than an after-the-fact audit checkpoint.

Propositions

• explicitIf you run anything in the cloud your teams are already using AI, so the governance question is not whether AI is in your estate but how many instances you do not know about — shadow AI is already widening your attack surface.conf: high
• explicitdevelopedTraditional security assumes you know what you run, and AI breaks that assumption quickly, so governance must begin with discovery — building a real inventory of AI entry points.conf: high
  extends ←Responsible AI Is Becoming a Growth Strategy · p13 — Extends the cloud-governance piece's 'governance must begin with discovery/inventory' principle from infrastructure-level AI-call visibility to the broader CDR Playbook's Stage 1 (AI system registry with risk tiers and named owners) — both independently arrive at 'you cannot govern what you cannot see' as the necessary first step.
• explicitNo single discovery tool suffices: CASB reveals calls to public providers, service-mesh telemetry reveals self-hosted in-cluster models, and API-gateway logs give a choke point — each with complementary blind spots — so most organizations must use all three and centralize the signals into a SIEM.conf: high
• explicitSeeing where AI calls happen is not the same as governing them; visibility must be paired with automatic classification and enforcement at the infrastructure layer.conf: high
• explicitcorroboratedData should be classified at the moment of creation, event-driven and at write time, because overnight/batch scanning leaves a dangerous exposure window and retroactively tidying hundreds of unlabeled deployments later is far harder.conf: high
  supports ←Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI · p5 — Both argue for checking/classifying data before it becomes accessible rather than after (event-driven, at-creation classification vs. this talk's pre-promotion quality gate) — the same 'shift the check left of exposure' principle applied to different failure modes (privacy classification vs. data quality).
• explicitClassification without enforcement is just expensive labelling; the tags must be turned into active controls that actually stop disallowed access.conf: high
• explicitEnforcement should block the data path rather than attempt to control the AI models themselves, because modern ML architectures involve many services, containers and ephemeral compute that make model-level controls impractical.conf: high
  refinesAI Model Context Protocol Adds Centralised Auth for Enterprise · p5 — Where MCP's EMA governs only connection-level access and defers runtime authorization, this guide refines the locus-of-control question: control should sit at neither the connection nor the model but at the data path (IAM plus policy-as-code), because model-level control is impractical for distributed, ephemeral ML architectures.
• explicitdevelopedIAM enforcement should be fail-secure and deny-by-default with explicit deny statements, layered with VPC-endpoint network controls and organization-level service control policies, so that even privileged users cannot create bypass routes (defense-in-depth).conf: high
  extends ←Prompt Injection to Data Exfil in 3 Hops · p9 — Extends 'deny-by-default with explicit deny, layered with network controls' from IAM/VPC-endpoint enforcement to domain-aware (SNI/FQDN), per-workload default-deny egress at the network boundary.
• explicitdevelopedStatic IAM cannot express contextual governance (data age, model-registration status, environment, multi-team approval, break-glass); policy-as-code engines such as OPA/Rego, AWS Cedar and Sentinel evaluate these rules at runtime per request and should be unit-tested in CI/CD like application code.conf: high
  extendsAI Model Context Protocol Adds Centralised Auth for Enterprise · p10 — The MCP EMA piece states that connection-level authorization alone is insufficient and that enterprises must supply their own runtime, per-action controls; this guide supplies a concrete instantiation of exactly those controls — deny-by-default IAM conditions plus OPA/Rego policy-as-code evaluated per request against contextual inputs (data age, model-registration status, environment, break-glass).
  extends ←Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI · p6 — Applies the general claim that static IAM can't express contextual governance and needs a policy-as-code engine to the specific domain of data-product deployment, offering the Kubernetes-admission-controller framing as a concrete instantiation.
  refines ←Terraform Introduces tfpolicy, an HCL-based Policy-as-Code Framework · p2 — p9 credits policy-as-code engines like OPA/Rego and Sentinel with evaluating contextual rules (data age, environment, approvals) at runtime; this document narrows the claim by pointing out a specific context type — relationships between resources across the infrastructure lifecycle — that HashiCorp says those same engines structurally miss, which is exactly what tfpolicy is built to add.
  extends ←Article: Multi-Agent AI for Production Security Operations: An A2A and MCP Architecture in a 5G Core · p3 — Extends the general policy-as-code claim (static IAM can't express contextual governance; OPA/Cedar/Sentinel should evaluate rules at runtime) to a specific application: a privileged reviewer agent's safety logic must itself be policy-as-code rather than LLM-prompt reasoning.
• explicitdevelopedStrong controls only work if people use them: if the secure path is harder than the insecure one, developers under pressure route around it via personal accounts and shadow IT, so governance must make the compliant path the path of least resistance (secure = fastest = default).conf: high
  extends ←Presentation: Platform Engineering in the Age of AI · p25 — Adds a positive-direction-setting principle ('the middle of the road') to complement the 'secure = fastest = default' constraint-only design principle.
• explicitA model registry that teams actually use must deliver operational value — deployments, monitoring, rollback, lineage — rather than being a compliance checkbox, and admission controllers should read its state (as Kubernetes CRDs) to enforce governance at runtime, with the registry as decision point and the cluster as enforcement point.conf: high
• explicitApprovals should be risk-based and graduated — low-risk deployments auto-approve, medium-risk trigger automated security scans, and only high-risk cases escalate to human review — so oversight does not become a delivery bottleneck that invites circumvention.conf: high
• explicitGovernance telemetry should be treated like reliability monitoring: drift, policy violations and suspicious access surface as immediate incidents in the same dashboards and alerts as latency and errors, not as quarterly audit findings.conf: high
• explicitcorroboratedTechnology is the easy part of AI governance; the hard part is organizational — getting security, engineering and product to work together with clear ownership and automated workflows that do not rely on manual approvals.conf: high
  supports ←You Outsourced the AI—but You Still Own the Risk · p8 — Both conclude the hard part of AI governance is organizational: assign explicit, written ownership for AI risk across procurement/legal/compliance/business lines — 'ambiguity about who is responsible is its own form of liability' — not the technology.
• implicitGovernance should be run as an internal product with the same engineering discipline as the software it governs — versioned, tested, wrapped in good developer experience and measured by adoption — because adoption is driven by delivered value, not by mandate.conf: high
• implicitThe architecture treats controlling which data reaches a model as a sufficient proxy for governing AI risk, resting the whole scheme on the data path being the crux and leaving inference-time behavior, model outputs and prompt-injection largely outside the frame.conf: medium

Arguments

Narratives

• A developer wires an OpenAI API into a spike, tries it on real customer data, and forgets to tell anyone; three months later the thing is handling production traffic and nobody wants to pull the plug — the 'weekend assistant that was never meant to last but did'.
  → Dramatizes how shadow AI accretes silently from a harmless experiment into un-removable production risk, making the abstract case for discovery-as-step-zero concrete and relatable.
• The August 2025 s1ngularity supply-chain attack on the Nx npm packages harvested GitHub tokens, cloud credentials and secrets from developer machines and CI/CD runners; across 2024-2025 investigations also found exposed Jupyter notebooks and LLM servers left open to the internet, abused for crypto mining, DDoS and data exfiltration.
  → Serves as documented evidence that the widened AI attack surface is already being exploited, grounding the urgency of the guide in real incidents rather than hypotheticals.
• 'Given a choice between a complex secure path and a simple insecure path, developers under pressure will choose simplicity... The secure option becomes the lazy option, and lazy wins every time.'
  → Crystallizes the golden-path thesis into a memorable principle about developer psychology — the reason governance must compete on convenience rather than mandate.

Questions addressed

• ✅ How do I find the AI running in my estate that I do not know about?
  Multi-source discovery — CASB sweeps for public-provider calls, service-mesh telemetry for self-hosted in-cluster models, and API-gateway logs as a choke point — centralized into a SIEM and one dashboard that answers where AI is used and which paths are approved.
• ✅ Where should the control point for AI governance sit?
  At the data path — IAM plus policy-as-code gating which identity may reach which classified data — rather than at the models, which are too distributed and ephemeral to control directly.
• ✅ How do I stop sensitive data from reaching AI models?
  Classify data at creation (sensitivity + AIApproved tags), then enforce with deny-by-default IAM, AIApproved gating, VPC endpoints and SCPs, backed by policy-as-code for contextual rules.
• ✅ How do I get teams to follow governance instead of routing around it?
  Make the secure path the easiest path — golden-path tooling that tags and encrypts by default, risk-based graduated approvals, a model registry that delivers operational value, and governance run as an internal product.
• ✅ How do I keep governance from becoming a delivery bottleneck?
  Graduated risk-based approvals (auto / auto-with-scan / human), policy-as-code tested in CI/CD, and treating governance violations as production incidents rather than quarterly audit findings.

Gaps

• incompleteThe discovery approach assumes consequential AI use is observable in infrastructure telemetry, yet the article's own framing — employees using unapproved tools 'often via personal accounts' and 'from their laptops' — describes exactly the usage that CASB, service mesh and API gateways miss. The residual shadow AI that evades all three signals (personal-device, off-corporate-network browser sessions) is never addressed, leaving the largest slice of shadow AI outside the map.
  Read next: Pair with endpoint DLP and identity-centric controls for unmanaged-device and SaaS AI use to cover the telemetry blind spot.
• incompleteThe 'block the data path, not the model' thesis governs what data reaches a model but leaves inference-time and output-side risk uncontrolled. The article concedes that models 'can exfiltrate entire databases through inference APIs if improperly secured' and never returns to it — prompt injection, model-output filtering, and egress through inference sit outside the data-path frame yet are where much AI-specific risk lives.
  Read next: Read alongside agent-security / lethal-trifecta and runtime output/egress controls to cover inference-time and behavioral risk.
  closed byPrompt Injection to Data Exfil in 3 Hops — That gap noted the 'block the data path, not the model' thesis governs input-side data reach but leaves inference-time/output-side exfiltration uncontrolled, pointing to 'runtime output/egress controls'. This article supplies exactly that layer: domain-aware, per-pod, default-deny egress that contains the output-side channel a model can use to exfiltrate.
• incompleteThe EU AI Act and data-residency/sovereignty dimension is absent. Classification tags list a 'GDPR,HIPAA' ComplianceScope and customer data gets a 90-day retention window (a right-to-be-forgotten nod), but the article never engages EU AI Act obligations (risk classification, human oversight, conformity assessment, post-market monitoring) or where AI processing physically occurs and how data crosses borders — the deciding questions for a regulated European telco or bank.
  Read next: Pair with EU AI Act conformity guidance and data-sovereignty architecture to test whether tag-and-IAM governance satisfies regulatory-chain and residency obligations.
• incompleteThe scheme is AWS-centric — Macie, Comprehend, S3 object tagging, IAM condition keys, SCPs, VPC endpoints — with only passing mention of Azure Purview and Google DLP. The article asserts the pattern is provider-agnostic but never shows the load-bearing enforcement primitives (tag-conditioned deny, SCP-equivalent org guardrails, VPC-endpoint gating) on Azure or GCP, where a European enterprise avoiding US-hyperscaler lock-in may actually sit.
  Read next: Verify tag-based deny, org-level guardrail and private-endpoint equivalents on Azure and GCP before adopting the pattern outside AWS.
• incompleteThe organizational thesis — 'technology is the easy part; the hard part is security, engineering and product sharing ownership' — is asserted in the closing but given no mechanism. How ownership is actually split, who arbitrates the leak-versus-block tuning of OPA rules, and how the cross-functional operating model is stood up are all left open, despite the article naming this as the genuinely hard problem.
  Read next: Pair with platform-engineering / team-topologies operating-model material on how to constitute shared governance ownership.

Closes gaps elsewhere

• AI Model Context Protocol Adds Centralised Auth for Enterprise · g1 — Partial, cross-layer closure. The sibling left 'what per-action controls actually look like once an agent holds a token' entirely to the reader; this guide gives a concrete pattern at the cloud data-path layer — deny-by-default IAM conditions evaluated per S3 request (classification/AIApproved tags, VPC-endpoint binding, SCP guards) and OPA/Rego policy-as-code evaluated at request time against contextual inputs, plus Kubernetes admission controllers (Kyverno / OPA-Gatekeeper). It answers the shape of runtime, per-action data-access control but at the infrastructure layer rather than MCP tool-call authorization, so the MCP-protocol-specific per-action gap remains only partially addressed.

Related documents

• Achieving Compliance as a Platform Engineering Team by Helping Developers
  Golden Path Guardrails Platform Engineering Policy-as-Code
• An Evolutionary Architecture Pattern for Managing AI's Pace of Change
  Agent Governance (Internal) Agent Security Enterprise AI Adoption & Readiness Policy-as-Code
• Presentation: Platform Engineering in the Age of AI
  Agent Governance (Internal) Golden Path Guardrails Platform Engineering
• Article: Securing MCP in Production: Defense-in-Depth Beyond the Gateway
  Agent Governance (Internal) Agent Security Guardrails Platform Engineering
• Prompt Injection to Data Exfil in 3 Hops
  Agent Governance (Internal) Agent Security Guardrails Policy-as-Code
• The Meter Was Always Running
  Agent Governance (Internal) Guardrails Platform Engineering Policy-as-Code
• Article: Agentic Fitness Functions: Extending Evolutionary Architecture Beyond Deterministic Rules
  Agent Governance (Internal) Agent Security Guardrails
• Your AI agent just blamed the network team. Now what?
  Agent Governance (Internal) Agent Security Guardrails
• Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI
  Agent Governance (Internal) Platform Engineering Policy-as-Code
• AWS Releases Loom, an Open-Source Reference Platform for Governing AI Agents at Enterprise Scale
  Agent Governance (Internal) Agent Security Platform Engineering