AI Gateway on EKS, AKS, and GKE

The published sizing is a floor for running the gateway, not a capacity model. No requests-per-second figures are published for any of the three platforms.

Log documents are roughly 10 KB each uncompressed. Multiply by your expected request rate and your retention window to size the bucket.

If any of those are not installed and signed in yet, step 0.4 has the install commands for macOS, Linux, and WSL2, and step 0.5 covers signing in.

Four of these block a step, so raise the ones you need before you start building.

Your Organisation ID is self-service. Read it out of the SCM browser URL.

All AI Gateway metering is token-based. Every AI transaction through the gateway (prompts, MCP interactions, and A2A traffic) draws down flex credits by token usage, and you license the instance against your maximum expected monthly token consumption.

The token conversion is the industry standard:

1 token = 4 characters of text

estimated monthly tokens = total characters sent and received per month / 4

billion tokens per month = estimated monthly tokens / 1,000,000,000

Example: 12,000,000,000 characters per month
  tokens   = 12,000,000,000 / 4 = 3,000,000,000
  billions = 3,000,000,000 / 1,000,000,000 = 3

To build the estimate, inventory the applications, agents, and MCP servers that will route through the gateway, and for each one approximate: requests per month, average prompt size (including system prompts and context), and average response size. Sum the character counts, divide by 4, and add headroom for growth.

The deployment profile is the licensing object that funds AI Gateway. You create it in the Customer Support Portal (not Strata Cloud Manager), from the credit pool that holds your flex credits. Creating it and finishing setup (next step) is what activates the gateway for your tenant.

1. Log in to the Palo Alto Networks Customer Support Portal.
2. In the left navigation, expand Products and select Software NGFW Credits. (Flex credits for every Software NGFW and Prisma AIRS product, AI Gateway included, come from the same pools, so this is the right page.)
3. Locate the credit pool that will fund the gateway and click its name (or its row) to open the pool page; if several are listed, open the one with the most remaining credits. Item 8 shows how many credits this profile will draw; if that number exceeds the pool's remaining credits, close the dialog without creating the profile and repeat from this item with another pool. The pool page shows total credits, the allocated and consumed meters, the expiration date, and, at the bottom, the Current Deployment Profiles table.
4. Before creating anything, scroll to the Current Deployment Profiles table. If a row with product type AI-GW and the Profile Name you chose (or planned to choose) already exists from an earlier attempt, do not create another; it already holds a credit allocation. Skip to Licensing step 3 and use that row's Finish Setup link. An AI-GW row with a different name belongs to another deployment; leave it alone and create yours. If you created a profile by mistake, open the vertical ellipsis (More Options) at the far right of its row and select Delete; Palo Alto Networks documents this control for other Software NGFW credit profiles, including Prisma AIRS ones. If the menu offers no Delete, open an admin case in the Customer Support Portal or ask your account team to remove the profile and return its credits.
5. Click Create New Profile (above the Current Deployment Profiles table).
6. In the Create Deployment Profile: STEP 1 dialog, the products are laid out in labeled columns. Find the column headed Prisma AIRS: and click AI Gateway within it so its option is selected. The dialog calls this the firewall type because the same wizard provisions Software NGFW products; AI Gateway is not a firewall, and selecting it here is correct. Click Next.
7. Complete the FORM stage:
   • Deployment — select the option matching your deployment model: Managed Service if you chose SaaS, Self Service (DIY) if you chose Hybrid. The field is informational and does not change the credit allocation or the usage limit; if undecided, choose Managed Service.
   • Profile Name — a descriptive name, for example AI-Gateway-Deployment-Profile.
   • Billion tokens per month — the billions figure you calculated in Licensing step 1.
   • Bundled — no action needed. Strata Cloud Manager Pro is bundled with AI Gateway automatically and cannot be deselected. Its cost is included in the credit estimate you see in the next item, so that estimate covers more than AI Gateway alone.
   • Notes — optional.
8. Click Calculate Estimated Cost. The dialog shows how many credits this profile will draw and how many remain available in the pool; this is the flex credit allocation.
9. Click Create Deployment Profile. The dialog closes and returns you to the Software NGFW Credits credit pool page, with your new profile listed in the Current Deployment Profiles table at the bottom.

Credit pool page with the Create New Profile button (steps 3 to 5):

STEP 1: find the Prisma AIRS column and select AI Gateway (step 6):

▶ See screenshots — the FORM stage field by field

FORM: choose the Deployment type (step 7):

FORM: name the profile (step 7):

FORM: enter the monthly token volume (step 7):

Estimated cost in credits (step 8):

Created profile row (verification):

Mapping the deployment profile to a Tenant Service Group (TSG) is the link that connects Strata Cloud Manager and the Strata Logging Service to your AI Gateway. The TSG you map becomes the AI Gateway organization. The activation form labels the TSG as the Tenant.

1. On the same Software NGFW Credits credit pool page from Licensing step 2, scroll to the Current Deployment Profiles table and click the Finish Setup link on your AI-GW profile row. If you navigated away after the last step, get back via Products → Software NGFW Credits and reopen the same pool. The Activate Subscriptions based on Deployment Profile(s) page opens.
2. Under Select Customer Support Account, choose the support account used to create the deployment profile.
3. Under Specify the Recipient, select the Tenant that will own the gateway. This is the TSG. To confirm which name is yours, open the hub with the account you used in the prerequisites above, select Common Services → Tenant Management, find the tenant you work in, and select the matching Tenant here. If only one tenant is listed, select it.
4. Under Select Region, choose your region (for example, United States - Americas). The form will not proceed without it.
5. Under Select Deployment Profile(s), check the AI Gateway profile you created (its services column reads Strata Cloud Manager Pro, AI Runtime Security Gateway). Profiles already associated with this tenant are pre-checked; leave them checked, so the selected count will read higher than one. Unchecking a pre-checked profile removes its association when you activate. Click Done.
6. Leave Data Loss Prevention and Additional Services (including Cloud Identity Engine) at their pre-populated defaults; AI Gateway uses neither. A greyed-out DLP field is expected.
7. Check Agree to the Terms and Conditions, then click Activate.

▶ See screenshots

Activation page: select the Customer Support Account (step 2):

Region is required before activation can proceed (step 4):

Locate the AI Gateway profile in the deployment profile table (step 5):

Check the AI Gateway profile and click Done; the other checked rows are existing associations, left in place on purpose (step 5):

Leave DLP and Cloud Identity Engine at their pre-populated defaults (step 6):

Agree to the terms and activate (step 7):

With licensing in place, complete setup from inside SCM:

1. Log in to Strata Cloud Manager.
2. In the left navigation, select AI Security → Home.
3. On the Home page, locate the AI Gateway card under Select a feature to begin onboarding. On a fresh tenant (status Not Started) it offers three actions: Go to Gateway Registration, Launch AI Gateway, and Deploy Hybrid. Seeing the card is all you need here; this guide installs the data plane with Helm rather than through those actions.

▶ See screenshots

The AI Security entry in the left navigation (step 2):

The AI Security menu with Home at the top and the AI Gateway section below (step 2):

The AI Gateway onboarding card with its three actions (step 3):

Eleven terms cover almost everything in this guide. Read them once and refer back when a later step uses one.

A cluster bills by the hour from the moment it is created, whether or not anything uses it. The charges do not stop when you close your laptop. Nothing in this phase is free tier on any of the three clouds once you add a second node and a load balancer.

Price the exact shape you intend to build before you build it, using the AWS calculator, the Azure calculator, or the Google Cloud calculator. Rates differ enough by region that a figure quoted here would mislead you.

This part is identical on EKS, AKS, and GKE. SERVICE_NAME is a label you choose. PORTKEY_CLIENT_AUTH and the registry credentials come from Palo Alto Networks. ORGANISATIONS_TO_SYNC is your Organisation ID from the SCM URL.

imageCredentials:
  - name: portkey-enterprise-registry-credentials
    create: true
    registry: https://index.docker.io/v1/
    username: <PROVIDED BY PALO ALTO NETWORKS>
    password: <PROVIDED BY PALO ALTO NETWORKS>

images:
  gatewayImage:
    repository: "docker.io/portkeyai/gateway_enterprise"
    pullPolicy: IfNotPresent
    tag: "2.13.0"          # chart default; check the changelog for current
  dataserviceImage:
    repository: "docker.io/portkeyai/data-service"
    pullPolicy: IfNotPresent
    tag: "1.9.0"
  redisImage:
    repository: "docker.io/redis"
    pullPolicy: IfNotPresent
    tag: "7.2-alpine"

environment:
  create: true
  secret: true
  data:
    ANALYTICS_STORE: control_plane
    SERVICE_NAME: <SERVICE_NAME>
    PORTKEY_CLIENT_AUTH: <PROVIDED BY PALO ALTO NETWORKS>
    ORGANISATIONS_TO_SYNC: <ORGANISATION_ID>

The MCP gateway is off by default. SERVER_MODE accepts "" for the AI gateway alone, "mcp" for the MCP gateway alone, and "all" for both. Leave MCP_GATEWAY_BASE_URL unset on the first install, because the load balancer that it points at does not exist yet. Set it on a second pass after step 6.

environment:
  data:
    SERVER_MODE: "all"
    MCP_PORT: "8788"
    MCP_GATEWAY_BASE_URL: "https://<MCP hostname>"   # Second pass only

The Data Service handles batch processing, fine-tuning, and log exports. Enable it only if you need one of those.

dataservice:
  name: "dataservice"
  enabled: true
  env:
    DEBUG_ENABLED: false
    SERVICE_NAME: "portkeyenterprise-dataservice"
  serviceAccount:
    create: false
    name: <SERVICE_ACCOUNT_NAME>

Two chart defaults are worth setting deliberately rather than inheriting. Both are visible in the chart's values.yaml, and neither is called out on the platform deployment pages.

The chart also defines liveness and readiness probes against /v1/health on the gateway port, so you do not need to write them. What you may want to change is the timing:

service:
  type: ClusterIP        # or LoadBalancer; never inherit the NodePort default
  port: 8787

livenessProbe:
  httpGet:
    path: /v1/health     # port omitted: the chart targets the active server port
  initialDelaySeconds: 5
  periodSeconds: 10      # chart default is 60
  timeoutSeconds: 3
  failureThreshold: 3
readinessProbe:
  httpGet:
    path: /v1/health
  initialDelaySeconds: 5
  periodSeconds: 10      # chart default is 60
  timeoutSeconds: 3
  successThreshold: 1
  failureThreshold: 3

At the chart's 60-second period and a failure threshold of 3, both probes take roughly two minutes to react. A wedged pod keeps its place in the Service endpoints and keeps taking requests for that whole window. Dropping the period to 10 seconds cuts it to about 30 seconds at a negligible cost, since /v1/health is a local check.

The chart defines no startupProbe. With liveness on the same endpoint, a pod that is slow to start gets restarted at around the two-minute mark and then restarts again on the next attempt. If your first sync with the management plane is slow, or the node is pulling the image cold, add a startupProbe on /v1/health with a high failureThreshold: Kubernetes suspends liveness until the startup probe passes, so a slow start no longer looks like a failure.

If you enabled the Data Service in step 4.4, note that it uses a different endpoint: /health on port 8081, not /v1/health on 8787. The chart already ships startup, liveness, and readiness probes for it on a 10-second period, so there is nothing to tune there.

kubectl get svc -n $namespace -o custom-columns=NAME:.metadata.name,TYPE:.spec.type,PORTS:.spec.ports[*].port,NODEPORTS:.spec.ports[*].nodePort

The chart defaults are sized to start, not to run in production. Left alone they give you one replica, no resource requests or limits, no autoscaling, no disruption budget, and a rolling upgrade that drops in-flight requests. None of this is called out on the platform deployment pages, and all of it is cheaper to set now than to diagnose later.

replicaCount: 3

resources:
  requests:
    cpu: 500m
    memory: 1Gi
  limits:
    memory: 2Gi

# Let the load balancer drain the pod before the process stops.
terminationGracePeriodSeconds: 45
lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 15"]

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

podDisruptionBudget:
  enabled: true
  minAvailable: 2

Treat the figures as a starting point rather than a sizing model. No requests-per-second guidance is published for any platform, so the CPU request and the autoscaling target both need revisiting once you have measured the customer's own traffic. Keep a CPU limit off unless you have a reason for it, because throttling a gateway under burst turns a latency problem into a timeout.

kubectl get deploy,hpa,pdb -n $namespace
kubectl get pod -n $namespace -o jsonpath='{.items[0].spec.terminationGracePeriodSeconds}{"\n"}'

helm repo add portkey-ai https://portkey-ai.github.io/helm
helm repo update

helm upgrade --install portkey-ai portkey-ai/gateway \
  -f ./values.yaml -n $namespace --create-namespace

On AKS and GKE substitute ${NAMESPACE} for $namespace, matching the variable you set in step 1.1.

kubectl get pods -n $namespace

If a pod sits in Pending, ImagePullBackOff, or CrashLoopBackOff, read the events and the logs before changing anything:

kubectl describe pod <POD_NAME> -n $namespace
kubectl logs <POD_NAME> -n $namespace --tail=100

ImagePullBackOff almost always means the registry credentials in step 4.1 are wrong or were never applied. CrashLoopBackOff with an authentication error in the logs points at the cache or log store identity from step 3.

Kubernetes permits all egress by default. If your cluster applies NetworkPolicies that restrict it, the gateway needs an explicit allowance or it will never register.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-gateway-egress
  namespace: portkeyai
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0

Narrow that CIDR to the destinations the gateway actually needs, which are the management plane endpoints, your cache, your log store, and every model provider you route to. A broad rule is the published example, not a recommendation.

The management plane endpoints are these. Allow all four before the first install, because a gateway that cannot reach them starts and then never registers, which looks like a chart problem rather than a network one.

The simplest path is over the internet. The gateway needs outbound access to two endpoints and nothing else changes in values.yaml.

• https://aigw.portkey.ai
• https://albus.portkey.ai

If you need the traffic to stay off the public internet, each cloud has a private outbound path. All three need Palo Alto Networks to allow-list your account first, so raise that request before you start.

All three platform deployment pages describe a second direction, running from the management plane inbound to your gateway over AWS PrivateLink, Azure Private Link, or GCP Private Service Connect, with IP whitelisting as the alternative. Do not build it.

Palo Alto Networks confirmed on 2026-09-27 that private link is not supported with Strata Cloud Manager today, and that the only network connectivity between a gateway and the cloud is outbound from the gateway. Those procedures describe the Portkey product, which shares the charts and the tooling but not this part of the architecture.

In practice that means three things you do not do:

• Do not create a VPC endpoint service, a Private Link Service, or a published Private Service Connect service for the gateway.
• Do not authorise arn:aws:iam::299329113195:root, or any Palo Alto Networks principal, into your account.
• Do not add 54.81.226.149, 34.200.113.35, or 44.221.117.129 to inbound-cidrs, azure-allowed-ip-ranges, spec.loadBalancerSourceRanges, or any security group rule. Your load balancer admits your own applications and nothing else.

A gateway registers and stays visible in Strata Cloud Manager on outbound connectivity alone. If it does not appear, the cause is in step 7.1, 7.2, or 7.3.

kubectl port-forward <POD_NAME> -n $namespace 9000:8787

In a second terminal:

OPENAI_API_KEY=<OPENAI_API_KEY>
PORTKEY_API_KEY=<GATEWAY_API_KEY>       # Created in Strata Cloud Manager

curl 'http://localhost:9000/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is a fractal?"}]
  }'

curl 'http://<LB_ADDRESS>:<LISTENER_PORT>/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is a fractal?"}]
  }'

On the LoadBalancer path the listener port is 80. All three platform pages set service.port: 80 against containerPort: 8787, and the AWS and GCP pages label that line as the load balancer listener port. On the Ingress path the Service is ClusterIP on 8787 and the listener comes from the Ingress and its annotations instead. Confirm against the object you actually created, because a values file you have edited governs over the vendor default.

Open Strata Cloud Manager and go to Logs. Your test request appears there, and what you see when you select the entry depends on the log store decision taken at kickoff.

Run this before handover, not after. It is the only check that proves the availability settings in step 4.6 actually hold, and it is the failure a customer notices first because it shows up as intermittent errors during a routine maintenance window.

1. Start a continuous load against the gateway from outside the cluster, through the same ingress real traffic uses. A request every second for a few minutes is enough.
2. While it runs, trigger a rolling upgrade: helm upgrade with the same chart version is sufficient, since the point is the pod replacement rather than the version change.
3. Count the failures.

# Terminal 1: continuous load, counting non-200 responses
while true; do
  code=$(curl -sS -o /dev/null -w '%{http_code}' \
    -H "Authorization: Bearer $GATEWAY_KEY" \
    https://<GATEWAY_HOST>/v1/health)
  [ "$code" = "200" ] || echo "$(date +%T) FAIL $code"
  sleep 1
done

# Terminal 2: force a pod replacement
kubectl rollout restart deploy/<gateway-deployment> -n $namespace
kubectl rollout status  deploy/<gateway-deployment> -n $namespace

The gateway pods are stateless, so scaling is a replica count. Set replicaCount in values.yaml and re-run the install, or attach a HorizontalPodAutoscaler against CPU. Keep replicas spread across zones with a pod topology spread constraint so that a zone failure does not take every replica.

No requests-per-second figure is published for any platform, so size from your own load test rather than from the documentation.

helm repo update
helm upgrade --install portkey-ai portkey-ai/gateway \
  -f ./values.yaml -n $namespace

There is no published compatibility matrix between chart versions and gateway image versions, and no tested rollback procedure. Pin your image tag, upgrade in a non-production cluster first, and keep the previous values.yaml so that helm rollback has something consistent to return to.

helm uninstall portkey-ai --namespace $namespace

helm uninstall removes the release. It does not remove the namespace, the cloud identity from step 3, the cache, the log bucket, or a load balancer created outside the chart. Remove those in the reverse order you created them, and delete only what this deployment created.

These questions come up in the field and the published material does not currently answer them. Raise them with the product team rather than inferring an answer, and treat anything below as unresolved when writing a customer commitment.

• Outbound private connectivity — whether the Portkey PrivateLink, Private Link and Private Service Connect endpoint services in step 7.3 accept a gateway managed by Strata Cloud Manager. The procedures are published by Portkey and target Portkey's own control plane; Palo Alto Networks has published no equivalent, and confirmed on 2026-09-27 that the inbound direction is not supported with Strata Cloud Manager. Confirm before designing around it or costing it.
• Throughput sizing — no requests-per-second figures for any of the three platforms. The published CPU and memory numbers are minimums to run, not a capacity model.
• Image and chart compatibility — versioned tags are published and the chart pins its own defaults, so pinning is straightforward (see step 4). What remains unpublished is whether the chart's pinned pairing is the supported pairing, and whether any matrix covers running a newer gateway image against an older chart.
• Upgrade and rollback — no supported upgrade path and no tested rollback procedure.
• TLS policy on ingress — the mechanism is settled: the chart exposes ingress.tls and each platform has a documented certificate source, all covered in step 6. What remains unpublished is the policy: no recommended cipher suite, no minimum TLS version, and no statement on whether the gateway expects TLS to terminate at the load balancer or pass through to the pod. Every vendor example still serves plain HTTP.
• Region placement and data residency — Strata Cloud Manager runs in the Americas only today, with other regions planned, and prompt bodies land there regardless of the log store setting. Retention and custody for that store are still unpublished.
• Air-gapped deployment — no documented configuration for a cluster with no path to portkey.ai.
• TLS inspection — behavior of the outbound links through an intercepting proxy is not described, and streaming through one is untested.
• Private outbound regions — the private endpoint for each cloud lives in one fixed region, and no other region is offered. Cross-region access works but adds a hop and a cost.

This guide is derived from the published self-hosting documentation for the three Kubernetes platforms.

• Amazon EKS — self-hosting/hybrid-deployments/aws/eks
• Azure AKS — self-hosting/hybrid-deployments/azure/aks
• Google GKE — self-hosting/hybrid-deployments/gcp
• Architecture — self-hosting/hybrid-deployments/architecture

The doc pages are published twice, on docs.portkey.ai/docs/aigw/ and on portkey.ai/docs/, and the two versions differ. See SOURCES.md in this directory for which to prefer and where the local mirrors are.