Skip to main content

AI Gateway on Kubernetes

Setup​

Prerequisites​

Before installing the Levo AI Gateway on Kubernetes, ensure you have:

  • Kubernetes version >= v1.25.0 (required by the chart — uses autoscaling/v2 and other APIs introduced in 1.25)
  • Helm v3 installed and configured
  • The Kubernetes cluster API endpoint is reachable from the machine running Helm
  • kubectl access to the cluster with cluster-admin permissions
  • An active Levo.ai account
  • Your Environment ID and Refresh Token from the Levo platform

1. Add the Levo Helm Repository​

helm repo add levoai https://charts.levo.ai && helm repo update levoai

2. Install the AI Gateway​

Create a values file with your credentials (avoids secrets appearing in shell history):

cat > levoai-values.yaml <<EOF
config:
  saas:
    environmentId: "<Environment ID>"
    refreshToken: "<Refresh Token>"
EOF
helm upgrade --install -n levoai --create-namespace \
  -f levoai-values.yaml \
  levoai-aigateway levoai/aigateway
info

You may need to set a different Levo base URL if your SaaS/dashboard account is on the India domain.

For example, if you are accessing the Levo dashboard with app.india-1.levo.ai, add the following to your values file:

config:
  saas:
    url: "https://api.india-1.levo.ai"
    environmentId: "<Environment ID>"
    refreshToken: "<Refresh Token>"

3. Verify connectivity with Levo.ai​

Wait a couple of minutes after installation, and check that the pod is running:

kubectl -n levoai get pods

If the AI Gateway is healthy, you should see output similar to the following:

NAME                                 READY   STATUS    RESTARTS   AGE
levoai-aigateway-6d9b8c7f4-xk2np     1/1     Running   0          2m

Verify connectivity to Levo.ai by running:

kubectl -n levoai logs deployment/levoai-aigateway | grep -Ei "saas|connected|polling"

Please contact support@levo.ai if you notice health/connectivity related errors.

4. Note the Gateway Endpoint​

The AI Gateway service is of type LoadBalancer on port 8080 by default. Retrieve the external address:

kubectl -n levoai get svc levoai-aigateway

The admin API is available on port 15000. You can verify it with:

kubectl -n levoai port-forward svc/levoai-aigateway 15000:15000
curl http://localhost:15000/healthz

AI Gateway Lifecycle Management​

Upgrade AI Gateway​

helm repo update levoai

helm upgrade -n levoai \
  levoai-aigateway levoai/aigateway

Uninstall AI Gateway​

helm uninstall levoai-aigateway -n levoai

Troubleshooting​

Check Pod Logs​

kubectl -n levoai logs -f deployment/levoai-aigateway

Enable Debug Logging​

Add the following helm option to enable debug logging:

helm upgrade --install -n levoai --create-namespace \
  --set extraEnv.RUST_LOG="debug" \
  levoai-aigateway levoai/aigateway

Need Help?​

For further assistance, please reach out to support@levo.ai.