Skip to main content

AI Firewall on Kubernetes

Setup​

Prerequisites​

Before installing the Levo AI Firewall on Kubernetes, ensure you have:

  • Kubernetes version >= v1.18.0
  • Helm v3 installed and configured
  • The Kubernetes cluster API endpoint is reachable from the machine running Helm
  • kubectl access to the cluster with cluster-admin permissions

1. Add the Levo Helm Repository​

helm repo add levoai https://charts.levo.ai && helm repo update levoai

2. Install the AI Firewall​

helm upgrade --install -n levoai --create-namespace \
  --set config.vigil.upstream.address="<Upstream LLM Host:Port>" \
  --set config.vigil.upstream.tls=true \
  --set config.vigil.upstream.sni="<Upstream LLM Host>" \
  levoai-ai-firewall levoai/ai-firewall

For example, to proxy traffic to OpenAI:

helm upgrade --install -n levoai --create-namespace \
  --set config.vigil.upstream.address="api.openai.com:443" \
  --set config.vigil.upstream.tls=true \
  --set config.vigil.upstream.sni="api.openai.com" \
  levoai-ai-firewall levoai/ai-firewall

3. Verify the Installation​

Wait a couple of minutes after installation, and check that the pod is running:

kubectl -n levoai get pods

If the AI Firewall is healthy, you should see output similar to the following:

NAME                                    READY   STATUS    RESTARTS   AGE
levoai-ai-firewall-7c8b9f6d4-p9mnz      1/1     Running   0          2m

Verify the health endpoint:

kubectl -n levoai port-forward svc/levoai-ai-firewall 8080:8080
curl http://localhost:8080/health

Please contact support@levo.ai if you notice health/connectivity related errors.

AI Firewall Lifecycle Management​

Upgrade AI Firewall​

helm repo update levoai

helm upgrade -n levoai \
  levoai-ai-firewall levoai/ai-firewall

Uninstall AI Firewall​

helm uninstall levoai-ai-firewall -n levoai

Troubleshooting​

Check Pod Logs​

kubectl -n levoai logs -f deployment/levoai-ai-firewall

Enable Debug Logging​

Add the following helm option to enable debug logging:

helm upgrade --install -n levoai --create-namespace \
  --set config.vigil.observability.log_level="debug" \
  levoai-ai-firewall levoai/ai-firewall

Need Help?​

For further assistance, please reach out to support@levo.ai.