Skip to main content

AI Gateway via Docker

Setup​

Prerequisites​

Before installing the Levo AI Gateway via Docker, ensure you have:

  • Met the system requirements listed in Install AI Gateway
  • Docker Engine 20.10.0 or higher
  • Admin privileges on the Docker host
  • Your Environment ID and Authorization Key from the steps in Install AI Gateway

1. Pull the AI Gateway Image​

docker pull levoai/ai-gateway:latest

2. Run the AI Gateway​

docker run -d \
  --name levoai-aigateway \
  --restart unless-stopped \
  -p 8080:8080 \
  -e LEVOAI_BASE_URL="https://api.levo.ai" \
  -e LEVO_ENVIRONMENT_ID="<Environment ID>" \
  -e LEVOAI_AUTH_KEY="<Authorization Key>" \
  levoai/ai-gateway:latest
info

If your Levo account is on the India domain (app.india-1.levo.ai), set LEVOAI_BASE_URL to https://api.india-1.levo.ai:

docker run -d \
  --name levoai-aigateway \
  --restart unless-stopped \
  -p 8080:8080 \
  -e LEVOAI_BASE_URL="https://api.india-1.levo.ai" \
  -e LEVO_ENVIRONMENT_ID="<Environment ID>" \
  -e LEVOAI_AUTH_KEY="<Authorization Key>" \
  levoai/ai-gateway:latest

3. Verify the Installation​

Check that the container is running:

docker ps -f name=levoai-aigateway

If the AI Gateway is healthy, you should see output similar to the following:

CONTAINER ID   IMAGE                       COMMAND                CREATED         STATUS         PORTS                    NAMES
a3f1c2b9d4e7   levoai/ai-gateway:latest   "/usr/local/bin/..."   1 minute ago    Up 1 minute    0.0.0.0:8080->8080/tcp   levoai-aigateway

Verify connectivity to Levo.ai by running:

docker logs levoai-aigateway | grep -Ei "connected|polling|saas"

If connectivity is healthy, you should see output similar to the following:

INFO aigateway: Connected to Levo SaaS. Polling for configuration every 60s.

Please contact support@levo.ai if you notice health or connectivity errors.

4. Configure Routes in the Levo Dashboard​

The AI Gateway polls the Levo platform every 60 seconds and automatically applies the latest configuration. Routes, LLM backends, and guardrail policies are all managed from the dashboard.

Refer to step 5 in AI Gateway on Kubernetes for configuration instructions and an example YAML — the configuration format is identical for Docker and Kubernetes deployments.

5. Point Your Application at the Gateway​

Update your application to send LLM requests to the gateway instead of directly to the LLM provider. Replace the LLM provider's base URL with http://<Docker-host-IP>:8080/v1.

For example, if you are using the OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    base_url="http://<Docker-host-IP>:8080/v1",
    api_key="<Your OpenAI API Key>",
)

AI Gateway Lifecycle Management​

Upgrade AI Gateway​

docker pull levoai/ai-gateway:latest

docker stop levoai-aigateway && docker rm levoai-aigateway

Then re-run the docker run command from Step 2.

Stop AI Gateway​

docker stop levoai-aigateway

Uninstall AI Gateway​

docker stop levoai-aigateway && docker rm levoai-aigateway

Troubleshooting​

Container Fails to Start​

Check the container logs for errors:

docker logs levoai-aigateway

Common causes:

  • Invalid credentials: Verify your LEVO_ENVIRONMENT_ID and LEVOAI_AUTH_KEY are correct and not expired.
  • Network connectivity: Confirm the host can reach api.levo.ai (or api.india-1.levo.ai) on port 443.

Enable Debug Logging​

Stop and re-run the container with RUST_LOG=debug:

docker stop levoai-aigateway && docker rm levoai-aigateway

docker run -d \
  --name levoai-aigateway \
  --restart unless-stopped \
  -p 8080:8080 \
  -e LEVOAI_BASE_URL="https://api.levo.ai" \
  -e LEVO_ENVIRONMENT_ID="<Environment ID>" \
  -e LEVOAI_AUTH_KEY="<Authorization Key>" \
  -e RUST_LOG="debug" \
  levoai/ai-gateway:latest

Need Help?​

For further assistance, please reach out to support@levo.ai.

Was this page helpful?