Skip to main content

AI Gateway for Cursor

Cursor's custom-inference setting accepts only a public HTTPS base URL, publishes no egress IP list to allowlist, and sends no credential other than the API key you paste into its settings.

This deployment turns those constraints into the design. The AI Gateway is the public endpoint: it authenticates the key Cursor sends, runs your guardrail pipeline, and forwards to a LiteLLM proxy that never gets a public listener.

Cursor ──HTTPS──► TLS edge ──► Levo AI Gateway ──► LiteLLM ──► provider
                  (Caddy or     authenticate      private,
                   Cloudflare)  + inspect         no host port

Cursor's key authenticates the developer to the gateway and never travels upstream — the gateway strips it and inserts LiteLLM's own master key. You can therefore issue one key per developer without provisioning anything in LiteLLM.

Setup​

Prerequisites​

  • A fresh Ubuntu, Debian, or RHEL VM with root or sudo access
  • An API key for your LLM provider (for example OPENAI_API_KEY or ANTHROPIC_API_KEY)
  • One of the following for TLS:
    • a DNS A record pointing at the VM, plus inbound ports 80 and 443, or
    • no DNS at all, in which case the installer uses a Cloudflare quick tunnel

Docker is installed automatically if it is missing.

1. Download the Installer​

git clone https://github.com/levoai/aigateway.git
cd aigateway/examples/cursor-litellm

2. Run the Installer​

Choose the option that matches your environment. Use sudo -E so that the provider keys you export survive into the script.

A. Your own DNS name — automatic Let's Encrypt certificate (recommended)

export OPENAI_API_KEY=sk-...
sudo -E ./setup.sh --domain llm.example.com --email you@example.com

B. No DNS record and no inbound ports — Cloudflare quick tunnel

export OPENAI_API_KEY=sk-...
sudo -E ./setup.sh --tunnel

C. Front a LiteLLM you already run privately

sudo ./setup.sh --domain llm.example.com --email you@example.com \
                --litellm-url litellm.internal:4000 \
                --litellm-key sk-your-litellm-master-key

D. Bundled LiteLLM, but a different OpenAI-compatible upstream

Use this when the upstream is something other than the OpenAI or Anthropic APIs — OpenRouter, OpenCode Zen, or a self-hosted proxy.

export CUSTOM_API_BASE=https://opencode.ai/zen/v1
export CUSTOM_API_KEY=sk-...
export CUSTOM_MODEL_NAME=code-supernova   # the name you will type into Cursor
sudo -E ./setup.sh --tunnel

The installer generates the Cursor-facing API key, renders all configuration into /opt/levo-cursor-gateway, starts the stack, runs a set of smoke tests, and prints the exact values to paste into Cursor.

info

Re-running the installer is safe. Generated secrets are reused unless you pass --regenerate-keys, and only the services whose configuration actually changed are restarted — a re-run that changes nothing restarts nothing. Pass --no-start to render the configuration without Docker, which is useful for reviewing what would be deployed.

3. Confirm the Smoke Tests Passed​

The installer asserts the security properties rather than simply reporting that the stack started:

RequestExpected
No Authorization header401
Wrong key401
Valid key, benign prompt200, proxied to LiteLLM
Valid key, prompt containing an AWS access key403, blocked before LiteLLM
Valid key, prompt containing an SSN200, SSN masked before the model sees it

Mismatches are reported as warnings rather than aborting, so a provider-side failure such as a bad upstream key does not hide the authentication results.

Please contact support@levo.ai if the first two checks return anything other than 401.

4. Point Cursor at the Gateway​

In Cursor, go to Settings → Models → OpenAI API Key and fill in the values the installer printed:

FieldValue
Base URLhttps://llm.example.com/v1
API Keythe generated sk-levo-… key

Click Verify, enable the key, and select a model that the bundled LiteLLM exposes — gpt-4o, gpt-4o-mini, or gpt-4.1. If you used option D above, add your CUSTOM_MODEL_NAME through Cursor's Add Model field, since it will not appear in the built-in dropdown.

note

Only OpenAI-family models route through an OpenAI base-URL override. Models from other families will not reach the gateway.

5. Watch Traffic in Levo​

Set the following before running the installer to connect the gateway to the Levo platform:

export LEVOAI_BASE_URL=https://api.levo.ai
export LEVOAI_AUTH_KEY=<Authorization Key>
export LEVOAI_ENVIRONMENT_ID=<Environment ID>

For accounts on the India domain, replace https://api.levo.ai with https://api.india-1.levo.ai.

Refer to Install AI Gateway for how to obtain these values. With them set, the gateway polls AI Policies every 30 seconds and hot-swaps the guardrail engine, and the static rules in the generated gateway.yaml are superseded by the policies you manage in the dashboard.

  • Login to Levo.ai.
  • Navigate to AI Gateway → Configuration.

To follow requests as they arrive:

cd /opt/levo-cursor-gateway && docker compose logs -f gateway

Add a Developer​

Each developer should get their own key, so that each has a separate audit trail. Add an entry under policies.apiKey.keys in /opt/levo-cursor-gateway/gateway.yaml with distinct metadata, then restart the gateway:

cd /opt/levo-cursor-gateway && docker compose restart gateway

apiKey.key and its metadata are available as CEL variables, so route-level authorization rules can act on a specific developer's key.

Configuration Settings That Must Not Change​

Four settings in the generated configuration are load-bearing. Changing them produces a configuration that looks complete but silently stops protecting the endpoint.

danger
  • apiKey.mode: strict — the default is optional, which allows requests carrying no credential at all. On a public listener that is an open relay into LiteLLM.
  • action: allow_with_inspection, not allow — request-body inspection only runs on allow_with_inspection. With allow, the guards stay configured but never execute on the request side.
  • The "*": passthrough route entry — for non-Bedrock providers with no explicit ai.routes, every path defaults to Completions, so Cursor's /v1/models probe would be parsed as a chat-completions body and fail.
  • The forward proxy stays off — with it on, a request matching no route is proxied to its own Host header, which is an open proxy for anyone who finds the endpoint.

Guardrails​

The generated policy uses secrets and pii_regex, which are pure regular expressions. No ML models are downloaded, so the VM stays small.

To run the ML content-safety scanners — prompt injection, toxicity, and similar — add an llm_bastion guard and mount the models at LEVOAI_MODELS_BASE_PATH. Note the higher resource requirements for ML guardrails in Install AI Gateway.

Known Limits​

  • LLM request bodies are capped at 2 MiB. The limit is fixed in the LLM path and the maxBufferSize setting does not raise it. Cursor agent-mode requests carrying large repository context can exceed it and be rejected.
  • Identity policies cannot match on the API key. No api_key principal is extracted at the identity boundary, so identity-policy rules never see it. Use route-level authorization CEL on apiKey.* instead.
  • localRateLimit is a per-route bucket, not per-key. Per-key limits require remoteRateLimit descriptors.
  • A Cloudflare quick-tunnel URL is ephemeral. It changes on every restart, so use --domain for anything beyond a trial.

Troubleshooting​

Smoke Tests Return HTTP 000​

The certificate is most likely still being issued. Check the TLS edge:

cd /opt/levo-cursor-gateway && docker compose logs caddy | tail -30

Confirm that your DNS A record points at this VM and that inbound ports 80 and 443 are open.

Cursor's Verify Button Fails​

Confirm that the Base URL ends in /v1 and that the API key is the generated sk-levo-… value rather than your provider key. Then check that the gateway is reachable from outside the VM:

curl -sS -o /dev/null -w '%{http_code}\n' https://llm.example.com/v1/models

Requests Are Rejected After Working Previously​

If you are using --tunnel, the quick-tunnel URL changes on every restart. Re-run the installer and paste the new Base URL into Cursor.

Please contact support@levo.ai if requests continue to fail after these checks.

Stop or Remove the Stack​

cd /opt/levo-cursor-gateway && docker compose down

Generated secrets live in /opt/levo-cursor-gateway/.env (mode 600).

Was this page helpful?