# AI models

Call an open model from your app with any OpenAI SDK. One nesly command adds it; the URL, key and model arrive as environment variables. Priced per token.

The model addon gives a project an OpenAI-compatible API to an open model that nesly runs on its own GPUs, in its own region. Your app needs no GPU and no CUDA: it calls an HTTP endpoint with the SDK it already uses. Prompts and answers are not logged; only token counts are kept, for the bill.

## Add a model

```
nesly models list
nesly addon add model qwen3-8b --yes --json
nesly deploy --yes --json
```

`nesly models list` shows every model, whether it can be added now, how loaded it is right now (stock: HIGH, MEDIUM, LOW, NONE) and its price per million tokens in your account's currency. The new variables reach the app on its next deploy.

## What your app gets

-   `NESLY_AI_BASE_URL` — The OpenAI-compatible endpoint, https://api.nesly.net/ai/v1
-   `NESLY_AI_API_KEY` — The project's key, sent as a bearer token. Keep it server-side.
-   `NESLY_AI_MODEL` — The model you added, e.g. qwen3-8b. Send it as "model".
-   `OPENAI_BASE_URL, OPENAI_API_KEY` — The same URL and key, so an OpenAI SDK works with no arguments. Left out if the project sets any OPENAI\_\* itself: yours wins, and our key is never sent to a URL you chose.

## Call it

JavaScript (`npm install openai`):

```
import OpenAI from "openai"

const client = new OpenAI() // reads OPENAI_BASE_URL and OPENAI_API_KEY
const reply = await client.chat.completions.create({
  model: process.env.NESLY_AI_MODEL,
  messages: [{ role: "user", content: "Summarise this ticket in one line." }],
})
console.log(reply.choices[0].message.content)
```

Python (`pip install openai`):

```
import os
from openai import OpenAI

client = OpenAI(base_url=os.environ["NESLY_AI_BASE_URL"], api_key=os.environ["NESLY_AI_API_KEY"])
reply = client.chat.completions.create(
    model=os.environ["NESLY_AI_MODEL"],
    messages=[{"role": "user", "content": "Summarise this ticket in one line."}],
)
print(reply.choices[0].message.content)
```

Any client that speaks the OpenAI API works the same way: LangChain, LlamaIndex, the Vercel AI SDK, or plain HTTP to `/chat/completions`, `/completions` and `/models`.

## Streaming

Pass `stream: true` and tokens arrive as server-sent events, as they are written. Usage is counted either way; you get the final usage chunk only if you ask for it with `stream_options: { include_usage: true }`. A stream that is cut before it finishes is not charged.

```
const stream = await client.chat.completions.create({
  model: process.env.NESLY_AI_MODEL,
  messages: [{ role: "user", content: "Write a haiku about deploys." }],
  stream: true,
})
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "")
```

## Price

Per million tokens: input $0.10, output $0.40 (USD list, no VAT; the TRY list is on the [pricing page](/pricing/)). AI models are available on the Pro plan; the free plan does not include them. On Pro every token is metered: the monthly credit covers it first, and it counts toward your spend cap. A request that fails is not charged. `nesly usage --json` shows the tokens used.

## Models that are not served yet

Bigger models are in the catalog with access `request`. They have no price and cannot be added; asking for one costs nothing and promises no date. We add a model when enough accounts ask for it.

```
nesly models request llama-3.3-70b --monthly-tokens 50000000 --json
```

## Errors

Errors have the OpenAI shape, so SDKs show them as they are: `{"error": {"code", "message"}}`.

-   `401 invalid_api_key` — Missing or wrong key. A rotated key, or one from a removed addon, stops working at once.
-   `403 model_not_available` — The model is listed but not served yet. Ask for it with models request.
-   `404 model_not_found` — No such model id. models list shows the catalog.
-   `403 plan_required` — Free plan: AI models are available on the Pro plan. The key is refused, nothing is charged; upgrade in the console.
-   `429 spend_cap_not_set` — Paid plan: the month's included tokens are used and no spend cap is set. Set one in the usage budget and requests go through as metered usage.
-   `429 spend_cap_reached` — Paid plan: this month's usage reached your spend cap. Raise it in the usage budget.
-   `429 rate_limit_exceeded / too_many_concurrent` — Too many requests in a minute, or in flight at once, from one account. Every 429 carries Retry-After.
-   `403 account_suspended` — The account is suspended; model requests are refused until it is reinstated.
-   `502 model_unavailable` — The model did not answer. Nothing was charged; retry.

## Keys

`nesly addon rotate model --yes` issues a new key; the old one stops working at once and the model stays. `nesly addon rm model --yes` removes the addon and its key. Preview deployments of the project use the same key, so an AI feature can be tried on a preview before it goes live; their tokens count on the same account.

## From an AI agent (MCP)

-   `list_models` — The catalog: availability, live stock (HIGH, MEDIUM, LOW, NONE), price per million tokens in the account's currency.
-   `add_addon type=model model=<id>` — Adds the model to a project. Adding it again with another id switches the model and keeps the key.
-   `request_model_access` — Asks for a model that is not served yet. No charge, no date.

See [for AI agents](/docs/agents/) and [addons](/docs/addons/).

---
Canonical: https://nesly.net/docs/ai-models/
Markdown: Accept: text/markdown on the HTML URL, or replace the trailing slash with .md