AI models

The model addon gives a project an OpenAI-compatible API to an open model that nesly runs on its own GPUs, in its own region. Your app needs no GPU and no CUDA: it calls an HTTP endpoint with the SDK it already uses. Prompts and answers are not logged; only token counts are kept, for the bill.

Add a model

nesly models list
nesly addon add model qwen3-8b --yes --json
nesly deploy --yes --json

nesly models list shows every model, whether it can be added now, how loaded it is right now (stock: HIGH, MEDIUM, LOW, NONE) and its price per million tokens in your account's currency. The new variables reach the app on its next deploy.

What your app gets

• NESLY_AI_BASE_URL — The OpenAI-compatible endpoint, https://api.nesly.net/ai/v1
• NESLY_AI_API_KEY — The project's key, sent as a bearer token. Keep it server-side.
• NESLY_AI_MODEL — The model you added, e.g. qwen3-8b. Send it as "model".
• OPENAI_BASE_URL, OPENAI_API_KEY — The same URL and key, so an OpenAI SDK works with no arguments. Left out if the project sets any OPENAI_* itself: yours wins, and our key is never sent to a URL you chose.

Call it

JavaScript (npm install openai):

import OpenAI from "openai"

const client = new OpenAI() // reads OPENAI_BASE_URL and OPENAI_API_KEY
const reply = await client.chat.completions.create({
  model: process.env.NESLY_AI_MODEL,
  messages: [{ role: "user", content: "Summarise this ticket in one line." }],
})
console.log(reply.choices[0].message.content)

Python (pip install openai):

import os
from openai import OpenAI

client = OpenAI(base_url=os.environ["NESLY_AI_BASE_URL"], api_key=os.environ["NESLY_AI_API_KEY"])
reply = client.chat.completions.create(
    model=os.environ["NESLY_AI_MODEL"],
    messages=[{"role": "user", "content": "Summarise this ticket in one line."}],
)
print(reply.choices[0].message.content)

Any client that speaks the OpenAI API works the same way: LangChain, LlamaIndex, the Vercel AI SDK, or plain HTTP to /chat/completions, /completions and /models.

Streaming

Pass stream: true and tokens arrive as server-sent events, as they are written. Usage is counted either way; you get the final usage chunk only if you ask for it with stream_options: { include_usage: true }. A stream that is cut before it finishes is not charged.

const stream = await client.chat.completions.create({
  model: process.env.NESLY_AI_MODEL,
  messages: [{ role: "user", content: "Write a haiku about deploys." }],
  stream: true,
})
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "")

Price

Per million tokens: input $0.10, output $0.40 (USD list, no VAT; the TRY list is on the pricing page). AI models are available on the Pro plan; the free plan does not include them. On Pro every token is metered: the monthly credit covers it first, and it counts toward your spend cap. A request that fails is not charged. nesly usage --json shows the tokens used.

Models that are not served yet

Bigger models are in the catalog with access request. They have no price and cannot be added; asking for one costs nothing and promises no date. We add a model when enough accounts ask for it.

nesly models request llama-3.3-70b --monthly-tokens 50000000 --json

Errors

Errors have the OpenAI shape, so SDKs show them as they are: {"error": {"code", "message"}}.

• 401 invalid_api_key — Missing or wrong key. A rotated key, or one from a removed addon, stops working at once.
• 403 model_not_available — The model is listed but not served yet. Ask for it with models request.
• 404 model_not_found — No such model id. models list shows the catalog.
• 403 plan_required — Free plan: AI models are available on the Pro plan. The key is refused, nothing is charged; upgrade in the console.
• 429 spend_cap_not_set — Paid plan: the month's included tokens are used and no spend cap is set. Set one in the usage budget and requests go through as metered usage.
• 429 spend_cap_reached — Paid plan: this month's usage reached your spend cap. Raise it in the usage budget.
• 429 rate_limit_exceeded / too_many_concurrent — Too many requests in a minute, or in flight at once, from one account. Every 429 carries Retry-After.
• 403 account_suspended — The account is suspended; model requests are refused until it is reinstated.
• 502 model_unavailable — The model did not answer. Nothing was charged; retry.

Keys

nesly addon rotate model --yes issues a new key; the old one stops working at once and the model stays. nesly addon rm model --yes removes the addon and its key. Preview deployments of the project use the same key, so an AI feature can be tried on a preview before it goes live; their tokens count on the same account.

From an AI agent (MCP)

• list_models — The catalog: availability, live stock (HIGH, MEDIUM, LOW, NONE), price per million tokens in the account's currency.
• add_addon type=model model=<id> — Adds the model to a project. Adding it again with another id switches the model and keeps the key.
• request_model_access — Asks for a model that is not served yet. No charge, no date.

See for AI agents and addons.