AI tools

Call OpenAI, Anthropic or Google from your app through a named AI tool, priced in credits.

A tool's definition lives on the server: its instructions, prompt template, inputs, model, caps, credit price and the access it requires. Your provider key stays on the server and never reaches the browser. A signed-in person's app sends the tool's name and inputs. Gemmein composes the request, spends the tool's credits, adds your key and streams the answer back.

Call a tool from your app

A tool file at gemmein/ai/tools/deep-research.json:

{
  "label": "Deep Research",
  "provider": "openai",
  "model": "gpt-4o",
  "credits": 20,
  "requires": "access:pro-max",
  "instructions": "You are a careful research assistant. Answer with sources.",
  "promptTemplate": "Research this for a {{audience}} reader:\n\n{{question}}",
  "inputs": [
    { "name": "question", "type": "text", "required": true, "maxLength": 2000 },
    { "name": "audience", "type": "text" }
  ],
  "bounds": { "maxOutputTokens": 4000 }
}

The app never sends a prompt or a provider request. It names the tool and passes the inputs the tool declares. Gemmein composes the request in the provider's own format, from the instructions and the template on the server. The answer comes back in the provider's own shape, so a streaming call streams:

const res = await g.ai.run("deep-research", { question: text, audience: "beginner" }, { stream: true })

for await (const chunk of res.body) render(chunk)   // the provider's SSE, byte for byte
res.headers.get("x-gemmein-credits-remaining")     // the balance after this call

For a non-stream answer as one string, whichever provider answered, and for the person's own history:

const answer = await g.ai.runText("deep-research", { question: text })
const { calls } = await g.ai.calls()   // what they ran, when, what it cost, how it ended

g.ai.runText reads choices[0].message.content from OpenAI, joins content[].text from Anthropic, and joins candidates[0].content.parts[].text from Google.

A provider's non-2xx answer is returned as it came by g.ai.run, and is not thrown. g.ai.runText, like g.ai.text, throws it as provider_error with the provider's status and message. A refusal by Gemmein throws GemmeinError with one of the codes below. An unknown input, a missing required one, a wrong type or a value over its cap is refused by name before anything is spent (invalid_inputs).

What the person sees:

Tool kinds and providers

A tool is one of two kinds. A chat tool answers now. A job tool (generate or transcribe) answers with a run. A provider serves every kind its key sells:

An OpenAI key makes chat, image, speech, transcription and video tools. A Google key makes chat, image, video and audio-transcription tools. external is your own pipeline.

When your own server makes the model call, or the call is neither chat nor a job (such as embeddings), call the provider directly with your key. Ask Gemmein's server methods in the SDK reference who the person is and what they hold. If your provider is not on the list, write to hello@gemmein.com.

Files a tool takes

A tool says which kinds of file a person may hand it: image, audio, video and document. Each is a tick in the tool's file ("accepts": ["image", "document"]) and on the dashboard's AI page. A file input takes the ref of a file the person already uploaded, to any collection: g.ai.run("describe-photo", { photo: upload.ref }).

Gemmein checks the file's kind, the tool's model and the file's size before any credit moves, then hands the file to the provider. Most providers fetch a short-lived signed link made for that one call. Small files ride inline. Gemini files over 14 MB stream from storage into Gemini's own file store. A file is never sent as a public link, never converted, and never held whole in Gemmein's memory.

A tick is offered only where the tool's provider and model read that kind:

npx gemmein dev and npx gemmein sync refuse a tick the model does not read and name the models that do. A file may be up to 500 MB of video, 100 MB of audio, or 25 MB of image or document, or the provider's own limit when lower. "inputMaxMb": { "video": 200 } in the tool's file sets a lower limit for one tool; the AI page has no size field. A tool that takes files is priced per run: a flat credit price per call, held and charged as that price whatever the file's size. A video input can cost more on your own provider key. A tool charged per token that ticks a kind is refused at gemmein dev, gemmein sync and on the AI page: "A tool that takes files is priced per run — set a credit price per call, and lower its file size limit if you want to cap your cost." Text-only tools keep per-token pricing. Files a person hands a tool count toward your app's storage; video fills it fastest, and you can add 100 GB blocks under Usage. With ticks set, a file input takes only a Gemmein file ref, never a link. A tool created before ticks takes exactly the files it took before.

Create a tool

Create a tool on the dashboard's AI page, or have your AI write a file at gemmein/ai/tools/<name>.json. npx gemmein sync pushes it to Development. npx gemmein sync --live pushes it to your live app with a sync key. Create the sync key under Live → Secret keys → Sync key with your sign-in code. It works for one hour, is shown once and is never saved.

The file owns the implementation on every sync. After creation, the dashboard owns the commerce: label, credits, unit and per, access, on/off, record calls, and a transcribe tool's delete-after-run. Every field is in the field table.

A tool can be defined before its provider key exists. Runs are refused ai_not_configured until the key is added on the AI page, and the dashboard shows the tool as waiting on a key until then. A tool's name is fixed once created, because it is what the app calls.

Model aliases

Provider keys

The AI page holds one key per provider, per environment. Paste it, make a test call, and it is set. A key is 8 to 512 printable characters. Replace it by pasting again. A job provider's key is pasted like a chat key; its test call is one authenticated read that spends nothing.

The key is write-only. No response, no audit record and no page shows more than its last four characters. The AI page shows the last four characters of a key of 16 or more characters (a shorter key shows dots) and the last call. A provider that echoes the key in a refusal reaches the app as ***<hint>.

Removing a key is a step-up action, and your app's AI features stop the moment it is removed.

Optionally, list up to 20 models the app may call; the test call uses the first. Any other model answers model_not_allowed. The model name is the provider's, 1 to 80 characters of letters, digits, dots, colons, underscores and hyphens. Google needs one; OpenAI and Anthropic refuse a missing one themselves.

Pricing

A tool holds one credit price. For a price that depends on the plan, make two tools, each with its own access requirement. How credits are bought, granted and spent is on Credits.

Per call

call is the default unit. The tool's credits are spent before the request is forwarded; the default tool spends one. A person short of the price is answered 402 credits_exhausted before anything is sent, with the balance in the message.

Per 1,000 tokens

With unit: "tokens", per is 1,000 by default. Before the call the tool reserves a ceiling: the price of the request's bytes ÷ 3 plus the output cap (4,096 when the tool sets none). When the answer ends cleanly, the call settles at the provider's usage, up to the ceiling, and what was not used returns. Per-token pricing is for text: a tool that takes files is priced per call.

The call costs the ceiling when the provider reports no usage, when a stream does not end cleanly, or when the call is abandoned mid-answer. A call abandoned before the provider answered costs the input estimate.

Per job unit

A generate tool meters images, seconds or characters (or call: one price per run). A transcribe tool meters seconds (or call). A tool's unit is the one its model meters: an image model per image, a video or transcription model per second, a speech model per character, or call. Any other unit is refused when the tool is saved, since a unit nothing meters would charge every person the ceiling.

A metered job tool reserves credits × ceil(bounds.maxUnits / per) when its run starts, and settles at what the provider metered when the run succeeds.

Refunds

Credits spent before the request are refunded only when the provider fails before its first byte, with a non-2xx answer or no answer at all. The response then carries x-gemmein-credit: refunded. A stream that dies after the first byte is not refunded, and hanging up early does not refund.

Response headers

Every answer that passed the spend carries x-gemmein-credits-remaining. x-gemmein-tool names the tool; absent on the implicit default.

Usage & billing

The Usage & billing page counts AI calls for the last 30 days: spent from your people's credits, priced by your provider. Gemmein meters the calls; your provider bills the tokens.

Image, audio and video runs

A job tool (kind generate or transcribe) answers with a run. The app calls the same endpoint, POST /ai/run/<tool>, and gets 202 with the run straight away. Gemmein follows the run until it ends.

The tool file, at gemmein/ai/tools/poster.json. The count placeholder stands alone in its string, so the provider receives a number:

{
  "label": "Poster",
  "kind": "generate",
  "provider": "replicate",
  "model": "black-forest-labs/flux-schnell",
  "credits": 2,
  "unit": "images",
  "params": { "prompt": "{{prompt}}", "num_outputs": "{{count}}", "aspect_ratio": "16:9" },
  "inputs": [
    { "name": "prompt", "type": "text", "required": true, "maxLength": 1000 },
    { "name": "count", "type": "number" }
  ],
  "bounds": { "maxUnits": 4 }
}

Starting it holds 8 credits. Two images back settle at 4 and return 4. In the app:

const run = await g.runs.start("poster", { prompt: "a lighthouse at dusk", count: 2 });
const done = await g.runs.watch(run.id, { onUpdate: (r) => setProgress(r.progress) });
if (done.status === "succeeded") img.src = done.result.files[0].url;   // a signed link, created for this answer
save(done.result.files[0].ref);   // the ref is what the app stores

A result file's url lapses at urlExpiresAt. Read the run again for a fresh one. The ref stays, and is what g.files reads. The run carries reserved, and once ended charged, units and unitCount.

A key on start (your own name for the run, up to 80 characters) makes a retried start of the same tool answer the same run. g.runs.list({ since }) is the person's own runs, newest first; g.runs.get(id) is one of them.

Runs are the person's, called from the browser. A secret key is refused, and another person's run is unknown_run. Ten runs may be open per person at a time.

Run refusals are in the codes table. start also meets every refusal g.ai.run does for the tool itself: unknown_tool, tool_disabled, entitlement_required, payload_too_large, ai_capped, session_required, scope_denied.

Your own pipeline

A start with no url set answers 409 tool_incomplete ("… names no URL — set where it runs") before any credit moves.

Agents

A relay's start_run starts this tool, or any tool, for that person when a record is written, a payment lands, a clock fires or a webhook arrives, with nobody pressing a button. run_started chains it to the next tool, one hop at a time, in a sequence your app composes. Gemmein owns no graph and runs none of the agent's code. It holds the run's reservation, person, deadline and result, and nothing else.

The hand-off

Gemmein signs the hand-off the same way it signs a relay's call_url. The two envelopes are what Gemmein sends your URL and what your pipeline sends back:

// Gemmein → your URL (X-Gemmein-Signature: t=…,v1=…)
{
  "run": { "id": "run_…", "tool": "poster", "kind": "generate", "units": "images", "reserved": 8, "createdAt": "…" },
  "params": { "prompt": "…" }, "person": { "id": "usr_…", "email": "…" }, "deadline": "…",
  "complete": { "url": "…/complete", "token": "…", "expiresAt": "…" },
  "progress": { "url": "…/progress" }, "fail": { "url": "…/fail" }
}

// your URL → Gemmein (Authorization: Bearer <token>)
{ "text": "made: a lighthouse at dusk", "units": 2 }

Answer a run

Your pipeline answers with two required calls and one optional one, all with the token the hand-off carried:

The pipeline helper

g.pipelines.handle does all of this in three lines (@gemmein/sdk ≥ 0.12.0):

export default g.pipelines.handle(process.env.PIPELINE_SECRET!, async (run, { progress }) => {
  await progress(50, "half way");
  return { text: `made: ${run.params.prompt}` };
});

g.pipelines.handle(secret, fn, options?) verifies the signature, acknowledges within Gemmein's 10-second window, runs fn in the background, and calls /complete or /fail for you. It is a plain Fetch-API handler that runs unmodified on Vercel, Cloudflare Workers, Deno, Bun or Node 18+.

secret is the tool's own signing secret (aisk_…), created with the tool, shown once on the tool's page and rotatable. It is never your app key. A bad or stale signature is refused before fn runs. A re-delivered hand-off (the same run.id, after a lost acknowledgement) runs fn once.

Local development

gemmein dev answers AI tools without a key: a fake provider echoes a stream, marked x-gemmein-ai: fake, and the tool's credits are spent from the local ledger. For a real call, set GEMMEIN_AI_KEY_OPENAI, GEMMEIN_AI_KEY_ANTHROPIC or GEMMEIN_AI_KEY_GOOGLE in the environment gemmein dev runs in. Keys never sync; your hosted app holds its own, pasted on the AI page.

Tools do sync. A file at gemmein/ai/tools/<name>.json hot-reloads like a relay, and the boot card prints a line per tool: AI tools: deep-research (20 credits · requires access:pro-max · openai).

Credits in Development

A fresh person's balance is zero, and AI tools refuse at zero locally and on Gemmein alike, so testing starts with a credit arriving. Locally that is the pay simulator. Declare a product with "Grants credits" in gemmein/payments.json (npx gemmein payments setup asks for it), call g.payments.buy("credits pack"), and open the URL it returns. The simulated payment travels the real signed-webhook path, and the credits land on that person's local ledger.

A relay's grant_credits is the other local way, for a receiver you have wired. On Gemmein the two ways are the same purchase, plus a comp you add by hand on that person's page under Customers. There is no comp in gemmein dev, because the dashboard is not part of it.

Runs and pipelines in Development

On gemmein dev without a key for the provider, a run ends succeeded with a generated image or a placeholder transcript. It is metered and settled, so the whole flow works before any key exists.

An external tool works the same way on gemmein dev, with loopback allowed. Your pipeline calls http://127.0.0.1:<port>/runs/<app>/<env>/<id>/complete with the token.

What happens if…

Reference

Tool fields

Numbers

• The credits the owner set for that tool; one by default.
• Up to 50 tools per environment; a name is at most 40 characters, a label at most 60.
• 20 calls per person per minute.
• 64 KB of inputs on a g.ai.run call.
• 256 KB raw request body on a g.ai.chat call, unless a tool sets a smaller cap (up to 262,144 bytes); nested at most 32 levels.
• 170 seconds in all; on a stream, 10 seconds to the first response headers.
• Every call counts toward the app's request band like any other request.
• One test call per minute from the AI page; a test call spends no credits.

Codes

Raw provider calls

g.ai.chat(body), where the browser sends the provider's own request, is off until you switch raw calls on for that key on the AI page. Without the switch it answers raw_calls_off. With the switch on, the body is forwarded as sent and the answer passed back byte for byte. A call that names no tool runs as the default tool for one credit.

gemmein dev's fake provider, with no key set, keeps raw calls open; there is no switch locally. ?provider= and ?stream=1 on the URL do what the body fields do.

Those three endpoints are the whole list. There is no field for a URL, and no other host is reachable through a chat call.

The ai_calls record

Every call lands as one record in the ai_calls collection, per person: the tool, the kind, the provider and model, tokens in and out when the provider said, the credits it cost, how it ended, the refusal code, the latency and the time. The prompt and the answer are kept only for a tool whose record calls switch is on.

A person reads their own with g.ai.calls(). You read them by app on the Records page and on the person's record. Erasure removes a person's records.