Call a tool from your app

Named AI tools, the kinds and providers, the files a tool takes, and model aliases.

Call a tool from your app

A tool file at gemmein/ai/tools/deep-research.json:

{
  "label": "Deep Research",
  "provider": "openai",
  "model": "gpt-4o",
  "credits": 20,
  "requires": "access:pro-max",
  "instructions": "You are a careful research assistant. Answer with sources.",
  "promptTemplate": "Research this for a {{audience}} reader:\n\n{{question}}",
  "inputs": [
    { "name": "question", "type": "text", "required": true, "maxLength": 2000 },
    { "name": "audience", "type": "text" }
  ],
  "bounds": { "maxOutputTokens": 4000 }
}

The app never sends a prompt or a provider request. It names the tool and passes the inputs the tool declares. Gemmein composes the request in the provider's own format, from the instructions and the template on the server. The answer comes back in the provider's own shape, so a streaming call streams:

const res = await g.ai.run("deep-research", { question: text, audience: "beginner" }, { stream: true })

for await (const chunk of res.body) render(chunk)   // the provider's SSE, byte for byte
res.headers.get("x-gemmein-credits-remaining")     // the balance after this call

For a non-stream answer as one string, whichever provider answered, and for the person's own history:

const answer = await g.ai.runText("deep-research", { question: text })
const { calls } = await g.ai.calls()   // what they ran, when, what it cost, how it ended

g.ai.runText reads choices[0].message.content from OpenAI, joins content[].text from Anthropic, and joins candidates[0].content.parts[].text from Google.

A provider's non-2xx answer is returned as it came by g.ai.run, and is not thrown. g.ai.runText, like g.ai.text, throws it as provider_error with the provider's status and message. A refusal by Gemmein throws GemmeinError with one of the codes below. An unknown input, a missing required one, a wrong type or a value over its cap is refused by name before anything is spent (invalid_inputs).

What the person sees:

Tool kinds and providers

A tool is one of two kinds. A chat tool answers now. A job tool (generate or transcribe) answers with a run. A provider serves every kind its key sells:

An OpenAI key makes chat, image, speech, transcription and video tools. A Google key makes chat, image, video and audio-transcription tools. external is your own pipeline.

When your own server makes the model call, or the call is neither chat nor a job (such as embeddings), call the provider directly with your key. Ask Gemmein's server methods in the SDK reference who the person is and what they hold. A chat model on any server that speaks the OpenAI Chat Completions format runs as provider openai-compatible (see Any OpenAI-compatible provider); a server that speaks another format runs as your own pipeline.

Files a tool takes

A tool says which kinds of file a person may hand it: image, audio, video and document. Each is a tick in the tool's file ("accepts": ["image", "document"]) and on the dashboard's AI page. A file input takes the ref of a file the person already uploaded, to any collection: g.ai.run("describe-photo", { photo: upload.ref }).

Gemmein checks the file's kind, the tool's model and the file's size before any credit moves, then hands the file to the provider. Most providers fetch a short-lived signed link made for that one call. Small files ride inline. Gemini files over 14 MB stream from storage into Gemini's own file store. A file is never sent as a public link, never converted, and never held whole in Gemmein's memory.

A tick is offered only where the tool's provider and model read that kind:

npx gemmein dev and npx gemmein sync refuse a tick the model does not read and name the models that do. A file may be up to 500 MB of video, 100 MB of audio, or 25 MB of image or document, or the provider's own limit when lower. "inputMaxMb": { "video": 200 } in the tool's file sets a lower limit for one tool; the AI page has no size field. A tool that takes files is priced per run: a flat credit price per call, held and charged as that price whatever the file's size. A video input can cost more on your own provider key. A tool charged per token that ticks a kind is refused at gemmein dev, gemmein sync and on the AI page: "A tool that takes files is priced per run — set a credit price per call, and lower its file size limit if you want to cap your cost." Text-only tools keep per-token pricing. Files a person hands a tool count toward your app's storage; video fills it fastest, and you can add 50 GB storage packs under Usage. With ticks set, a file input takes only a Gemmein file ref, never a link. A tool created before ticks takes exactly the files it took before.

Create a tool

Create a tool on the dashboard's AI page, or have your AI write a file at gemmein/ai/tools/<name>.json. npx gemmein sync pushes it to Development. npx gemmein sync --live pushes it to your live app on its own approval: you approve the push in the browser, item by item, and anything different from what you approved is refused.

The file owns the implementation on every sync. After creation, the dashboard owns the commerce: label, credits, unit and per, access, on/off, record calls, and a transcribe tool's delete-after-run. Every field is in the field table.

A tool can be defined before its provider key exists. Runs are refused ai_not_configured until the key is added on the AI page, and the dashboard shows the tool as waiting on a key until then. A tool's name is fixed once created, because it is what the app calls.

Model aliases