Evaluate

Typed decisions from an evaluation model.

01Overview

Evaluation models return choices, scores, and boolean probabilities instead of generated text. Send the shared `state` plus a map of typed `questions`; every question is evaluated independently and returned under its own id. The upstream path is configured on the channel and defaults to /evaluate, so providers that expose a different path keep working.

POST/api/v1/evaluate

Base URL https://capi.aiAPI version v1Authentication Bearer YOUR_API_TOKEN

02Parameters

Name	Type	Description
model*	string	An enabled evaluation model ID, e.g. typesafe-ai/jev.
state*	string \| object \| array	The shared input every question is evaluated against.
questions*	object	Map of question id to a question of type boolean, choice, or score. Up to 20 per request.

03Request body

JSON

{
  "model": "typesafe-ai/jev",
  "state": "I was charged twice for my subscription this month.",
  "questions": {
    "refund": { "type": "boolean", "instructions": "Is the customer asking for money back?" },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["Low, no impact", "Medium, degraded experience", "High, blocking with financial loss"]
    }
  }
}

04Notes

Requires the llm.evaluate scope on the API key.
Evaluation models are not language models: /api/v1/chat/completions rejects them.
Billed from the reported usage like any other model; some evaluation models charge input tokens only.

Example

Evaluate

curl -X POST https://capi.ai/api/v1/evaluate \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model":"typesafe-ai/jev","state":"I was charged twice for my subscription.","questions":{"refund":{"type":"boolean","instructions":"Is the customer asking for money back?"}}}'

HTTP 200· OK

{
  "model": "typesafe-ai/jev",
  "answers": {
    "refund": { "type": "boolean", "probability": 0.98 },
    "urgency": { "type": "score", "score": 2.1, "probabilities": { "0": 0.05, "1": 0.2, "2": 0.75 } }
  },
  "usage": { "inputTokens": 275, "outputTokens": 20 },
  "cost": { "amount": 0.0001, "currency": "USD" }
}

All endpoints →