Bifrost now routes TypeSafe's Jev, with a decisions API and LLM fallbacks

Bifrost, the open-source AI gateway, now routes requests to Jev, the decision model TypeSafe AI released in early access on 15 September 2026. Support shipped in Bifrost HTTP v2.2.2 on 23 September: a TypeSafe provider, a pass-through route that lets TypeSafe’s own Python and JavaScript SDKs work by changing one base URL, and a new POST /v1/decisions endpoint that puts Jev behind the same virtual keys, budgets, logs and fallback chains that already govern a team’s LLM traffic.

For teams piloting Jev, this matters because a decision model tends to end up in the hot path of ordinary software: triage, routing, moderation, guardrails. Calls like that multiply fast, and they carry customer data. Running them through the gateway that already holds provider credentials and budgets means the new model arrives with the same controls as the old ones, instead of as one more API key in an environment variable.

This piece covers what shipped and when, the one-line SDK change, the second, gateway-native decisions route, what a gateway adds to decision calls, how Jev and LLM traffic share one gateway, and the limits that remain as of 29 September 2026.

What shipped, and when

Bifrost is an open-source AI gateway written in Go and maintained by Maxim AI. It sits between applications and model providers, exposes one API, and applies identity, budgets, routing, failover and logging to each call. Readers new to the category can start with the site’s explainer on what an AI gateway is.

The TypeSafe work landed in four pull requests merged on 21 and 22 September 2026 and shipped together in the Bifrost HTTP v2.2.2 release, published on 23 September. The release notes list four things:

• A TypeSafe provider that holds TypeSafe API keys and maps requests to TypeSafe’s native POST /v1/systemone endpoint.
• A /typesafe integration that mirrors TypeSafe’s native API so the official SDKs work unchanged.
• A /v1/decisions endpoint, Bifrost’s own contract for “judgment models”, with a normalised answer shape and fallback support.
• Decision emulation: providers without native decision support answer decision requests through forced tool-calling on their chat model, whether used as the primary model or as a fallback.

The notes add that decision requests “are priced from the datasheet and logged with their answers”. A follow-up release, v2.2.3 on 24 September, adjusted emulation for gpt-oss models on Bedrock Mantle. Two further changes are merged to the main branch but, as of 29 September 2026, not yet in a tagged HTTP release: native Jev routing through OpenRouter (merged 27 September) and a compatibility fix that brings Bifrost’s validation in line with TypeSafe’s JavaScript SDK v0.6.0 (merged 28 September). Both are covered under limits below.

Bifrost is not the only gateway to add Jev. TypeSafe’s own Python SDK documentation gained “examples for usage with AI gateways” in SDK v0.7.1 on 21 September, and as of this writing shows OpenRouter, Vercel AI Gateway and Pydantic AI Gateway. All three are hosted services. Bifrost is the self-hosted option in that set, which is the property that matters to teams that want decision traffic, logs and keys inside their own network. The broader field is mapped in this rundown of open-source LLM gateways for self-hosted deployments.

Jev in one paragraph

Jev is a “System One” model: it takes unstructured state plus a set of typed questions and returns typed answers with probabilities and confidence in one pass, instead of generating text token by token. TypeSafe’s launch post reports end-to-end response times of 70 to 500 ms, and the models page lists jev-1.13.0 at $0.042 per million input tokens with output tokens free, a 64k-token context, text-only input, and rate limits of 250,000 tokens per second and 1,200 requests per minute that TypeSafe says “are adjusting dynamically”. There are three question types: a Noul (probability of yes), a Choice (one option from a set, up to 255) and a Score (a rubric of 2 to 10 ordered levels). Speed, cost and accuracy figures are TypeSafe’s own and have not been independently reproduced. The full launch story is in TypeSafe launches Jev, and where the model fits is covered in what Jev is good for.

The one-line change: TypeSafe’s SDKs through Bifrost

The TypeSafe SDK integration works because Bifrost exposes TypeSafe’s native API one-to-one under a /typesafe prefix. The SDK keeps its question builders, retry policy and types; only the host changes.

Both SDKs read TYPESAFE_BASE_URL from the environment. TypeSafe’s Python SDK defines it as a public constant alongside TYPESAFE_API_KEY, and the JavaScript client falls back to it when no baseURL is passed. For a service already calling Jev, the zero-code-change setup is one environment variable:

export TYPESAFE_BASE_URL="http://localhost:8080/typesafe"
export TYPESAFE_API_KEY="sk-bf-v1-..."   # a Bifrost virtual key, not a TypeSafe key

The second line is the part that changes the security model. When Bifrost authentication is on, the SDK’s Authorization: Bearer header authenticates the caller to Bifrost, not to TypeSafe. The real TypeSafe key is configured once on the Bifrost provider, and Bifrost selects and injects it on the upstream call. Passing a virtual key as the SDK’s API key means the application never holds a TypeSafe credential, and “governance (budgets, rate limits, model restrictions) applies per key”.

In Python, with the typesafe-sdk package, the explicit form looks like this:

from typesafe_sdk import TypeSafeClient, Noul

with TypeSafeClient(
    base_url="http://localhost:8080/typesafe",
    api_key="sk-bf-v1-your-virtual-key",  # Bifrost virtual key
) as client:
    response = client.system_one(
        state={"document": "I was charged twice. Please fix this ASAP."},
        questions={"billing": Noul(instructions="Is this ticket about billing?")},
    )
    print(response.nouls["billing"].noul)

And in TypeScript, with @typesafe-ai/sdk:

import { TypeSafeClient, choice } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  baseURL: "http://localhost:8080/typesafe",
  apiKey: "sk-bf-v1-your-virtual-key",
});

const response = await client.systemOne({
  state: { document: "I was charged twice. Please fix this ASAP." },
  questions: {
    category: choice("What is this ticket about?", {
      billing: null, technical: null, other: null,
    }),
  },
});

On the Bifrost side, the provider configuration is the usual key block, with the secret read from the environment:

{
  "providers": {
    "typesafe": {
      "keys": [
        { "name": "TypeSafe API Key", "value": "env.TYPESAFE_API_KEY", "weight": 1, "models": ["*"] }
      ]
    }
  }
}

Bifrost’s documentation is specific about fidelity. Success responses are “shape-compatible” with TypeSafe’s API: the native answers object with noul, choice or score fields and input_tokens and output_tokens usage. They are byte-identical only on the raw-response passthrough path; otherwise the response is rebuilt, so key order in the JSON can differ. Errors come back in TypeSafe’s native {"detail": {"error_type", "message"}} shape with upstream status codes (401, 422, 429, 529) preserved, so SDK exception handling keeps working. Model IDs work bare (jev-1.13.0, jev-latest, jev-preview) or with a typesafe/ prefix.

This is the same pattern Bifrost already uses for the OpenAI, Anthropic and Google SDKs, described in its drop-in replacement guide: point the existing client at the gateway and keep the application code.

The second route: POST /v1/decisions

The native route is for code that already speaks TypeSafe. The decisions endpoint is Bifrost’s own contract, and the docs point to it explicitly “for provider-routed access with fallbacks and Bifrost’s normalized response shape”. It takes the same three question types, with kind in place of TypeSafe’s type:

curl http://localhost:8080/v1/decisions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-bf-v1-your-virtual-key" \
  -d '{
    "model": "typesafe/jev-1.13.0",
    "state": "Customer message: I was double charged and nobody replied to my two emails. I want a refund today.",
    "questions": {
      "is_frustrated": { "kind": "noul", "instructions": "Is the customer frustrated?" },
      "category": {
        "kind": "choice",
        "instructions": "Pick the ticket category",
        "criteria": { "billing": "charges and refunds", "bug": "product defects", "other": "anything else" }
      },
      "urgency": {
        "kind": "score",
        "instructions": "Rate how urgently this needs a human reply",
        "criteria": ["can wait a week", "should be answered soon", "needs a reply today"]
      }
    }
  }'

The documented response gives every answer a single value field, whatever its kind, and keeps TypeSafe’s confidence, probabilities and legend metadata when the provider supplies them:

{
  "model": "jev-1.13.0",
  "answers": {
    "is_frustrated": { "kind": "noul", "value": 0.98 },
    "category": { "kind": "choice", "value": "billing", "confidence": 1,
                  "probabilities": { "billing": 1, "bug": 0, "other": 0 } },
    "urgency": { "kind": "score", "value": 2, "confidence": 1,
                 "probabilities": { "0": 0, "1": 0, "2": 1 },
                 "legend": { "0": "can wait a week", "1": "should be answered soon", "2": "needs a reply today" } }
  },
  "usage": { "prompt_tokens": 437, "completion_tokens": 72, "total_tokens": 509 }
}

The TypeSafe provider page documents the full field mapping. The differences that matter in practice:

Two details in the mapping are worth knowing. Aliases resolve upstream, so a request for jev-latest reports jev-1.13.0 in the response; logging that field records which model version actually answered. And every requested question must produce an answer of its declared kind: “a missing answer, a kind mismatch, or a missing value fails the request rather than returning partial results”. For code that branches on the answers, a clean failure is easier to handle than a half-filled object.

Figure 1: A Jev call through Bifrost passes the same identity, policy and logging steps as an LLM call; only the upstream contract differs.

Why route a decision model through a gateway

A gateway adds a network hop and one more component to run, so the case for it has to be concrete. For decision traffic, five arguments carry weight, and they are the same ones this production-ready comparison of LLM gateways uses to separate gateways from thin proxies.

One credential store

Without a gateway, every service that calls Jev holds a TypeSafe key. With Bifrost, the key lives in one provider configuration, applications hold virtual keys, and revoking a leaked credential is a gateway operation rather than a hunt through deployment secrets. The provider page also documents key rotation behaviour specific to TypeSafe: on a 401 the failing key is marked and other keys rotate in; on a 429 Bifrost retries with backoff and rotates keys, because TypeSafe’s limits are per key; on a 529 (“service overloaded”) it retries on the same key, because the problem is capacity, not the credential.

Virtual keys, budgets and rate limits

Budgets and limits in Bifrost attach at the virtual key, team and customer levels, with reset windows from one minute to one year, and every applicable budget must have headroom before a request proceeds. Rate limits count both requests and tokens per period at the virtual key and provider-config levels.

A worked example shows why this matters for Jev specifically. Take the documented request above: 437 prompt tokens. At TypeSafe’s list price of $0.042 per million input tokens, and assuming Bifrost’s datasheet carries the same price, one call costs about $0.000018, and a million such calls about $18. Cost is rarely the constraint at that price. Throughput is. TypeSafe’s published limit of 1,200 requests per minute is 20 requests a second, and it is shared by everything using that key. A batch job that fans out over a backlog can consume all of it and starve the real-time triage service. A request-per-minute limit on the batch job’s virtual key (say 600) reserves the rest for production, and the check happens before any upstream call.

Logging decisions next to everything else

Bifrost’s built-in observability records inputs, outputs, tokens, cost, latency, the provider and model that served each request, and, on retries, an attempt trail showing which keys were tried and why each failed. The v2.2.2 notes say decision requests are “logged with their answers”. For a decision model that is more useful than it sounds: the probability Jev gave for “is this a refund request” on a given ticket is exactly what someone will ask about when a ticket is mis-routed. Having it in the same log store as the LLM call that drafted the reply turns a two-system investigation into one query.

Retries and fallbacks

Bifrost’s retries and fallbacks separate per-key failures (401, 402, 403, 429) from transient server failures (5xx, network), retry with exponential backoff, and move to the next model in a fallback chain once retries are exhausted, with each fallback getting its own retry budget. Retries are configured per provider in network_config, and the default is max_retries: 0, so they have to be switched on for the TypeSafe provider. On /v1/decisions, the fallbacks field takes a list of provider and model pairs and is “consumed by fallback routing, never sent upstream”.

Governance and security in one place

Beyond routing, Bifrost applies governance controls and security controls centrally: virtual keys and budgets in the open-source build, plus guardrails and audit logs in the enterprise edition. Bifrost Edge, currently in alpha, extends that same governance and security to AI traffic on employee machines, with endpoint enforcement on each device. For Jev the gateway layer is the relevant one today, because decision calls come from services rather than desktop apps. The endpoint problem is covered in the site’s piece on shadow AI governance.

Decisions and LLM traffic on the same gateway

The less obvious part of the release is that /v1/decisions is not TypeSafe-only. Pull request #7361 changed what happens when a decision request reaches a provider with no native decision support. Before, it returned unsupported_operation. Now Bifrost intercepts that error, builds a single forced function tool (emit_decision) from the question set, calls the same provider’s chat model, and maps the tool-call arguments back into a decision response. The same path runs whether an LLM is named as the primary decision model or reached as a fallback after a native provider fails. A follow-up change normalises probabilities to sum to exactly 1 and rejects any Choice answer whose selected option is not the most probable one.

In practice this means a fallback list can end in a chat model:

{
  "model": "typesafe/jev-1.13.0",
  "fallbacks": ["openai/gpt-4o-mini"],
  "state": "...",
  "questions": { "...": "..." }
}

(gpt-4o-mini is the model Bifrost’s own integration tests use for emulation; any tool-capable chat model on a configured provider works the same way.)

That is useful for availability. If TypeSafe returns 529s during a capacity crunch, a triage workflow keeps producing typed answers instead of stopping. It also lets a team run the same question set against Jev and an LLM through one API and compare the answers, which PR #7361’s own test suite does side by side.

It comes with a caveat that the gateway cannot solve. TypeSafe’s central claim is that Jev’s probabilities are calibrated: higher confidence means higher accuracy. The launch post says prompted LLMs “tend to be overconfident and inconsistent” and that its own wrapper for extracting decisions from LLMs “tends to be slower and more expensive”. Bifrost’s normalisation makes an LLM’s numbers well-formed; it does not make them calibrated. A threshold tuned on Jev (act automatically above 0.9, send to a human below 0.6) may behave quite differently when a fallback LLM supplies the number. The practical fix is simple: the response’s model field says which model answered, so code should branch on it, apply per-model thresholds, or route every fallback answer to review.

There is a second, quieter benefit of one gateway for both kinds of traffic. A typical Jev workflow is a hybrid: a decision model classifies or scores, and an LLM writes the prose only when prose is needed. TypeSafe’s own docs describe patterns like intent routing and screening LLM input and output with one decision call. With both calls on one Bifrost AI gateway, one virtual key can carry a budget that covers the whole workflow, and one log view shows the decision and the generation that followed it.

Figure 2: One gateway, three routes. The virtual key, the budget and the log are shared; the contract and the calibration are not.

A worked example: ticket triage with one virtual key

The following sketch combines the two documented routes. It uses TypeSafe’s Python SDK through Bifrost for the decision and the OpenAI SDK through Bifrost’s OpenAI-compatible route for the reply, with the same virtual key on both. The question builders and answer fields (nouls, choices, scores, .confidence) are from TypeSafe’s SDK reference; the thresholds are illustrative and belong in code, tuned on a team’s own data.

import os
from openai import OpenAI
from typesafe_sdk import TypeSafeClient, Noul, Choice, Score

BIFROST = "http://localhost:8080"
VK = os.environ["BIFROST_VIRTUAL_KEY"]          # one key, one budget, one log

decide = TypeSafeClient(base_url=f"{BIFROST}/typesafe", api_key=VK)
llm = OpenAI(base_url=f"{BIFROST}/openai", api_key=VK)

def triage(ticket: str) -> dict:
    r = decide.system_one(
        state={"ticket": ticket},
        questions={
            "refund": Noul(instructions="Is the customer asking for a refund?"),
            "team": Choice(instructions="Which team should handle this?",
                           criteria={"billing": "charges, refunds, invoices",
                                     "technical": "bugs, outages, integrations",
                                     "sales": "pricing, upgrades"}),
            "urgency": Score(instructions="How urgently does this need a reply?",
                             criteria=["this week", "today", "within the hour"]),
        },
    )
    team = r.choices["team"]
    if team.confidence < 0.6:                    # uncertain: a person decides
        return {"route": "human_review", "model": r.model}
    reply = None
    if r.nouls["refund"].noul > 0.9 and team.choice == "billing":
        reply = llm.chat.completions.create(     # prose only when prose is needed
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": f"Draft a short refund acknowledgement for: {ticket}"}],
        ).choices[0].message.content
    return {"route": team.choice, "urgency": r.scores["urgency"].score,
            "model": r.model, "draft": reply}

What the gateway contributes here is not visible in the code, which is the point. The application holds one credential. Both the Jev call and the chat call are checked against the same virtual key’s allowed providers, budget and rate limits. If the refund acknowledgement starts costing more than expected, the budget stops it at the gateway rather than on the next invoice. And when a ticket is mis-routed, the log holds the Jev probabilities and the drafted reply side by side. The model field is carried through so the calling code can tell a Jev answer from anything else.

Limits and open questions

The integration is a week old and the underlying model is two weeks old. Several things are worth checking before depending on it.

A few of these deserve a sentence more. On latency, the gateway’s own overhead is negligible next to a 70 to 500 ms model call, according to Bifrost’s published benchmarks. The hop is not always negligible: a gateway in Frankfurt calling a service on the US West Coast adds a transatlantic round trip that no benchmark on mocked calls captures. On OpenRouter, pull request #7589 routes TypeSafe models on OpenRouter to OpenRouter’s native decisions endpoint and maps OpenRouter’s billed cost into Bifrost’s logs and budgets. Once released, that gives a second network path to the same Jev model, which is provider-path diversity without the calibration problem of an LLM fallback. It is not yet in a tagged release.

Two broader questions remain open. First, no independent benchmark of Jev’s accuracy or calibration exists yet; the numbers in circulation are TypeSafe’s, and the company itself notes that its workflow evaluations use GPT-6 Astra and Claude Fable 5.1 as reference answers. A gateway makes it cheap to collect a team’s own evidence, since every answer is logged, but it does not substitute for that evidence. Second, the /v1/decisions contract is Bifrost’s own. It is well documented, but code written against it is code written against a gateway API rather than TypeSafe’s; teams that want to keep the option of calling TypeSafe directly may prefer the native route and accept that fallbacks are not documented there.

Who should use it, and how to start

For a team already running LLM traffic through a gateway, the answer is straightforward: put Jev behind it too, from the first pilot, so that decision calls never exist outside the budget and log system. For a team calling Jev directly from one service, the gateway starts to pay when a second service, a batch job, or a compliance requirement to log every automated decision appears. The same reasoning applies to choosing among AI gateway options for production generally.

A practical sequence:

1. Upgrade to Bifrost HTTP v2.2.2 or later, add the TypeSafe provider with the key read from the environment, and set max_retries on its network_config.
2. Create a virtual key per workload (real-time triage, batch backfill, evaluation) with request-per-minute limits that add up to less than TypeSafe’s account limit, and a budget that covers both Jev and any LLM calls in the same workflow. The Bifrost governance overview shows how keys, teams and customers nest.
3. Switch existing code by environment variable: TYPESAFE_BASE_URL pointing at /typesafe, TYPESAFE_API_KEY set to the virtual key. Nothing else changes.
4. Use /v1/decisions for new code that needs fallbacks, and decide up front what happens when the answer comes from a non-Jev model.
5. Pin jev-1.13.0 rather than jev-latest if confidence thresholds were tuned on it; TypeSafe warns that aliases move when a new release ships.
6. Log the model field and the probabilities, and review a sample of automated decisions weekly until there is enough data to trust the thresholds.

Teams comparing self-hosted gateways for this kind of workload can read this survey of open-source LLM gateways teams can run themselves, request a Bifrost demo, or start from the open-source repository and the TypeSafe provider documentation linked above.