Private endpoints
Frontier and open-weight models reachable without traffic leaving your network.
Recorded with the runEndpoint and region
Infrastructure
Context Inference
Serve your models where your data lives.
Managed inference inside your VPC: frontier models over private endpoints and the distilled models you own on dedicated capacity, behind one router that selects the lowest-cost route validated for each workflow.
Context Inference
Example workflow
Request
Review vendor contract
legal · high quality
Max latency8s
Data regionUS
Threshold0.91
Eligible routesExample
Owned legal model
1.0×0.94 qualitySelected
Frontier private
4.8×0.96 qualityAvailable
Fast general
0.6×0.86 qualityBelow bar
Route decision
Quality bar cleared
Lowest-cost eligible route selected.
Latency2.8s
Routeowned
Tracerecorded
How it works
From request to a reviewable result.
Each run keeps the request, actions, evidence, and final decision together.
01 · Classify
Read the workflow and quality tier.
The router receives the task class, latency budget, allowed models, and eval threshold configured for the workflow.
Output · Eligible routes
02 · Route
Choose a validated model.
Context sends the request to the lowest-cost eligible route with current eval evidence for that task class.
Output · Model selected
03 · Measure
Record quality, cost, and latency.
The run joins operational telemetry with the evaluator result so routing policy can be reviewed and updated.
Output · Comparable result
What you can inspect
Capability is only useful when it leaves evidence.
Dedicated capacity for owned models
The distilled models trained on your work, served on capacity you control, at cost you can predict.
Recorded with the runCapacity and utilization
Cheapest-sufficient routing
Each request routes to the least expensive model verified against your rubrics for that workflow tier.
Recorded with the runRoute reason and eval
Cost telemetry per team
Spend broken down by team, workflow, and model tier, so finance sees where inference dollars go.
Recorded with the runCost, tokens, and latency
Deployment boundaries
Control is part of the product.
Context Inference can run as a standalone surface, while its traces, evals, and policy decisions remain part of the same Context system.
Routes require evidence
A model becomes eligible only after it clears the eval threshold configured for that workflow and version.
Fallbacks are explicit
Teams choose whether a failed or unavailable route may retry on another model, wait, or stop.