OpenRouter adds benchmarks for model routers

A comparison page for model routers

OpenRouter introduced a benchmark page comparing router performance across quality, speed, and cost. It says the Router Index combines those dimensions into a score, with default weights of 60% for quality, 20% for time per task, and 20% for cost. Users can adjust the weights on the page.

The company says the benchmark covers general tasks, not necessarily a user's own work, and that the listed routers are available on OpenRouter. It says they can be swapped in for a model in an app, harness, or through the API.

Sources: Model Router Benchmarks

Router types affect billing

The post distinguishes provider routing, which selects an inference provider for a model, from model routing, which chooses a model to answer a request. It groups the tested model routers into blends of models, per-turn model selection, and switching between a predefined set of models.

OpenRouter says its Auto Router and Jev Router select a model for each turn, with transparent selection and billing at the standard rate for the selected model. Other approaches may bill at standardized per-token rates or switch between paired models as an agent works through a task.

Sources: Model Router Benchmarks

Routing can add cost and latency

The post notes that switching models can make requests more expensive while input cache is rebuilt, and that routers may have too little information to judge task complexity from a prompt. It also says routing decisions can add latency. The benchmark is presented as a way to compare those trade-offs, rather than a guarantee that routing will improve results for every workload.

Sources: Model Router Benchmarks