Edition

RSS

Topic

Benchmarks

11 stories on this topic, newest first.

  1. ToolsExplainer

    Is LiteLLM written in Rust? What the codebase and migration tracker show

    LiteLLM Rust, checked: the repo is 3.11% Rust, the migration tracker shows 24% of Major APIs, and Chat Completions still runs on Python.

    20 min read

  2. ToolsAnalysis

    The '4.5x more requests per dollar' claim is a model, not a measurement

    LiteLLM cost claims for its Rust gateway rest on an assumed 4 vCPU / 16 GB instance. What the 4.5x, 5.8x and 4.4x ratios measure, and what is missing.

    24 min read

  3. ToolsExplainer

    Peak RSS vs memory at rest: reading LiteLLM's '9x less memory' figure

    LiteLLM's homepage labels its memory figure 'memory at rest', but the data is peak RSS. What the files show, why Bifrost sat near 282 MB, and how to read RSS.

    20 min read

  4. ToolsComparison

    LiteLLM Rust vs LiteLLM Python vs Bifrost: what the data means for teams running LiteLLM today

    In LiteLLM's own sweep the Python LiteLLM proxy adds 546 ms of p99 at c=64 and serves 179.8 RPS, and main:latest still runs chat routes on Python.

    23 min read

  5. ToolsExplainer

    Throughput vs latency: why LiteLLM Rust and Bifrost tie at 2,722 vs 2,719 RPS

    Throughput vs latency for LLM gateways: why 2,722 vs 2,719 requests per second is a tie, how RPS and p99 interact, and how to read a throughput tile.

    22 min read

  6. ModelsExplainer

    Claude Sonnet 5.5: a practical guide to price, effort, benchmarks and migration

    Anthropic's mid-tier model costs $2 in and $10 out per million tokens with a 1M context. What changed from Sonnet 5, when to pick it over Opus 5.5 or Haiku 5.5, and how to migrate.

    24 min read

  7. ModelsComparison

    Best LLM for coding in October 2026: ten models ranked on the evidence

    Opus 5.5, GPT-6.1 Sol, Gemini 4 Argon and seven more ranked on Terminal-Bench 4.0, DeepSWE and cost per task, with each score's harness and source.

    23 min read

  8. ResearchExplainer

    What is an agent harness? The loop, tools and context around the model

    An agent harness is everything around the model: the loop, tools, context, memory, permissions and sandbox. Why it moves benchmark scores as much as the model.

    24 min read

  9. ModelsComparison

    Top 10 AI models in October 2026, ranked by what they are good at

    Claude Opus 5.5, GPT-6.1 Sol, Gemini 3.8 Flash, Kimi K3 and six more, ranked on independent evals, coding, context, price and speed, with every figure sourced and dated.

    25 min read

  10. ModelsExplainer

    How to read a model card without being fooled: the 2026 field guide

    System cards now run past 200 pages. We take five current cards apart section by section, show how one benchmark is reported five different ways, and set out a ten-step reading protocol.

    30 min read

  11. IndustryEditorial

    Benchmarks became a marketing channel. Read the footnotes, then run your own

    Launch tables report the same benchmark under different attempts, tools and thinking budgets. How measurement became collateral, what independent evaluators find, and how to run your own.

    21 min read