IndustryAnalysis

Databricks AI Index: what enterprise token spend data shows

Databricks' new AI Index draws on Unity Gateway traffic from what it says are 10,000+ companies plus an internal coding benchmark. We checked its headline numbers against the charts' own data and footnotes, and set out what the sample can and cannot tell you.

Databricks published the Databricks AI Index on 16 November 2026, a public page that tracks which AI models its customers use, roughly what they spend and how coding agents compare on cost and quality. The usage half comes from Unity Gateway, Databricks’ product for routing and governing model calls, which Databricks says more than 10,000 companies use. The benchmark half comes from coding tasks built from Databricks’ own pull requests. Databricks says it will update both weekly.

The headline findings are striking and, where we could check them, accurate to the index’s own data. Claude Opus 5.5 went from 7.2 per cent to 37.9 per cent of proprietary-model tokens in five weeks. Open-weight models went from 7.3 per cent to 30.9 per cent of tokens between July and November. The share of active companies that had used two or more providers roughly doubled, to 71.1 per cent. And in the coding benchmark, two models with the same 89.1 per cent pass rate differ in cost per task by a factor of about 30.

The useful work is reading the footnotes. Each chart measures a different population: some include traffic routed to Anthropic and OpenAI, one covers only models Databricks hosts, spend is priced at list rates, and the multi-provider measure is cumulative. All of it is a vendor’s view of its own customers. This analysis sets out what data the index draws on, what each headline number measures once its conditions are attached, what the coding benchmark can and cannot tell a buyer, and how to use the index without over-reading it. GenAI Brief has not had access to the underlying data; the chart values below are the ones embedded in the index page as published, read on 18 November 2026.

What the index draws on

The launch post, credited to “The Databricks AI Product and Engineering Team”, describes two sources.

Usage and spending. “Aggregated, anonymized data from 10,000+ companies using Unity Gateway, spanning digital natives, Fortune 500, and Global 2000 enterprises, including traffic to external providers such as Anthropic and OpenAI.” The same post’s “About the data” section and the index page both say “thousands of enterprises” instead. Databricks does not say how many companies contribute to each weekly figure.

Coding-agent evaluations. These build “on the coding-agent benchmark we published in July” and use “real coding-agent workflows from Databricks engineers, captured through Unity Gateway”, including creating and reviewing pull requests and investigating issues. Tasks come from Databricks’ private codebase with withheld tests, which Databricks says “helps reduce the risk of benchmark contamination”. Results are graded “against hidden tests rather than an LLM judge”.

The index page breaks this into five sections and nine charts. Each chart has a one-line note on what it counts, and those notes are the methodology:

ChartWhat its note says it countsLatest point
Token share, proprietary modelsInput plus output tokens through Unity Gateway, including traffic to external providersWeek of 2 Nov
Wallet share, proprietary modelsTokens priced at each provider’s list price, cache reads at the cache-read rate; “not what Databricks bills”Week of 2 Nov
Token and wallet share, open-weight modelsSame as above, by open-weight provider, including external trafficWeek of 2 Nov
Companies using multiple providersAmong companies active each week, every provider a company has used since 20 July 2026; Gemini and Gemma count as one provider, as do GPT and GPT-OSSWeek of 2 Nov
Proprietary vs open-weight token shareInput plus output tokens “to Databricks-hosted models on Unity Gateway”Week of 2 Nov
Weekly user spend percentilesEach user’s input, cached and output tokens at list price; “human users only”Week of 2 Nov
Coding cost and quality frontierInternal benchmark from Unity Gateway traces of Databricks engineering tasks; each model shown in the harness, Claude Code or Codex, closest to the frontierUpdated 10 Nov

Notes quoted or paraphrased from the Databricks AI Index page, read 18 November 2026.

The time windows are not quite consistent. The usage charts run in 16 weekly points labelled from 20 July to 2 November. The post’s “About the data” section says “The current dataset covers September 21 through November 7, 2026”, which matches the six-week comparisons the post highlights but not the July baselines it also quotes. Readers should take each number with the window attached to it.

The headline numbers, with their conditions

The post’s main claims can be checked against the values the index page embeds for its charts. All of the following match, with the qualifications in the right-hand column.

Claim in the launch postValue in the index dataCondition the chart attaches
Opus 5.5 “surged from 7.2% to 37.9% of proprietary-model tokens” in six weeks7.2% in the week of 21 Sep; 37.9% in the week of 26 Oct; 35.8% in the week of 2 NovProprietary models only; 26 Oct is the peak
Anthropic’s share rose “from 57.6% to 68.8%”57.6% (21 Sep) to 68.8% (26 Oct); 61.5% on 2 NovSame chart; sum of Anthropic models
About 64% of companies used one provider in July; 28% by 2 Nov2+ providers: 36.0% (20 Jul) to 71.1% (2 Nov), implying 64.0% and 28.9% single-providerCumulative count of providers used since 20 July; the data gives 28.9%, which the post states as 28%
Open-weight share “grew from 7.3% in July to 30.9% in November”7.3% (20 Jul) to 30.9% (2 Nov); peak 33.2% (14 Sep)Databricks-hosted models only
Median weekly spend “more than doubled from $1.10 over six weeks” to $2.49; P99 $699.74, about 281 times the medianMedian $1.10 (21 Sep) to $2.49 (2 Nov); P99 $699.74List-price estimate; human users only
Fable 5.1’s share of proprietary tokens fell “from 5.5% to 0.6% over 6 weeks”5.5% (21 Sep) to 0.6% (2 Nov)Proprietary models only

GenAI Brief reading of the chart data embedded in the index page on 18 November 2026. Shares in per cent of the chart’s own denominator.

The independent coverage we found mostly repeats these figures. StartupHub.ai restates the post’s numbers. The venture investor Tomasz Tunguz built his own analysis from the same series and added the caveat that matters most: “Unity Gateway reflects Databricks’ enterprise customer base (data-platform & agentic-coding workloads) rather than the entire enterprise AI market.”

Model churn among proprietary models

The proprietary token-share series is the most dynamic part of the index, and it is the best-supported finding: customers move volume to new models within days of release. Opus 4.8 held 49.8 per cent of proprietary tokens in the week of 20 July and 1.3 per cent by 2 November. Opus 5 jumped to 27.0 per cent the following week and then faded to 2.1 per cent. Opus 5.5 appeared at 7.2 per cent in the week of 21 September and became the largest single model within two weeks.

At the provider level, the picture depends on the week chosen. On the index’s numbers, Anthropic’s share of proprietary tokens was 80.0 per cent on 20 July, 57.6 per cent on 21 September, 68.8 per cent on 26 October and 61.5 per cent on 2 November. OpenAI’s moved the other way, from 16.4 per cent in July to 37.0 per cent on 2 November, which supports the post’s statement that OpenAI’s share “grew meaningfully”. The post’s own explanation is that the shift was “driven largely by its 6 series Sol and Astra models”, and that “model quality continues to strongly influence provider market share”. Those are Databricks’ interpretations; the data shows the movements, not their causes.

Databricks also uses the series to describe its own decisions, which are company claims rather than index measurements. Its Day 1 rollout post of 28 September reports that, after reweighting sessions by type, Opus 5.5 averaged $4.23 per session against $5.94 for Opus 4.8, 29 per cent less, and that GPT-6 Sol averaged $2.34 against $4.52 for GPT-5.6 Sol. The same post says Opus 5 “was more expensive and ranked lower on both quantitative and qualitative quality scores among our engineers compared with Opus 4.8”, which is why, the index post says, Databricks “decided to stay on 4.8” until Opus 5.5.

Tokens are not spend

The wallet-share charts price the same tokens at list rates, and they tell a different story about who gets paid. For the week of 2 November:

Provider or modelShare of proprietary tokensShare of estimated proprietary spend
Anthropic (all models)61.5%70.1%
OpenAI (all models)37.0%27.4%
Gemini and Grok1.4%2.7%
GPT 5 Nano7.1%0.3%
GPT 6 Luna4.8%0.6%
GPT 6 Astra1.9%8.5%
Claude Opus 5.535.8%34.7%

Index data for the week of 2 November 2026. Provider totals are sums of the index’s model rows, calculated by GenAI Brief.

The gaps follow from price per token. OpenAI’s cheap models carry volume without much spend: GPT 5 Nano is 7.1 per cent of tokens and 0.3 per cent of estimated spend. Premium models do the opposite: GPT 6 Astra is 1.9 per cent of tokens and 8.5 per cent of spend. Tunguz, working from the same rows, also puts Anthropic at about 70 per cent of proprietary spend. Because all of it is priced at list rates, a customer on a negotiated discount with one provider would see a different split on its own invoices.

Open-weight growth, on Databricks’ own servers

The open-weight figure is the one most likely to be repeated out of context. The post says open-weight models’ “share of enterprise token traffic grew from 7.3% in July to 30.9% in November, more than quadrupling in less than four months.” The chart’s note is narrower: it counts “Input plus output tokens to Databricks-hosted models on Unity Gateway”, and classifies GLM, Kimi, DeepSeek, Qwen, Llama, GPT-OSS and Gemma as open-weight and Claude, GPT, Gemini and Grok as proprietary.

That denominator excludes the traffic Unity Gateway routes to external providers, which the post elsewhere says is included in its usage data. So the 30.9 per cent is a share of tokens on models Databricks serves itself, where open-weight models are a natural fit, not a share of everything that passes through the gateway, and still less of the market. It is also not a straight-line rise. The series peaked at 33.2 per cent in the week of 14 September and has moved between 29.0 and 32.6 per cent since. Tunguz noted the stall coincided with OpenAI price cuts; that is his reading, not something the index tests.

Databricks’ own earlier account of the July jump does not line up neatly with the index either. Its 4 November post on open-weight models says GLM 5.2 became available “in mid-July” and that “Just one month after first being offered, GLM 5.2 already accounts for ~40% of open-weight tokens used on Unity Gateway”. The index’s open-weight series puts GLM 5.2 at 92.3 per cent of open-weight tokens on 20 July and 58.1 per cent on 10 August. The two posts may use different denominators or windows; neither says, which is one more reason to quote these shares only with their source and date.

Within open-weight traffic, which this time does include external routing according to the chart note, Zhipu’s GLM family dominated in the week of 2 November with 60.4 per cent of open-weight tokens, led by GLM 5.3 Flash at 33.3 per cent and GLM 5.3 at 21.6 per cent. DeepSeek V4.1 Flash had 28.0 per cent and Moonshot’s Kimi K3 7.9 per cent. On estimated spend, Kimi K3’s share was 21.0 per cent against its 7.9 per cent of tokens, while DeepSeek V4.1 Flash’s was 9.1 per cent against 28.0. The open-weight market inside the index is therefore highly concentrated and moves as fast as the proprietary one: GLM 5.2 held 92.3 per cent of open-weight tokens on 20 July and 5.4 per cent on 2 November.

Our analysis of the open-weight gap argues that open models trail the frontier by months for structural reasons. The index does not contradict that: rising usage of open-weight models on a platform built to host them is evidence about cost-sensitive workloads, not about capability parity.

What “multi-provider” means here

The post says “approximately 64% of enterprises using Unity Gateway relied on just one model provider” in July, falling to 28 per cent by 2 November. The chart behind it counts, “Among companies active each week, … every provider a company has used since July 20, 2026.” Two features of that definition push the number up over time.

First, it is cumulative. A company that tried a second provider once in August still counts as multi-provider in November. For a stable set of companies the measure can only rise, so part of the increase from 36.0 to 71.1 per cent with two or more providers reflects the counting window lengthening from one week to sixteen. The chart does not separate companies actively using several providers in the same week from those that ever did.

Second, the provider grouping counts open-weight siblings with their proprietary parents: “Gemini and Gemma count as one provider, as do GPT and GPT-OSS.” A company that runs Claude and a Databricks-hosted Llama counts as using two providers. That is a fair definition of supplier diversity, but it is not the same as a company splitting production work between Anthropic and OpenAI.

The deeper tiers rose too: 47.9 per cent of active companies had used three or more providers by 2 November, and 31.4 per cent four or more, up from 12.4 and 4.4 per cent in July. Those numbers support the post’s argument that enterprises are “moving beyond a single AI provider”, with the caveat that the customers who choose a multi-provider gateway are not a random sample of enterprises.

Spend per user is small for most and large for a few

The weekly spend chart prices “Each user’s input, cached and output tokens” at list prices for “human users only”, which excludes applications and service accounts. Its series:

Week ofMedian90th percentile99th percentile
20 Jul$0.40$49.47$511.44
21 Sep$1.10$95.66$781.48
2 Nov$2.49$95.96$699.74

Index data, estimated weekly spend per human user at list prices.

The post’s reading is that “AI consumption is growing among typical users, but a small group still accounts for most of the AI bill.” The six-week window it uses fits that: the median more than doubled while the 90th percentile was flat and the 99th fell about 10 per cent. Over the full 16 weeks the picture is broader growth. The median rose about sixfold, and the 90th percentile almost doubled from $49.47.

The ratio of the 99th percentile to the median, 281, comes straight from $699.74 divided by $2.49. The post does not say how “user” is defined across customers, whether inactive users with zero spend are included, or how users at companies with very different headcounts are weighted. A median of $2.49 a week, about $130 a year at list prices, is a useful anchor for budgeting a broad rollout of assistants. A 99th percentile of about $700 a week, the kind of all-day coding-agent use the post contrasts with occasional assistant questions, is the line that drives a budget. Our breakdown of the real cost of an AI product covers why the tail, not the average, sets the bill.

The coding benchmark: what it measures

The index’s fourth section ranks model and harness combinations on tasks from Databricks’ codebase. The July benchmark post describes how the tasks were built. Databricks filtered recent, human-written merged pull requests with “high-quality tests”, rewrote each pull-request description as a task prompt without the solution, held the tests out, and ran the agent until it said it had finished. Then it applied the held-out tests. Tasks span Scala, Rust, TypeScript and React, protobuf and gRPC, and Bazel. Databricks says it reviewed each sample by hand, sometimes rewrote tests to allow alternative implementations, and “sealed git history” after finding that agents could recover the merged solution from the repository’s history.

The November index data, updated on 10 November, contains 20 model entries, each shown in the harness that sits closest to the cost and quality frontier:

Model (harness)LabPass rateCost per taskAverage turnsAverage minutes
Claude Fable 5.1 (Claude Code)Anthropic96.0%$7.4715.722.4
Muse Wonder (Codex)Meta94.9%$1.5334.35.2
DeepSeek V4.1 Flash 1017 (Codex)DeepSeek93.1%$1.0265.143.5
GLM 5.3 (Claude Code)Zhipu93.1%$2.4094.338.0
Claude Fable 5.2 (Claude Code)Anthropic92.6%$5.6943.249.9
GPT 6.1 Sol (Codex)OpenAI92.0%$0.7633.610.3
GPT 6 Astra (Claude Code)OpenAI92.0%$3.2822.611.9
Kimi K3.1 Pro (Claude Code)Moonshot91.4%$0.9727.724.4
Gemini 4 Carbon (Claude Code)Google90.3%$1.9062.026.6
GPT 6 Luna (Claude Code)OpenAI89.1%$0.0537.111.0
GPT 6.1 Terra (Claude Code)OpenAI89.1%$0.1215.05.3
GLM 5.3 Flash (Claude Code)Zhipu89.1%$0.1868.933.2
Claude Opus 5.5 (Claude Code)Anthropic89.1%$1.6229.818.0
Claude Sonnet 5.5 (Claude Code)Anthropic88.0%$0.3316.25.4
Gemini 4 Argon (Codex)Google85.7%$4.8442.923.4
Claude Haiku 5.5 (Claude Code)Anthropic85.1%$0.1631.410.8
Gemini 3.8 Flash (Codex)Google82.9%$11.97140.253.5
Grok 4.7 (Codex)SpaceXSI82.3%$1.7029.819.5
Muse Spark Open (Claude Code)Meta80.6%$0.4219.737.8
Beam-501B-A23B (Claude Code)Reflection AI77.7%$0.4266.313.5

Index benchmark data, updated 10 November 2026, read by GenAI Brief on 18 November. Costs rounded to the cent. Model names as the index gives them, except that the index writes “Gemini 4 carbon” and “Gemini Argon 4”.

The post’s claims about the benchmark all match this table. Five models sit on the cost and quality frontier, from four labs: GPT 6 Luna, GPT 6.1 Sol, DeepSeek V4.1 Flash, Muse Wonder and Claude Fable 5.1, which is what a check of the table confirms. Luna and Opus 5.5 share an 89.1 per cent pass rate at $0.05 and $1.62 per task, “a roughly 32x difference”. Fable 5.1’s 1.1-point lead over Muse Wonder “comes at nearly five times the cost”. And Gemini 3.8 Flash, despite a low token price, “averaged 140 turns per task” and cost $11.97. The post’s “three open-weight models” in the top ten would be DeepSeek V4.1 Flash, GLM 5.3 and, by the index’s own family list, Kimi K3.1 Pro; the post calls the first one “DeepSeek V4.1”, while the data names the Flash variant.

Where the benchmark is strong

The design avoids two common problems with public coding benchmarks. Tasks are private, so they cannot have leaked into training data, and grading uses tests rather than a model judge, which Databricks says “rewards sounding right over being right”. The July post also reports catching and fixing a git-history leak, which suggests the team inspected traces rather than trusting scores. And it reports cost, turns and time alongside pass rate, which is what a buyer needs. Our piece on benchmarks as marketing argues for exactly this kind of task-level, cost-aware evaluation on a team’s own work.

Where it is limited

It is one company’s codebase. The tasks come from Databricks’ monorepo and its engineers’ workflows. The July post says this was the point: public benchmarks “weren’t representative for our codebase”. By the same logic, Databricks’ results are not representative of anyone else’s.

The size of the task set is not published. Neither post gives the number of tasks, the number of runs per model or confidence intervals. Four models tie at exactly 89.1 per cent and two at 93.1 per cent, which is typical of pass rates computed over a fixed, modest set of tasks. Without the denominator, readers cannot tell whether 92.0 and 93.1 per cent are different. The July post was more careful on this point, describing GLM 5.2 as “statistically tied with Opus 4.8”.

One harness per model. Each entry is shown “in the harness, Claude Code or Codex, that sits closest to the frontier”. That is a sensible way to show a model at its best, but it means the table mixes harnesses, and the July post found harness choice could change cost per task by “more than 2x in some cases” for the same model and effort. It also found that a minimal harness, Pi, sent “about 3x less context per turn”. Pi is not in the November index.

Costs are list-price estimates. As with the usage charts, cost per task is computed from token counts at published prices, so it reflects caching behaviour but not discounts.

Some entries cannot be checked anywhere else. The index includes models from Meta, Reflection AI and SpaceXSI whose public documentation we did not review for this article; their results here are Databricks’ measurements only. One result is notable for its direction: Claude Fable 5.2, the newer model, scores 92.6 per cent at $5.69 per task, below Fable 5.1’s 96.0 per cent. The post discusses Fable 5.1 and does not mention Fable 5.2. Databricks’ Day 1 post also says Fable was “not currently rolled out internally due to Anthropic data retention policies”, so Fable’s benchmark runs do not reflect daily use at Databricks.

Why a gateway vendor publishes this

The index is marketing as well as data, and the post makes no secret of it. Unity Gateway is a commercial product; its product page says it “governs more than 1 quadrillion coding agent and tool tokens per year”, a figure Databricks does not break down. A section near the end of the post is headed “Put these insights to work with Unity Gateway” and recommends the product’s model switching, usage dashboards, Smart Routing, and budgets and spend caps. The findings it highlights, that customers switch models often, use several providers, adopt open-weight models and overspend on premium models for routine work, are the conditions under which a multi-provider gateway with routing is most useful.

That alignment does not make the numbers wrong. Where we checked, the post reports its own data accurately, and the chart notes are candid about what is and is not counted, including the statement that wallet share is “not what Databricks bills”. But it should shape how the index is read. The sample is self-selected twice: companies chose Databricks, then chose to route model traffic through its gateway. Companies that call Anthropic or OpenAI directly, use a different gateway, or buy AI as per-seat subscriptions such as chat assistants and coding plans are absent. A data and analytics platform’s customers may also skew toward data engineering and coding workloads, as Tunguz noted.

The index also lacks some things a neutral industry index would usually provide. We did not find a downloadable dataset, a methodology document beyond the chart notes, the number of companies per week, or any weighting scheme: whether a large customer’s tokens count the same as a small customer’s in the share charts is not stated. Databricks says the index is “a living index” with weekly updates, so readers should expect the numbers quoted here to change, and should date any figure they reuse.

How to use the index

Read carefully, the Databricks AI Index is the best public time series we have seen of how a large set of companies actually split model traffic week to week. It is not a market-share survey. Practical ways to use it:

  1. Use it as evidence of churn, not of rankings. The clearest finding is how fast volume moves to new releases, within days in Databricks’ own rollouts. If your architecture assumes one model for a year, the index is a reason to make switching cheap.
  2. Quote shares with their denominators. “30.9 per cent of tokens to Databricks-hosted models” and “61.5 per cent of proprietary-model tokens through Unity Gateway” are accurate. “Open-weight models now have 31 per cent of enterprise AI” is not.
  3. Budget on percentiles, not averages. The median user’s estimated weekly spend of $2.49 and the 99th percentile’s $699.74 are list-price estimates from one platform, but the shape, a long tail driven by heavy agent users, is consistent with Databricks’ own account of why it set per-user budgets.
  4. Compare cost per task, not price per token. The benchmark’s most transferable lesson is that a cheap-per-token model that takes 140 turns can be the most expensive option, while two models with equal pass rates can differ thirtyfold in cost. That lesson holds even though the specific numbers apply to Databricks’ codebase.
  5. Build your own version. The July benchmark post makes the case that “Any team with a backlog of merged PRs is sitting on a benchmark already that no model has trained on.” The index is a template for that exercise, not a substitute for it.
  6. Check the date. Databricks updates the index weekly. Every figure in this article is as of the week of 2 November 2026 for usage, and 10 November 2026 for the benchmark.

The index’s real contribution is to show, from live traffic, that the unit of AI cost that matters is the task and the user, not the token. Its limitation is that the traffic belongs to one vendor’s customers, measured by that vendor. Both things can be true, and the chart footnotes say so.

Sources

  1. Introducing the Databricks AI Index: The state of enterprise AI tokenomics Databricks
  2. Databricks AI Index Databricks
  3. Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase Databricks
  4. How Databricks rolls out frontier models to 12,000 employees on Day 1 Databricks
  5. The Enterprise Case for Open-Weight Models on Databricks Databricks
  6. Unity Gateway Databricks
  7. Strategic Decision or Structural Advantage? Tomasz Tunguz
  8. Databricks index shows enterprise AI churn StartupHub.ai

Frequently asked questions

What is the Databricks AI Index?

A public page Databricks launched on 16 November 2026 that tracks model usage and estimated spend across companies using its Unity Gateway, plus a cost and quality benchmark of coding agents built from Databricks' own codebase. Databricks says it will update the index weekly.

How many companies does the Databricks AI Index cover?

The launch post says the usage data comes from 10,000+ companies using Unity Gateway, spanning digital natives, Fortune 500 and Global 2000 enterprises. The same post and the index page also say thousands of enterprises. Databricks does not publish the exact number of companies in each weekly figure.

Is the AI Index a measure of the whole enterprise AI market?

No. It measures traffic from Databricks customers that route model calls through Unity Gateway. Companies that call model providers directly, use other gateways or buy AI through subscriptions outside the gateway are not in the data, and some charts cover only models hosted by Databricks.

Are the AI Index spend numbers what companies actually paid?

No. The index prices each user's input, cached and output tokens at each provider's published list price. Its chart notes say the figures are not what Databricks bills, and they do not reflect negotiated discounts, committed-use agreements or subscription seats.

Which model topped the Databricks coding benchmark?

As of the 10 November 2026 update, Claude Fable 5.1 in Claude Code had the highest pass rate, 96.0 per cent, at $7.47 per task. Meta's Muse Wonder in Codex was second at 94.9 per cent and $1.53 per task. The benchmark uses tasks from Databricks' private codebase graded by hidden tests.

Did open-weight models really reach 30.9 per cent of enterprise tokens?

Of tokens sent to Databricks-hosted models on Unity Gateway in the week of 2 November 2026, according to the index chart's footnote. The launch post describes it as enterprise token traffic, but the chart excludes traffic routed to external providers such as Anthropic and OpenAI.

All industry →