Mistral Large 4 open weights: free to run in-house, licensed to resell

Mistral Large 4’s weights are free to download, modify, fine-tune and build products on, at any company size. The one condition is about what you do with them. If your company, or any affiliate, runs a Model-as-a-Service business, it must sign a separate commercial licence with Mistral before using the model commercially. The licence file defines Model-as-a-Service as giving a third party inference or post-training access “in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data.” Internal use is excluded. So are products where the model’s capabilities sit inside specific features, and plain relaying of requests to models someone else hosts.

That is a different kind of restriction from the licences on Mistral’s two previous large open models. Mistral Large 3 shipped under Apache 2.0, with no conditions on use. Mistral Medium 3.5’s modified MIT licence blocked companies above $20 million in monthly revenue. Large 4 has no size test at all. It tests activity: a two-person inference startup needs a commercial licence, and a bank with a large internal deployment does not.

Serving is the other half of the decision. The FP8 checkpoint is about 1,060 GB and the NVFP4 checkpoint about 610 GB. vLLM’s recipe validates one four-GPU GB300 tray as the smallest setup. On Hopper, vLLM validates FP8 only across two eight-GPU H200 nodes. On the API, Mistral lists $0.70 per million input tokens, $0.07 cached and $2.10 output. This explainer goes through the licence clause by clause, sorts common deployments by whether they trigger it, works out the memory from the published configuration file, and runs one cost comparison with explicit hypothetical inputs. Licence text, prices and serving recipes were read on 12 November 2026.

What was released on 29 October, and which dates to trust

The open-weight announcement carries no date. It says the weights of the improved model are out “today”. Hugging Face’s launch blog post is dated 29 October 2026, and Artificial Analysis gives the same release date. Other dates in circulation mean different things, so we have put them in order.

The preview post had promised the weights “by the end of the month”. According to the announcement, the open release is not the same model as the preview. Mistral says it ran three more weeks of reinforcement learning, which it credits with lifting the model’s Terminal-Bench 4.0 score from 28% to 41% while cutting output tokens by 30%. The chart labels the 28% preview figure as self-reported. It also renamed the lowest reasoning mode: “Low” replaces the preview’s “None”. A team that tested the preview on 6 October has tested a different checkpoint.

The published numbers do not always match each other. The model card and the NVFP4 card say the model was trained on 3,800 NVIDIA Grace Blackwell GPUs. The technical report says 3,840 GB200 GPUs, over 45 days and about 4.15 million GPU-hours. The card gives 49 billion active parameters per token, or 52 billion counting embeddings and output layers. Mistral’s documentation lists 1.05 trillion total parameters, and Artificial Analysis lists 1.1 trillion. None of these differences changes a deployment decision, but they are worth knowing before you copy a number into a capacity plan.

The licence, clause by clause

The Mistral Ecosystem Open-Weight License is short: a grant, three numbered conditions and a warranty disclaimer. It is worth reading in full, because each sentence does a specific job.

The grant. It starts from MIT’s wording and adds rights written for model weights. The licensee may “use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Model,” and may “run, deploy, further train or otherwise modify the Model and create derivative works from it.” The Model is defined to include the weights, parameters, configuration files and documentation. Fine-tuning, distillation into a derivative and redistributing a quantised build are all inside the grant.

Section 1, notice. The copyright notice and permission notice must travel with “all copies or substantial portions of the Model.” A GGUF or NVFP4 conversion you publish needs the LICENSE file alongside it, and the official NVFP4 repository ships its own copy.

Section 2, the Model-as-a-Service trigger. “If the Licensee or any of its affiliates operates a Model as a Service business, the Licensee must enter into a separate commercial license with Mistral before using the Model or its derivative works for any commercial purpose.” The definition follows: giving a third party access to the Model or its derivatives “for inference or post-training by any means (for instance via API or hosted endpoints), in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data.” It “does not include (a) end-user products with model capabilities solely embedded within specific features or harnesses, or (b) mere relaying of requests to models hosted by others.”

Section 3, the carve-outs. Section 2 does not apply to “(a) internal use of the Model or its derivative works, defined as any use that does not make the Model or its derivative works available to third parties; or (b) any use of the Model accessed through Mistral’s official products.”

The disclaimer. The Model “and any output therefrom” is provided as is, with no warranty of merchantability, fitness or non-infringement. The model card adds a line that is not in the LICENSE file: “You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights.”

Three features of this drafting matter more than the headline.

First, the trigger is a status, and the obligation is broad. Read literally, Section 2 asks whether the licensee or an affiliate operates a Model-as-a-Service business. If so, the commercial licence is required before using the model “for any commercial purpose”, and not only for the hosted offering. Section 3 then removes internal use from that requirement. On this reading, a company that hosts Large 4 for customers also needs the commercial licence for a customer-facing product feature built on the same weights, but not for an internal tool. This is our interpretation, not Mistral’s, and one to confirm with counsel.

Second, the definition is about control, not about access. Every product that calls a model gives users some influence over its inputs. The test is whether a third party can exercise “meaningful control” over inputs, parameters or training data. An API that accepts arbitrary prompts, sampling parameters and fine-tuning datasets is clearly on the wrong side of that line. A form that turns a filled-in invoice into a structured summary is clearly on the right side. Most real products fall somewhere between the two.

Third, “the Model” is a defined term. Model-as-a-Service is defined as giving access to “the Model or its derivative works”, meaning Large 4 itself. We read Section 2 as triggered by hosting Large 4 or a derivative for third parties, not by hosting models in general. A cloud company that hosts other open models but deploys Large 4 only internally would then sit outside the trigger. The phrase “a Model as a Service business” can also be read more broadly, as describing the company’s line of work. That ambiguity is one for counsel, and we flag it again below.

How this compares with Mistral’s previous open licences

Sources: Large 3 model card, Medium 3.5 LICENSE, Large 4 LICENSE, all read on 12 November 2026.

The Medium 3.5 approach left large enterprises out. The Large 4 approach targets the companies that would compete with Mistral’s own API. For a mid-sized or large company building internal tools, this is a loosening: an enterprise above the $20 million monthly line could not use Medium 3.5 at all without a deal, but can run Large 4 internally without asking. For inference providers it is a tightening compared with Large 3, which any host could serve under Apache 2.0.

The Medium 3.5 licence also contains a drafting slip. It refers to “This restriction in (b)” when there is no clause (b). It is a minor error, but it suggests these licences are drafted release by release, and the next one may be worded differently again.

Open weights, not open source

The Open Source Definition says a licence “must not restrict anyone from making use of the program in a specific field of endeavor.” Large 4’s licence imposes a condition on one field, model hosting, so it does not meet that criterion. That matters in two practical places. Some procurement policies treat “OSI-approved” as an automatic pass, and a model governed by this licence should go through legal review. Some regulatory provisions also turn on whether a model is released under a free and open-source licence: Article 53(2) of the EU AI Act is one, as our guide to the GPAI obligations explains, although that exemption does not apply to models with systemic risk. Our analysis of open-weight releases covers why none of 2026’s major open models meets the OSI’s separate Open Source AI Definition.

Which deployments need a commercial licence

The table below is our reading of the licence text against common deployment patterns. It is not legal advice, and Mistral has not published guidance or examples beyond the licence and the one-sentence summaries on the model card and announcement. The confidence column says how directly the text answers each case.

The market already shows how Section 2 plays out. On 12 November 2026, OpenRouter’s listing showed Mistral as the only provider for Large 4, at $0.70 input and $2.10 output per million tokens, and $0.77 and $2.31 for EU in-region routing. The announcement says the model is available through NVIDIA-hosted endpoints and Microsoft Foundry, and calls Microsoft the first cloud provider to offer it. NVIDIA’s build page repeats the Model-as-a-Service condition and tells anyone who wants to “deploy and customize the model in your environment” to contact Mistral. Neither page says what terms those partners agreed. We think a model of this size under Apache 2.0 would have attracted independent hosts quickly, as Large 3 could be served by anyone. Two weeks after release, Large 4 had none on OpenRouter.

The phrases a lawyer will ask about

Five parts of the licence are open to more than one reading. A team whose business sits near any of them should settle the question before deploying, not after.

1. “Meaningful control.” No published guidance says how much control counts. Free-form prompts? A temperature slider? A system prompt the customer writes? Retrieval over a corpus the customer uploads? Each is “control over the inputs or parameters” to some degree.
2. “Solely embedded within specific features or harnesses.” “Harness” can describe a narrow wrapper or a general agent loop. A coding agent that lets users run arbitrary tasks against their repository is a harness in the engineering sense, and possibly also a way for a third party to steer inference.
3. “Third parties.” The internal-use carve-out depends on not making the model “available to third parties”. The licence does not say whether contractors, a parent company’s other subsidiaries or an outsourced support team count. Other model licences sometimes define these terms. This one does not.
4. “Operates a Model as a Service business.” As discussed above, this can be read narrowly (hosting Large 4) or broadly (being in the model-hosting business). The broad reading would reach many cloud and AI infrastructure companies even when their Large 4 use is otherwise internal.
5. What the commercial licence costs. Neither the licence nor the announcement describes the price, structure or availability of the separate commercial licence. The licence points to a contact form. Medium 3.5’s licence said a commercial licence would be granted at Mistral’s “sole discretion”. Large 4’s licence does not use that phrase, but it does not promise terms either.

Three things are clear from the text. The licence has no revenue threshold or user cap. Outputs are not restricted: the only mention of output is in the warranty disclaimer. The Model-as-a-Service trigger reaches derivatives as well as the base model, because Section 2 and the definition both cover “the Model or its derivative works”.

What one trillion total and 52 billion active means in memory

A mixture-of-experts model only computes with a fraction of its weights for each token, but every weight has to sit in GPU memory. For Large 4 that fraction is very small, and the published configuration file shows why.

params.json in the main repository lists 72 layers with a hidden dimension of 6,144 and 64 attention heads. The first layer is a dense feed-forward layer with an inner dimension of 24,576. The other 71 layers each have 128 routed experts plus one shared expert, each expert with an inner dimension of 6,144, and a router that picks four routed experts per token. The vocabulary has 204,800 entries, and input and output embeddings are not tied.

We reconstructed the parameter count from those values. This is our arithmetic, not a figure Mistral publishes, and it ignores norms, routers, the sparse-attention indexers, the DSpark draft head and the 1.6-billion-parameter vision encoder that Mistral’s documentation lists.

Two practical points follow. About 98% of the weights are routed experts, and each token uses about 3% of them. Serving is therefore constrained mostly by memory capacity and bandwidth, not by compute per token, and we think that is why Mistral’s quantisation work is aimed at the experts. The NVFP4 card says the MoE experts are stored in NVFP4 and the other linear layers in FP8, and that the checkpoint needs an FP8 KV cache. The technical report describes the result as “a mixed BF16–FP8–NVFP4 precision checkpoint at 606 GB” and says naive abs-max quantisation, calibrated on 200 million tokens from Mistral’s supervised fine-tuning mix, “matches the FP8 reference in all evaluations”. That is Mistral’s claim. The card shows the comparison only as a chart.

The file listing agrees. Through the Hugging Face API, the main repository’s 54 safetensors files add up to 1,060.6 GB, and the NVFP4 repository’s 32 files to 610.0 GB. The vLLM recipe gives about 128 GiB of weights per GPU across eight GPUs for FP8 and about 76 GiB for NVFP4. AMD’s Quark MXFP4 build for MI350-series GPUs is about 578 GB, according to the same recipe.

That makes the arithmetic for Hopper simple. An H200 has 141 GB of HBM3e, so an eight-GPU node has 1,128 GB. At the 0.85 memory-utilisation setting in Mistral’s sample command, vLLM would plan to use about 959 GB. That is less than the FP8 weights alone, which is consistent with vLLM validating FP8 on H200 only as two nodes, either 16-way tensor parallelism or two pipeline stages of eight. Hugging Face’s blog describes an Inference Endpoints recipe on a single 8 x H200 node serving up to 131K context and 16 concurrent requests. By our arithmetic the FP8 weights take about 94% of that node’s memory, so the recipe must use a higher memory setting than Mistral’s sample command; the blog does not publish its settings. NVFP4 needs hardware support that Hopper lacks: the NVFP4 card targets B200, B300, GB200 and GB300.

The KV cache is where the 1M context gets expensive

Weights are a fixed cost. The KV cache grows with every token of context held for every active request, and it decides how many long agent sessions a node can hold at once. Large 4 cuts the cache in three ways, all visible in params.json and described in the technical report.

Latent attention. The configuration sets kv_lora_rank to 512 with a 64-dimension rotary component. Each token stores a compressed 576-value latent per layer instead of full keys and values for 64 heads.

Cross-layer sharing. The kv_sharing map shows nine of the 72 layers (39, 43, 47 and every fourth layer up to 71) reading layer 35’s cache instead of writing their own. Only 63 layers store KV. The report describes this as YOCO sharing kept among the second-half global-attention layers. The sharing also explains a limitation in SGLang’s cookbook: pipeline parallelism must keep layer 35 and the layers that read its KV cache on one stage.

Sparse attention. The dsa block configures DeepSeek Sparse Attention with a top-k of 2,048. A lightweight indexer picks the 2,048 most relevant earlier tokens for each query, so attention compute stays bounded as context grows. The indexer keeps its own small FP8 key cache.

By our arithmetic, at one byte per value in FP8, this comes to roughly 36 KB per token for the latent cache (63 layers x 576 bytes) plus about 6 KB for indexer keys (the dsa sharing map gives 44 layers their own indexer, each with a 128-dimension key). That is about 42 KB per token before quantisation scales and allocator overhead. A single 1M-token session therefore needs about 42 GB of cache. This is an estimate, not a measured figure. It is consistent with vLLM’s measurement of 3.60 million tokens of KV capacity with eight-way tensor parallelism on two GB300 trays (NVFP4, no speculative decoding, --max-model-len 65536): about 151 GB per GPU, which is roughly what a 288 GB GB300 has left at 0.85 utilisation after about 82 GB of NVFP4 weights and runtime overhead. The FP8 checkpoint’s 2.38 million tokens on the same hardware, and SGLang’s 2.49 million tokens of FP8 cache on eight B300s, fit the same arithmetic.

There is a catch. With plain tensor parallelism, every GPU keeps a full copy of the latent cache. The technical report says naive TP “suffers from KV replication across TP ranks, severely limiting the system’s concurrency and KV-cache hit rate.” Decode context parallelism (DCP) splits the cache across the sequence instead. vLLM measured capacity rising from 3.60 million to 28.76 million tokens on the same hardware with DCP8. Mistral’s report says that, with context parallelism, a native FP8 cache and offloading, it gets about 20 million tokens of KV-cache space on four GB300s in production.

The recipe also states the trade-off: DCP decode is slower than TP8 at inputs up to 60K with 128 sequences. It pays off only when KV capacity is the constraint. A team running short chat turns should leave DCP off. A team running long agent sessions with large shared prefixes probably needs it.

Validated hardware, by source

The table lists configurations that a named source says it has validated or measured, as of 12 November 2026. “Validated” is each source’s own word, and none of these are tests by this publication.

The bottom two rows are for experiments, not production. Mistral publishes no evaluation of the 2-bit quantisations, and 10 to 15 tokens per second is too slow for agent loops that generate tens of thousands of reasoning tokens per task. The rows that matter are GB300, B300 and B200 in NVFP4, and H200 in FP8 if Hopper is what you have.

Running it on vLLM and SGLang

The model card recommends vLLM and gives an SGLang path, and both point to dedicated container images rather than a stock install. The vLLM recipe is tagged for vLLM 0.34.0, but also says “No vLLM release or nightly wheel includes Mistral Large 4 support yet”. Support ships in dedicated container images, vllm/vllm-openai:mistral-large-4 and vllm/vllm-openai-rocm:mistral-large-4. SGLang’s equivalent is lmsysorg/sglang:mistral-large-4, and both need mistral_common 1.13.0 or later to load the tekken.json tokenizer. Pin the image digest in production. A tag like this can be rebuilt underneath you as support moves upstream.

A minimal vLLM command for the NVFP4 build, from the NVFP4 card, with the DSpark flag the card says to add:

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 --ipc=host \
  vllm/vllm-openai:mistral-large-4 \
  --model mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_ds_mla \
  --tool-call-parser mistral \
  --reasoning-parser mistral \
  --enable-auto-tool-choice \
  --gpu-memory-utilization 0.85 \
  --speculative-config '{"method": "dspark", "num_speculative_tokens": 3}'

What the flags do, and what happens if you leave them out:

• Parsers. --tool-call-parser mistral and --reasoning-parser mistral (or auto in SGLang) split tool calls and reasoning from the visible answer. The SGLang cookbook warns that without them, reasoning and tool calls come back as raw text in content. With them, the model card’s vLLM examples read the reasoning from message.reasoning, SGLang returns it in reasoning_content, and the answer arrives in content.
• KV cache type. The NVFP4 card says the checkpoint needs --kv-cache-dtype fp8_ds_mla, an FP8 KV cache for the latent-attention layers. vLLM’s recipe says the image already uses that format by default, so the flag is harmless there but not required.
• DSpark. The checkpoint includes a speculative-decoding draft head: three dense latent-attention layers in BF16 that propose three tokens at a time. Mistral’s card reports “up to 1.9x higher throughput” than the target model alone. vLLM’s recipe measured 3.13 ms time per output token at concurrency 1 on 8 x GB300 in NVFP4 (3.56 ms in FP8), with an acceptance length of about 2.9, and puts the gain at 1.9 to 2.2 times output throughput. SGLang reports about 3.7 accepted tokens per four-token verify step. DSpark does not change the target model’s outputs, according to the card. It does not yet run on a DCP decode node in a prefill-decode split, and vLLM has not validated it with pipeline parallelism.
• Context length. vLLM’s default recipe sets --max-model-len 262144 and says serving at 1M is not validated. SGLang says to leave --context-length unset for the native 1,048,576 tokens. NVIDIA’s hosted endpoint lists a 262,144-token context. If your product depends on the 1M window, test that exact length on your stack. Do not assume it works because the checkpoint advertises it.
• Reasoning effort. The parameter takes low, medium or high, with high as the default. SGLang returns HTTP 400 for none. The Hugging Face blog says reasoning cannot be disabled in Transformers, so none still runs at high. max_completion_tokens counts reasoning tokens as well as the visible answer, so a budget set for a non-reasoning model will cut answers short.
• Sampling. Mistral recommends temperature 1.0 and top-p 0.95 at every effort level.

Beyond one node, the vLLM recipe documents prefill-decode disaggregation presets (separate GPU pools for prompt processing and generation). In an agentic preset replayed from real coding-agent traces (mean input about 101K tokens) at 32 concurrent sessions, median time to first token fell from 14.95 seconds to 0.72 seconds when the decode side used DCP4, because the decode node no longer ran out of KV cache. For production at scale, the technical report says Mistral runs on llm-d for routing and NVIDIA’s Model Express for peer-to-peer checkpoint transfer, which brings new replica start times “below 120 seconds”.

API versus self-hosting: a worked example

Self-hosting is cheaper than the API only when a node stays busy. The example below shows how busy, using published throughput and Mistral’s list prices. Its hourly hardware cost is a hypothetical input that you should replace with your own quote. For GPU rental prices as of October 2026 (H100 and B200, not GB300), see our self-hosted LLM guide.

Throughput input. SGLang’s cookbook reports 2,812 total tokens per second per GPU on 4 x GB300, NVFP4, TP4, with DSpark at concurrency 64. “Total” counts input and output tokens. The benchmark used 1,024-token random prompts, 512 output tokens, 64 requests per concurrency point and a flushed cache. Four GPUs come to 11,248 tokens per second, or about 40.5 million tokens an hour, two-thirds input and one-third output.

API price for the same traffic. At $0.70 input and $2.10 output with no cache hits, the same mix costs about $1.17 per million tokens. An hour of saturated traffic would cost about $47.24 on the API. That figure is the break-even: if the tray costs less than $47.24 an hour all in (hardware or rental, power, networking, staff time) and runs flat out, self-hosting is cheaper.

Utilisation. Real traffic does not run flat out. The table divides three hypothetical all-in hourly costs by the throughput actually used.

The same calculation with SGLang’s 8 x B300 FP8 figure (1,018 total tokens per second per GPU with DSpark, same benchmark) gives about 29.3 million tokens an hour on eight GPUs, worth about $34.20 at API prices. The NVFP4 tray does more work on half the GPUs, so we think the FP8 checkpoint makes economic sense mainly on hardware you already own.

Why this example flatters self-hosting:

• Short prompts. The benchmark used 1,024-token prompts. Agent sessions run tens of thousands of tokens. vLLM’s recipe shows output throughput per GPU in its expert-parallel preset falling from 209.8 tokens per second at 8K input to 50.6 at 32K.
• No caching. The benchmark flushed the cache. Prefix caching also helps a self-hosted server, but the API bills cache hits at a tenth of the input price, which Artificial Analysis uses to reach a $0.40 blended rate.
• Reasoning tokens. At high effort, reasoning tokens are billed as output on the API and cost decode time on your own GPUs. Artificial Analysis reports that running its Intelligence Index at high effort generated 107 million output tokens, at an average of $2.06 per task.

Why this example understates self-hosting:

• Data control. Keeping prompts and fine-tuning data inside your own environment is the reason many teams self-host. It does not show up in a per-token price.
• Prefill-decode splits and DCP. vLLM reports 241 output tokens per second per GPU at 32K input with a 4+4 GB300 split, and says disaggregated 4+4 serving beats colocated eight-GPU serving by four to five times at 32K and 60K input.
• Data residency. Mistral’s EU in-region route on OpenRouter costs $0.77 and $2.31, 10% above the standard rate.

The licence adds a further cost line for some teams. A company that self-hosts to resell access needs the commercial licence, and its price is not public. Until it is, nobody outside Mistral can calculate the economics of hosting Large 4 for others, and comparisons that leave it out are incomplete.

For internal use the calculation is clearer. A team that can keep a GB300 tray busy with its own workloads, at an all-in cost well under $47 an hour, should beat Mistral’s list price on short-prompt traffic like the benchmark’s. A team with spiky traffic, long contexts and heavy cache reuse will find the API cheaper.

What remains unknown, and what would change this assessment

• The commercial licence. Price, structure, the definition of affiliate and whether the terms are standard or negotiated are all unpublished. A published price list or standard agreement would make the Model-as-a-Service economics calculable.
• Mistral’s reading of “meaningful control”. An FAQ with worked examples, or a statement on general chat products, would settle the low-confidence rows in our deployment table. Until then, a team in that zone either asks Mistral in writing or treats itself as covered by Section 2.
• Whether NVFP4 really matches FP8 on your workload. Mistral’s evidence is a chart on the NVFP4 card and one sentence in the report. An independent comparison on long-context agent tasks would confirm or overturn the claim. A team can run its own: the same 200 production prompts through both checkpoints at the same effort level, scored on task success.
• Independent capability numbers. Many headline results in the announcement come from third-party indexes, such as Artificial Analysis and Cognition’s FrontierCode, but Mistral chose which to show, and the preview comparison point is self-reported. Artificial Analysis independently scores the model 26 on its Intelligence Index v5.0.1. It reports 119 output tokens per second and 31.65 seconds to the first answer token on Mistral’s API at high effort, averaged over the previous five days of runs. This article’s conclusions do not depend on Mistral’s coding claims, but a team’s choice between the API and self-hosting does depend on whether the model is good enough for the job in the first place. Our guide to reading a model card covers how to weigh vendor-reported results.
• 1M-context serving. vLLM has not validated the full window, and NVIDIA’s hosted endpoint lists 262K tokens. A published vLLM recipe at 1,048,576 tokens with throughput figures would change the hardware guidance for long-context users.
• Upstream support. vLLM’s container-only support is a maintenance risk. A tagged vLLM release with Large 4 support would remove it.

How to decide

1. Classify your use against Section 2 before you size any hardware. If you or an affiliate would give third parties control over inputs, parameters or training data, contact Mistral first. The licence says the commercial agreement must be in place “before using the Model” commercially.
2. For internal use, write the boundary down. Record who can reach the endpoint, and keep that record. “Internal” ends wherever the model becomes “available to third parties”, and contractors and partner organisations are where that line most often gets crossed.
3. Start on the API and measure your traffic. Record input, cached input, reasoning and output tokens per task at the effort level you will actually use. That is the number to compare with the table above.
4. Size self-hosting on NVFP4 and Blackwell if you can. One four-GPU GB300 tray is the validated minimum. FP8 on H200 takes two nodes in vLLM’s validated setups. Treat 2-bit GGUFs as research tools.
5. Turn on DSpark, set the parsers, and test the context length you need. Do not count on 1M tokens until you have run it on your own stack.

Mistral has put trillion-parameter weights on Hugging Face without a revenue threshold, which is more permissive than its previous licence for any company that wants to run the model for itself. The restriction is aimed squarely at hosts who would compete with Mistral’s API. For most builders, the practical questions are whether their product gives users “meaningful control” over the model, and whether they can keep four GB300s busy.