dgx_spark_benchy

Best model for...

Different from the No.1 picks above on purpose — each card here picks a winner for one specific thing you might care about, straight from the same raw measurements, instead of one composite number trying to represent everything at once. A model can win a category here and rank modestly elsewhere, or vice versa.

Top 5 by the numbers

A faster-to-scan companion to the detail tables below, not a replacement — each panel ranks every model with data for that one metric and shows the top 5, bar length and the number at the tip both carrying the same value. Hover or tab through a bar for the full model name if it's cut off.

Overall quality ranking

General-purpose accuracy across 22 graded tasks (tool use, coding, safety, instruction-following, and more). LocalScore is one 0-100 number combining how often the model got the task right (Quality), how consistent it was across repeats (Reliability), how good its accuracy-per-token was (Efficiency), and how fast it started responding (Responsiveness). A different 0-100 number from Hermes Score below — see that section for what that one means. Avg tok/s here is a blended average across every decode-speed reading recorded for this model (short quick tasks and long-context runs together), so it can land well below the model's best case — that peak number is what Best model for → Fastest and the Decode peak column further down report instead.

• * No vLLM-compatible tool-call/reasoning parser exists for this model's custom "Onyx ATEM" chat template -- confirmed 2026-08-16, no matching vLLM plugin available. A harness-side workaround (spark_bench_plus.py's chat_stream) strips its raw reasoning text before grading, which should help plain-text domains, but its tool calls are also raw text that never reaches vLLM's tool_calls field, so tool-use-dependent numbers specifically may still understate this model's real capability. See METHODOLOGY.md.

How many people can use it at once

The peak number of simultaneous requests of that traffic type this box was actually tested against — orchestrator is multi-step tool-chain traffic (the shape an autonomous agent sends), coding_agent is code-generation requests, chat_agent is casual back-and-forth conversation. All three are swept over the same concurrency levels (1, 2, 4, 8, 16, 32), so the numbers are directly comparable to each other.

Hermes Benchmark — best model for a personal-agent harness

One 0-100 number for "how good would this model be behind a Hermes-style personal agent": 50% how well it actually completes real agent tasks (tool use, web search, remembering things across a conversation), 30% how many concurrent sessions it sustains, 20% how quickly it starts responding.

Smallest model that's actually good enough (70+/100): google/gemma-4-12B-it (~12B, score 79.0) — the one to reach for if you're tight on memory. Largest model tested: nvidia/Gemma-4-31B-IT-NVFP4 (~31B, score 80.6) — the highest-quality option this box can run.

Context scaling — does speed hold up as documents get longer?

Same-model decode speed at its smallest vs. largest tested context, both from the same speed sweep run. Stability is 100 × (tok/s at largest context) / (tok/s at smallest context) — 100% means no slowdown at all as the prompt grows; a low number means the model falls off hard on long documents even though it might look fine on a short prompt. Peak is the single fastest decode_tps ever recorded for this model at any context size, no quality filter. reasoning? is derived from whether any measured reasoning time was ever recorded for this label, not a name guess — note that a reasoning-capable model's thinking-ON and thinking-OFF entries can have very different stability curves, so check the label, not just the model name.

Reasoning models average stability: 46% (n=7) vs. non-reasoning: 90% (n=7). Most stable at long context: RedHatAI/Muse-Glimmer-30B-NVFP4 (muse-glimmer-30b-nvfp4) (96%). Fastest single-context peak (no quality filter): RedHatAI/Muse-Glimmer-30B-NVFP4 (muse-glimmer-30b-nvfp4) (210 tok/s).

Detailed inference performance

Best-case single-stream timing at the smallest context tested, the largest context that actually returned a real answer, and the GPU memory budget each model was served at. TTFT = time to first token, TPOT = time per output token (decode latency), prefill = how fast it reads the prompt before answering. Max document size is the actual measured prompt token count of the largest successful run, not the round number passed to --contexts — different tokenizers turn the same target size into different real token counts, so this can land a bit above or below what was requested.

Machine telemetry

Real GPU behavior sampled every 5s for the duration of each model's benchmark run (not the box's own hermes-vllm production traffic) — how hot, how loaded, and how much memory it actually used, versus the --gpu-memory-utilization flag we asked vLLM to target. Only present for runs after telemetry capture was added; older runs show —.