# SCRIPT — FINAL

# Just because you believe something doesn't make it true.
## Ep. 01: On-prem AI — Belief vs. Benchmark

GATE2b approved by Chetan via octopai-cc with one viewer-copy amendment:remove the Router measurement-path disclaimers. Router narration and end credit are unchanged. No Router performance number is claimed. Main208s;native4:5 teaser56s. Audio and all63main/10teaser caption units are unchanged from the approved draft.

Eight commitments remain 2.510–2.841s, with hard pre-stamp music drops verified in the finished WAV and encoded AAC. FP8 is an exact 5s interlude, with 100–160ms subtitle lead. Actual scene/cue timings below are authoritative rather than the earlier proposal subslots.

## 00:00.00–00:26.00 — OPEN

FACTS/source keys: hw-server;credit-bench;historical references in SOURCES.
Visual: original typography / local conceptual imagery / brand geometry.

**Narration**

> Are you benchmarking and deploying your on-prem AI models correctly? Or do you only believe you do? Buying GPUs you think are full? Scheduling a reboot? New observations challenged Earth-centred astronomy. Medical handwashing met resistance. We believed some GPU rules too. Then we benched a Dell PowerEdge R770. We measured it with Metrum AI Bench-CLI, our open-source benchmark tool. Can zero and a hundred both mislead?

**Viewer copy**

- Are You Benchmarking And Deploying Your On-Prem AI Models Correctly?
- Or Do You Only Believe You Do?
- 0%? 100%?
- Buying More GPUs? Scheduling A Reboot?
- Observation Challenged The Belief.
- New observations challenged Earth-centred astronomy. A medical handwashing policy met resistance before germ theory was widely accepted.
- We Believed Some Rules Too. So We Measured.
- Dell PowerEdge R770 with 2× NVIDIA RTX PRO 6000 Blackwell Server Edition.
- Can Both Mislead? Different Tests.
- Just because you believe something doesn't make it true.
- OPEN / ON-PREM AI: BELIEF VS. BENCHMARK

## 00:26.00–00:39.00 — CHAPTER 1

FACTS/source keys: mig-reboot.
Visual: reboot.
Commitment: 00:29.95–00:32.66;reveal/stamp 00:32.66.

**Narration**

> Changing hardware partitions sounds disruptive. Must the host reboot? In our test, no. Each layout step took under a tenth of a second. Model restarts were separate.

**Viewer copy**

- MYTH: Changing MIG Means A Host Reboot.
- A · Host Reboot.
- B · No Host Reboot.
- A Or B?
- No Host Reboot.
- <0.1 s
- Each Layout Step.
- In our test: enable / create / destroy / disable, 0.02–0.07 s each. One boot ID. Restarting model servers is separate and slow.
- MIG enable / create / destroy / disable. Test the host reboot assumption.
- Metrum AI Bench-CLI v1.6.0 · open source
- CHAPTER 1 / ON-PREM AI: BELIEF VS. BENCHMARK

## 00:39.00–00:57.00 — CHAPTER 2

FACTS/source keys: mig-more-total;mig-quarter-performance.
Visual: concurrency.
Commitment: 00:41.93–00:44.66;reveal/stamp 00:44.66.

**Narration**

> Two lanes sound faster. Predict the busy case. In our test, two halves helped at light load, then lost at high load. A lone request barely noticed the slice. Single-user speed is not combined capacity.

**Viewer copy**

- MYTH: More Slices Means More Capacity.
- A · More Total Throughput.
- B · No Added Capacity.
- A Or B?
- Load Decides.
- 1.18×
- At 16 Requests. Not At Every Load.
- Two half slices / whole GPU, same total requests. Qwen3.8-27B NVFP4; eager; 8k context; 1024 in / 128 out. In our test. Ratios, not normal serving speed.
- Two half slices versus one whole GPU at the same total request load.
- CHAPTER 2 / ON-PREM AI: BELIEF VS. BENCHMARK

## 00:57.00–01:15.00 — CHAPTER 3

FACTS/source keys: mig-noisy-neighbour;mig-shared-wreck.
Visual: isolation.
Commitment: 01:01.37–01:04.16;reveal/stamp 01:04.16.

**Narration**

> A wall feels safer. Flood the neighbour: which probe slows? In our small test, neither did. But partitions still share one power and clock budget. No slowdown here is not a universal guarantee.

**Viewer copy**

- MYTH: Only MIG Makes Sharing Safe.
- A · Shared GPU Slows.
- B · Neither Probe Slows.
- A Or B?
- Neither Probe Slowed Here.
- 119.6 → 119.1
- ms/token, MIG Probe.
- Shared probe: 120.0 → 119.7 ms/token. One probe user, 6 requests; ≥50/64 neighbour running verified. About 154 → 301 W is MIG whole-card ONLY. Tails not ruled out; heavier mixed loads untested.
- One probe user, six requests; neighbour flood verified at ≥50/64 running.
- CHAPTER 3 / ON-PREM AI: BELIEF VS. BENCHMARK

## 01:15.00–01:31.00 — CHAPTER 4

FACTS/source keys: mig-27b-small-slice;sw-models.
Visual: quarter.
Commitment: 01:19.19–01:21.78;reveal/stamp 01:21.78.

**Narration**

> Twenty-seven billion parameters. A quarter slice. Does it fit? In our test, yes—with four-bit weights and a short context. We had to fit the serving configuration, not just the model.

**Viewer copy**

- MYTH: A 27B Model Needs A Bigger Slice.
- A · Cannot Fit.
- B · Can Fit.
- A Or B?
- It Fit. Configuration Matters.
- 19.0 GiB
- Weights In A 23.6 GiB Quarter Slice.
- Qwen3.8-27B NVFP4; 1g.24gb; 4k context; up to 8 requests; kernel autotuning off; 1024 in / 128 out; one user. In our test. Initial 8k: autotuner scratch allocation failure, not weight OOM. Cold first response: 103 s.
- Qwen3.8-27B NVFP4 on one 1g.24gb quarter slice. Test the serving configuration.
- CHAPTER 4 / ON-PREM AI: BELIEF VS. BENCHMARK

## 01:31.00–01:55.00 — CHAPTER 5

FACTS/source keys: mig-smi-blind;util-100;util-100-hook.
Visual: counters.
Commitment: 01:35.88–01:38.66;reveal/stamp 01:38.66.

**Narration**

> Those zero and hundred readings. Surely one tells us the GPU is full? Under MIG, zero hid active SMs. In a different test, a hundred still left tensor cores mostly idle. The memory pipe was the bottleneck. Read what is waiting.

**Viewer copy**

- MYTH: Utilisation Tells Us If The GPU Is Full.
- A · One Reading Is Enough.
- B · Read The Bottleneck.
- A Or B?
- 0% ≠ Idle.
- nvidia-smi: 0%. GPM SM active: 64%. Without MIG: 98%.
- About 86%
- Tensor Idle. nvidia-smi: 100%.
- 79.4%
- Memory Bandwidth. Tensor Active: 13.6%.
- MIG layout test: two halves, 128 requests; eager 1024 in / 128 out. Whole-card counters. SM active is not tensor activity.
- DIFFERENT serving test: 64 users, Qwen3.8-27B NVFP4, 1024/1024, GPU 0; 74 one-second GPM sample means. Tensor active: 13.6%; memory bandwidth: 79.4%. Decoding was memory-bandwidth-bound here.
- Read The Bottleneck.
- CHAPTER 5 / ON-PREM AI: BELIEF VS. BENCHMARK

## 01:55.00–02:13.00 — CHAPTER 6

FACTS/source keys: mtp-spec.
Visual: mtp.
Commitment: 01:58.79–02:01.53;reveal/stamp 02:01.53.

**Narration**

> Extra guesses sound cheap. Are they free in a full batch? Spare seats help; full buses pay. Two draft tokens nearly doubled one-user output. At 256 users, the gain disappeared—and first-token tails grew.

**Viewer copy**

- MYTH: Turn On Speculative Decoding Everywhere.
- A · Free Extra Speed.
- B · Depends On The Batch.
- A Or B?
- Spare Compute Helps.
- 1.95×
- One User. 0.991× At 256 Users.
- Native MTP, two draft tokens versus default; 1024/1024. At 256 users, p99 time to first token: 20.4 → 105.6 s.
- Two draft tokens; spare compute versus a full batch; 1024/1024.
- CHAPTER 6 / ON-PREM AI: BELIEF VS. BENCHMARK

## 02:13.00–02:18.00 — 5-SECOND QUICK HIT

FACTS/source keys: fp8-kv.
Visual: fp8.

**Narration**

> FP8 KV? More capacity. Almost no speed gain in this test.

**Viewer copy**

- Capacity ≠ Speed.
- 1.853×
- KV Capacity. Output: 1.016×.
- 128 users; 1024/1024. No output gain at 1 or 32 users. Capacity and output are different measures.
- 5-SECOND QUICK HIT / ON-PREM AI: BELIEF VS. BENCHMARK

## 02:18.00–02:34.00 — CHAPTER 7

FACTS/source keys: b200-vs-r770;hw-b200;sw-vllm.
Visual: b200.
Commitment: 02:21.90–02:24.41;reveal/stamp 02:24.41.

**Narration**

> One multiplier makes sizing easy. Does it survive a load sweep? The B200 lead ranged from 2.45 to 4.15 times. Compare the workload, not a fixed hardware promise.

**Viewer copy**

- MYTH: Size With One GPU Multiplier.
- A · One Fixed Multiplier.
- B · Compare Each Load.
- A Or B?
- Compare Each Load.
- 2.45–4.15×
- The Lead Changes With Load.
- One NVIDIA B200 (cloud) versus one RTX PRO 6000. Same Qwen3.8-27B NVFP4, software, prompts and Bench-CLI options; 1024/1024. Range across 1–256 users; measured endpoints only.
- One NVIDIA B200 versus one RTX PRO 6000. Same model, software, prompts and options.
- CHAPTER 7 / ON-PREM AI: BELIEF VS. BENCHMARK

## 02:34.00–02:50.00 — MODEL CLOSER

FACTS/source keys: bigger-slower;sw-models.
Visual: model.
Commitment: 02:36.19–02:39.03;reveal/stamp 02:39.03.

**Narration**

> More parameters, more work? The thirty-billion model used only three billion per token—and ran faster. Measure what is read, not just what is stored. Then measure quality beside speed.

**Viewer copy**

- MYTH: Pick The Smaller Model For Speed.
- A · 27B Dense.
- B · 30B Total / 3B Active.
- A Or B?
- Read Less. Run Faster.
- 301.5 vs 62.7
- tok/s, One User.
- NVIDIA Nemotron 3.5 Lightning 30B-A3B NVFP4 versus Qwen3.8-27B NVFP4; 1024/1024. GPQA: Qwen 79.8% of 198; Nemotron 65% of a fixed 40-question subset. Different samples; speed is not overall quality.
- 30B total / 3B active versus 27B dense; one user, 1024/1024.
- Speed ≠ Overall Quality.
- GPQA Diamond: Qwen 79.8% of 198.
- Nemotron 65% of a fixed 40-question subset.
- Different Evaluation Sample Sizes.
- MODEL CLOSER / ON-PREM AI: BELIEF VS. BENCHMARK

## 02:50.00–03:09.00 — BENCHMARK HABITS

FACTS/source keys: derived habits;credit-bench;product repositories.
Visual: original typography / local conceptual imagery / brand geometry.

**Narration**

> Like the history we opened with, test the belief. Sweep concurrency. Read GPM and memory bandwidth. Flood the neighbour in your real layout. Report per-load ratios. Measure quality beside speed. Configure concurrency and output-length checks in Bench-CLI. For deployment, Metrum AI Router routes requests among configured models.

**Viewer copy**

- Test The Belief.
- 1. Sweep Concurrency.
- 2. Read GPM And Memory Bandwidth.
- 3. Flood A Neighbour In Your Real Layout.
- 4. Report Ratios Per Load.
- 5. Measure Quality Beside Speed.
- Metrum AI Bench-CLI · open source
- Metrum AI Router
Routes Requests Among Configured Models.
- Measured with Metrum AI Bench-CLI (open source).
- BENCHMARK HABITS / ON-PREM AI: BELIEF VS. BENCHMARK

## 03:09.00–03:23.00 — THE COROLLARY / A WINK

FACTS/source keys: campaign/corollary/kicker.
Visual: original typography / local conceptual imagery / brand geometry.

**Narration**

> Just because you believe something doesn't make it true. Just because you disagree with something doesn't make it wrong. If anything, it might make it more likely to be right.

**Viewer copy**

- Just because you believe something doesn't make it true.
- Just because you disagree with something doesn't make it wrong.
- …if anything, it might make it more likely to be right. A Wink, Not A Law.
- THE COROLLARY / A WINK / ON-PREM AI: BELIEF VS. BENCHMARK

## 03:23.00–03:28.00 — METRUM AI

FACTS/source keys: credit-bench;credit-smi;brand assets.
Visual: original typography / local conceptual imagery / brand geometry.

**Narration**

(No narration.)

**Viewer copy**

- Just because you believe something doesn't make it true.
- Powering Enterprise AI From Insight to Infrastructure.
- Measured with Metrum AI Bench-CLI (open source)
- github.com/metrum-ai/bench-cli
- Also from Metrum AI: Router · open source
- github.com/metrum-ai/router
- Next episode: AI agents and CPU cores
- www.metrum.ai · © 2026 Metrum AI, Inc.
- GPU telemetry: all-smi, Metrum AI fork
- METRUM AI / ON-PREM AI: BELIEF VS. BENCHMARK

## Native4:5 teaser —56s

Independent layout and approved whole-sentence excerpts;no new voice generation.
- 00:00.00–00:04.01: Are you benchmarking and deploying your on-prem AI models correctly?
- 00:04.06–00:06.15: Or do you only believe you do?
- 00:06.16–00:11.14: We measured it with Metrum AI Bench-CLI, our open-source benchmark tool.
- 00:14.65–00:17.51: Under MIG, zero hid active SMs.
- 00:17.56–00:21.66: In a different test, a hundred still left tensor cores mostly idle.
- 00:23.00–00:24.70: Must the host reboot?
- 00:27.03–00:28.73: In our test, no.
- 00:34.53–00:39.24: The thirty-billion model used only three billion per token—and ran faster.
- 00:41.00–00:44.16: Just because you disagree with something doesn't make it wrong.
- 00:44.17–00:47.97: If anything, it might make it more likely to be right.

### Teaser scene1 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Benchmarking Your On-Prem AI Correctly? Or Only Believing?
- Buying More GPUs? Scheduling A Reboot?
- Metrum AI
- Bench-CLI
- v1.6.0 · open source
- Are you benchmarking and deploying your
- on-prem
- AI models correctly?
- Or do you only believe you do?
- We measured it with Metrum AI
- , our open-source benchmark tool.

### Teaser scene2 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Can Both Counters Mislead?
- A · One Reading Is Enough.
- B · Read The Bottleneck.
- 0% ≠ Idle.
- GPM SM Active: 64%.
- About 86%
- Tensor Idle In A DIFFERENT Test.
- Read The Bottleneck.
- MIG: two halves, 128 requests, eager 1024/128; without MIG: 98%. DIFFERENT 64-user serving test: 100% util, 13.6% tensor active, about 86% idle; 79.4% bandwidth; 74 one-second means.
- Metrum AI
- Bench-CLI
- v1.6.0 · open source
- Under MIG, zero hid active SMs.
- In a different test, a hundred still left tensor cores mostly idle.

### Teaser scene3 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Must The Host Reboot?
- A · Reboot.
- B · No Host Reboot.
- No Host Reboot.
- BUSTED: 0.02–0.07 s each; one boot ID. In our test. Model-server restarts are separate and slow.
- <0.1 s
- Each MIG Layout Step.
- Must the host reboot?
- In our test, no.

### Teaser scene4 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Which Model Is Faster?
- A · 27B Dense.
- B · 30B Total / 3B Active.
- Nemotron 3.5 Lightning 30B-A3B NVFP4 vs Qwen3.8-27B NVFP4; 1024/1024. GPQA: Qwen 79.8% of 198; Nemotron 65% of a fixed 40-question subset. Different quality samples.
- Active Work Matters.
- 301.5 vs 62.7
- tok/s, One User.
- The thirty-billion model used only three billion per token—and ran faster.

### Teaser scene5 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Just because you disagree with something doesn't make it wrong.
- If anything, it might make it more likely to be right. A Wink, Not A Law.
- If anything, it might make it more likely to be right.

### Teaser scene6 viewer copy

- EP. 01 / BELIEF VS. BENCHMARK
- Just because you believe something doesn't make it true.
- Measured with Metrum AI
- Bench-CLI
- (open source)
- github.com/metrum-ai/bench-cli
- Also from Metrum AI: Router · open source
- github.com/metrum-ai/router
- Powering Enterprise AI From Insight to Infrastructure.
- Next episode: AI agents and CPU cores
- www.metrum.ai · © 2026 Metrum AI, Inc.
