WEBVTT

1
00:00:00.000 --> 00:00:04.014
Are you benchmarking and deploying your on-prem AI models correctly?

2
00:00:04.064 --> 00:00:06.150
Or do you only believe you do?

3
00:00:06.160 --> 00:00:11.141
We measured it with Metrum AI Bench-CLI, our open-source benchmark tool.

4
00:00:14.655 --> 00:00:17.511
Under MIG, zero hid active SMs.

5
00:00:17.561 --> 00:00:21.661
In a different test, a hundred still left tensor cores mostly idle.

6
00:00:23.000 --> 00:00:24.700
Must the host reboot?

7
00:00:27.030 --> 00:00:28.730
In our test, no.

8
00:00:34.530 --> 00:00:39.244
The thirty-billion model used only three billion per token—and ran faster.

9
00:00:41.000 --> 00:00:44.164
Just because you disagree with something doesn't make it wrong.

10
00:00:44.174 --> 00:00:47.974
If anything, it might make it more likely to be right.
