WEBVTT

1
00:00:00.046 --> 00:00:02.113
Zero percent. A hundred percent.

2
00:00:04.035 --> 00:00:06.531
Under MIG, nvidia-smi read zero

3
00:00:06.705 --> 00:00:08.423
while GPM showed the SMs

4
00:00:08.493 --> 00:00:10.037
active sixty-four percent of the

5
00:00:10.095 --> 00:00:10.513
time.

6
00:00:10.815 --> 00:00:11.871
In a different test, a

7
00:00:11.953 --> 00:00:14.147
hundred percent utilisation still left

8
00:00:14.182 --> 00:00:16.155
tensor cores idle about eighty-six

9
00:00:16.202 --> 00:00:17.247
percent of the time.

10
00:00:20.058 --> 00:00:21.533
To carve up this GPU,

11
00:00:21.625 --> 00:00:23.146
must we reboot the server?

12
00:00:23.425 --> 00:00:24.365
What would you guess?

13
00:00:24.679 --> 00:00:26.281
In our test, no. Every

14
00:00:26.339 --> 00:00:27.546
layout step took less than

15
00:00:27.581 --> 00:00:28.591
a tenth of a second.

16
00:00:30.045 --> 00:00:31.212
Then surely a bigger model

17
00:00:31.269 --> 00:00:31.886
is slower?

18
00:00:32.148 --> 00:00:33.954
Not this one. Nemotron's thirty

19
00:00:34.011 --> 00:00:35.703
billion parameters include only three

20
00:00:35.749 --> 00:00:37.623
billion active per token. It

21
00:00:37.680 --> 00:00:39.005
served one user faster than

22
00:00:39.074 --> 00:00:41.211
dense Qwen. Count what's read,

23
00:00:41.360 --> 00:00:42.549
not just what's stored.

24
00:00:43.000 --> 00:00:44.323
Just because you disagree with

25
00:00:44.358 --> 00:00:45.914
something doesn't make it wrong.

26
00:00:46.192 --> 00:00:47.365
If anything, it might make

27
00:00:47.400 --> 00:00:48.259
it more likely to be

28
00:00:48.317 --> 00:00:48.747
right.
