Video Generation Times That Would Not Settle on a Single RTX 3090

This page contains advertising (affiliate links). See our Privacy Policy for details.

I have been running ComfyUI on an RTX 3090 in my Ubuntu machine for a while. While trying out video generation, the time each run took moved around a lot, even though nothing about the settings changed. The first run took 72.1 seconds. The fourth took 32.1 seconds. That is more than double the gap.

My first thought was that the card had warmed up and got faster. But a few things did not fit that story.

This article works out what was behind the swing in video generation time, using the temperature and VRAM logs. I compared two models of different sizes on the same RTX 3090.

Why do the times line up on some runs and not on others, on one and the same graphics card?

The previous article covered why ComfyUI can hand you a false number when you time it.

Sponsored

How I measured this

Everything here came off one machine.

Machine Graphics card VRAM
Ubuntu box RTX 3090 24GB (24,576MiB)

VRAM is the memory that lives on the graphics card itself. It is separate from the memory in the computer, and image and video models are loaded into it to do their work.

The software was ComfyUI. Video generation used a setup built from two 14B (14 billion parameter) models. Image generation used SDXL.

Each condition was repeated 12 times. A single run cannot tell you whether something was slow that once or slow in general.

Sponsored

Twelve video generation runs in a row

The same video, the same settings, twelve times. The first run is kept separate because it includes loading the model.

Run Time GPU temperature Clock
1 72.1 s 74°C 1,735MHz
2 40.0 s 75°C 1,715MHz
3 36.0 s 75°C 1,687MHz
4 32.1 s 75°C 1,694MHz
5 32.1 s 76°C 1,693MHz
6 32.0 s 75°C 1,720MHz
7 failed
8 40.0 s 75°C 1,679MHz
9 36.0 s 75°C 1,668MHz
10 32.0 s 75°C 1,704MHz

Does temperature account for it?

The card was already at 74°C on the first run and barely moved afterwards. The clock stayed between 1,668 and 1,735MHz, which is not much of a change either.

Getting faster as it heats up does not make sense to begin with. Silicon slows down when it gets hot, not the other way round. The logs do show throttling from the second run onwards. The times still came down.

There is a second thing. Look at what happens after run 7 fails: the sequence starts over from 40.0 seconds. The temperature runs straight through, but the timing goes back to the beginning. If heat were the cause, it would not reset like that.

Sponsored

VRAM, sampled every half second

To find whatever else was moving, I logged VRAM usage at a fine interval.

What was logged Value
Logging window 146 seconds
Drops of 2GB or more 14
Minimum VRAM 403MiB
Maximum VRAM 23,719MiB (of 24,576MiB)

VRAM climbs to nearly full, then falls away sharply. That happened 14 times in 146 seconds. The model does not fit, so parts of it are being unloaded and loaded back in.

# log VRAM every half second
while true; do
  echo "$(date +%s) $(nvidia-smi --query-gpu=memory.used 
    --format=csv,noheader,nounits -i 0)"
  sleep 0.5
done

The video model was a setup of two 14B (14 billion parameter) models. On disk they are 13.3GB each, but they expand when loaded, and together they go well past the 24GB on an RTX 3090.

Sponsored

Does a model that fits behave differently?

If the reading so far is right, a model with room to spare in VRAM should not swing about on the same RTX 3090. So I checked.

A 6.5GB model (SDXL) generating images 12 times in a row on that same RTX 3090. Everything was held constant except the prompt and the seed. The seed changes every run because ComfyUI will otherwise reuse the earlier result.

Run Time VRAM used Drops of 2GB+ GPU temperature
1 18.1 s 9,907MiB 1 61°C
2 6.0 s 9,907MiB 0 64°C
3–11 6.0 s throughout 9,907MiB 0 66–72°C
12 6.0 s 9,907MiB 0 74°C

The first run carries the model load. From the second on, it was 6.0 seconds, eleven times running, a spread of 0.0%. No drops either.

Temperature rose 13 degrees, from 61°C to 74°C, and the time did not move by a single second. That settles the temperature question a second time.

Sponsored

Fitting against not fitting

Same RTX 3090. The only thing that differs is how big the model is.

Does not fit Fits
VRAM used 23,719MiB (97%) 9,907MiB (40%)
Drops of 2GB+ 14 in 146 s 0
Spread in time ±25.2% ±0.0%
Failures 2 of 12 0
What changes is whether the model fits in VRAM
It fits
The model stays in VRAM
Nothing needs unloading
→ times line up
It does not fit
The overflow gets unloaded
and loaded back when needed
→ times swing, and runs fail
Sponsored

How running short of VRAM actually shows up

The symptoms were not what I expected. I assumed it would stop with an error. What happens instead is that it keeps running, but slowly and unevenly. A run fails now and then, and the next one works again.

That is a hard shape to read. It invites the thought that the card is unstable. What is more likely is that the card is fine, and this is what asking it to hold a model it cannot hold looks like.

Whether the model you want will fit in the VRAM you have is something you can work out before buying anything.

What this cannot tell you

I compared two models on an RTX 3090: 6.5GB, and one that goes past 24GB when loaded. Nothing in between was tried. Where the behaviour starts to change is unknown.

Whether other graphics cards do the same thing has not been checked. Telling whether this belongs to the model or to NVIDIA cards in particular would take measurements on a card from another maker.

The VRAM drops come from a half-second sampling interval. Anything shorter than that was not caught.

Sponsored

In summary: what was actually moving

Temperature was not what made the times swing on one RTX 3090.

It reached 74°C on the first run and hardly moved after that, and the times kept falling anyway. Measured again with a model that fits, 13 degrees of warming left eleven consecutive runs at 6.0 seconds.

The cause was VRAM capacity. Because the model would not fit, unloading and reloading happened 14 times in 146 seconds. The spread in time was ±25.2%, and 2 runs out of 12 failed.

The same graphics card gives you different results depending on whether the model fits. Running short of VRAM did not stop the work. It slowed it down and made it uneven.

What running local video generation looks like in practice is covered here.

How quantization changes the way VRAM gets used is here.

Sponsored