Measured on this card

NVIDIA GeForce RTX 5060 Ti

16 GB · Blackwell sm_120 · 448 GB/s

Qwen3.8-27B UD-Q3_K_XL: token generation

Typical app Our engine + card profile t/s faster 1.8×

Model: Qwen3.8-27B UD-Q3_K_XL
File: 13.2 GB
Memory used: 14.5 GB / 16 GB
A 500-word answer: 20.2 s → 11.0 s

Memory: 16GB
Bandwidth: 448GB/s
Power limit: 180W
Architecture: Blackwell sm_120
PCIe: gen4 x8

What fits

16 GB of memory. Each block is a model loaded by our engine, with its context and KV cache.

Qwen3.8-27B UD-Q3_K_XL

14.5 GB

file 13.2 GB used 14.5 GB

LFM2.5-24B Q4_K_M

14.2 GB

file 14.5 GB used 14.2 GB

Models for this card

Every measured model that fits in the card’s memory. The best one first.

Qwen3.8-27B

UD-Q3_K_XL 13.2 GB Licence: Apache 2.0: commercial use allowed

Our pick for this card
Writing speed

1.8× faster

Same card, same model, same script. Only the software changes.

Honestly: the details

Q4 does not fit in 16 GB. Upstream MTP with the default -fit drops to 7.7 t/s here.

Typical app[27.9–28.1] 28.1 t/s

Best flags[34.0–42.8] 37.3 t/s

Our engine[48.2–59.9] 50.8 t/s

Our engine + card profile[49.6–59.9] 51.7 t/s

Typical app: llama.cpp with default settings, like a typical app (on the RTX 3090 the same as Ollama) · Best flags: upstream llama.cpp with the best flags we know (e.g. MTP speculation) · Our engine: our engine with the same flags plus our patches · Our engine + card profile: our engine with the same flags plus our patches · spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median (A0 typowa aplikacja: 28.07) (A1 najlepsze flagi upstream: 37.32) (A2 nasz silnik (fork + draft vocab): 50.8) (nasz profil NVIDIA (A2 + GATE_UP_Q8=0): 51.66)

A 500-word answer
The bars run in real time: this is how long the answer takes at the measured speed. spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median

in Polish

in English

Replay at real speed

The same answer, written out at the measured speed.

Typical app: 28.1 t/s, Our engine + card profile: 51.7 t/s. spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median

Typical app UD-Q3_K_XL

28.1 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·5.6 s
Our engine + card profile UD-Q3_K_XL

51.7 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·3.0 s 1.8×
Conditions

spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median
- 32k context
- KV q8_0
- MTP n=4
- -fit off
- A0: default -fit
- KV f16
spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median · 180 W, PCIe gen4 x8, GDDR7 448 GB/s, container limited to 7.68 CPU (EPYC 7502), driver 580.173.02. :65-69
About the model

A dense 27B model with hybrid attention (Gated DeltaNet + full attention) and a built-in MTP head that guesses several tokens ahead.

Qwen at Q3 is the largest model that fits in 16 GB with a 32k context.
- chat
- code
- quality
app/landing/src/lib/data/models.jsonapp/landing/src/lib/data/models.json (raport 06 §2.4.2)
All measurements
Instruments

Typical app → Our engine + card profile
Tokens per step: tokens accepted per model step (MTP speculation) · Step time: The step is longer because it yields ~2.5 tokens. Per token: 35.7 → 19.3 ms (derived: step time ÷ tokens per step). · Prefill 512: reading the prompt, tokens per second · First token: ~2k-token prompt, no cache · Memory used: 32k context, KV q8_0, MTP n=4, -fit off; A0: default -fit, KV f16 (the source gives GiB; shown as GB, the way the vendor labels the card memory)

Tokens per step

1.00 2.48

↗ better
Step time

35.7 47.9 ms

↘ longer step, ↗ more tokens
Prefill 512

1,048 1,032 t/s

≈ no change
First token

2.4 2.7 s

↘ worse
Memory used

13.9 14.5 GB

↘ worse
Long context
Generation as the conversation grows (llama-bench, no speculation). The gain here is small, and we show it as it is. KV q8_0.

our engine upstream context length (linear scale) →
Where the gain comes from

Typical app 28.1 t/s

upstream flags 1.3× MTP + -fit off

our engine 1.4×

total 1.8×
Same model, same quality: measured

Honestly

Saluki 27B

KLD ours vs upstream 0.00065

upstream noise 0.00061
Bonsai 2 27B

KLD ours vs upstream 0.00032

upstream noise 0.00021

In plain mode our engine does not produce bit-identical text to upstream (patches 0001/0002, plus 0013 for Qwen). So we measure quality with KLD: on the RTX 3060 the difference stays within upstream’s own noise.
LFM2.5-24B-A2B

Q4_K_M 14.5 GB Licence: LFM Open License v1.0: commercial use only below USD 10M annual revenue
Writing speed

1.4× faster

Same card, same model, same script. Only the software changes.

Honestly: the details

Q5 does not fit in 16 GB.

Typical app[146.5–150.2] 147.7 t/s

Best flags 144.5 t/s

Our engine 191.0 t/s

Our engine + card profile[206.7–209.1] 207.9 t/s

Typical app: llama.cpp with default settings, like a typical app (on the RTX 3090 the same as Ollama) · Best flags: upstream llama.cpp with the best flags we know (e.g. MTP speculation) · Our engine: our engine with the same flags plus our patches · Our engine + card profile: our engine with the same flags plus our patches · spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median (A0 typowa aplikacja: 147.69) (A1 najlepsze flagi upstream: 144.47) (A2 nasz silnik: 190.99) (nasz profil NVIDIA (A2 + GATE_UP_Q8=0 Q8_IDS=0): 207.86)

A 500-word answer
The bars run in real time: this is how long the answer takes at the measured speed. spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median

in Polish

in English

Replay at real speed

The same answer, written out at the measured speed.

Typical app: 147.7 t/s, Our engine + card profile: 207.9 t/s. spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median

Typical app Q4_K_M

147.7 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

155 / 156 tokens done ·1.1 s
Our engine + card profile Q4_K_M

207.9 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·0.8 s 1.4×
Conditions

spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median
- 32k context
- KV q8_0
- no speculation
- A0: default -fit
- KV f16
spec_sweep through the server: 6 prompts × 3 seeds, T=1, 400-token answers, short context, median · 180 W, PCIe gen4 x8, GDDR7 448 GB/s, container limited to 7.68 CPU (EPYC 7502), driver 580.173.02. :65-69
About the model

A mixture of experts (MoE): 24B parameters, about 2B active per token. That is why it writes several times faster than a dense model.
- speed
- chat
app/landing/src/lib/data/models.jsonapp/landing/src/lib/data/models.json (raport 06 §2.4.2)
All measurements
Instruments

Typical app → Our engine + card profile
Step time: with MTP a step takes longer but accepts several tokens at once · Prefill 512: reading the prompt, tokens per second · First token: ~2k-token prompt, no cache · Memory used: 32k context, KV q8_0, no speculation; A0: default -fit, KV f16 (the source gives GiB; shown as GB, the way the vendor labels the card memory)

Step time

6.8 4.8 ms

↗ better
Prefill 512

4,148 4,160 t/s

≈ no change
First token

0.6 0.5 s

↗ better
Memory used

14.5 14.2 GB

≈ no change
Long context
Generation as the conversation grows (llama-bench, no speculation). The gain here is small, and we show it as it is. KV q8_0.

our engine upstream context length (linear scale) →
Where the gain comes from

Typical app 147.7 t/s

upstream flags (no gain) 0.98×

our engine 1.4× sampler ~+25%, fusions +5%, profile +5–6%

total 1.4×
Other LFM formats on this card

The same weights in other files. For both, md5 is identical to upstream.
QAD-Q4_0 (oficjalny Liquid), 13.6 GB · NVFP4 (społeczność), 13.6 GB

QAD-Q4_0 (official Liquid)

164.5 → 226.8 t/s

KLD vs Q8_0 0.203

md5 identical to upstream
NVFP4 (community)

156.0 → 198.0 t/s

KLD vs Q8_0 1.183· poor quality

md5 identical to upstream
Same model, same quality: measured

Honestly

Saluki 27B

KLD ours vs upstream 0.00065

upstream noise 0.00061
Bonsai 2 27B

KLD ours vs upstream 0.00032

upstream noise 0.00021

In plain mode our engine does not produce bit-identical text to upstream (patches 0001/0002, plus 0013 for Qwen). So we measure quality with KLD: on the RTX 3060 the difference stays within upstream’s own noise.

What next

Compare with another card Check your hardware

NVIDIA GeForce RTX 5060 Ti

Writing speed

A 500-word answer

Replay at real speed

Conditions

About the model

Instruments

Long context

Where the gain comes from

Honestly