Serving 8 users on a DGX Spark, Demerzel delivers 35% to 67% more throughput than llama‑server on every model tested; against vLLM it is faster for a single user on 3 of 4 models and at 16 users on 2 of 4, compared on equal‑size 4‑bit files.

Serving on an NVIDIA DGX Spark (GB10, 128 GB)

Each engine serves the same OpenAI‑compatible streaming requests: distinct 82‑token prompts, 128 tokens generated greedily, no prompt cache. Bars show throughput for all users together, relative to llama‑server on the same file (1.0). vLLM cannot load GGUF files, so it runs the model maker’s own 4‑bit checkpoint (BF16 for Llama‑3.2‑1B); models with no vLLM bar have no vLLM checkpoint in this set.

Lemonade Server 2026.44.1 (AMD’s llama.cpp‑based server; its CUDA build, llama.cpp b11395) was measured on 3 of these files, tuned to 16 parallel slots, a 65,536‑token context and flash attention; its ratio is against Demerzel measured the same night. At 8 and 16 users Demerzel is 24%–67% ahead; for one user the two are within a few percent (108%, 96%, 99%). As shipped, Lemonade serves one request at a time (--parallel 1), so its throughput stays at the one‑user figure however many users wait.

Aggregate tokens per second; green where Demerzel is ahead, red where it is behind. Time to first token is the median over requests. Lemonade columns: see the table above.

One user, raw engine speed

Demerzel’s CUDA engine on a 540‑token prompt and a 128‑token generation, against llama‑bench pp512 / tg128 on the same GGUF file (flash attention on, all layers on the GPU).

CPU and AMD

Demerzel against llama.cpp with the GPU disabled on both sides, 8 threads, 2 interleaved rounds. Generation on this one‑memory‑channel laptop is bound by memory bandwidth for both engines.

Same streaming client for every engine: a 553‑token prompt, 128 tokens generated, 1 user and 4 users. vLLM 0.32’s CPU build runs the BF16 weights, so the half‑precision rows compare like with like.

An eight‑year‑old GPU pair on a 2012 FX‑4300 host, model split across both cards. Demerzel against llama.cpp’s Vulkan build, 3 interleaved rounds.

Many models, one server

20 users on Llama‑3.2‑1B and 2 users on Qwen3‑30B‑A3B at the same time, on one DGX Spark: the same streaming client and prompts for every engine (50‑word prompts, 256 tokens, greedy), both models warmed first, means of 2 interleaved rounds (31 October 2026). Demerzel serves both from one process and keeps tuning each model to the load and to the other model sharing the GPU; llama‑server’s router runs a child server per model; vLLM serves one model per instance.

Vision and video: Qwen3‑VL‑8B

Qwen3‑VL‑8B Instruct reading charts, photos and video clips on one DGX Spark, one user. Same OpenAI‑API requests to every engine (media sent inline, greedy decoding), 200 questions per dataset. Demerzel and llama.cpp run Qwen’s own 4‑bit GGUF (Q4_K_M) with its F16 vision projector; vLLM cannot load those files and runs the 4‑bit AWQ checkpoint. Video clips are 32 frames at 2 frames per second. First token is the median time to the first answer token. Bold marks the best value in each row. Per‑dataset detail for each size below.

Qwen3‑VL in detail: 30B‑A3B, 8B and 4B

Qwen3‑VL 8B and 4B Instruct reading pictures and video clips on one DGX Spark. Every engine gets the same OpenAI‑API requests (images and clips sent inline, greedy decoding) over 200 questions per dataset. Demerzel, llama‑server and Lemonade Server run Qwen’s own 4‑bit GGUF (Q4_K_M) with its F16 vision projector; vLLM cannot load those files and runs the 4‑bit AWQ checkpoint. Video clips are 32 frames at 2 frames per second; vLLM’s container cannot decode H.264, so it reads near‑lossless MPEG‑4 copies of the same frames (Demerzel scores the same on both). Bold marks the best value in each row.

Generated tokens per second for all users together, and the mean time to first token (s), with 1 and 4 users asking at once.

Generated tokens per second for all users together, and the mean time to first token (s), with 1 and 4 users asking at once.

Generated tokens per second for all users together, and the mean time to first token (s), with 1 and 4 users asking at once.

Video generation: Wan2.2

Wan2.2 TI2V‑5B turning text into video on one DGX Spark: the 16 VBench prompts, 832×480, 33 frames, 30 steps, guidance 5, shift 5, seed 42. Demerzel and diffusers use the UniPC sampler; stable‑diffusion.cpp offers Euler at the same 30 steps (the same number of model passes). Demerzel’s output matches the fp32 diffusers reference (cosine 0.99999 at Q8_0; VAE 70.8 dB PSNR). Times are per video with the model loaded. CLIPScore (ViT‑L/14, how well the frames match the prompt; higher is better) is scored on 8 evenly spaced frames of each video and averaged over the 16 videos.