Two 300B-class MoE models run on two 128 GB Strix Halo minis; GLM-5.3-Flash prefill about 580 tok/s and MiMo-V2.6-Flash decode up to 44 tok/s, using EXL3 weights and a ROCm engine.
Read the original at www.reddit.com→We built an engine, Kyojin, on top of ExLlamaV3 for Strix Halo (gfx1151, ROCm), and packed two 300B-class MoE models so each fits one 128 GB machine. First release, all measured on Ryzen AI Max+ 395. Model...
Original headline: "Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine"
Coverage timeline
- Oct 3, 14:16 UTC r/LocalLLaMA lead source Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine
- Oct 5, 15:25 UTC r/LocalLLaMA Qwen3.8-Flash-Next (125B) on a single Strix Halo mini PC: 44-59 tok/s with speculative decoding, ~1,400 tok/s prefill, engine is open