LIVE · refreshes every 20 min

updated Nov 14, 08:03 UTC

110010.art / topic

Llama

Oct 21

24d ago How to squeeze the max token from M1 Max 64gb? r/LocalLLaMA → story

Oct 20

24d ago EAGLE-4 release achieves faster performance than DFlash2 and DSpark, with MTP, DSpark, DFlash 2, and EAGLE-4 benchmarks showing speedups; Qwen 3.8-27B currently supported r/LocalLLaMA → story

24d ago Geezuz kernel: Qwen3.8-27B runs at up to 190 tok/s on two RTX 3090s (620 tok/s at 8 requests) r/LocalLLaMA → story

24d ago Used HP Z640 with 2x E5-2680 v4, 64GB ECC, and Quadro M6000; plan to add two 3060s for Qwen 3.8 Flash Next on Strata, llama.cpp, and PyTorch workloads under Ubuntu. r/LocalLLaMA → story

25d ago 24GB is enough for engines that take turns, says system service that manages VRAM allocation and swaps, avoiding collisions and orphaned processes r/LocalLLaMA → story

25d ago Budget upgrade options for a 31B model: PCIe x1 bottleneck and GPU setup considerations for 16K+ context inference r/LocalLLaMA → story

Oct 19

25d ago GameSummoner: ease of local use update r/LocalLLaMA → story

25d ago RTX 5090 Laptop on Linux experiences GPU hangs (Xid 8) during local LLMs; CUDA graphs issue noted with GGML and a workaround is to disable CUDA graphs. r/LocalLLaMA → story

25d ago Llama.cpp support for Cohere models r/LocalLLaMA → story

25d ago Alternative approaches to base llama.cpp for RDNA2 GPUs in local LLM usage; user notes WMMA requires RDNA3+ and seeks broader RDNA2 support r/LocalLLaMA → story

25d ago Strata with multi-GPU explanation r/LocalLLaMA → story

25d ago Llama server default port changed from 8080 to 9931 (ggml) inside the container starting v0.7.0 r/LocalLLaMA → story

26d ago SonicBoom v0.0.3 outperforms llama.cpp in decoding on a desktop RTX 3050 8 GB, with benchmark results shown for Gemma 4 E4B Q3_K_M and Qwen2.5-0.5B. r/LocalLLaMA → story

Oct 18

26d ago New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K r/LocalLLaMA → story

Oct 17

27d ago CUDA shows big speedup for low-bit MMVQ types like Q2_K and Q3_K in llama.cpp pull request 30517 r/LocalLLaMA → story

27d ago Kolibri-1 (Aleph Alpha) support added to llama.cpp with European 78B A3.4B model; GGML community pulls request 30673 r/LocalLLaMA → story

27d ago Russian guide explains how to run DeepSeek locally with Ollama, llama.cpp, LM Studio, vLLM, and Open WebUI GitHub Trending (AI) → story

←1 23→