LIVE · refreshes every 20 min

updated Nov 14, 08:03 UTC

110010.art / topic

Llama

Oct 30

14d ago DFlash2-enabled Qwen3.8-Flash-Next shows up to 39% speed gains over DFlash for certain runs, with llama.cpp v0.8.0 now supporting DFlash2 for FN r/LocalLLaMA → story

15d ago 10 coding agents, one Qwen 3.8 27B, with identical llama-server; tokens and cache reuse vary more than pass rates r/LocalLLaMA → story

Oct 28

16d ago Show HN: ThreadShelf enables Obsidian and AI chats with vector search, MCP, and llama.cpp Hacker News (AI) → story

16d ago CUDA: fuse MoE gate/up matmuls and GLU in MMQ for llama.cpp r/LocalLLaMA → story

16d ago mtmd: enable video input support for LFM3 model family in llama.cpp r/LocalLLaMA → story

16d ago Two days tuning a single Tesla P40: 35B MoE, full 262k context, ~60 t/s short and ~30 t/s at 100k deep; no sub-Q4 quants r/LocalLLaMA → story

Oct 27

17d ago Tensor split over PCIe 3.0 x1 shows good results r/LocalLLaMA → story

17d ago I made Pumpkin, a graphical manager for llama.cpp, for my girlfriend and released it on GitHub. r/LocalLLaMA → story

18d ago Merge-based vocabulary extension stalls at the pre-tokenizer for Nepali, delaying English-preserving repair during continued pretraining arXiv cs.CL → story

18d ago Zest bundles Lemonade server, Pi agent, and a friendly GUI for local model running; aims to simplify setup for new users r/LocalLLaMA → story

Oct 26

18d ago Nanbeige4.3-33B runs on MacBook Pro M5 Max; mlx-lm, llama.cpp, and Kapitsa compare prefill and decode from 512 to 32K tokens r/LocalLLaMA → story

19d ago Ternary Nemotron 30B-A3B runs on 12 GB GPU with 368K context in GGUF; supports Stock llama.cpp and PrismML fork/Bonsai r/LocalLLaMA → story

19d ago llama.cpp vulnerability CVE-2026-116096 allows out-of-bounds write in Pocket-TTS via local access; upgrade to b11796 recommended. r/LocalLLaMA → story

Oct 25

19d ago Qwen3.5-9B fine-tuning with QLora on 500 GB of raw data using an Asus ESC4000 G3 with 8 T4s; 87k context per slot and 44 slots, TTFT 3–5 seconds, 35–40 TPS per user. r/LocalLLaMA → story

19d ago Llama Swap mobile controller: developer builds apk for local monitoring and logs while on the go r/LocalLLaMA → story

19d ago Backup your quants! r/LocalLLaMA → story

Oct 24

20d ago TensorSharp adds MiniMax-M3.1-Flash support on Day 1; 64K executor prefill speeds and TTF token times improve vs llama.cpp branch r/LocalLLaMA → story

20d ago High-Performance llama.cpp fork adds Kepler support, CUDA Graphs & Windows NCCL for legacy GPUs r/LocalLLaMA → story

20d ago Local inferencing in one JVM JAR, dependency free, faster than llama.cpp Hacker News (AI) → story

20d ago Meta trained small AI models to read raw bytes instead of tokens; they start worse but the paper predicts they end up better r/LocalLLaMA → story

Oct 23

21d ago ISTA-DASLab releases DeepSeek-V4.1-Flash-1017-GSQ-RCO with GSQ-RCO smart quantization; Llama.cpp support not yet merged, Strata noted in the model card. r/LocalLLaMA → story

21d ago Qwen4exp: build the QSA selection without the kq_mask; ggml-org/llama.cpp PR 30965 r/LocalLLaMA → story

21d ago Vibecoding your own inference engine by cloning llama.cpp and running gguf q4 paged into webgl2 embedded compute buffers with GLSL in HTML; tokenizer and vocab remain in debug hell. r/LocalLLaMA → story

22d ago Probing and steering for safety-relevant behaviours in large language models arXiv cs.LG → story

22d ago GPU prices dip as crypto demand wanes; researchers explore llama-generated content safety r/LocalLLaMA → story

Oct 22

22d ago Massive B70 spec-decoding improvement as Intel XMX matrix engines refactored in llama.cpp Pull Request 29864 r/LocalLLaMA → story

22d ago Muse Spark 816B runs across 10 NVIDIA A40 GPUs; benchmarks show differing optimal multi-GPU strategies for prompt processing versus token generation r/LocalLLaMA → story

22d ago Ling-3.0-tiny-UBTree DFlash draft for stock mainline llama.cpp (+8% prose, +61% code speedup) r/LocalLLaMA → story

22d ago Snapdragon mini PC results update; £300 device achieves 315/43 Tok/s and 46/20 Tok/s read/gen with ToRc engine in development r/LocalLLaMA → story

22d ago Llama.cpp PR: how to beat StrataDecode performance with a R9700 r/LocalLLaMA → story

22d ago wici-ai/inferssd: run models larger than RAM or VRAM by streaming MoE experts from SSD; built on llama.cpp GitHub Trending (AI) → story

22d ago Carbon-X, a 14B model, runs locally with llama.cpp and combines DNA and natural language; claims open-source DNA language model outperforms open lms on five DNA benchmarks at lower cost r/LocalLLaMA → story

22d ago Llama.cpp adds WebGPU Q2_0 support with testing on RTX 3060 using native Dawn WebGPU r/LocalLLaMA → story

23d ago Android AI chat app for Arderno GPU and Hexagon NPU; project seeks testing on Android 14+ Snapdragon devices per author’s disclosure r/LocalLLaMA → story

Oct 21

23d ago Llama.cpp fork AbliteratedOutTheBox performs inference-time abliteration on models; Inference time abliteration reports attention-based inhibitions and speed gains GitHub Trending (AI) → story

23d ago llampart 2.0.1 released; a self-hosted Web UI for chatting with LLMs via llama-server, vLLM, and OpenAI-compatible APIs r/LocalLLaMA → story

23d ago 8B Llama model runs at 17,000 tokens per second on a single chip r/LocalLLaMA → story

24d ago Fast LoRA in llama.cpp lacks fast kernels, according to updates and examples showing LoRA usage with Qwen3.8-Flash-Next models r/LocalLLaMA → story

24d ago Strata speeds up on Apple Silicon? r/LocalLLaMA → story

24d ago High-dimensional layer-state trajectories reveal the internal dynamics of transformer inference arXiv cs.LG → story