LIVE · refreshes every 20 min
updated Nov 14, 07:43 UTC
110010.art(ificial intelligence)
AI products, research, and launches — clustered and ranked, not just re-blogged.
110010.art / topic

Llama

Nov 12
1d ago Krafton (PUBG) releases Raon-1 nano under Apache 2.0 with E1.5B, 262K context, and 2.08GB GGUF; requires llama.cpp fork for now, with benchmark status on 8GB cards unknown. r/LocalLLaMA → story
2d ago Qwen4 is coming; pull requests spotted on github r/LocalLLaMA → story
2d ago Gemma4-31b latent space discovery: caring and contempt for user controlled by the same 3 neurons; how to find your own steering recipes and export to GGUF steering vector for llama.cpp r/LocalLLaMA → story
Nov 11
3d ago llama.cpp: add FP8 KV-cache r/LocalLLaMA → story
3d ago Third Eye: immediate gains from self-update are near zero; further native updates can improve performance, according to tests on 3-8B backbones across math and code arXiv cs.LG → story
Nov 10
3d ago XeMaya, an Intel Arc inference engine, enables native running of DeepSeek V4.1 and GLM 5.3 via SYCL and Level Zero r/LocalLLaMA → story
3d ago Lokey is an open-source Android keyboard that runs on-device AI for word prediction, autocorrect, dictation, and translation. r/LocalLLaMA → story
3d ago DNA Factory open-sources a fine-tuning framework with five training methods and opinionated defaults r/LocalLLaMA → story
3d ago MACaQ achieves SOTA model-agnostic quantization for GGUF, enabling local inference and beyond for text, image, video, and audio models with llama.cpp, vLLM, Diffusers, and PyTorch GitHub Trending (AI) → story
4d ago ggml-org/llama.cpp adds --shard support to load models from multiple shard locations and enables PLE (engram) file loading from separate GGUFs r/LocalLLaMA → story
4d ago Multiverse Computing releases Comet 60B (A12B) GGUFs with MTP head for stock llama.cpp; user converts Nemotron 3.5 Super compressed model to GGUF r/LocalLLaMA → story
4d ago CUDA+ROCm dual-backends LLM inference for mixed-vendor GPUs (up to 48% faster) r/LocalLLaMA → story
Nov 9
4d ago VRAM usage spikes to 0.5–1 GB during prompt processing on Vulkan Llama.cpp with 7900 XTX r/LocalLLaMA → story
4d ago RustingStrata runs DeepSeek-V4.1-Flash (249 GB) on a single RTX 3060 with 30 GB RAM using CUDA and Vulkan backends r/LocalLLaMA → story
5d ago Forcing a letter suppresses refusal: mechanistic evidence from WMDP arXiv cs.CL → story
Nov 8
5d ago Upgrade your GPU? run local LLMs on both GPUs by splitting inference workloads across them. r/LocalLLaMA → story
5d ago Fine-tuned Qwen3.5-0.8B turns plain English into bash commands with a deterministic risk checker; dale achieves 89.2% accuracy on 471 requests across English and Spanish. r/LocalLLaMA → story
6d ago Two PCs with different specs considered for pooled resources to run large MoEs; user contemplates whether to pool via direct Ethernet and use llama.cpp RPC or distribute GPUs across machines r/LocalLLaMA → story
Nov 7
6d ago One token at the beginning of a prompt changed 6 of 15 LLM responses; does it matter? r/LocalLLaMA → story
Nov 6
7d ago moe-cache: LRFU/MQ-like hybrid cache, ClassLRU for llama.cpp improves decode and reduces uploads vs SLRU r/LocalLLaMA → story
7d ago CyberTiel35BA3B works on Blender rigging via MCP with 8GB VRAM and 16GB RAM; llama.cpp used r/LocalLLaMA → story
8d ago Bring Agentic Coding on-device with Foundry Local; 35-135% improvements (decode/tg) on 5090 & DGX Spark, and possibility to add to llama.cpp r/LocalLLaMA → story
Nov 5
8d ago Nemotron 3.5 Bolt E4B runs at 27.7 tok/s on Jetson Orin Nano Super, outperforming the official release on Llama CPP by about 1.8x r/LocalLLaMA → story
8d ago Hugging Face hosts quantization livestream with contributors from Red Hat and others r/LocalLLaMA → story
Nov 4
9d ago LFM3 1.7b performs well; 2b VL also good but with half the context, and is fast on CUDA with RTX 3090 Ti r/LocalLLaMA → story
10d ago Hacking or just failing? A more capable LLM monitor better separates reward hacking from honest failure arXiv cs.AI → story
10d ago New inference engines proliferate beyond llama.cpp-based stacks, from ollama and koboldcpp to vllm, exl3+tabbyapi, litert, and newer entrants like Strata, Basalt, Maya, Colibri, Ninfer, and Infernix r/LocalLLaMA → story
Nov 3
10d ago Llama.cpp fork combines QFN, specific quant, and ultra-specific hardware config for high pp and tg; upvote prompts, ParaboloidalCrest reports r/LocalLLaMA → story
10d ago Gemma4: graph optimizations yield massive gains on pp for E2B and E4B in llama.cpp Keras? Not. Wait. r/LocalLLaMA → story
10d ago mtmd: add Nemotron 3.5 Transcribe support to llama.cpp; GGML core updates for offline transcription model with 40-language support and 65,536-token context r/LocalLLaMA → story
10d ago Olla v0.1.0 releases as a lightweight open-source LLM proxy for homelabs and on-prem inference, with failover, sticky sessions, and load balancing (Ollama, vLLM, SGLang, llama.cpp, oMLX). r/LocalLLaMA → story
10d ago Qwen3.8-Flash-Next (local) vs Claude in Claude Code: user compares code reviews on a Swift on-device LLM chat app with a module move and a 783-line search harness on a large context model r/LocalLLaMA → story
Nov 2
11d ago Llama.cpp: fix unified KV for multi-seq with flash_attn_ext_rows on CUDA r/LocalLLaMA → story
12d ago Berth, a lightweight Go-based GPU resource manager, coordinates GPU access via leases, priorities, and a queue to manage when apps use VRAM and hand control to the next task. r/LocalLLaMA → story
Nov 1
12d ago LFM3-1.7b shows strong performance for compression and summarization; 4.1k t/s prefill and ~78 t/s decode on Strix Halo iGPU, with higher coherence when tensor-split across 7900 XTX + XT r/LocalLLaMA → story
12d ago The bespoke engine race continues as multiple forked models vie for faster performance; user notes llama.cpp lagging behind newer forks and upgrades r/LocalLLaMA → story
13d ago Qwen-flash-next Q4 with Expert cache on Llama.cpp setup; 19-21 t/s at 262K context using Qwen Flash Next Q4 model r/LocalLLaMA → story
Oct 31
13d ago A game scripting language runs speech-to-text 5x faster on CPU than llama.cpp on CPU Lobsters (AI) → story
13d ago What to run on 2x16GB VRAM and 128GB RAM for coding/agentic workflows with current LLMs and hardware. r/LocalLLaMA → story
13d ago Ternary Bonsai 2 runs with native CUDA offload and fast CPU in ik_llama.cpp, bringing full compatibility outside PrismML’s fork. r/LocalLLaMA → story