Tag

#local LLM inference

Browsing posts matching the tag #local-llm-inference

Jul 14, 2026 • 5 min read

Bonsai 27B: The First Frontier-Class LLM That Runs on Your Phone

PrismML announces Bonsai 27B, bringing 27B-class reasoning and multimodal capability to iPhones and laptops through extreme quantization. With intelligence density 10x better than full-precision models, this is the first frontier-class LLM that runs on a phone.

Aby Varghese

Jul 14, 2026 • 6 min read

Tencent Hy3: Run a 295B Flagship MoE Model on a Single GPU with GGUF and llama.cpp

Tencent's Hy3 is a 295B Mixture-of-Experts model that activates only 21B parameters per token — and with new 1-bit and 4-bit GGUF quantizations, it can now run on a single GPU via llama.cpp, with MTP speculative decoding for extra speed.

Aby Varghese