Bonsai 27B: The First Frontier-Class LLM That Runs on Your Phone
Today, PrismML announced Bonsai 27B, based on Qwen3.6 27B, marking a watershed moment in on-device AI: the first model of its capability class to run natively on a smartphone. This isn't a stripped-down variant or a research curiosity. It's a full-featured 27B model with multimodal vision, structured tool calling, multi-step reasoning, and agentic capabilities—compressed to fit inside the memory budget of an iPhone 17 Pro.
The breakthrough comes from extreme quantization without sacrificing intelligence. Bonsai 27B ships in two purpose-built variants:
- Ternary Bonsai 27B (5.9 GB): Uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, delivering 1.71 effective bits per weight. This is the quality-oriented variant, retaining 95% of full-precision performance while running on everyday laptops.
- 1-bit Bonsai 27B (3.9 GB): Uses binary {−1, +1} weights with the same scaling approach, delivering 1.125 effective bits per weight. It fits inside the ~4 GB memory ceiling of an iPhone 17 Pro with room for KV cache and activations, retaining 90% of baseline performance.
Both variants are fully multimodal, with vision encoders shipped in compact 4-bit form, supporting 262K-token context windows and speculative decoding for lossless speed gains. Everything is available today under the Apache 2.0 License.
The Intelligence Density Frontier: 10x Better Than Full-Precision
What sets Bonsai 27B apart is its performance per gigabyte—what PrismML calls "intelligence density." Here's where it gets remarkable: the 1-bit variant delivers 0.53 capability points per GB, more than 10 times the full-precision baseline and roughly 2.7x the best conventional low-bit alternatives.
Across a 15-benchmark suite covering math, coding, tool calling, instruction following, knowledge, and vision tasks, the results tell a sharp story:
- Math & Coding (hardest tasks): Nearly untouched. Ternary retains 93.4% on math benchmarks; 1-bit stays at 91.7%.
- Tool Calling & Agentic Reasoning: Ternary (74.0%) stays within touching distance of the full-precision baseline (80.0%), the exact frontier that agentic workloads depend on.
- Vision Tasks: 1-bit Bonsai 27B scores 59.6% on vision benchmarks versus 72.6% for the baseline—a meaningful loss, but coherent multimodal operation is preserved.
- Overall: Ternary hits 80.5% across 15 benchmarks; 1-bit hits 76.1%. For a model 18x smaller than conventional full-precision equivalents, this is unprecedented.
For comparison, aggressive conventional low-bit quantization of the same base model scores significantly lower while occupying 2.5x more disk space. Bonsai's approach wins on both axes.
Why This Changes Everything: From Cloud Calls to Local Agents
The seismic shift isn't about speed or cost alone—it's about the workload shape itself. Modern AI deployments are moving from single-turn responses to sustained work: multi-step agentic loops, tool use chains, real-time document synthesis, and reasoning workflows that span hundreds of model calls.
Cloud-only execution imposes hard constraints: every step is a network round-trip, per-token cost compounds across iterations, and every intermediate result—including the user's private files and screenshots—crosses the network boundary. The math changes radically when a capable model lives on the device itself.
Local agentic execution with Bonsai 27B unlocks:
- Zero marginal cost for multi-step loops. A hundred-step reasoning chain costs nothing after the model loads.
- Privacy by construction. User data, files, and screens never leave the machine.
- Hybrid deployment patterns. Route routine and privacy-sensitive tasks to local Bonsai; reserve frontier cloud models for the hardest reasoning steps. This collapses cost-per-task for entire categories of agentic systems.
- Offline-first workflows. Agents that reason over local data without network dependency.
- Persistent on-device assistants. Assistants that stay coherent across sessions without the latency penalty of cloud calls.
Speed is solid: on an NVIDIA GeForce RTX 5090, Bonsai 27B reaches 163 tok/s in 1-bit and 134 tok/s in Ternary. On an Apple M5 Max, it hits 87 tok/s (1-bit) and 58 tok/s (Ternary). On a phone, inference speed is conservative by design—the focus is on sustainable work, not throughput.
Multimodal and Tooling: The Full Stack on Device
A key detail: the model runs end-to-end in low-bit precision. No escape hatches to FP16 for attention or embeddings. The vision tower is compact 4-bit, enabling on-device workflows over screenshots, documents, and camera input. Structured tool calling stays coherent—critical for agents that interact with APIs, file systems, and external services.
The 262K-token context window and support for speculative decoding compound the speed advantage. Draft-and-verify acceleration adds lossless speedup without sacrificing quality.
Platform Coverage
Bonsai 27B runs natively on Apple devices (Mac, iPhone, iPad) via MLX and NVIDIA GPUs via CUDA, leveraging custom low-bit kernels built for its hybrid-attention architecture. A free, limited-time developer preview API is available for early exploration.
Full technical details on compression methodology and evaluation processes are in PrismML's whitepaper.
The Intelligence-Density Frontier Moves Left
PrismML's thesis is compelling: intelligence density—the capability per gigabyte—is becoming one of the defining axes of AI progress. Raw capability determines what a model can do; density determines where it can do it.
Bonsai 27B is the largest step yet on this frontier. Earlier Bonsai releases moved the needle on smaller models. This release crosses a practical threshold: full reasoning, multimodal understanding, tool use, and agentic capability now fits on devices people already own.
The methodology is architecture-agnostic. Larger models and new architectures are already in progress.
Related Reading
- Apple M7 Ultra: 1.5TB of Unified Memory Could Make On-Device AI Unstoppable
- Meta Unveils Muse Spark 1.1: A New Agentic Powerhouse for Coding and Computer Use
- Turning Video into Searchable Intelligence: How VSS Skills Power Autonomous AI Agents
- Soofi S 30B: Germany's Open-Source LLM That Beats OLMo 3 and Apertus 70B
