Bonsai 27B: The First Frontier-Class LLM That Runs on Your Phone

Bonsai 27B: The First Frontier-Class LLM That Runs on Your Phone

PrismML recently announced Bonsai 27B: The First Frontier-Class LLM That Runs on Your Phone. This release marks a major milestone for on-device AI capabilities.

Unlike stripped-down variants, this is a full-featured 27B model. It includes multimodal vision, tool calling, and agentic reasoning. Furthermore, it fits entirely within an iPhone 17 Pro’s memory budget.

Extreme Quantization Variants

The breakthrough comes from extreme quantization without sacrificing intelligence. Therefore, PrismML ships two purpose-built variants to suit different hardware needs.

First, the Ternary Bonsai 27B uses 5.9 GB of storage. It delivers 1.71 effective bits per weight while retaining 95% of full-precision performance. This quality-oriented variant runs smoothly on everyday laptops.

Second, the 1-bit Bonsai 27B requires only 3.9 GB. It fits inside mobile memory ceilings with room for cache. Despite its size, it retains 90% of baseline performance.

Both variants support 262K-token context windows and speculative decoding. Additionally, everything is available today under the Apache 2.0 License.

Intelligence Density and Benchmark Performance

What truly sets Bonsai 27B apart is its intelligence density. The 1-bit variant achieves over 10 times the capability per gigabyte compared to full-precision baselines.

Benchmark results across 15 tasks tell a compelling story. For math and coding, the ternary variant retains 93.4% of baseline performance. Meanwhile, the 1-bit version stays at 91.7%.

Tool calling remains robust for agentic workloads. The ternary model scores 74.0%, staying close to the 80.0% baseline. Vision tasks show more loss but preserve coherent multimodal operation.

Overall, these models are 18x smaller than conventional equivalents. However, they outperform aggressive low-bit quantization methods that use significantly more disk space.

Enabling Local Agentic Workflows

This shift changes the fundamental shape of AI workloads. Modern deployments now require sustained multi-step reasoning rather than single-turn responses.

Cloud execution imposes latency and privacy constraints for these tasks. Consequently, local execution unlocks zero marginal cost for reasoning loops. User data also never leaves the device.

Developers can now build hybrid deployment patterns. Routine tasks run locally, while cloud models handle the hardest steps. This approach dramatically reduces costs for agentic systems.

Speed remains solid across platforms. On an NVIDIA RTX 5090, the 1-bit variant reaches 163 tokens per second. Apple M5 Max devices achieve 87 tokens per second.

Full Multimodal Stack on Device

Crucially, the model runs end-to-end in low-bit precision. There are no escape hatches to higher precision for attention or embeddings.

The compact 4-bit vision tower enables workflows over screenshots and documents. Structured tool calling also stays coherent for API interactions. Finally, speculative decoding adds lossless speedups without quality sacrifice.

Platform Availability and Future Outlook

Bonsai 27B runs natively on Apple devices via MLX and NVIDIA GPUs via CUDA. A free developer preview API is currently available for early exploration.

You can find full technical details in PrismML’s whitepaper. Intelligence density is becoming a defining axis of AI progress.

This release crosses a practical threshold for consumer devices. Full reasoning and agentic capability now fit on hardware people already own. Larger models using this architecture-agnostic methodology are already in progress.