PrismML has released Bonsai 27B, a 27-billion-parameter AI model that can run on consumer smartphones, including iPhones. The model requires roughly 3.9 GB of memory to run, significantly less than the 54 GB needed for a standard 27-billion-parameter model.
The compression method used to achieve this reduction in size involves reducing each model weight from 16 bits of floating-point precision to a single sign, either +1 or -1 in the binary build, or one of three values in the ternary.
Model Performance
Bonsai 27B has been evaluated on 15 benchmarks and averages 80.49, or 94.6% of the full-precision model. The model also performs well on tasks such as coding and general knowledge, scoring 86 points in coding and 77% on general knowledge.
Commercial Interest
PrismML CEO Babak Hassibi has confirmed that the company is in early talks with Apple, which is evaluating the compression technology for potential on-device use. A compressed Gemma model is next in the pipeline, followed by larger frontier models.
Availability
The 1-bit Bonsai 27B model is available for free download now under Apache 2.0. PrismML also ships a DSpark speculative decoding layer alongside the model, which can provide a throughput boost on certain hardware.
Based on reporting from crypto.news.
