BaseRT
BaseRT is a high-performance runtime designed specifically for Apple Silicon, enabling fast inference of open-source language models.
Our verdict
Who it's for
Fast open-source LLM inference for Apple Silicon, on-device
Who should look elsewhere
Only compatible with Apple Silicon (M-series) hardware.
Pros
- +Fast inference speeds, especially on prefill and decode phases.
- +Keeps data fully local, enhancing privacy and security for sensitive work.
- +Free and open-source, with active community support.
Cons
- −Only compatible with Apple Silicon (M-series) hardware.
- −Model support may not cover all open-source models available in other runtimes.
Key features
- ✓Supports a wide range of models: Qwen, Llama, Gemma, Mistral, Phi, Nomic BERT, and more.
- ✓Serves models via a simple `basert serve` command for integration with coding agents.
- ✓Installs with a single curl command for quick setup.
- ✓Delivers significantly faster prefill tokens per second compared to MLX and llama.cpp.
- ✓Community-driven development with documentation, GitHub repository, and Discord chat.
Pricing
| Plan | Price | Billing | Notes |
|---|---|---|---|
| BaseRT | — | month | The fastest LLM runtime on Apple Silicon; Prefill up to 6.4x faster than llama.cpp and 3.9x faster than MLX; Decode up to 33% faster than MLX |