I used Ollama, MLX, and llama.cpp for the same broad goal: run coding-focused language models on Apple Silicon and make them available to development tools. They overlap, but they optimize for different operator experiences.
The useful comparison was not “which runtime wins?” It was which layer of control I needed for a particular experiment. Sometimes I wanted a model online quickly. Sometimes I needed an MLX-native package. Sometimes the exact GGUF quantization and runtime controls were the point of the test.
Ollama: the shortest path to a service
Ollama gave me the simplest operational loop. Model management, starting the service, and exposing a consistent API require fewer moving parts than my lower-level experiments. That made it well suited to testing a coding tool or comparing model behavior before tuning the runtime.
Its OpenAI-compatible endpoints are particularly useful when a client already supports a configurable base URL. I could keep development on Windows while serving inference from the Mac across the private network.
The tradeoff is abstraction. Ollama packages models and runtime defaults behind its own model-management experience. That is usually helpful, but it can make the path from a raw model artifact to the final runtime configuration less direct. When I needed to inspect a particular quantization or adjust lower-level behavior, I moved closer to MLX or llama.cpp.
MLX and mlx-lm: native experimentation
MLX is designed for Apple Silicon, and mlx-lm provides model loading, generation, conversion, serving, and cache tooling around it. I used it when I wanted to work with MLX-native models and observe prompt processing and memory behavior more directly.
This mattered on a 16 GB machine because prompt cache and KV cache are part of the same unified-memory budget as the model. mlx-lm exposes concepts such as maximum KV size and cache quantization. I did not treat those as magic flags; they trade memory behavior against how much prior context remains available and how it is represented.
Model loading also depends on the exact MLX package. A model being available in another format does not mean the MLX conversion has identical characteristics. I kept the model artifact, runtime version, and context settings together in my notes so that a successful run could be reproduced.
llama.cpp: GGUF choice and low-level controls
llama.cpp was the most useful when GGUF support and quantization flexibility mattered. The project supports Metal on Apple Silicon and exposes detailed server and inference controls. That makes it possible to reason more explicitly about context, cache, threading, and model loading.
The cost of that flexibility is operational complexity. I had to understand more of the model metadata and runtime configuration rather than depending on a managed model package. That is worthwhile for targeted experiments, but it is not always necessary for simply providing a stable endpoint to a coding client.
GGUF availability was also a practical advantage. Many model variants are distributed in multiple quantizations, making it easier to test how a smaller weight representation changes fit and output quality. Quantization still does not determine total memory on its own; cache, runtime overhead, model architecture, and context remain part of the equation.
A practical comparison
| Concern | Ollama | MLX / mlx-lm | llama.cpp |
|---|---|---|---|
| First model to API | Simplest | More setup | More setup |
| Model format focus | Managed runtime packages | MLX-native models | GGUF |
| Apple-Silicon focus | Good platform support | Designed around Apple Silicon | Metal backend available |
| Low-level control | Intentionally abstracted | Direct experimentation | Extensive controls |
| Remote coding tools | Straightforward API | Server available | Server available |
| Best fit in my lab | Stable daily endpoint | MLX and cache experiments | Quantization and runtime experiments |
This table describes my use, not a universal ranking. Runtime capabilities and model support continue to change, so I validate the current release before depending on a specific feature.
What went wrong in testing
The easiest mistake was comparing unlike configurations. A model served by Ollama and a similarly named GGUF or MLX conversion might differ in quantization, prompt template, context defaults, or packaging. If those inputs are not controlled, the result says little about the runtime itself.
The second mistake was using a short prompt. Coding workloads can include large system prompts, tool schemas, source files, and accumulated agent history. A configuration that answers one question may fail once the cache grows.
I therefore recorded four things for each useful test: the exact model artifact, its quantization or MLX format, the context configuration, and the client workload. I avoided turning the observations into benchmark claims because that was not the goal and the test conditions were not a controlled benchmark suite.
When I would choose each runtime
I choose Ollama when I want the fastest route to model management and a dependable local API for development tools.
I choose MLX / mlx-lm when the model is MLX-native or when prompt processing, cache behavior, and Apple-Silicon-specific experimentation are central to the work.
I choose llama.cpp when I want a particular GGUF quantization, broad GGUF compatibility, Metal acceleration, and explicit runtime controls.
The choice can change as the experiment matures. I may validate a model quickly in Ollama, inspect an MLX version for memory behavior, and use llama.cpp to compare GGUF quantizations. That is not duplication; each runtime answers a different operational question.