Three Weeks In: M5 Ultra Mac Studio Is Becoming a Serious Local AI Workstation

Three weeks after the first 256 GB M5 Ultra Mac Studios reached users, the picture is much clearer. Large 150 to 190 GB models are running locally with 128K to 200K contexts at usable speeds, optimized Qwen workloads now exceed 100 tok/s, and the latest short-request concurrency tests reach roughly 325 aggregate tok/s. This is not an NVIDIA replacement, but it is becoming one of the most interesting single-box platforms for private LLM, coding-agent and RAG workloads.

The main architectural advantage is straightforward: up to 256 GB of unified memory with 1.2 TB/s bandwidth, accessible directly by CPU and GPU. There is no separate 32 GB or 48 GB VRAM ceiling followed by a slow PCIe path into system RAM. That difference matters once models and KV caches become large.

One of the clearest improvements over the previous M3 Ultra generation is prompt processing. MacStories measured roughly 2.5x faster prefill on M5 Ultra using Qwen3.8-Flash-Next. Time to first token at 64K context dropped from about 59 seconds to 24 seconds, at 128K from roughly 121 seconds to 50 seconds, and at approximately 256K from around 245 seconds to about 102 seconds.

For coding agents this can matter more than headline generation speed. Agents repeatedly process system prompts, tools, source code, retrieved documents, history and memory. Faster prefill directly improves responsiveness.

Generation performance has also moved quickly as MLX and oMLX have matured. Optimized Qwen3.8-Flash-Next configurations regularly exceed 100 tok/s, including large contexts. Recent 4-bit runs have reported around 127 tok/s at 64K, around 101 tok/s at 128K, and close to 90 tok/s near 200K on the 64-GPU-core M5 Ultra.

The latest 13 October concurrency test is particularly interesting. With Qwen3.8-Flash-Next 4-bit on the 64-core M5 Ultra:

1 request: 140.2 tok/s
2 requests: 160.2 tok/s aggregate
4 requests: 192.8 tok/s aggregate
8 requests: 324.9 tok/s aggregate

The same test with the 8-bit version scaled much less:

1 request: 124.9 tok/s
8 requests: 153.9 tok/s aggregate

This does not mean 4-bit is universally twice as good for concurrency. The tests use short synthetic requests, and speculative decoding behavior matters. But it shows clearly that quantization, runtime configuration and memory footprint can affect real multi-agent throughput as much as raw hardware.

This leads to one of the strongest conclusions from the first three weeks: there is no meaningful single M5 Ultra tokens-per-second number.

MLX versus llama.cpp, 4-bit versus 6-bit versus 8-bit quantization, MTP, DFlash, KV-cache compression and custom kernels can change performance dramatically. In several cases, the same hardware has moved from ordinary performance to more than twice the throughput purely through software changes.

The more important story, however, is not small models running faster. It is the ability to run models that are simply too large for consumer GPUs.

DeepSeek V4 Flash configurations around 156 to 173 GB have been tested successfully on the 256 GB Mac Studio. At 128K context, generation rates in the region of 48 to 56 tok/s have been measured depending on model packaging and runtime.

GLM-5.3-Flash pushes memory even harder. Recent 4-bit tests use roughly 180 to 190 GB of model memory at large contexts and still deliver interactive generation, commonly around 60 to 80 tok/s, with some speculative-decoding runs substantially higher.

This is the real architectural point: a model approaching 180 GB, plus a very large context, can remain entirely inside GPU-accessible memory on one desktop machine.

The limitation is also becoming clear. With GLM-class models, total system memory usage can exceed 220 to 230 GB. At that point 256 GB stops looking generous. Running embeddings, a vector database, rerankers, browsers, development tools and additional models simultaneously requires careful memory management.

This is where the forthcoming 512 GB configuration becomes interesting. It should not automatically make the same model faster, but it will provide much more operational headroom and allow higher-precision or multiple large models to remain resident.

Long-context testing has gone much further than expected. Fireside Labs pushed Qwen, GLM and DeepSeek to around one million input tokens.

Qwen 8-bit required approximately 5.3 minutes for the initial million-token prefill and then generated at around 29 tok/s. GLM took approximately 13 minutes and generated at around 31 tok/s. DeepSeek required roughly 57 minutes for the cold prefill and produced about 22 tok/s afterward.

This sounds impractical until caching is considered.

In the same work, a 250K Qwen prompt took 234 seconds when processed cold, while a cached follow-up required only about 0.64 seconds.

For serious coding agents and RAG systems, this may be one of the most important findings so far: prefix and KV caching are architectural requirements, not optional optimizations.

The same project also performed multi-hour coding-agent tests rather than short benchmark loops. Optimized DeepSeek under oMLX reached approximately 348 agent steps per hour, compared with about 241 steps per hour using llama.cpp. Hidden functional tests were also used to check output quality.

These longer runs exposed another important issue: runtime stability.

One DeepSeek configuration showed Metal buffer usage growing by roughly 52 GB over about an hour, eventually causing memory pressure, a failed request and model eviction. An earlier oMLX release candidate reportedly completed similar three-hour workloads without the same problem.

This reinforces a point that benchmark charts cannot show. For unattended local agents, stable operation for six or twelve hours matters more than achieving another 20 tok/s for a short test.

NVIDIA remains stronger in several important areas.

When a workload fits entirely inside an RTX 5090's VRAM, CUDA hardware is typically faster. MacStories showed the 5090 leading the M5 Ultra in several Qwen tests where model and cache stayed inside VRAM.

Once the workload spills into ordinary PC system memory, performance can collapse because data must cross PCIe. The Mac avoids that boundary because the GPU works directly across the unified-memory pool.

The comparison is therefore better framed this way:

RTX 5090 is generally faster when the workload fits. M5 Ultra can keep workloads GPU-accessible that simply do not fit inside consumer GPU VRAM.

Dual DGX Spark testing exposed another distinction. The M5 Ultra can be competitive in single-stream generation, particularly with MLX, but dual Sparks are much stronger at cold prompt processing and scale better under heavier concurrent workloads.

That makes the likely positioning clearer.

The M5 Ultra is looking very strong as a personal or small-team private AI workstation with one large model and several cooperating agents.

NVIDIA server architectures remain better suited to serving many independent users or long-context sessions simultaneously.

Power and thermals still need better testing. Some sustained comparisons have measured Mac Studio system load above 400 W, while shorter MLX tests have shown both nominal and elevated macOS thermal states depending on workload. There is still no definitive multi-hour study combining wall power, clocks, temperatures, memory pressure and token throughput.

So after three weeks, the major conclusions are fairly consistent:

256 GB unified memory changes what is practical on a desktop
150 to 190 GB models can run locally at interactive speeds
128K to 200K contexts are now practical
optimized Qwen workloads can exceed 100 tok/s
short-request batching can exceed 300 aggregate tok/s
MLX software maturity is improving performance very rapidly
prefix caching is critical for long-context agents
256 GB becomes tight with the largest models and full RAG stacks
concurrency and cold prefill remain weaker than NVIDIA server platforms
sustained runtime stability still needs more work

My conclusion is not that the M5 Ultra is the fastest AI system available. It is something more specific.

It may be one of the first compact desktop systems where running genuinely large, private LLMs becomes a normal daily workstation workflow rather than a hardware experiment.

For private coding assistants, RAG over confidential engineering or business data, document agents and local automation, the combination of large unified memory, high bandwidth and a rapidly improving MLX ecosystem is increasingly compelling.

The next tests I would like to see are less about peak tok/s and more about production behavior: 4 to 8 coding agents running for many hours, persistent RAG databases, embeddings and rerankers operating simultaneously, cache reuse, automatic recovery, wall-power measurements and controlled comparisons against RTX Pro and multi-GPU NVIDIA systems.

Three Weeks In: M5 Ultra Mac Studio Is Becoming a Serious Local AI Workstation

Sources