M5 Ultra Mac Studio for Local AI: 256 GB vs 512 GB Starts to Become a Real Question

The latest M5 Ultra tests finally make the 512 GB discussion more concrete. Independent testing now shows that the 256 GB Mac Studio can be extremely competitive for local AI when the software stack is well optimized. At the same time, the first 512 GB community results are showing workloads consuming more than 400 GB of unified memory. The distinction is becoming clear: 512 GB is primarily about expanding what can fit, not automatically making models that already fit run faster.

Independent testing: M5 Ultra versus NVIDIA GB10

The most important recent test comes from Tom’s Hardware, which evaluated an M5 Ultra Mac Studio with 36 CPU cores, 80 GPU cores and 256 GB of unified memory against NVIDIA GB10 systems.

The first comparison used Qwen3.8-27B with multi-token prediction enabled and llama.cpp on both platforms. Tom’s Hardware used Artificial Analysis AgentPerf-Local, which replays inference traces from agentic workloads and places significant emphasis on prompt processing.

The M5 Ultra completed the workload in approximately 555 seconds, compared with 1,020 seconds for a single GB10 system.

Wall-power measurements also allowed total task energy to be calculated:

M5 Ultra: 139 kJ
GB10: 166 kJ

In this particular workload, the Mac therefore completed the job about 1.8 times faster while consuming around 16 percent less total energy.

This is more useful than another isolated tokens-per-second figure because it measures the energy required to finish a fixed workload.

It should not be generalized to every model. The rest of the Tom’s Hardware testing shows why.

Large models expose the importance of the inference engine

Tom’s Hardware also tested DeepSeek V4.1 Flash and GLM-5.3 Flash, models large enough to make the memory architecture genuinely relevant.

DeepSeek exposed a weakness in the current Apple software ecosystem.

On the M5 Ultra, the available configuration relied on llama.cpp and could not take advantage of the same optimized speculative-decoding setup used by the two-GB10 cluster. At long contexts the difference became substantial.

At 128K context, DeepSeek time-to-first-token on the Mac was approximately 274 seconds, compared with about 66 seconds on the dual-GB10 setup. At 256K, the Mac required more than ten minutes before the first token.

This is not simply a hardware comparison. The two systems were running different serving recipes, which is precisely the point: local AI performance is increasingly determined by the complete hardware-software combination.

GLM-5.3 provides a much better example of what happens when Apple Silicon gets a well-optimized MLX-native serving stack.

On the same M5 Ultra, moving from llama.cpp to an optimized TensorFold/MLX configuration changed the results dramatically.

At 128K context:

Generation: 34.1 tok/s to 96.5 tok/s
TTFT: 118.3 s to 66.3 s

At 256K context:

TTFT: 484.6 s to 137.1 s

The hardware did not change. The inference stack did.

This is probably the most important conclusion from the recent M5 Ultra testing.

Buying more memory or a faster processor does not guarantee good local-AI performance. Model support, quantization, speculative decoding, KV-cache handling and MLX-specific kernels can determine whether a large model feels responsive or frustrating.

oMLX continues to move quickly

The open-source MLX ecosystem is also evolving rapidly.

Recent oMLX releases have added or improved multi-token prediction, lower-memory KV caches, multi-model stability, DeepSeek optimizations, distributed inference and even the ability to use a second Mac as a dedicated prefill accelerator.

For example, oMLX 0.7.4.dev1 demonstrates a configuration where a large-memory Mac retains the model and KV cache while a second Mac with a faster GPU processes the first layers of prompt prefill.

Developer testing reports substantial prefill improvements for several large models. The important architectural idea is that decode remains on the high-memory host while another machine accelerates the compute-heavy prompt-ingestion phase.

This creates interesting alternatives to simply buying the largest possible single machine, although these distributed configurations still need broader independent testing.

512 GB finally demonstrates a real capacity advantage

Until recently, most discussion around the 512 GB M5 Ultra was theoretical.

A model that uses 100 GB will not automatically become faster because the machine has 512 GB rather than 256 GB. Both configurations use the same M5 Ultra compute architecture and memory-bandwidth class.

The extra memory becomes valuable only when it enables a workload that cannot operate comfortably within 256 GB.

We now have an example.

A new oMLX community submission dated 11 November reports DeepSeek V4.1 Flash oQ4e with Lightning MTP running on an 80-core M5 Ultra with 512 GB unified memory.

At 64K context it reports:

4,523 prompt tok/s
79.7 generation tok/s
14.5 s TTFT
408.6 GB peak MLX memory

At approximately 195K context, prompt processing remained above 4,300 tok/s and generation around 79 tok/s, while the model footprint stayed around 408 GB.

Earlier results from the same model family using oMLX 0.7.3 reported roughly 411 GB of peak memory, so this appears to be a genuinely capacity-class workload rather than a small model running on an unnecessarily large machine.

This is the first result in this monitoring period that provides a convincing technical reason for 512 GB.

The configuration, as tested, simply cannot reside completely inside 256 GB.

There is an important qualification: these are community submissions. The hardware configuration is self-reported, and we do not yet have an independent review reproducing the same workload on a delivered retail 512 GB machine.

The result should therefore be treated as promising evidence, not final verification.

Today’s 256 GB results are also revealing

New oMLX submissions on 12 November continue to show how capable the 256 GB configuration remains when the workload fits.

An 80-core M5 Ultra running Qwen3.8 Flash-Next at 8-bit recorded approximately 5,355 prompt tok/s and 99 generation tok/s at 64K, using around 187 GB of model memory. At 128K it remained above 5,200 prompt tok/s and approximately 96 tok/s generation.

A separate 6-bit Qwen configuration on the same class of machine reported more than 110 tok/s generation at 64K, with approximately 148 GB peak model memory.

These are community benchmarks rather than production tests, but they illustrate why 256 GB continues to look attractive. A large and high-quality model can remain fully resident while leaving enough memory for macOS, development tools, embeddings, retrieval services and other components of a private AI stack.

What 512 GB does and does not buy

The emerging distinction is straightforward.

For a workload occupying 100 to 180 GB, the 512 GB version should not be expected to provide a major speed advantage purely because it contains more memory.

Its advantages are instead:

models or higher-precision quantizations exceeding 256 GB;
much larger KV caches and context windows;
several large models remaining resident simultaneously;
more concurrent agents and supporting services without memory pressure;
avoiding model eviction and reload cycles;
experimentation with 300 to 450 GB-class inference footprints.

This also means that memory capacity must be evaluated against the intended workload, rather than purchased as a proxy for performance.

For many private coding and RAG deployments, 256 GB remains a very large memory pool.

Where NVIDIA still has an advantage

The Tom’s Hardware comparison also prevents an overly simple conclusion.

The M5 Ultra is extremely strong in some local inference workloads, particularly when memory bandwidth and a good MLX serving recipe align with the model.

NVIDIA still benefits from a broader and more mature inference ecosystem. New models frequently receive optimized CUDA serving recipes quickly, and multi-system clustering is already a familiar deployment model.

With DeepSeek V4.1, the two-GB10 configuration clearly benefited from that software maturity.

For users buying hardware specifically as an inference server, model support may therefore matter as much as theoretical memory capacity or bandwidth.

The Mac Studio becomes particularly compelling when local AI is one part of a broader workstation role: software development, engineering, media work, document analysis and private AI can all coexist on the same machine.

Current assessment

The evidence through 12 November suggests three fairly clear conclusions.

First, the 256 GB M5 Ultra is already a serious local-AI workstation. With optimized MLX software, it can run large models, long contexts and agent workloads at genuinely interactive speeds.

Second, software maturity remains critical. The difference between an unsuitable generic inference configuration and a model-specific optimized MLX stack can exceed the difference between expensive hardware platforms.

Third, 512 GB now has a technically credible use case. A DeepSeek V4.1 configuration consuming more than 400 GB demonstrates that the extra memory can enable workloads physically impossible on 256 GB as configured.

That still does not make 512 GB the automatic choice.

For most deployments where the primary model, KV cache and supporting RAG services remain comfortably below 256 GB, the smaller configuration continues to look like the more rational balance of cost and capability.

The real case for 512 GB begins when the workload itself crosses the boundary.

The next evidence we want to see is therefore not another short benchmark. It is a delivered 512 GB system running a 300 to 450 GB-class model for several hours with real coding or RAG agents, while measuring wall power, thermal behavior, memory pressure, latency, concurrency and output quality.

That will tell us whether 512 GB is simply an impressive capacity specification or the foundation of a genuinely new class of single-box private AI workstation.