We benchmarked open speech models on a Mac mini VM — the numbers nobody had published

The model card says 3,300 times faster than real time. We measured 37.

Both numbers are true. They are just answers to different questions — and if you plan capacity off the first one, you will be wrong by two orders of magnitude.

We run an agent system that lives on text-to-speech and speech-to-text. Every voice note, every audio digest, every “listen while you run” doc goes through a speech model somewhere. Right now that somewhere is a cloud API. We wanted a private endpoint instead — one we control, one that doesn’t leak a transcript of everything the agents say to a third party.

So we asked a frontier chat model for a stack. Then we fact-checked its advice before writing a line of code. Then we built the real thing in one night, on a Mac mini running a macOS VM. Here’s what came out.

Why a private speech endpoint

A private speech endpoint is a self-hosted server that turns text into audio and audio into text without sending either through a third-party API. For a system where agents narrate their own work and listen to voice notes, that’s not a nice-to-have. It’s the difference between “our internal reasoning stays internal” and “every voice note is also a line in someone else’s logs.”

The obvious place to run it was a Mac mini we already had sitting idle. The obvious way to keep it clean was a VM — the heavy, churn-prone Python environment stays disposable, separate from anything else on that box.

We asked a frontier model, then fact-checked it

Before touching any code, we asked a frontier chat model to recommend a local speech stack for an Apple Silicon Mac mini. It came back with a coherent architecture: a router in front of specialized workers, a pinned hot model with LRU eviction, pronunciation dictionaries for correction. That part held up well.

The model and serving picks did not. We ran every recommendation against GitHub repos, Hugging Face model cards, the live Open ASR Leaderboard, and vendor docs — and found real problems with the two anchor picks.

Its top server recommendation had no Metal path at all. The suggested server runs on faster-whisper, and that library’s underlying engine has no Metal or MPS backend — its documented device options are cpu and cuda, nothing else. On a Mac, that server runs entirely on the CPU, with the GPU and Neural Engine sitting idle. Its own install docs never mention macOS.

Its default speech-to-text pick, Whisper large-v3-turbo, is no longer competitive on accuracy. Pulling the live leaderboard showed roughly ten Apache-2.0 open models ranked ahead of it on word error rate. One of them was Granite-Speech-4.1-2b — the exact model the frontier model said it wouldn’t personally deploy, and one of the highest-ranked open entries on the board, with a documented path onto Apple Silicon.

Licensing was never mentioned once. Several of its alternate suggestions carry non-commercial or bespoke research licenses that would block us from ever publishing audio built on them. That’s the kind of thing that eliminates a model before performance does, and it wasn’t on the transcript’s radar at all.

The fact-check also surfaced the deployment question that ended up mattering most: bare metal, or a VM? That single distinction reorders almost everything, because of one hardware fact.

What we actually built, in one night

Metal works inside a macOS VM on Apple Silicon. The Neural Engine does not.

That’s the load-bearing finding underneath everything else. Apple’s Virtualization framework gives a macOS guest real, GPU-backed Metal access — not a stub. We confirmed it live: every model we tested ran on GPU-backed MLX inside the VM, with no special configuration and no VM-specific slowdown beyond ordinary cold-start weight loading. The Neural Engine, by contrast, isn’t exposed to the guest at all — an ANE-dependent model silently falls back to a slower CPU or GPU path with no error, which is worse than a crash because it looks like it’s working.

The practical upshot: pick pure-Metal MLX models for anything inside the VM, and save Neural-Engine-dependent tooling for a future host-side tier.

We stood up mlx-audio, an MIT-licensed, OpenAI-compatible speech server, inside the guest (32 to 48GB allocated). Getting there took a clean uv-managed Python 3.12 environment — the guest’s system Python was too old — plus a handful of undocumented surprises: Kokoro needed an extra text-processing package that dragged in roughly a gigabyte of side dependencies; Voxtral needed an explicit voice parameter with no default; one model’s exact Hugging Face repo id didn’t exist and needed a community substitute; and one STT model accepts only 16kHz mono input, which the model card doesn’t say.

We verified four models end-to-end, from a real remote client, over an actual network hop — not simulated:

• Kokoro-82M (TTS)
• Voxtral-4B-TTS (TTS)
• parakeet-tdt-0.6b-v3 (STT)
• granite-speech-4.1-2b-nar (STT)

A real round trip worked: text in, speech out via Kokoro, speech back in as text via Parakeet, both calls returning success over the network, both live-verified.

The numbers

Every number below is a first-published measurement for these models on a Mac-mini VM. Source text was a real technical paragraph, tested at two lengths: a short 82-word clip and a longer 501-word clip.

Text-to-speech

RTF below 1.0 means faster than real time; lower is better.

One correction we owe our own first pass. Kokoro’s 0.21 short-text RTF came from a 5.25-second generation that turned out to be the first call after loading the model, not a steady-state one. Re-running the same short request against the already-running server returned in 2.0 seconds end-to-end, HTTP included. So 0.21 is a cold-ish upper bound, and Kokoro’s true steady-state short-text speed sits much closer to its long-text figure of 0.07 — about fourteen times faster than real time.

That correction widens the gap between the two TTS models rather than narrowing it. On long text, Voxtral spends roughly six times the wall clock per second of audio produced, and about four times the memory, for its extra expressiveness. Kokoro is the default; Voxtral is the one you reach for when you want the voice, not the throughput — though its license, as it turns out, has something to say about that.

Speech-to-text

It also works on real people

The first version of this benchmark had an obvious hole: we generated audio with our own TTS models and fed it straight back into our own STT models. That only proves the two halves agree with each other. Synthetic speech is clean, evenly paced, and free of the things that actually break transcription.

So we ran both models against a genuine 67-second human voice note — phone mic, casual speech, filler words, mid-sentence self-corrections, the works. Parakeet transcribed it at 37.3x real time; granite-nar at 33.7x. That’s modestly slower than the 44-to-53x the synthetic clips managed — the same order of magnitude, not a different regime. The round-trip setup wasn’t flattering the numbers much.

The transcripts were near-identical and genuinely readable, both models catching the hesitations and corrections rather than smoothing them away. Two differences worth naming. Granite-nar inserted one hedge word that wasn’t spoken — a small hallucination-style addition rather than a dropped-content error — and it emits lowercase only, with no case model, consistent with what we saw on the synthetic clips. And both models rendered a slash-command reference as ordinary words, which is exactly what you’d expect from models that have no concept of chat-app commands; a domain dictionary or hotword hint would fix it, and mlx-audio’s transcription CLI already exposes a flag for that.

The honest verdict is that parakeet and granite-nar are close enough on this hardware that speed and licensing should drive the pick, not accuracy. We haven’t compared either against a cloud transcription service on the same audio — that’s still open.

The headline finding: the vendor-number gap

Both speech-to-text models publish figures in the thousands — parakeet-tdt-0.6b-v3’s model card reports 3,332x real time, granite-nar’s about 1,820x. On this hardware, running one request at a time, they landed between 34x and 53x.

That is not a rounding error. At the narrow end it’s a 34-fold gap; on real human audio against parakeet’s published figure, it’s nearly 90-fold. The vendor numbers were almost certainly measured on datacenter GPUs with large batches running at once, not a lone call against a Mac mini. Both models remain comfortably faster than real time for a voice-agent workload — a minute of audio transcribes in about two seconds — just not by four digits. If you’re planning capacity off a model card’s RTFx figure, this is the correction: expect low tens for single-stream Apple Silicon inference.

Does the server add a tax?

We benchmarked the models in-process for a fair model-to-model comparison, which raises the fair question of whether the real HTTP path costs anything. We re-ran the short TTS and STT calls through the running server, timed client-side end to end, and the answer is essentially no: server-path latency sat inside call-to-call noise of the in-process numbers at these payload sizes — hundreds of kilobytes to a few megabytes of audio. No meaningful serialization tax. The server is fine to build a real client against as it stands.

Licensing

The Voxtral row is the second correction we owe our own first pass. We had it filed as Apache-2.0 like the rest of the Voxtral line, on the strength of a secondary write-up. The model card says otherwise: its reference voices are CC-BY-NC-4.0 and the model inherits that license. Voxtral stays interesting for the voice, and it is off the table for anything we publish.

Which is the whole point. None of this made it into the frontier model’s original recommendations — and one of them slipped past our own fact-check too. For anything destined for a public post or a product surface, licensing is the constraint that should come first, not last.

Honest limits

A benchmark that hides its own limits isn’t a benchmark.

We tested one request at a time. Nothing here measures concurrent load, batching, or what happens when a router has to juggle simultaneous calls.

We didn’t measure streaming latency. The server writes a complete audio file rather than streaming it, so the cold-start numbers above are an upper bound on first-byte latency, not the real streaming figure.

We didn’t compare against a cloud baseline. The real-audio test tells us the two local models agree with each other on human speech; it doesn’t tell us how either compares to a hosted API on the same clip.

Memory numbers are relative, not OS ground truth. Peak RSS was sampled inside the benchmarking process — good for comparing models to each other, not for capacity planning against the machine.

We didn’t push disk or model count. We stayed conservative and verified four models rather than the full candidate list; a wildcard TTS model and the Neural-Engine host tier both got skipped for the night.

What’s next

Three things are queued: true first-byte latency for streaming requests instead of full-file wall time, a host-side Neural Engine tier for the always-on low-power path, and a capability router — a thin layer that lets a caller ask for “fast narration” or “accurate transcription” without knowing which model or which machine answers.

The bigger point survives all the caveats. A real private speech endpoint runs comfortably on a Mac mini inside a VM. Metal works exactly as advertised and the Neural Engine doesn’t come along for the ride. The numbers on the model cards are not the numbers you’ll get — and now we have our own, on synthetic audio and on a real human voice alike.