Skip to content
← All articles

What Actually Works in On-Device Speech Recognition

Apple's new speech API, Whisper, and what the independent benchmarks actually show, for anyone building something that listens.

By Nelson, maker of Aurator

Aurator listens to you all day. Every dictation starts with speech recognition, before any cleanup, before any coaching, before any of it. So I have spent a lot of time this year reading the actual documentation and the actual benchmarks behind the models that turn speech into text.

Not the marketing pages. The API references, the WWDC sessions, the model cards, and the benchmark repositories that publish their own numbers.

Here is what I found, and what it changed about how Aurator listens.

The one-minute wall is gone

For years, Apple's speech API had a quiet limit built in. Apple's own documentation for the old SFSpeechRecognizer says it plainly, plan for a one minute limit on audio duration, because speech recognition places a high burden on battery and network, and the framework stops a recognition task once it runs past that.

A one minute cap is fine for a text message. It is not fine for a meeting, a lecture, or a long dictation.

So in 2025 Apple shipped a new API, SpeechAnalyzer, with a module called SpeechTranscriber that does the listening. Apple's own words from the session that introduced it, the new model is both faster and more flexible than the one previously available, and it is good for long form and distant audio, lectures, meetings, conversations. No one minute wall.

The one minute wall is gone

It runs on the device, and it does not cost your app anything to carry

The new model is private by design. Apple's own words, we wanted to keep speech private, our new on device model achieves all of that. The audio never has to leave the Mac.

And it is light in a way that matters to anyone shipping an app. Apple's own words again, the model is retained in system storage and does not increase the download size, the storage size, or the runtime memory of your application, it operates outside your application's memory space entirely. The operating system owns the model. Your app just asks it to listen.

How it actually compares to Whisper

I did not want to take Apple's word for how good the new model is, so I went looking for an independent run.

Argmax, the team behind WhisperKit, benchmarked Apple's new on device model against Whisper on an M4 Mac mini, on a slice of a real conversational dataset. Apple's model scored a 14.0 error rate at a 70 times speed factor. Whisper's small model scored a slightly better 12.8, but at only 35 times speed, half as fast. Whisper's base model was faster again, 111 times, but with a worse 15.2 error rate.

Argmax's own conclusion, in their own words, Apple matches the speed and accuracy of mid tier OpenAI Whisper models on long form conversational speech transcription. Not a toy. A real contender, on a Mac, for free, built into the operating system.

Apple's model against Whisper

Whisper itself is not one model, it is a ladder

If you reach for Whisper directly, OpenAI's own model table matters more than people think. It runs from a 39 million parameter tiny model up to a 1.55 billion parameter large model, and there is a turbo version in between, 809 million parameters, with the decoder cut from 32 layers down to 4. OpenAI's own words, the model is way faster, at the expense of a minor quality degradation.

On the Open ASR Leaderboard, large v3 scores a mean error rate of 7.44. Turbo scores 7.83, close behind, for roughly 38 percent more throughput. For most real work, that trade is worth taking.

Running it yourself, on a Mac, is genuinely fast now

Two open projects make this practical. whisper.cpp, maintained in the ggml org, is a from scratch C and C plus plus implementation with no dependencies. Its own words, Apple Silicon first class citizen, the encoder can run on the Neural Engine through Core ML, more than three times faster than CPU alone.

MLX Whisper, published by Apple's own MLX team, is the other path. One independent benchmark, run on an M5 Mac, had Whisper large v3 turbo transcribing continuous conversational audio at a real time factor of 0.0266. That is almost 38 times real time. An hour of audio, transcribed in under two minutes, entirely on the machine in front of you.

Running Whisper locally on a Mac

The limit nobody mentions, the language has to be chosen first

This is the one that actually shaped how Aurator works.

Apple's on device model has to be told what language it is listening for before it starts. It cannot detect the language mid sentence the way a person can. Most apps solve this by asking the user to pick one language and leave it there.

That does not work for anyone who genuinely speaks two languages in a day. English is my second language, Spanish is my first, and I write to my family in one and to clients in the other, often within the same hour.

So Aurator runs your two most used languages side by side, over the same recording, entirely on device, and keeps whichever one actually reads like real words. No cloud call, no menu to dig through, no guessing out loud. It costs a little more processing for every dictation, and I think it is worth it.

The other quiet lever, biasing the model toward your own words

Apple's speech API lets you hand it a list of phrases it should recognize even when they are not in the system vocabulary, officially called contextualStrings. It is a bias list, not a guarantee, but it is the difference between a transcript that gets a product name or a person's name right, and one that does not.

Aurator feeds this from your own personal dictionary. Add a word once, and the model leans toward hearing it correctly from then on, at the point where the words first become text, not as a find and replace afterward.

What it actually costs

On device transcription has no per minute cost at all, because nothing is sent anywhere. A cloud api is a different story. OpenAI's own Whisper API is priced at 0.006 dollars a minute. That is small for one dictation. It is not small for an app that listens all day.

What I would choose, and why

If I am building something that listens on a Mac or an iPhone today, here is the order I would reach for things in.

Apple's own SpeechAnalyzer first, because it is free, private, already on the device, and the independent numbers say it now holds its own against Whisper's mid tier models.

MLX Whisper or whisper.cpp next, when I need a language Apple does not yet cover, or a model size Apple does not offer, and I am willing to ship and manage the model myself.

A cloud API last, only when the audio has to be processed somewhere other than the user's own device, and I am willing to pay for every minute, forever.

Aurator's own cleanup and coaching sit on top of whichever one is listening. But the listening itself, I now believe, is a solved problem on a Mac. The honest work left is in what you do with the words once they land.

If you want to see what that looks like in a real app, Aurator is at tryaurator.app. Seven days free, then US$29 once, and the listening happens on your Mac by default.

And if you have found different numbers in your own testing, I would really like to hear them.

Sources, all checked on 3 October 2026