Automatic Speech Recognition (ASR)
The task and technology of converting spoken audio to written text, including foundation ASR models like Whisper and their evaluation via word-error-rate benchmarks.
Also known as: ASR, speech recognition, speech-to-text, Whisper, OpenAI Whisper, NB-Whisper, word error rate, WER, transcription accuracy, speech-to-text models
How authors use it
Oldest first (extraction date when no publication date is known).
| Author | Their term | Their meaning |
|---|---|---|
| Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges Per E Kummervold, Javier de la Rosa, Freddy Wetjen, Rolv-Arild Braaten, Per Erik Solberg · 2024-02-02 | Word Error Rate (WER) | The paper's central metric, computed with the JiWER package after lowercasing and stripping punctuation, deliberately reported non-normalized so scores are directly comparable across the three test datasets; the authors note that a stronger normalization (e.g. spelling out numbers, as in a companion Wav2Vec2 paper) would have produced lower, but less comparable, scores. |
Documents (2)
- Generative AI in the Real World: Local Voice AI with Pete Warden Ben Lorica and Pete Warden · 2026-09-10 Reader 🖥️
Local AI is no longer a technical compromise on either text or voice: open-weight models already fit consumer hardware and track frontier quality closely, so the remaining gap is usability polish and a marketing incentive structure that rewards subscription clouds, not model capability. Voice interfaces have earned their bad reputation from bad implementations (high latency, brittle recognition), not from any inherent dislike of talking to machines; fixing latency, designing for graceful failure, and removing per-use cost thresholds unlocks a different, larger space of viable applications. Chaining specialized models through a shared text interface ('compound AI') is the honest architecture most of the field actually uses; 'end-to-end' single-model pipelines are as much a funding narrative for capital-intensive labs as a technical necessity. And the economic logic increasingly favors owning your own inference: the durable advantage in an AI product is the compounding usage loop you build around a model, not the model itself, and only self-hosting protects that loop from a vendor's unilateral pricing, deprecation, or capability changes. - Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges Per E Kummervold, Javier de la Rosa, Freddy Wetjen, Rolv-Arild Braaten, Per Erik Solberg · 2024-02-02 Reader 🖥️
NB-Whisper — OpenAI's Whisper fine-tuned by the National Library of Norway on roughly 28,000 hours of curated, iteratively-cleaned Norwegian audio-text data from four sources (NRK subtitles, audiobooks, the NST corpus, and the Stortinget speech corpus) — substantially reduces word-error-rate on Norwegian Bokmål and Nynorsk transcription (e.g. 10.4→6.6 on Fleurs, 6.8→2.2 on NST, 30.0→12.6 on Common Voice Nynorsk) by explicitly targeting the dual written standards and pervasive dialectal/orthographic variation that generic Whisper handles poorly, though an inherent ~1-2% WER floor from legitimate Norwegian spelling variation and unresolved hallucination/streaming limitations mean the problem is substantially advanced rather than solved.
Propositions touching this concept (8)
By publication month, oldest first (extraction month when no publication date is known).
2024-02
- explicitNB-Whisper Large reduces Norwegian Bokmål WER on the Fleurs dataset from 10.4 (OpenAI Whisper Large v3) to 6.6.
- explicitNB-Whisper Large reduces Norwegian Bokmål WER on the NST dataset from 6.8 (OpenAI Whisper Large v3) to 2.2.
- explicitOn Common Voice Nynorsk, NB-Whisper Large achieves 12.6% WER versus 30.0% for OpenAI Whisper Large v3 — a proportionally larger improvement than on Bokmål datasets.
- explicitAt the Tiny model size, NB-Whisper achieves 28.0% WER on Common Voice Nynorsk while OpenAI Whisper Tiny exceeds 100% WER (more insertions/errors than reference words) — i.e., stock small Whisper models are close to unusable for Nynorsk.
- explicitdevelopedASR performance scales strongly and roughly monotonically with model size for Norwegian (Tiny through Large), for both OpenAI Whisper and NB-Whisper, but the absolute and relative gap between the two narrows as size increases.
- explicitNB-Whisper's architecture (autoregressive decoder, 30-second audio chunks) is not well-suited to real-time/live transcription, and would benefit from an alternative architecture for that use case.
2026-09
- explicit'End-to-end' speech-to-speech models are largely a marketing framing that favors well-capitalized labs, since only they can afford to train a single model spanning an entire pipeline; a 'compound AI' architecture instead chains specialized speech-to-text, LLM, and text-to-speech models through a shared text interface, trading paralinguistic expressiveness (tone, intonation) for independent trainability and upgradability of each component.
- explicitGeneral-purpose multimodal models are not consistently better than dedicated ASR models at speech transcription — behavior varies by model and language (a roughly 1.7-billion-parameter Qwen multimodal model reportedly outperforms Whisper, notably on Mandarin), while other general models like Gemma can misfire by treating transcribed speech content as a fresh prompt to answer rather than text to return; general-purpose models are also non-viable at all on extremely constrained edge hardware.
Open gaps (7)
- incompleteNo quantitative benchmark, error-rate, or task-success number is offered anywhere in the episode — every capability claim ('a few months behind', 'wipes the floor', 'does a better job', 'significantly better on some languages') is a qualitative impression from someone with a direct commercial stake in local voice AI (Moonshine.ai) succeeding.Generative AI in the Real World: Local Voice AI with Pete Warden — read next: Read against extract-local-ai-loukides#p9 ('published benchmarks are an unreliable basis for model selection; domain-specific evaluation is necessary') and extract-experiences-with-local-models-for-coding#p17 (benchmark-free claims are close to meaningless without task and methodology) — this episode's claims are exactly the unaudited pattern that corpus already treats skeptically.
- incompleteLorica's direct question about whether AMD or other vendors will diversify AI hardware away from NVIDIA goes essentially unanswered; Warden's response is almost entirely about Apple, leaving the 'beyond NVIDIA' hardware landscape one-sided and the original question unaddressed.Generative AI in the Real World: Local Voice AI with Pete Warden — read next: Would need a comparable industry voice on the AMD/alternative-accelerator side to balance Warden's Apple-centric hardware forecast.
- incompleteDespite framing dialectal diversity as the central Norwegian ASR challenge (with the 38-pronunciation 'hvem' example), the paper never reports WER broken down by dialect group — only by dataset and written standard (Bokmål/Nynorsk). It is impossible to tell from this paper whether remaining errors concentrate on non-Oslo or minority dialects, which would be the direct empirical test of the paper's own motivating claim.Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges — read next: Look for NB-Whisper follow-up work or model-card updates using the Nordic Dialect Corpus (or NST's dialect metadata) as a stratified evaluation set.
- incompleteNo comparison against any Norwegian ASR system besides the authors' own prior NB-Wav2Vec2 1B model — no commercial cloud ASR (Google, Azure, Amazon) and no other multilingual open models (e.g. Meta's MMS or SeamlessM4T) are benchmarked, so NB-Whisper's standing relative to the broader Norwegian ASR market is unknown from this paper alone.Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges — read next: Search for independent third-party benchmarks comparing NB-Whisper against commercial cloud ASR APIs for Norwegian.
- incompleteNo ablation studies isolate which of the four data sources, or which of the four cleaning criteria (fuzzy matching, insertion detection, omission detection, NER filtering), actually drove the WER improvement. The recipe is reported in full but never decomposed, so a reader cannot tell which ingredient mattered most or whether some contributed little or even hurt.Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges — read next: Check whether the NB-Whisper GitHub repo (nostram) or any successor NB AI Lab paper includes ablations over data sources or cleaning filters.
- illogicalThe 'clear trend' language used to describe model-size scaling (Section 6) treats single-run WER point estimates — with no reported variance, confidence intervals, or repeated-seed training — as sufficient evidence of a definitive, generalizable relationship between model size and accuracy for Norwegian specifically.Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges — read next: Compare against papers that report WER variance across training seeds or use statistical significance testing (e.g. bootstrap resampling over test-set WER) for ASR scaling claims.
- incompleteThe real-time/streaming transcription limitation is named but not engaged with relative to existing community workarounds (e.g. sliding-window/VAD-based chunked streaming used by tools like whisper.cpp or faster-whisper) that already mitigate this for the base Whisper architecture the paper builds on.Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges — read next: Look at whisper-streaming / faster-whisper VAD-chunking implementations to see how the wider Whisper ecosystem addresses real-time use without an architecture change.