**SozAI Accuracy Test: Word Error Rate in 10 Languages**
Source: https://sozai.app/sozai-accuracy-test-10-languages/

[![SozAI](https://sozai.app/wp-content/themes/sozai/assets/images/logo.svg)](https://sozai.app/)

- [Log in](https://web.sozai.app/)
- [Start free](https://web.sozai.app/)

 [Get App](https://sozai.app/download/)

[Transcription](https://sozai.app/category/transcription/)

# SozAI accuracy test: 10 languages × 20 files

Oct 24, 2026  • 6 min read • 30 views  • Last updated: Oct 24, 2026

![Bar chart: word error rate of SozAI transcription in ten languages](https://sozai.app/wp-content/uploads/post-sozai-accuracy-test-10-languages-1280x720.webp)

We ran 200 audio files in 10 languages through SozAI to check, with numbers anyone can reproduce, how close its transcripts come to what was actually said. In eight of the ten languages, at least 14 of the 20 files had fewer than one word in ten wrong (for Japanese, one character in ten). Arabic and Kazakh did worse, and we explain why below. Every reference, every transcript and every per-file score are published with this article.

## What we tested

**The audio.** We used FLEURS, an open multilingual speech dataset ([Conneau et al., 2022](https://arxiv.org/abs/2205.12446)), released under CC BY 4.0. Each clip is a sentence of news, travel or encyclopedia-style text read aloud by a native speaker, with a reference transcript. We used the test split at one fixed revision.

**The languages.** English (US), Spanish (Latin America), Russian, Arabic (Egypt), Portuguese (Brazil), French, German, Turkish, Japanese and Kazakh. The set follows where SozAI’s users are, with Kazakh added as a lower-resource language with far less public speech data.

**The sample.** For each language we took the 20 lowest sentence ids and, for each sentence, the recording whose file name sorts first. Nothing was picked by hand or re-run. Because FLEURS sentences are translations of the same source, the ten languages read largely the same 20 sentences. In total: 200 files, 42 minutes 22 seconds of audio.

**The pipeline.** Each file went through exactly the request SozAI sends for an uploaded file or an app recording: the same speech-recognition engine SozAI uses for uploaded files, automatic language detection on, speaker labels on, no language passed in. SozAI does not ask which language you speak, so neither did we.

**The scoring.** We lowercased the reference and the transcript, replaced all punctuation with spaces and collapsed repeated spaces. Numbers were left as written, so “13” against “thirteen” counts as an error. Then we computed:

- **WER** (word error rate): words substituted, dropped or inserted, divided by the number of words in the reference. Lower is better; 5% means about one wrong word in twenty.
- **CER** (character error rate): the same on characters. For Japanese, which does not separate words with spaces, CER is the only meaningful measure.

The headline figure per language is the corpus rate: all errors in the 20 files divided by all reference words. We also give the median file and the worst file, because an average can hide one disaster.

## Results

| Language | Audio | WER | Median file | Worst file | CER | Wrong language detected |
| --- | --- | --- | --- | --- | --- | --- |
| Spanish (Latin America) | 4:12 | 1.8% | 0.0% | 10.5% | 0.5% | 0 of 20 |
| German | 5:03 | 3.0% | 0.0% | 19.1% | 0.8% | 0 of 20 |
| English (US) | 3:23 | 3.6% | 0.0% | 13.3% | 1.6% | 0 of 20 |
| French | 3:26 | 4.9% | 4.3% | 17.2% | 1.5% | 0 of 20 |
| Russian | 3:58 | 5.5% | 5.0% | 17.6% | 1.6% | 0 of 20 |
| Turkish | 4:25 | 7.6% | 6.5% | 21.7% | 2.0% | 0 of 20 |
| Portuguese (Brazil) | 4:41 | 8.0%\* | 4.0% | 100.0%\* | 5.8%\* | 0 of 20 |
| Arabic (Egypt) | 3:39 | 19.6% | 15.2% | 54.5% | 6.9% | 0 of 20 |
| Japanese | 4:36 | n/a | CER 3.0% | CER 18.6% | 4.8% | 0 of 20 |
| Kazakh | 5:00 | 50.0% | 50.0% | 117.7% | 26.3% | 5 of 20 |

\* One Portuguese speaker read the sentence twice; the transcript correctly contains it twice, the reference only once, so the second reading is scored as 100% inserted words. We kept the file in the table. Without it, Portuguese is at 3.9% WER and 1.5% CER.

In Spanish, German and English, at least half of the files had no word errors at all. Most of the remaining errors in these languages are names and spelling choices rather than misheard words: “Kochamo Tal” for “Cochamó-Tal”, “Josemite” for “Yosemite”, “dragonflies” where the reference writes “dragon flies”. Russian and Turkish errors are similar. “Wild Card” became “вилдкарт”, and “on üç koloni” (thirteen colonies) became “13 koloni”, which our scoring counts as a wrong word even though nothing was misheard.

## Where it is weak

### Kazakh

Kazakh is the weak language in this test, for two separate reasons.

**Language detection.** In 5 of the 20 files, automatic detection decided the speaker was talking Azerbaijani, a related Turkic language, and returned Azerbaijani-looking text in Latin script. Those five transcripts are unusable:

> Reference: Барлық зат есімдер, сонымен бірге «сіз» дегенді білдіретін «Sie» сөзі, тіпті сөйлемнің ортасында да бас әріптен басталады.
>
>
>
> SozAI: Varlıq zat yesimcəri son və mən bir gəz siz deyəndə bildirədim C sözü tipdə söyləyiminin ortasında da baz arptayam başlaladı.
>
>
>
> (“All nouns, including the word Sie, meaning ‘you’, always start with a capital letter, even in the middle of a sentence.”)

Detection was unsure even when right: median confidence 0.63, against 0.99 or more for every other language.

**Recognition.** On the 15 files that were detected as Kazakh, WER is 34.5% and CER is 7.5%. The gap between these two numbers is typical of Kazakh: words are long and carry several suffixes, so one wrong letter in a suffix makes the whole word wrong. The text stays readable but needs editing; names suffer most:

> Reference: Романтизмде Гете, Фихте, Шлегель сияқты жазушылардан алынған мәдени детерминизмнің үлкен элементі болды.
>
>
>
> SozAI: Романтизмді геоты, фихты, шілегіл сияқты жазушылардан алынған мәденит де термініңіздің нүкен элементі болды.
>
>
>
> (“Romanticism had a large element of cultural determinism, drawn from writers such as Goethe, Fichte and Schlegel.”)

If you record in Kazakh, expect a draft to correct, not a finished transcript, and check that the result is in Cyrillic. A wrong-language result is easy to spot, since it comes back in another alphabet.

### Arabic

Arabic scores 19.6% WER, but a large part of that is spelling rather than hearing. Most references carry vowel marks such as tanwin (رقمًا), which spoken-language transcripts usually leave out:

> Reference: لم يحدد رقمًا للتخفيضات، قائلًا إنها ستعتمد على الناتج الاقتصادي للصين.
>
>
>
> SozAI: لم يحدد رقم التخفيضات قائلا أنها ستعتمد على نتائج الاقتصادية للصين
>
>
>
> (“He did not set a figure for the cuts, saying they will depend on China’s economic output.”)

Six of the eleven words count as errors here, yet the meaning is mostly intact: three are vowel marks or hamza spelling, and the remaining slips are small changes of form. When we strip vowel marks from both sides, Arabic WER drops to 12.8%. We kept the stricter number in the table. Egyptian speakers reading Modern Standard Arabic is also not the same as Egyptian dialect in conversation, which this test does not cover.

## What this test does not show

- **Real recordings.** These are short sentences (typically 9 to 15 seconds) read from a script by one person in a quiet room. Meetings, lectures, calls, crosstalk, noise and long files are harder, and we did not measure them.
- **Speaker separation.** Every clip has one speaker, so the test says nothing about how well SozAI tells speakers apart.
- **Small differences.** Twenty files per language is a small sample; one bad file can move a language by several points, as Portuguese shows. A gap of one to three points between two languages means little.
- **YouTube links and summaries.** Both go through separate steps and are not measured here.
- **The future.** The recognition engine is updated over time. These results are from 24 October 2026.

## How to reproduce it

The file list, every reference and transcript and the per-file scores are here: [results.json](https://sozai.app/wp-content/themes/sozai/assets/data/accuracy-2026-10/results.json) (200 files) and [summary.json](https://sozai.app/wp-content/themes/sozai/assets/data/accuracy-2026-10/summary.json) (per language). The FLEURS references inside them are CC BY 4.0.

1. Take the FLEURS test split at the pinned revision and, for each language, select the 20 lowest sentence ids, taking the recording whose file name sorts first for each. 2. Send each file unconverted, with automatic language detection and speaker labels on and no language specified. Keep the plain-text result and the detected language. 3. Normalise both sides: Unicode NFKC, lowercase (Turkish dotted and dotless i handled separately), punctuation to spaces, whitespace collapsed, digits untouched; for Japanese, remove all spaces. 4. Score with the open-source jiwer library (version 4.0.0): corpus WER and CER per language, plus median and worst file.

The run covers 42 minutes of audio and costs a few cents. If you get different numbers, tell us.

## Related

- [Audio to text](https://sozai.app/audio-to-text/) — upload a recording and get the transcript.
- [Transcription languages](https://sozai.app/transcription/)
- [About SozAI](https://sozai.app/about/)

![Merey Tleugazin](https://sozai.app/wp-content/themes/sozai/assets/images/merey-tleugazin-2026.webp)

[Merey Tleugazin](https://sozai.app/about/merey/)

Founder of SozAI. Building tools that turn speech into text for professionals worldwide.

- [Instagram](https://instagram.com/mereytleugazin)
- [X](https://x.com/mereytleugazin)

## Related Articles

- [Jan 2 19 min ### Ultimate Guide to Speech to Text Conversion Tools: Transform Your Voice into Written Words](https://sozai.app/speech-to-text-conversion/)
- [Jul 30 6 min ### You Recorded a Two-Hour Meeting. How Do You Find the Right Minute?](https://sozai.app/find-a-moment-in-a-long-recording/)
- [Aug 2 8 min ### How to Quote a Video in an Essay Without Retyping It](https://sozai.app/quote-a-video-in-an-essay/)

[![SozAI](https://sozai.app/wp-content/themes/sozai/assets/images/appicon.png) SozAI — Free Download Transcribe audio & video instantly Get App](https://sozai.app/download/)

## Transcription, notes and quizzes — on all your devices

Free on iOS and Android — 30 minutes of transcription included.

[Start in the browser](https://web.sozai.app/)

- [Download on the **App Store**](https://sozai.app/download/ios/)
- [Get it on **Google Play**](https://sozai.app/download/android/)

Product

- [Audio to Text](https://sozai.app/audio-to-text/)
- [Video to Text](https://sozai.app/video-to-text/)
- [YouTube Transcript](https://sozai.app/youtube-transcript/)
- [Meetings](https://sozai.app/meeting-recorder/)
- [Note Taker](https://sozai.app/meeting-notes/)
- [Quizzes and flashcards](https://sozai.app/quizzes-and-flashcards/)
- [Audio Translate](https://sozai.app/audio-translator/)
- [Subtitles](https://sozai.app/subtitle-generator/)
- [Languages](https://sozai.app/transcription/)

Platforms

- [Web version](https://sozai.app/web-version/)
- [iPhone and iPad](https://sozai.app/download/ios/)
- [Android](https://sozai.app/download/android/)
- [Mac](https://sozai.app/download/macos/)
- [Chrome extensions](https://sozai.app/download/)
- [All apps & extensions →](https://sozai.app/download/)

Solutions

- [For students](https://sozai.app/lecture-notes/)
- [For teams](https://sozai.app/meeting-notes/)
- [For journalists](https://sozai.app/interview-transcription/)
- [For doctors](https://sozai.app/medical-transcription/)
- [For lawyers](https://sozai.app/legal-transcription/)
- [For podcasters](https://sozai.app/podcast-transcription/)

Free Tools

- [WhatsApp Voice to Text](https://sozai.app/tools/transcribe-whatsapp-audio/)
- [Voice to Text (Dictation)](https://sozai.app/tools/voice-to-text/)
- [Subtitle Generator](https://sozai.app/subtitle-generator/)
- [YouTube Transcript](https://sozai.app/youtube-transcript/)
- [Subtitle Converter](https://sozai.app/tools/subtitle-converter/)
- [Word Counter](https://sozai.app/tools/word-counter/)
- [All Free Tools →](https://sozai.app/tools/)

Resources

- [Help Center](https://sozai.app/help/)
- [Case Studies](https://sozai.app/cases/)
- [Blog](https://sozai.app/blog/)
- [Compare](https://sozai.app/compare/)
- [Alternatives](https://sozai.app/alternatives/)

Company

- [About](https://sozai.app/about/)
- [Pricing](https://sozai.app/pricing/)
- [Contact](https://sozai.app/contacts/)
- [Press kit](https://sozai.app/press/)
- [Changelog](https://sozai.app/changelog/)

SozAI is a product of VAST FLOW, TOO · © 2026

- [Privacy](https://sozai.app/privacy/)
- [Terms](https://sozai.app/terms/)
- [DMCA](https://sozai.app/dmca/)

- [EN English](https://sozai.app/sozai-accuracy-test-10-languages/)
- [DE Deutsch](https://sozai.app/de/)
- [ES Español](https://sozai.app/es/)
- [FR Français](https://sozai.app/fr/)
- [IT Italiano](https://sozai.app/it/)
- [HE עברית](https://sozai.app/he/)
- [JA 日本語](https://sozai.app/ja/)
- [KO 한국어](https://sozai.app/ko/)
- [NL Nederlands](https://sozai.app/nl/)
- [PL Polski](https://sozai.app/pl/)
- [PT Português](https://sozai.app/pt/)
- [RU Русский](https://sozai.app/ru/)
- [SV Svenska](https://sozai.app/sv/)
- [TH ไทย](https://sozai.app/th/)
- [UK Українська](https://sozai.app/uk/)
- [VI Tiếng Việt](https://sozai.app/vi/)
- [ZH 中文 (中国)](https://sozai.app/zh/)
- [AR العربية](https://sozai.app/ar/)
- [DA Dansk](https://sozai.app/da/)
- [KK Қазақ тілі](https://sozai.app/kk/)
- [MS Bahasa Melayu](https://sozai.app/ms/)


---
This is the markdown twin of https://sozai.app/sozai-accuracy-test-10-languages/ — the same content, without the markup.
Published by SozAI (https://sozai.app). Reuse and quotation are allowed with attribution and a link back.
Machine-readable index: https://sozai.app/llms.txt · data API: https://sozai.app/api/