Arabic voice AI benchmark 2026: speech-to-text, voices and latency, measured on Lebanese Arabic

Hephon is an AI agency in Dubai (Hephon L.L.C-FZ) that builds AI voice agents and chat assistants for businesses in the UAE, Saudi Arabia and Lebanon. Before choosing the parts of our own voice engine in October 2026, we measured them: which engines transcribe Lebanese Arabic best, which voices a native listener hears as Lebanese, and how long each part takes to respond. This page publishes those measurements, how we took them, and what they do not show.

One limit up front. Everything here was measured on Lebanese Arabic, the dialect of our first voice deployment. Gulf dialects (Emirati, Saudi Najdi and Hijazi, Kuwaiti) have different vocabulary and sounds, and we have not run the same tests on them yet. Do not read these rankings as Gulf results. We will publish a Gulf round when we have run it the same way.

How did we measure?

• Dates: speech-to-text batch test on 1 October 2026; streaming speech-to-text, voice latency, model latency and two blind listening rounds on 23 October 2026; natural voice notes on 23 and 24 October 2026.
• Where from: a machine in Lebanon calling each vendor’s public endpoint. Absolute latencies include that network path and will differ from a server in another region, so compare engines against each other rather than reading the numbers as universal.
• Speech-to-text sample: 60 Lebanese Arabic clips (11.2 minutes in total) with human-made reference transcripts, scored by word error rate (WER) and character error rate (CER). A second sample of 25 natural, unscripted voice notes from one native speaker.
• Voice sample: 7 items (4 read-aloud lines and 3 caller replies) rendered by each voice and rated by one native Lebanese listener, the author, for how Lebanese and how natural each clip sounded, on a 1 to 5 scale.
• Latency: time to first audio for voices (6 runs each, warm connection), time to first token and first clause for language models (3 caller lines, 9 runs each, minimal reasoning).

Which speech-to-text engine is most accurate on Lebanese Arabic?

In the 1 October batch test on the 60 clips, OpenAI’s gpt-transcribe had the lowest word error rate. Its lead over ElevenLabs Scribe v2 held in 99% of bootstrap resamples, so it is not noise, but the whole field sits within nine points.

English leaderboards rank several of these engines differently. On this dialect the order flips, which is the main reason to test on your own customers’ speech rather than trusting a general ranking.

How fast is live (streaming) transcription?

A phone agent cannot wait for a whole recording, so on 23 October we replayed the same 60 clips in real time to each engine’s streaming endpoint and timed how long after the end of speech the final text arrived.

On clean clips, Soniox looked like the clear pick: within two points of the most accurate engine and four to five times faster to final text than OpenAI’s realtime transcription. Deepgram’s Lebanese-specific model was the least accurate here.

Does the ranking hold on natural speech?

No, and this is the most useful finding on the page. The 60 clips are clean recordings. People calling a business do not speak like that. So we ran 25 natural voice notes from one native speaker through the same engines.

Soniox went from 28.9% on read-aloud clips to 38.9% on natural speech, while gpt-live-transcribe held at 27.0%. A caveat on this table: the reference transcripts were built by picking and correcting outputs from four of the engines (gpt-transcribe, gemini-3.8-flash, Scribe and Soniox), so exact matches lean towards those four. The comparison between engines is still fair; the absolute numbers are less certain than in the 60-clip test.

One failure every engine shared: Lebanese words get rewritten into formal Arabic in the transcript (for example الدجيج becomes الدجاج, and فيني becomes فينا). A voice agent should never read the caller’s words back exactly as transcribed, or it will correct their dialect to their face.

Which voices sound Lebanese to a native listener?

We rendered the same 7 items with each voice and played them blind to one native Lebanese listener (the author). Each clip got two scores out of 5: how Lebanese it sounded and how natural. Round 1 had 49 clips. Round 2 had 63 clips and added two voices from Hamsa, a Lebanese-focused vendor, including one we knew to be Syrian as a dialect control: a listener who rates the Syrian voice as Lebanese is not telling the dialects apart.

Round 1 (49 clips, blind)

Round 2 (63 clips, same listener, not fully blind)

Round 2 was not fully blind because the listener had seen round 1’s results. The control still behaved: the Syrian voice scored about a point below the Lebanese voice from the same vendor. Gemini 3.8 Flash TTS was the top pick on 4 of the 7 items and scored 5 on six of them. In round 1 its clips sometimes read the English style instruction aloud; separating the directions from the text to be spoken fixed that.

Two lessons from the ratings. Newer was not better: Gemini 3.8 Live scored below the older 3.1 Flash Live in both rounds. And a voice can sound natural without sounding Lebanese: ElevenLabs’ Rami scored 4.14 for natural and 2.0 for Lebanese in round 2.

What the follow-up voice lab found

• With Gemini 3.8 Flash TTS the accent came from the model and the prompt, not from the voice: 23 of its 30 voices passed (12 of 16 male, 11 of 14 female).
• An instruction about the accent mattered. A prompt describing a persona (a young host from a Beirut neighbourhood) and a prompt listing Beiruti pronunciation cues both beat the round 2 prompt, and the control with no accent instruction failed.
• ElevenLabs’ Jasmine passed on v4 and v4 Turbo and failed on v3, Multilingual v2 and Flash 2.5. Of the male ElevenLabs voices tried, only one shared-library voice (Dr. Munir Lawand) passed.
• ElevenLabs renders varied: the same voice with identical settings passed in one batch and not in another.

How fast does each voice start speaking?

Time to first audio decides how long a caller hears silence after the agent has decided what to say. Six runs per voice on a warm connection, for a short and a long line:

This is the trade-off in one table: the voice our listener rated most Lebanese costs about 0.6 seconds more per reply than ElevenLabs v4 Turbo. Pre-recorded openers in the same voice and speaking clause by clause hide part of that gap.

How fast do the language models answer?

Three caller lines, nine runs each, with reasoning set to the minimum. First token is when text starts arriving; first clause is when there is enough to start speaking.

What about speech-to-speech models?

Speech-to-speech models (one model that hears and speaks, with no separate transcription or voice step) were faster end to end in our tests, around 1.6 seconds. But on 24 October they failed our listener’s ear on 3 of 4 test lines, even with an accent instruction in the prompt. For Lebanese callers we kept the three-part pipeline (listener, model, voice) and worked on making it fast instead.

What did we choose, and how fast is it end to end?

For live calls in Lebanese Arabic we chose gpt-live-transcribe to listen (close to the best character error rate on natural speech, about one second to final text, with partial text during speech), gemini-3.1-flash-lite to answer, and Gemini 3.8 Flash TTS to speak. On synthetic test calls heard from Lebanon, the full turn took about 1.9 to 2.7 seconds. That is slower than the best speech-to-speech systems and it sounds like a Lebanese person, which for our callers mattered more.

If you are choosing a voice stack for Arabic callers, three things from this work carry over to any dialect: test speech-to-text on natural speech from your own callers, not on read-aloud clips; have a native speaker of that dialect rate the voices blind, with a control voice from a neighbouring dialect; and measure first-audio time from the region your agent will run in. Our AI voice agent page explains how we build these, with market pages for Lebanon, the UAE and Saudi Arabia.