A product claiming support for Arabic or Turkish can still perform poorly on the accents, locations and code-switching patterns that actually matter to a customer.
Define the target population before the model
For Turkish, an Istanbul newscaster recording and a mixed Turkish-German factory-floor call may be very different inputs. For Arabic, Modern Standard Arabic alone is not a proxy for Egyptian, Gulf, Levantine or Maghrebi dialects. Build evaluation sets by dialect, age-appropriate recording context, speaker gender where consented, microphone and noise. Use explicit locale codes when a provider requires them; never assume an “Arabic supported” badge is a quality guarantee for every dialect.
Create an ethically collected reference set
Recruit participants with informed consent for evaluation and any storage or transfer. Obtain double-annotated transcripts and an adjudication procedure for proper names, script transliteration, numerals and common borrowings. Hold out a blind test set. Mark difficult segments such as overlapped voices, clipped syllables and language switching rather than silently deleting them. Do not use a purchased stock recording as evidence of model consent for speaker cloning or biometric identification.
Measure the right error types
Compute word error rate using a stated normalization protocol and report character error rate when relevant, but also review critical semantic errors. Missing negation or an incorrect price can be more damaging than punctuation. Compare code-switched phrases and terminology separately. Report confidence intervals for meaningful sample sizes, language mix, hardware, endpoint version, account region and date. There are no measured vendor scores in this article; it is a protocol, not a ranking.
Account for product version and geography
Microsoft and Tencent publish capability tables, but regional eligibility and language behavior can differ by product path. A batch recognizer may expose a language that is unavailable or differently priced in realtime. Keep a matrix of endpoint, language code, channel count, codec, tariff and data-processing geography. Publish “not confirmed” when a critical item cannot be documented. Recheck entitlements before rollout, rather than promoting all catalog languages to production-ready.
Turn an experiment into an operational gate
Define maximum human correction minutes per finished transcription hour, a tolerable critical-term error rate and a callback procedure for ambiguous phrases. Have native listeners evaluate both final text and any simultaneous captions. Choose the service that meets the measured business requirements and the organization’s security obligations, not the one with the largest marketing language count.
Evidence, availability and commercial-rights boundary
Research cutoff: 25 October 2026. Provider documentation is evidence of its published interface or policy, not independent product testing. No performance score, hands-on trial, customer consent, territory-wide entitlement or legal clearance is claimed. Confirm the exact version, selected region, voice rights, data flow, account limits and current service terms before deployment. Unknown model release dates, account prices and plan limits are deliberately left unasserted; monetary claims belong to the separately dated AI10 pricing ledger. Where documentation is undated or dynamic, the public page is not treated as an archival snapshot of every earlier tier.