Claims such as ‘human-like’, ‘most accurate’ and ‘sub-100-millisecond’ describe different properties. A credible buyer’s guide must separate vendor documentation from a reproducible listening or recognition experiment.
Quality must be task-specific
A documentary TTS voice is judged for sustained character, clarity, pronunciation and emotional pacing. A call-center prompt is judged for intelligible numbers, first-audio delay and consistency across interruptions. Music generation needs harmonic structure, usability in a mix, lyrical control and rights evidence. A single star rating across these unrelated tasks hides the real failure mode. Describe the intended job first.
Build the reference set
Use balanced text passages containing names, quantities, abbreviations, quoted dialogue and language switches; get approval to use any recorded human reference. For ASR, make expert-reviewed transcripts for noisy and clean recordings, with a declared policy on disfluencies, punctuation and overlapping voices. For diarization, label reference speaker turns without publishing people’s private identities. Split development and holdout audio so vendor tuning cannot contaminate the final test.
Measure the right thing
For recognition, publish word or character error rate together with language and accent slices. For diarization, calculate an explicitly specified speaker-turn error metric. For TTS, use blind ratings with multiple human listeners and separate pronunciation checks. For real-time systems, report median and 95th-percentile end-to-end latency and interruption recovery, not model inference alone. Never compare results obtained on different source sets as if they were controlled.
Track versions and spend
Log exact model IDs, immutable snapshot where available, date, endpoint region, codec, sample rate and API settings. Cost should count re-renders, failed requests, human correction hours, download allocations and storage. Per-character TTS, per-audio-minute ASR and token-priced interactive agents are different accounting units. Refer price claims to AI 10 with a timestamp and region.
Make non-testing explicit
This article provides a protocol, not an evaluation result. No provider audio was synthesized and no independent listener panel was run for this archive. Provider claims are listed as first-party claims, not established comparative conclusions. Publishing an honest ‘not tested’ alongside a useful procurement framework is more valuable than a fabricated benchmark or claimed screenshot.
Worked production example
A defensible vendor test might use sixty licensed recordings, divided equally among six languages, with reference transcripts double-checked by independent editors. A recognition provider is evaluated on names, dates, disfluencies and noisy rooms; a TTS provider gets matched prompts delivered in randomized blind order; a voice agent is tested over a simulated poor mobile connection. None of those scores should be mixed into one rating. Publish the dataset definition and uncertainty rather than cherry-picking the only language in which a model performed well.
Practical acceptance gate
An operations reviewer should calculate accepted output per dollar and editor-hour, not just list credits or milliseconds from product pages. Model versions, configuration, language and region must be frozen for a comparative round. Remove personal information from shared test corpora and ensure test contributors consented. If these tests were not performed, present only the testing procedure. This package intentionally contains no claimed empirical score, because inventing one would make the directory unreliable.
Region, plan and model-version notes
Documentation checked on 2026-10-25. Exact applicability: availability varies by account/endpoint and jurisdiction. Version: methodology. Plan: not independently verified for a specific plan. Monetary prices and quotas are not asserted unless separately verified; check the canonical AI 10 pricing ledger before purchase or publication of a tariff.
Sources & Further Reading
- Eleven Tts — official documentation (checked 2026-10-25)
- Assembly — official documentation (checked 2026-10-25)
- Cart Sonic — official documentation (checked 2026-10-25)
- Deep Nova — official documentation (checked 2026-10-25)