A defensible evaluation starts with the image task, not the name of the tool. If no controlled prompts, saved images or human annotations exist, a feature table is documentation-based discovery—not a hands-on test or a ranking.
Construct a use-case-balanced corpus
Build tasks for product photos, characters, exact copy, multilingual posters, inpainting, wide outpainting and low-quality-image enlargement. Include high-contrast and low-contrast scenes, difficult hands, fine product detail and licensed reference images. Store the original prompt, original image when applicable, intended target and a human-reviewed ground truth checklist. Never use hidden copyrighted example portfolios without permission.
Freeze operating conditions
Record the actual product build, model snapshot, region, account tier, resolution, seed when supported, number of runs and all reference settings. A product’s marketing model family is not a stable comparison endpoint. Measure elapsed job time and costs from actual logs, not from a launch benchmark. The same output count and retry policy must apply to all evaluated vendors or those comparisons cannot be called fair.
Blind review and categories
Randomize image order and hide vendor names. Ask multiple reviewers to rate composition, adherence, reference preservation, text exactness, visible artifacts and potentially misleading alterations separately. Include a failure flag for wrong product facts or added logos. Use inter-rater disagreement and confidence intervals when sample sizes justify them. Keep company self-reported human Elo charts labeled as vendor claims unless you reproduce the actual protocol.
Publish only supported conclusions
Separate documented operation support from observed success rates. A matrix entry may say “supports outpainting per official page” while also truthfully saying “not tested.” Do not compute an overall winner score by averaging noncomparable use cases or imputing unknowns as zero. Pass objective price input and rate units to AI 10 and cross-provider recommendations to AI 09; this article defines the experimental method only.
Worked scenario and failure test
A design team comparing three providers should select ten representative tasks before accessing the apps, not only cherry-pick a vendor’s finest demo. For each task, log the exact model, prompt, reference permission, number of generations and rejected images. Randomize final candidate order; two reviewers should independently mark text fidelity, subject continuity, image defects and misleading content. Publish disagreement and missing-data flags. If a product is inaccessible in a particular region, record ‘not evaluated’ instead of inventing a quality score. Supported feature documentation alone is not a substitute for saved output evidence.
Decision guide and practical handoff
No test corpus or original outputs were produced in this run; comparison flags remain not_tested=true. A responsible review packet names the exact source or model version, the permitted input assets, the requested operation and the image characteristics a reviewer must check. It should also contain an unaltered reference, the accepted derivative, and a short record of what changed. A supported feature is not necessarily a reliable one for every brief, and the team must approve both technical appearance and real-world truthfulness before distribution.
Availability, price and geographic limitations
Research cutoff: 25 October 2026. Release dates and historical tier details above are explicitly dated. Unless independently confirmed for a named country and account tier, current availability, billing currency, credit consumption, API quota and commercial entitlement are unknown—not universal. Consult official regional terms and AI 10’s dated price snapshots before procurement; do not treat a historical announcement as a live entitlement. No application, model, API response or image output was hands-on tested for this article.
Sources & Further Reading
- Ideogram 3.0 release — published/updated 2025-03-26 (consulted at research cutoff 2026-10-25).
- FLUX.1 Kontext announcement — published/updated 2025-05-29 (consulted at research cutoff 2026-10-25).
- Recraft V3 launch — published/updated 2024-10-30 (consulted at research cutoff 2026-10-25).
- Seedream 4.0 release — published/updated 2025-09-09 (consulted at research cutoff 2026-10-25).