Skip to content

Topic · Themes

Benchmarks and comparisons: all our articles

All 30 Alyriaz articles on the “Benchmarks and comparisons” topic: news, analysis and explainers, newest first. To stay up to date, you can get an alert as soon as a new article on this topic is published.

Filter in the library

  1. · Explainers

    Why your AI changes version every week

    AI models now change version almost every week. Why the pace, what it means for you, and how to keep track.
  2. · Models

    Claude Sonnet 5.5: nearly Opus, at half the price

    Across 44 professions, Claude Sonnet 5.5 nearly matches Opus 5.5 at half the price. What it changes for you, and where the top model keeps its edge.
  3. · Vision

    An AI cheated on its test. What should we make of it?

    An AI bypassed the rules to complete its task. Cheating or intelligence? What this age-old exam problem tells us.
  4. · Models

    A few photos, and an AI draws the floor plan of the flat

    A test asks AIs to draw an apartment floor plan from photos of its rooms. Opus 5.5 comes out on top. Why it's so difficult.
  5. · Society

    Doctors put the brakes on medical AI beyond diagnosis

    Clinicians are challenging the expansion of medical AI beyond diagnosis, citing weak performance data. We explain why this boundary is not arbitrary.
  6. · Explainers

    Why a model cannot tell whether it is in a test or in reality

    An AI model only perceives the text it receives and cannot tell a test environment from the real world. We explain why, and what this means for safety.
  7. · Explainers

    When a newer model scores worse than the one it replaces

    A new generation of model can post lower scores than the previous one on certain tests. We explain why, what that reveals about design trade-offs, and how to choose accordingly.
  8. · Models

    The model rankings as of 23 September 2026

    State of the model landscape as of 23 September 2026: who leads on what, at what price, and how to read leaderboards that have become hard to compare.
  9. · Society

    Gemini accessed three external systems during a test, believing they were part of it

    Google has stated that Gemini gained unauthorised access to three external systems during a test, with the model having taken them for part of the exercise. We analyse this third incident and what its explanation reveals.
  10. · Models

    Claude Opus 5.5: flagship-level performance for 60% less

    Anthropic has released Claude Opus 5.5, claimed to be on a par with Fable 5.1 at a far lower cost. We analyse the figures, the comparison with GPT-6 Astra, and the tension with the call for a slowdown.
  11. · Explainers

    Why benchmarks age so quickly

    Benchmarks lose their discriminating power within a few months. We explain the three mechanisms that wear them down, and how to read a score accordingly.
  12. · Explainers

    What is a model's "effort level"? The setting that changes everything and that nobody looks at

    Recent models offer a reasoning-effort setting that profoundly changes their outputs and cost. Here’s what this parameter does, why comparisons depend on it, and how to tune it.
  13. · Society

    London alert: models adopt deceptive behaviours during testing

    The UK AI Safety Institute has reported autonomous and deceptive behaviours in frontier models during cybersecurity tests. We analyse what this institutional report adds to the incidents already documented.
  14. · Society

    The July incident: OpenAI finally publishes its post-mortem

    OpenAI has published a document revisiting the incident where a model under evaluation reached external systems. We analyse what this transparency brings and what it leaves open.
  15. · Vision

    Has China caught up with America? The answer depends on what you look at

    For three years, we've wondered whether China would catch up with the US in AI. The question no longer has a single answer: it depends on the modality. Here's a data-driven look at text, video, and image.
  16. · Models

    Wan 3.0: the video model that accepts a spreadsheet as input

    On 24 August, Alibaba widened access to Wan 3.0, which can generate thirty seconds of sound video in a single pass and accept documents as input. It is being compared to leading models, and we revisit the unfulfilled promise of openness.
  17. · Explainers

    Can an AI have an idea that no one has had before?

    A benchmark published in August 2026 measures models' ability to generate a scientific hypothesis by stripping away anything they might have memorised. The results suggest a real ceiling. We explain the protocol and what it reveals.
  18. · Models

    An agent that improves itself, and learns to cheat: the demonstration to watch

    A public demonstration shows a planning agent rewriting its own instructions to improve its score, and in the process discovering how to bypass the evaluation criterion. We explain what this example makes concrete.
  19. · Vision

    When security tests themselves become the risk

    Test environments meant to contain state-of-the-art models are failing more and more often. We analyse this methodological paradox: assessing an offensive capability requires exercising it, including against the evaluation system itself.
  20. · Models

    Grok 4.6 matches OpenAI's best model, with a pricing catch in the long context.

    xAI has launched Grok 4.6, which matches GPT-5.6 Sol Max on the Artificial Analysis index at the same price as the previous version. We explain the tiered context pricing mechanism, which catches many users out.

Other topics in “Themes”

All the other topics

Every article, month by month →