The agent shopping your store is not a superhuman buyer

Every week someone tells me to optimize my store for AI agents. The advice is always structural: add schema, publish an llms.txt, keep your markup clean. Useful, and beside the point.

The question nobody answers is simpler. When an agent lands on your product page, how good is it actually?

We now have numbers.

It fails where the shopping gets real

On 10 August a benchmark called ComboShoppingBench was published. It does not test whether an agent can find one product. It tests what people actually do: build a basket of items that work together, inside a budget, with coupons that may or may not combine.

The numbers, in short:

• 61.2 percent success for the strongest agent tested, GPT-5.5 with extended thinking enabled
• 291 distinct shopping goals, from setting up a phone to arranging a group meal
• 22 agent configurations across eleven model families, each run with and without extended thinking
• 41 percent of the time agents fall for a single classic dark pattern, measured separately

Read the first number again with your own store in mind.

The best available shopping agent gets roughly four out of ten realistic baskets wrong. Not obscure edge cases. Setting up a new phone. Arranging a group meal. The failures cluster exactly where commerce gets complicated: compatibility between items, coupon rules, budget limits.

It falls for the same tricks your customers do

There is a comfortable assumption behind most agent-readiness advice: that an agent reads your page coldly, immune to the persuasion and the pressure that work on humans.

That assumption has been measured, and it is wrong.

Researchers built a controlled store environment called TrickyArena and ran six popular web agents across three different models against classic dark patterns, using an instrumentation framework they call LiteAgent. With a single dark pattern present on the page, agents fell for it 41 percent of the time on average. The work is accepted at IEEE Security and Privacy 2026.

An agent is not a crawler with a credit card. It is a distractible reader that happens to be fast.

We have watched this happen to our own agent

One number in this post has a version I can describe from our side rather than from a paper.

Earlier this year a BuyerEyes report told a client their site was missing its tax registration number. It was not. The number sat in the footer, in plain view, the whole time. Our agent had not invented anything. It was working from viewport screenshots on a mobile-sized window and never scrolled to the bottom of the page, so it judged the page from the slice it could see and reported that slice as the whole thing.

We fixed it by feeding the agent the full page text and the footer as ground truth, and by adding a hard check that throws out any claim about something being absent when the thing is demonstrably there.

The lesson generalises past our own bug. An agent's confidence tells you nothing about how much of your page it actually saw.

What this actually means for your store

Put those two findings next to each other and the advice changes shape.

If the agent's weak point were parsing, then schema and clean markup would fix it. But the agent is not failing to parse your page. It is failing to reason about it under constraints, and it is being nudged by the same layout decisions that nudge people.

That makes agent readiness a comprehension problem, not a markup problem. The question is not whether a machine can read your page. It is whether anything reading your page, human or otherwise, can answer: what is this, does it fit what I need, what does it cost me in total, and what happens if I am wrong.

A page that answers those four questions plainly does well with both audiences. A page that hides the total until checkout, buries compatibility in a spec table, or leans on urgency to close, is exploiting a weakness that agents share with your customers. You may currently be reading that as a conversion win.

The honest limits of this evidence

Both studies ran in simulated environments, not on live stores. The benchmark's basket tasks are synthesized, not scraped from real orders. The dark pattern number is for one pattern at a time; the paper notes that combining patterns changes agent behavior, without giving a headline figure for it.

There is also a gap worth naming. Writing this in mid-August 2026, I went looking for measured work on two questions that matter more to store owners than anything above: how far an agent's view of a page diverges from a human's, and whether an agent's stated reason for choosing a product matches the real one. There is nothing that clears a reasonable evidence bar on either. Plenty of blog posts. No numbers. Our own footer incident is a case of the first question, not a measurement of it, and I am not going to pretend otherwise.

So treat the two findings as what they are: the first hard measurements of a thing that has been discussed mostly on vibes.

Where I would start

Not with schema. Start by asking what your product page looks like to a reader with no patience, no memory of your brand, and a constraint to satisfy. Then check whether your page's most persuasive moment is also its least honest one.

That is the same audit either way. The agent just made it measurable.