90 Days of an Onsite AI Shopping Assistant: The Whole Funnel, Including What Broke

Every vendor in this category publishes a conversion lift. Not one publishes the funnel underneath it. So here is all seven steps of a live onsite AI shopping assistant over 90 days, the money it cost to run, the errors it threw, and the three places our own measurement is wrong.

The deployment is a hard-luggage retailer in India running the assistant on its Shopify storefront. The numbers below were read from its live analytics dashboard on 24 September 2026 with the range set to 90 days. We are not naming the brand, because our own rule is that we do not name a customer who is not already on our case studies page.

We published an audit of every conversion claim in this category last week and ended it with five questions to put to any vendor. This post is us answering our own.

How this was measured

Window: 90 days ending 24 September 2026. Source: chat_events and chat_messages, the deployment's own event tables. Denominator: stated per line below. The funnel is normalised to sessions that sent a message, not to all visitors, and we say so at every step. Attribution window: none applied. An order counts when the assistant recorded it as settled. Comparison group: none. There was no holdout arm. Other changes live during the window: ordinary merchandising and promotional activity on the storefront, not controlled for. This is associative, not causal. Nothing below establishes that the assistant caused an order.

The funnel, all seven steps

Total user messages across the window: 4,045. Order values are the merchant's to disclose, so we publish the counts and not the revenue.

Two things to sit with.

The drop is at search to product. 55% of engaged sessions run a search and 20.9% look at a product. Just over half the shoppers who ask for something never open one of the things they were shown. That is a merchandising and card-design problem, not a model problem, and it is invisible unless you instrument every step.

7.3% is the add-to-cart rate, not a conversion rate. We flag this because it is exactly the row a headline gets taken from. If you see a number in this range quoted as "conversion" by anyone, including us, ask which step it came from. Here the order rate is 0.5%.

Where our own instrumentation is wrong

Three faults, named, because a dashboard you cannot audit is a brochure.

Chat opened reads 234 against 1,332 sessions that sent a message. The open event does not fire on every entry path, so the first row of the funnel is undercounted by roughly a factor of six. The dashboard normalises to the peak step rather than to sessions, which hides it.

Step ordering is not enforced. In the seven-day view, checkout-started (46) exceeded added-to-cart (33). Over 90 days the two are the right way round, but a funnel that can invert is a funnel with independent event streams rather than a state machine.

The voice numbers are not trustworthy. The voice view reports 1,228 calls with a total duration of 1 minute 3 seconds and one distinct browser. Both are impossible. It also reports paid orders of its own, and we are not publishing those as fact while the same view gets call length wrong by three orders of magnitude. Whether they overlap the six above cannot be determined from the dashboard, so we do not add them together.

If we were selling you this tool, the incentive would be to quote the voice orders and stay quiet about the call-length bug. The reason to do the opposite is that one of these two habits survives contact with a technical buyer.

What it cost to run

This is the number nobody in the category publishes, and it is the one we found most interesting.

Rs 171.61 is the entire model bill for 90 days of a storefront assistant that handled 4,045 shopper messages. We are not pairing it with a revenue figure to produce a return multiple: the orders are associative rather than caused, and the merchant's revenue is theirs to disclose, not ours. What the number establishes on its own is enough - model cost is not the constraint on this class of product. It was comfortably smaller than the payment processing on the same orders.

Most shopper turns never reach a model

The finding underneath that cost. Deterministic fast paths handled more turns than the LLM did:

Roughly three thousand turns were served by routing rules at zero model cost, against 1,115 that needed generation. Greetings, "show me more", browse-by-category, sale listings and checkout hand-offs are deterministic problems wearing conversational clothes.

That has a design consequence worth more than the cost saving: a fast path cannot hallucinate. Every turn you can route is a turn that cannot invent a product, and it is faster. Any vendor whose architecture sends all of it to a model is paying twice, in money and in risk.

The errors

134 errors across 116 sessions in 90 days. Of those, 0 were transient infrastructure, 0 were rate limits, 6 were a billing quota lapse, and 128 were our own bugs.

The largest single class, 30 occurrences and last seen on 23 September, is no final_text and no tool calls produced output - the model returning nothing at all. The next largest is a salvage path firing: empty LLM reply; salvage search served 5 products for 'something expandable for shopping'. The salvage behaviour is right, and the fact that it has to exist is the honest read on where this technology is in 2026.

"Something expandable for shopping" is also a perfectly reasonable thing for a human to say to a luggage shop.

What the query log was worth

We argued last week that an assistant does four jobs and gets bought for one. Job four was the queries your site search never records. Here is what that looks like in practice.

The head is what you would guess: show me cabin luggage 237 asks, more 230, show me medium luggage 151, show me your highest rated products 130, show me items on sale 104, show me cheaper options 67.

The tail is where the value is:

• Explicit budgets. "show me travel bags under 3000 rupees", "hey my budget is 3000 show me luggage under 3000", "Cabin trolley bag under 2000". Shoppers state a ceiling to an assistant that they never type into a search box.
• Features that may not be catalog fields. "do you have any luggage with tsa lock", "show me products which have tsa lock", "45 Litar capacity bags", "Back bag 35lites". Each is a request to filter on an attribute. If it is not structured in the catalog, the assistant cannot answer it and neither can ChatGPT or Gemini.
• Sizing language that is not yours. "Any size smaller than cabin size."
• Post-sale arriving in a pre-sale tool. "Any order tracking link", "Any tracking link", "Bag is locked". Shoppers do not know your tool's scope.
• Distribution and retail questions. "any distribution in kerala".
• Negotiation. "I want more discount".

Three of those categories are catalog work, one is a support-scope decision, and one is a pricing conversation. None of them appears in a search log. This is the output we would defend hardest if someone told us to cut the assistant tomorrow.

Language mix on the voice side, which we do trust because it is a count rather than a duration: 1,171 English, 54 Hinglish, 3 Hindi. Hinglish is not a rounding error.

What we will not claim

• Not a lift. No holdout, no matched cohort, no causal claim. Six settled orders in 90 days is what the assistant recorded, not what it can be shown to have added.
• Not a benchmark. One deployment, one category, one country. A luggage catalog with a few hundred SKUs is not a beauty catalog with three thousand.
• Not the voice figures. Not until call-length instrumentation is fixed.
• Not "AI shopping assistants convert at X%". This is one number from one store, and the honest use of it is as a floor to argue with, not a rate to plan against.

What happens next

The fixes this data demands, in order: make the chat-open event fire on every entry path so the funnel has a real first step; enforce step ordering; fix voice call-length capture; and start a forward holdout, withholding the assistant from a random share of sessions, because it is the only design that supports the word lift without a footnote.

If you are evaluating assistants, the useful thing to take from this is not the 0.5%. It is that you can ask any vendor for this exact table, all seven steps, the model bill, and the error classes, and watch what happens. We think that is a fairer test than any number on a homepage.

The agentic readiness scan checks whether your catalog can answer the constraint queries in that tail, which is the precondition for any of this working at all. And if the question is whether shoppers are reaching you in the first place, Citation Rank measures that side, with House of Zelena as the worked example.

Sources

• Live analytics dashboard for the deployment, read 24 September 2026, 90-day range. Underlying tables chat_events and chat_messages.
• Model cost is the dashboard's own estimate, stated as plus or minus 5%, against aistudio.google.com billing as ground truth. Currency converted at a flat Rs 84 to the dollar, as the dashboard does.
• Vendor comparison claims referenced here are quoted verbatim in the conversion-claims audit, captured 23 September 2026.