How to choose an AI chatbot for Shopify without getting sold to

First, decide which job you're hiring for

'AI chatbot' covers at least three different products, and buying the wrong category is the most expensive mistake available. A support deflection tool optimises for closing tickets. A live-chat inbox optimises for routing humans. A shopping assistant optimises for helping someone buy.

They overlap, but their defaults conflict. A deflection tool measures success by conversations that ended without a human — which is also what happens when a shopper gives up. Know which outcome you're paying for before you compare features.

Grounding: the question that separates real tools from wrappers

Any tool can put a language model behind a chat bubble. The difference is what it knows. Ask specifically how answers are grounded in your data, and be suspicious of vague answers.

• Does it read live catalogue data, or a snapshot from onboarding? Prices and stock change daily.
• How fast do product edits propagate — webhooks in seconds, or a nightly re-crawl?
• Can it cite which product or policy page an answer came from?
• What does it do when it doesn't know? 'Makes something plausible up' is the wrong answer and a common one.

Order access and identity

Order tracking is the single highest-volume support question in ecommerce, so it's worth probing carefully. Can the assistant look up real fulfilment status, and how does it verify that the person asking owns the order? A tool that reads out order details to anyone who types an order number is a data-leak waiting for its first complaint.

Attribution you'd be willing to defend

Ask how the tool proves it made money, then listen for whether the answer involves real orders. Estimated influence, deflection savings, and 'conversations that showed purchase intent' are all ways of not answering.

What you want is a chain from conversation to session to completed Shopify order, with the amount taken from the order total. If a vendor can't explain that chain in a sentence, assume the number in their dashboard is generous.

Understand the pricing model, not just the price

Chat tools meter on wildly different units, and the unit matters more than the headline figure. Compare on your own traffic, not on the tier names.

• Per resolution or per conversation: predictable, but check what counts as one — a session, or every message.
• Per seat: fine for a human inbox, irrelevant for automation.
• Per message or per AI credit: hardest to forecast, and the model that surprises people in month three.
• Traffic-banded: sized by monthly visitors, so cost tracks the store rather than the conversation volume.

The five demo questions that expose a weak tool

Insist on testing against your own catalogue, not a sandbox store. Then ask these:

• A question about a product you discontinued last month.
• A price question about a variant whose price you changed this week.
• 'Can I return this if I've opened it?' — checks policy grounding, not product data.
• Something in the second language your customers actually use.
• A request for a discount you don't offer, to see whether it invents one.

Boring things that matter in month six

Ask where conversation data is stored and for how long, whether it's used to train models, and what happens to it when you uninstall. Ask how the widget affects page weight. Ask what the escalation path to a human looks like on a Sunday. None of this demos well, and all of it determines whether you still like the tool next quarter.