Milly Software
InsightsEngagementGaps and Weekly Recap: Two New Operational Pages for Your Shopify Chat
Engagement··6 min read

Gaps and Weekly Recap: Two New Operational Pages for Your Shopify Chat

Two new analytics pages: the Gaps page surfaces what the AI couldn't answer and tracks fixes; the Weekly Recap is a 7-day standup view of what shipped, what converted, and what didn't.

V
Viet Le
co-founder · Milly Software

Most chat-widget dashboards answer the same two flavors of question: how many? (sessions, messages, conversions) and what trend? (up vs last week, line chart over days). They rarely answer two more useful questions: what is the AI failing on? and what should I look at this week?

The Gaps and Weekly Recap pages exist to answer those two. Gaps surfaces unanswered queries with an explicit mark-as-addressed workflow. Weekly Recap is the 7-day standup — chat volume change, top queries, top gaps, leads, ATC, chat-assisted revenue — laid out as a Monday-morning read.

Gaps: what the AI couldn't answer

A "gap" is a customer query that came back with zero results from product search or knowledge search. The system already detects these — they fall into one of two categories:

  • zero_results — the AI ran a product search and got nothing back. Either the product doesn't exist in the catalog, or it exists but the search couldn't find it (synonym miss, missing tag, weak embedding).
  • no_kb_match — the AI looked in the knowledge base and found no match. Either the topic isn't covered, or the KB entry exists but the retrieval missed it.

The Gaps page lists these queries, grouped by query text and sorted by frequency. A query that ten different visitors asked with no result counts more than a one-off typo. Greetings, follow-ups, and off-topic queries are filtered out — they don't represent product or KB gaps.

Mark-as-addressed: the workflow

Surfacing gaps is half the value. The other half is closing the loop — the merchant adds a KB entry, updates a product tag, fixes a synonym, and the gap stops being a gap. To keep the page useful over time, gaps need a way to fall off the list once they're fixed.

Each gap row has a Mark Addressed action that stamps a gap_addressed_at timestamp. Future fetches filter addressed gaps out by default. A toggle on the page lets the merchant view addressed gaps if they want history — for example, to remind themselves what they've already worked on this month — but the default view is just the open ones.

The implication for merchant workflow: the Gaps page is a backlog. New gaps appear as the AI runs into things it can't answer; addressed gaps disappear as the merchant fills the holes. A merchant doing weekly KB maintenance can use it as their work queue.

Weekly Recap: the Monday-morning view

Weekly Recap is a single page that loads once and shows the week:

  • Chat volume this week vs last week, with the percentage change. The headline number — "up 24% this week" — sets the frame for everything else.
  • Top 10 queries the AI handled this week, excluding greetings and off-topic. Indicates what visitors are actually asking about — surfaces seasonal shifts (sale weeks see more "promo" queries; product launches spike new-product searches).
  • Top 5 gaps from this week specifically. Same gap definition as the Gaps page, scoped to the last 7 days. The merchant sees the freshest backlog without opening a separate tab.
  • Leads captured, products added to cart, chat-assisted revenue for the week. The conversion side of the chat funnel.
  • Thumbs up / down ratings on AI responses. A blunt quality signal that pairs with gaps — gaps mean the AI didn't answer; thumbs-down means it answered and the answer was wrong or unhelpful.

The page isn't deeply customizable. It's a deliberately fixed shape — what changed, what they asked, what we missed, what converted. Designed to be readable in 90 seconds and useful in a Monday standup.

Both as the operational layer

The other analytics pages — Overview, Speed, Queries, Zero Results, Impact — are diagnostic and decision-support tools. Open them when you have a specific question. Gaps and Weekly Recap are different: they're the routine pages, the ones you check on schedule because something always changes.

That distinction shapes how they should be designed. Diagnostic pages can be dense, drill-down-heavy, full of filters. Operational pages need to load fast, show a fixed shape, and have one or two obvious actions per row. Mark Addressed on Gaps; click-through-to-replay on Weekly Recap items. The merchant should be able to use them without thinking about how to use them.

Together they round out a multi-page analytics layout that spans the diagnostic and operational ends of the spectrum. Speed answers "why was the widget slow yesterday?" Queries answers "what are people searching for?" Impact answers "is the widget worth it?" And Gaps + Weekly Recap answer "what should I do this week to keep the AI getting better?" — the question the rest of the dashboard isn't built for.

Try Milly Chat

Want to see how this fits your store?
We'll set up a working session.

Get it on ShopifyTalk to sales →