08 - Analyst & PM playbook

Daily / weekly rhythms

Role	Cadence	Actions
Analyst	Weekly	Scenario funnel (impressions, views-as-clicks, carts, purchases); investigate CTR>100% or 0 impressions
Analyst	Ad hoc	ClickHouse recipes in 10; compare slug vs empty-slug volume
PM	When editing	Change one lever at a time; note slug + algorithm; QA 3–5 seeds in UI
Both	After train/reindex	Spot-check ML / vector scenarios for cold vs heavy users

---

“Expected item missing” checklist

1. Available? Catalog available / stock flags.
2. Filtered by nextQL? Temporarily loosen filter.
3. Diversity / purchase cap? Check category pile-up and user’s purchase history.
4. Score threshold (vector)? Too high → empty neighbors → category/min fill.
5. Wrong logic/algorithm? Scenario slug on the placement.
6. Seed quality? Bad/missing embedding for seed.
7. Timing? Just indexed? Wait for Qdrant/CH catch-up.

---

Interpreting metrics safely

Observation	Likely cause	Action
CTR ≫ 100%	Views with slug ≫ impressions	Audit host trackView / attribution (09)
Impressions ≫ 0, clicks ≈ 0	No slug on views; or users don’t open PDPs	Check SDK click/view tagging
Carts/purchases tiny	Funnel real or events not sent	Verify trackCart / trackPurchase
Rail looks generic	Cold start / fallback / missing factors	Check algorithm + training; identity merge
Sudden empty rails	Filter too strict / threshold / outage	Admin preview + engine logs / Qdrant health; consider autoRelaxFilters + clause order (durable left, soft right) — see 04 Query planning

Remember: Admin “clicks” = type=view + slug.

---

Scenario edit recipes

Goal	Lever
Hide OOS / wrong region	nextQL filter
Prefer featured / margin SKUs	booster (soft)
Stop one category dominating	maxPerCategory
Show newer catalog	freshnessBoostDays
Discoverability for obscure SKUs	injectLongTail + percentage
Fewer repurchase suggestions	purchasedItemsMaxPercent + pass userId
Stricter similar	raise score_threshold
Richer cart rail	multi-seed-hybrid + adaptive

Change one of {filter, boosters, constraints, algorithm, qdrant} per experiment when possible.

---

Experiments (high level)

If the tenant uses Admin experiments / variants:

• Keep scenario_slug consistent on impressions and attributed interactions.
• Don’t change instrumentation mid-test without splitting analysis windows.
• Primary metrics: impression volume, attributed view rate (once instrumentation is correct), cart/purchase rates.

---

Geo ranking v1 — sending country

Clients send optional ISO-2 country on:

• Ingest: POST /ingestion/interactions and POST /ingestion/impressions (same field on each event).
• Recommend: POST /recommendations body; legacy GET query (user-to-item, item-to-item, similar-products, recently-viewed); POST /recommendations/items-to-items (body wins, else query).
• SDKs: optional country on track + recommend; JS setCountry() / ctor default applies to both.

Missing, invalid, or OTH → global. Recommend country does not backfill ingest tags.

7-day warmup: fill and lift read 7d tagged interactions. Historical rows stay country='' (global only). Do not expect IN lift in QA until IN-tagged events exist.

Do not use a nextQL country filter as a substitute (e.g. 'allowed_countries' contains 'IN'). That removes SKUs that are sellable but untagged / not in a catalog geo attr. Geo v1 reweights candidates; it does not change filter support. Use nextQL country filters only for hard catalog policy, not to “turn on” regional ranking.

QA checklist

1. Ingest a few interactions with country=IN; wait until they are in ClickHouse.
2. Recommend with country=IN vs omit / OTH (expect _global_ when omitted/OTH).
3. weighted-history: personal order unchanged; only fill IDs/order may change.
4. i2i / similar: an item weak in IN can rank lower (lift < 1).
5. recently-viewed: country accepted, rank unchanged.
6. i2is body IN + query DE → effective IN.
7. Do not treat a large lift on a rare SKU as a bug (no cap in v1).