Unified technical and operational view of the WaveSearch + WaveStore platform.
AKS wave-dev ARM in-cluster builds STS audience + scope authEach host is an ingress entry point into a specific frontend/API service chain. Shopper and operator traffic share the same ingress layer but route to different service surfaces and scopes.
| Public host | Primary service | Downstream dependencies |
|---|---|---|
| store.retail.demos.wavefunctionlabs.com | wavestore-frontend | wavesearch-api, wavestore-erp-api, wave-sts |
| erp.retail.demos.wavefunctionlabs.com | wavestore-erp-frontend | wavestore-erp-api, wave-sts |
| labs.retail.demos.wavefunctionlabs.com | wavesearch-frontend | wavesearch-api, wavestore-erp-api, wave-sts |
| sts.retail.demos.wavefunctionlabs.com | wave-sts | user/audience/scope admin UI + token issuance/validation |
| search-api.retail.demos.wavefunctionlabs.com | wavesearch-api | runtime index, append logs, wave-sts |
| erp-api.retail.demos.wavefunctionlabs.com | wavestore-erp-api | ERP state store, wave-sts |
Call relationships between shopper/admin surfaces, core services, and the optional Azure Foundry AI harness used for reranking and intent classification.
| Layer | What is used | How it fits |
|---|---|---|
| UI surfaces | FastAPI + server-rendered HTML + Bootstrap Wave styling | Storefront, ERP admin, Search admin, and this unified doc surface. |
| Search runtime | CatalogRuntime + postings + term-frequency + doc length + variant links | Serves low-latency candidate retrieval and ranking directly from in-memory structures. |
| Ranking model | BM25-style term scoring + merchandising/inventory overlays | Converts lexical relevance into final ordered results; supports server-side filter constraints. |
| State + telemetry | Catalog snapshots, append logs (events/rules/inventory), analytics overlays | Supports replay/debug and relevance tuning loops using behavior and operator actions. |
| Platform | AKS + shared ingress + ACR | Single routed platform with multi-service hostnames and ARM-compatible deployments. |
The search platform now supports an optional second-stage AI pass after lexical retrieval. It keeps the fast server-side candidate retrieval/filtering path, then uses Azure Foundry to improve precision.
| Capability | How it works | Response visibility |
|---|---|---|
| Intent classification | Classifies query intent and proposes inferred filters from the request query/filters payload. | Returned under ai.intent in /search/query responses. |
| LLM reranking | Reranks top-N lexical candidates for semantic relevance while preserving backend constraints and filter boundaries. | Returned under ai.rerank; reordered results in the same query response. |
| Safety fallback | If the model call fails or times out, lexical ranking is preserved and the error is surfaced in metadata. | ai.rerank.error / ai.intent.error (when applicable). |
Foundry configuration (environment):
Exact prompts/templates used (verbatim from wavesearch_api/app.py):
| Call | System prompt | User payload shape |
|---|---|---|
| Intent classification | "You classify retail search intent. Return strict JSON with keys: intent (string), confidence (0-1), inferredFilters (object), notes (string)." | {"query": <search text>, "filters": <active facet filters>} |
| LLM reranking | "You rerank retail candidates for user relevance. Return strict JSON with keys: rerankedIndexes (array of integers, unique, subset/permutation of provided indexes), rationale (string)." | {"query": <search text>, "candidates": [{"index","id","title","description","categories","brands","tags","availability","price"}, ...]} |
Both calls request response_format: json_object (with a plain-text fallback if the model rejects that parameter), run concurrently via asyncio.gather, and are capped at RETAIL_V2_LLM_TIMEOUT_SECONDS each; reranking only sends the top RETAIL_V2_LLM_RERANK_TOP_N candidates to bound prompt size and cost.
Every /search/query call is recorded server-side with its lexical (pre-AI/BM25) result order and its final (post-AI) result order, so operators can inspect exactly what reranking and intent classification changed for any real search.
| Endpoint | Purpose | Where visible in Search Admin GUI |
|---|---|---|
| GET /search/admin/diagnostics | Lists recent searches per tenant (query, result count, whether rerank/intent applied, whether order changed). | Search Explorer paged table. |
| GET /search/admin/diagnostics/{id} | Returns full pre-AI and post-AI result lists plus rerank rationale and intent classification detail for one search. | Search diagnostic comparison modal (side-by-side before/after tables with rank deltas). |
Diagnostics are held in an in-memory ring buffer per tenant (default 50 entries, configurable via RETAIL_V2_SEARCH_DIAGNOSTICS_MAX) so recent activity is inspectable without impacting query latency.
Pure lexical/BM25 search can only find products that share at least one token with the query — a query like "something to keep me warm on a freezing winter hike" shares zero words with a product titled "Aurora Storm Shell Jacket" and would previously return almost nothing. Hybrid search adds a second, independent retrieval path over vector embeddings and fuses the two rankings, so semantically related products surface even with no keyword overlap.
| Stage | Detail |
|---|---|
| Embedding model | text-embedding-3-small, deployed on the same Azure OpenAI/Foundry resource as the gpt-5-nano rerank model (env RETAIL_V2_EMBEDDING_DEPLOYMENT). |
| Index build | Every product's title/description/categories/brands/tags text is embedded once and cached in-memory as a single L2-normalized numpy matrix (vector_matrix, shape N×D, plus a parallel vector_ids list), rebuilt automatically on every catalog ingest while the toggle is on, or on-demand the moment the toggle is switched on if the index is empty. |
| Query time | The query text is embedded (one call), L2-normalized, and compared against every indexed product vector via a single vectorized matrix-vector multiply (numpy/BLAS) rather than a Python loop — comfortably sub-100ms up to roughly 100k-1M vectors on one core. Still "brute-force" in that every vector participates (no ANN/graph index skips comparisons); results respect the same category/brand/tag/availability/price/stock filters as the lexical path. |
| Fusion | Reciprocal Rank Fusion (RRF) combines the lexical rank and vector rank by position, not raw score (BM25 and cosine similarity are on incomparable scales): score = 1/(k + lexical_rank) + 1/(k + vector_rank), k configurable via RETAIL_V2_RRF_K (default 60, the standard literature value). Existing LLM reranking, if also enabled, then runs on top of the fused order exactly as before. |
Off by default (env RETAIL_V2_VECTOR_ENABLED), toggled live per the same pattern as LLM rerank/intent via POST /search/admin/ai-toggle (vectorEnabled) from the AI harness panel in Search Admin, with index size/availability visible in GET /search/admin/config. POST /search/admin/vector-index/rebuild forces a manual rebuild.
Verified live: querying "something to keep me warm on a freezing winter hike" (2 lexical BM25 candidates only) returned Aurora Storm Shell Jacket plus several hiking boots pulled in purely by vector similarity (flagged vectorOnly: true in the response) — at ~270-320ms per query, dominated by the single query-embedding call, well under the ~1-1.8s LLM rerank path.
This demo's catalog is ~100 products. A real retail customer can have millions of SKUs and variants, and "brute-force" here means exactly that: every query compares against every stored vector, with no index structure skipping comparisons. Worth being explicit about where that breaks and what the honest production answer looks like.
| Concern | Why it breaks at millions of SKUs | Production answer |
|---|---|---|
| Compute | Even a vectorized numpy matmul is O(N×D) per query — linear in catalog size. Fine at 100k-1M vectors on one core; a query against 10M+ vectors on every request stops being "fast enough." | An Approximate Nearest Neighbor (ANN) index — HNSW (graph-based) or IVF (cluster-based) — that answers in roughly O(log N) by skipping the vast majority of comparisons, at the cost of being approximate. |
| Memory | 1M products × 1536 dims × 4 bytes (float32) ≈ 6GB just for vectors, held in one process. Doesn't fit comfortably in a single pod, and this demo's in-memory index is process-local (one copy per replica, same limitation as the catalog runtime itself). | An externalized, purpose-built vector store: Azure AI Search vector index, Azure Cosmos DB (DiskANN vCore), or self-hosted pgvector/FAISS service — something that shards/persists vectors outside of each API replica's process memory. |
| Freshness | Re-embedding the whole catalog on every ingest (this demo's approach) is fine at 100 products (~1.3s); at millions of SKUs a full re-embed is a substantial batch job. | Incremental/delta embedding: only re-embed products that actually changed since the last ingest, tracked by a content hash or updated-at timestamp. |
This ties directly to the same architectural gap already tracked for the catalog runtime itself (see the single-writer/multi-reader scaling item below): a real deployment needs the vector index externalized and shared across replicas, not held per-pod in memory, for exactly the same reasons.
Rerank and intent classification ask the LLM to reason fresh on every query — cost and latency that scale with query volume. But relevance-widening work that doesn't depend on the specific query — synonyms, use-cases, occasions, audience/gift framing a shopper might type but a title/description never uses, plus category-typical features that are true in general but never actually written down (hiking boots are expected to be water-resistant with ankle support; the raw catalog description often just says "Footwear item N for everyday retail browsing") — can be done once per product, at ingest instead, and reused by every future query for free.
| Aspect | Detail |
|---|---|
| What it generates | For each product, the LLM returns two things: (1) 5-10 additional search keywords/phrases not already present in the title/description (synonyms, use-cases, seasons, audience/gift framing); (2) 2-5 category-typical inferred features the description is missing (e.g. "water-resistant", "ankle support" for a hiking boot) — explicitly scoped to genuinely standard-for-the-category attributes, not invented specifics like exact certifications or ratings. |
| Where it lands | Keywords merge into the product's tags list (deduped, case-insensitive); inferred features are appended to the description as a clearly-labeled "Typically includes: ..." sentence (never rewritten into the description as an unqualified factual claim, and guarded against duplicate re-appending on repeat ingests). Both happen before the catalog runtime is built and before vector embeddings are computed, so lexical (BM25) postings and vector embeddings benefit automatically with zero changes to either retrieval path. |
| Batching | One LLM call enriches a whole batch of products at once (default 15/call, env RETAIL_V2_LLM_ENRICH_BATCH_SIZE), with up to RETAIL_V2_LLM_ENRICH_MAX_CONCURRENCY (default 6) batches running concurrently — cost/latency scales with catalog_size ÷ batch_size, not with catalog size directly, and pays out at ingest time, off the query-serving path entirely. |
| Separate token/timeout budget | A 15-product enrichment batch generates far more output than a single rerank/intent call — reusing the query-time max_completion_tokens/timeout (tuned tight for per-query latency) silently truncated the JSON and/or timed out on every batch. Enrichment now has its own, much larger budget: RETAIL_V2_LLM_ENRICH_MAX_COMPLETION_TOKENS (default 300 × batch_size) and RETAIL_V2_LLM_ENRICH_TIMEOUT_SECONDS (default 60s) — since ingest is off the query-serving path, there's no latency pressure to keep these tight. |
| Cost curve vs. rerank/intent | Rerank/intent cost is proportional to query volume (paid forever, on every search). Enrichment cost is proportional to catalog size and only re-paid when the catalog changes — a catalog changes far less often than searches arrive, so the amortized per-query cost trends toward zero as traffic grows, the opposite of the rerank/intent curve. |
Off by default (env RETAIL_V2_LLM_ENRICH_ENABLED), toggled via POST /search/admin/ai-toggle (enrichEnabled) from the same AI harness panel; runs automatically on the next POST /search/ingest/catalog or Ingest from ERP action once switched on, with an enrichment block (products enriched, descriptions augmented, batch count/errors) returned in the ingest response.
Verified live: re-ingesting the 103-product demo catalog with enrichment on took ~22s (7 concurrent batches) and enriched all 103 products; a generic "Summit Boot 132" (originally "Footwear item 132 for everyday retail browsing and search.") gained tags including hiking boot, boot waterproof, ankle support and a description suffix "Typically includes: hiking boot, sturdy outsole, ankle support, lace-up design, outerwear of footwear." — so a query for "waterproof hiking boots" now matches lexically, not just via the vector path.
wavesearch-api's ingest-time enrichment (above) only ever touched its own transient in-memory copy of the catalog -- re-ingesting from ERP without ERP's own data being enriched would just bring the bland originals back. The real fix is enriching the ERP's own product records directly, once, so the improved description/categories are permanent and every future ingest (from any consumer) already gets the enriched version for free.
| Aspect | Detail |
|---|---|
| What it does | POST /erp/admin/enrich-catalog on wavestore-erp-api (requires erp.write) rewrites each product's description into 2-3 natural sentences that weave in category-typical features (e.g. "typically provide ankle support, durable outsole grip, and reliable weather resistance" for boots) and expands categories into a richer taxonomy (e.g. Outdoor > Footwear > Boots, Outdoor > Hiking), persisting both directly into the ERP's own product store. |
| Idempotent by default | Each enriched product is marked _llmEnriched: true; subsequent calls skip already-enriched products ({"applied": false, "reason": "no products need enrichment"}) unless called with {"force": true}, so re-running it doesn't keep re-writing descriptions on every call. |
| Same batching/config pattern | Reuses the identical batched-call design as wavesearch-api's enrichment (default 15 products/call, up to 6 concurrent batches, its own generous token budget/timeout independent of any query-time path) against the same Foundry gpt-5-nano deployment. |
| Completes the loop | Once ERP is enriched, Ingest from ERP (POST /search/ingest/from-erp) pulls the permanently-improved descriptions/categories into the search index automatically -- no per-ingest LLM cost needed on the wavesearch-api side anymore for products already enriched at the source. |
Verified live: seeded ERP's (previously empty) product store from the 103-product demo catalog, ran /erp/admin/enrich-catalog (~13s for all 103, 7 batches), confirmed a second call correctly no-ops, confirmed /erp/export/catalog (ERP's own source-of-truth export) already returns the enriched description/categories, then re-ran Ingest from ERP and confirmed the search index picked up the same permanently-enriched text with zero wavesearch-api-side LLM calls needed.
Vertex AI Search for Retail's recommendation models let an operator choose an optimization objective (click-through rate, conversion rate, revenue) for the trained model to target. This platform now has a real (if intentionally small) trainable model too: an online two-tower recommender, trained record-by-record as events arrive -- no batch job, no offline training run, no model registry hot-swap. A heuristic views/clicks/purchases ratio is kept as the automatic cold-start fallback until a tenant's model has learned enough to be trusted.
| Aspect | Detail |
|---|---|
| Model | Per-tenant, per-objective (ctr/conversion/revenue) two-tower model: a d=16 embedding + bias per product, a d=16 embedding per visitor. score = dot(user, item) + bias. Trained with a pairwise Bayesian Personalized Ranking (BPR) SGD step -- a click/add-to-cart/purchase event is paired against a few randomly sampled unclicked products, and one gradient step pushes the real event's score above the sampled negatives'. See retail_v2/ml_ranker.py. |
| Real-time retraining | Every qualifying event trains inline on the request path inside _track_visitor_event -- a few ~16-dim vector operations, sub-millisecond. There is no separate training job or schedule: the model a query sees reflects events logged moments earlier. |
| Tenant-wide vs. personalized scoring | The ranking-objective step is a tenant-wide POLICY (same order for every shopper), so it scores using the item's learned bias term alone (a visitor-agnostic "this item generally wins for this objective" signal) rather than any one visitor's embedding -- the full personalized dot-product is available in the same model for a future recommend-style use case, just not applied here. |
| Cold start & fallback | A product the model has never trained on scores exactly 0.0 and is treated as "no signal" -- ranking falls back to the heuristic views/clicks/purchases ratio (_objective_score) until a tenant's model for that objective has processed ≥ MIN_EVENTS_FOR_ML (20) events. GET /search/admin/ml-model reports events trained, loss, and whether each objective is "ready" (ML-driven) or still on the heuristic. Every ranking-objective response also carries a method: "ml" | "heuristic" field -- never a silent black box. |
| Where it applies | /search/query, /search/browse, and /search/recommend all blend the objective score into the existing relevance order via Reciprocal Rank Fusion (same technique as hybrid search and personalization) -- nudging results, not fully overriding them. Applied AFTER AI rerank but BEFORE personalization/merchandising, same layering as before. |
| Admin controls | GET/POST /search/admin/ranking-objective (pick the objective), GET /search/admin/product-performance (raw counters), GET /search/admin/ml-model (model stats/readiness), POST /search/admin/ml-model/reset (wipe a tenant's model and restart learning from scratch) -- all in the Ranking objective panel in Search Admin. |
| Persistence | Model embeddings/biases, product performance counters, and the selected objective are all snapshotted to blob storage (every 10 trained events, to bound write volume) and reloaded on startup -- survives pod restarts. |
Verified live with scripts/simulate_ranking_traffic.py (a repeatable traffic-simulation test script -- logs in for a search.admin token, fires simulated view/click/add_to_cart/purchase events for a chosen "boot" query's lowest-ranked product from 25+ distinct simulated visitors, sweeps all three objectives, and asserts the target moved up in rank for each): the target product started at rank #10 of 10 for both a "boot" query (SKU-187) and a "jacket" query (SKU-141); after firing simulated traffic, all three objectives reported method: "ml" (the model, not the heuristic, drove the ranking) and moved the target to roughly rank #4-#6 for every objective -- a real, measurable, reproducible ranking shift driven by online learning, not a canned demo.
python scripts/simulate_ranking_traffic.py --base-url https://search-api.retail.demos.wavefunctionlabs.com --query boot --events 25 re-runs this end-to-end check against the live deployment at any time.
POST /search/visual closes the last Vertex AI Search for Retail gap (multi-modal/image search) without provisioning any new infrastructure. Two input modes feed the same downstream pipeline:
| Mode | Detail |
|---|---|
| Text description | A shopper types a vague or detailed visual description (e.g. "something that looks like a shoe", "a red waterproof hiking jacket"). gpt-5-nano expands it into the concrete category/color/material/style/use-case words a real product listing would contain. |
| Uploaded photo | A shopper uploads a real image. Rather than a separate CLIP-style image-embedding model/index, the same gpt-5-nano deployment analyzes the photo directly via native vision input (confirmed working with a direct test call -- no new deployment needed) and describes what it sees in the identical category/color/material/style vocabulary. |
| Why not generate an image | A literal "generate a picture, then search by image similarity" round-trip needs gpt-image-1 (DALL-E 3 is retired), which isn't deployable in this resource's region -- would require a second cross-region Azure OpenAI resource. Both text and photo modes here get the same practical outcome (rich visual terms feeding hybrid search) using only what's already deployed. |
| Shared pipeline | Whichever mode produces the expanded query, it's run through the exact same _run_search_pipeline as /search/query (hybrid retrieval, rerank, personalization, merchandising, dynamic facets) -- extracted into a shared function specifically so this didn't need a second copy of the ranking pipeline. |
Verified live: "something that looks like a shoe" expanded to "shoe-like footwear, ... sneaker-inspired silhouette ..." and returned Summit Boots (the closest footwear in this catalog); uploading a real product photo of a blue/orange color-block waterproof jacket correctly identified it ("Outerwear / Jacket", "color: blue, rust-orange", "material: waterproof/windbreaker fabric") and returned the Aurora Storm Shell Jacket and Trail Ridge Shell products at the top of results. Wired into the WaveStore home page as a "Describe & search" box with an "Or upload a photo" file picker.
Three more gaps closed from the Vertex AI Search for Retail comparison, all reusing the platform's existing bolt-on-layer pattern (retrieval untouched, additional passes composed on top).
| Feature | Detail |
|---|---|
| Personalization | POST /search/events was previously a write-only sink -- clickstream telemetry was logged but never used. It's now public (anonymous shoppers can log events without auth, same posture as search) and every view/click/add-to-cart/purchase event builds a per-visitor category/brand/tag affinity profile (weighted by intent strength: purchase > add-to-cart > click > view). /search/query, /search/recommend, and the new /search/browse all accept an optional visitorId and blend that affinity into ranking via Reciprocal Rank Fusion -- nudging results toward a shopper's taste without fully overriding relevance or the query itself, applied after AI rerank but before merchandising (an explicit business rule still has the final say). Persisted to blob storage, surviving restarts like everything else. |
| Dynamic faceting | Beyond the existing static category/brand/availability facets, every /search/query and /search/browse response now includes price-range buckets and a top-tags facet computed from the actual candidate pool -- and any facet dimension with fewer than 2 distinct values in the current result set is dynamically omitted (e.g. no pointless "Category: Outdoor (12)" facet once already filtered to Outdoor). |
| Dedicated browse endpoint | POST /search/browse -- category/collection navigation with no free-text query, mirroring Vertex's browse concept: same filters/sort/facets/merchandising/personalization pipeline as search, minus lexical retrieval and minus rerank/intent/vector (nothing for an LLM or embedding to usefully rerank without a query). Defaults to in-stock-first ordering when no explicit sort is given -- an honest proxy for "revenue-optimized" browse ranking given this demo has no real sales/margin data to rank by. |
| Storefront wiring | The shared, hardcoded "wave-store-visitor" constant was replaced with a real persistent per-browser visitor id (localStorage-backed), and the storefront now actually fires search/view/add_to_cart/purchase events on the corresponding real actions -- previously wired up as an API capability nobody called from the live UI. |
Verified live: simulated a visitor viewing/clicking/purchasing three jacket products (Contoso Trail brand); an "Outdoor" browse for that visitor then returned Trail Ridge Shell jackets ahead of Summit Boots in the top 5, versus a mixed boots/jackets baseline with no visitor history. A "boot" query's facets included price ranges (Under $25, $50-$100) and top tags (hiking boot, waterproof boot, ...) alongside the existing category/availability facets.
Every ingest previously re-embedded and re-enriched the entire catalog from scratch, every time -- adding one new product to a large catalog paid the full LLM/embedding cost of the whole catalog again. And since the catalog runtime, vector index, merchandising rules, redirects, and promotions were all process-local in-memory state, a pod restart (or a routine redeploy) silently reset everything back to empty/default.
| Aspect | Detail |
|---|---|
| Content-hash delta detection | Each product's indexable fields (title/description/categories/brands/tags/availability/price) are hashed (sha256 of a canonical JSON form). A product whose hash is unchanged since the last ingest skips both LLM enrichment and re-embedding entirely, reusing its cached tags/description/vector. |
| Self-referential re-ingest handled correctly | Re-posting the currently-served (already-enriched) catalog back into /search/ingest/catalog -- a very plausible "refresh" action -- is recognized as unchanged too: each cache entry stores both the original pre-enrichment sourceHash and the resulting outputHash, and incoming content matching either is treated as a cache hit. |
| Real persistence backend | A new AzureBlobStore adapter (retail_v2/azure_blob_store.py) implements the same BlobStore protocol as the existing filesystem-backed dev adapter, backed by a real Azure Storage Account + container. Selected automatically when RETAIL_V2_BLOB_ACCOUNT_NAME/_ACCOUNT_KEY are configured; falls back to the local filesystem adapter otherwise. Unlike a Kubernetes emptyDir volume (survives container restarts but not a full pod reschedule/rollout), a real blob container survives every redeploy. |
| What's persisted | The last-ingested catalog itself, the vector embedding cache, the enrichment cache, merchandising rules, query redirects, and promotions -- all rehydrated automatically on @app.on_event("startup"), before the first request is served. |
Verified live: a cold ingest of the 103-product catalog took ~22s (full LLM enrichment + embedding); re-ingesting the identical catalog afterward took 248ms with 104/104 products reused from cache for both enrichment and vectors. After a full pod restart, vectorIndexSize was already 104 before any ingest ran, and a subsequent re-ingest again reused all 104 vectors with zero new embedding calls -- confirming the cache and the underlying catalog both survive a real redeploy, not just a within-process cache. A merchandising rule and a query redirect created before a restart were both still present and enforced afterward.
A gap analysis against Vertex AI Search for Retail surfaced a real bug, not a missing feature: POST /search/admin/rules has always accepted and stored boost/bury/pin rules, and both the Search Admin GUI and this doc described them as live ranking controls -- but no query path ever actually read them. Setting a rule had zero effect on search or recommendation ordering.
| Aspect | Detail |
|---|---|
| Where it's applied | As the final ranking step in both POST /search/query and POST /search/recommend, after lexical/vector retrieval and after any LLM rerank. Business rules represent an explicit merchandising decision and should override AI/relevance ranking, not be undone by it. |
| Semantics | Pin: forced to the top, in the order pinned. Boost: moved ahead of neutral results, relative order preserved. Bury: moved to the bottom. A product listed in both boost and bury (contradictory rules) is treated as buried -- an explicit "hide this" decision wins over "promote this". Rules only re-rank products already present in the result set; they never inject unrelated products that didn't match the query/filters. |
| Transparency | Every response now includes a merchandising block (applied, pinned/boosted/buried counts) so it's immediately visible whether rules fired and how many products they touched. |
| Rule cleanup | Added the missing DELETE /search/admin/rules/{'{'}id{'}'} (rules previously had no way to be removed once created). |
Verified live: searching "boot" gave baseline order [SKU-192, SKU-197, SKU-172, SKU-162, SKU-187]; after pinning SKU-187 and burying SKU-192, the same query returned [SKU-187, SKU-197, SKU-172, SKU-162, SKU-192] -- pinned item moved to first, buried item moved to last, everything else kept its relative order. Confirmed the new delete endpoint correctly reverts merchandising.applied back to false.
A gap analysis against Google Vertex AI Search for Retail's feature set flagged these as the platform's biggest missing category: query understanding beyond raw keyword matching. Three features close that gap, all bolted on as a separate layer without touching the core BM25 engine (same design principle as hybrid search and rerank).
| Feature | Detail |
|---|---|
| Autocomplete | GET /search/autocomplete?query=...&limit=... (public, proxied via GET /v2/search/autocomplete on the storefront) returns prefix-matched product title, category, and brand suggestions from a vocabulary rebuilt on every catalog ingest. Wired live into the WaveStore search box as a debounced (180ms) dropdown. |
| Suggestion diversification | Patterned SKU-variant catalogs (e.g. "Summit Boot 102", "Summit Boot 117", "Summit Boot 122", ...) would otherwise flood autocomplete with near-duplicate suggestions. Titles are collapsed to one suggestion per "base" (trailing SKU number/size-color suffix stripped) before returning, mirroring Vertex's diversification concept. |
| Spelling correction / typo tolerance | Query terms with zero matches anywhere in the indexed vocabulary are fuzzy-matched (difflib.get_close_matches, stdlib, no new dependency) against real indexed terms and substituted before the lexical search runs (e.g. "jaket" → "jacket"). Applied only to the lexical path, not the vector-embedding query text -- embeddings already tolerate minor typos via subword similarity, and "correcting" a valid natural-language word based on a fuzzy string match risks corrupting a semantic query far more than it helps. spellCorrection is returned in every /search/query response for transparency. |
| Query redirects | POST /search/admin/rules's sibling for navigation: POST /search/admin/redirects maps an exact (tokenized) query to a fixed URL (e.g. "summer sale" → /collections/summer-sale), managed from a new Query redirects panel in Search Admin. Every /search/query response includes a redirect block so the storefront can choose to navigate instead of (or alongside) showing ranked results. |
Verified live: autocomplete for "sum" returns a single diversified "Summit Boot 102" suggestion (not 20 near-duplicates); autocomplete and full search for the misspelling "jaket" both correct to "jacket" and return the Aurora Storm Shell Jacket; a "summer sale" redirect was created, matched on the next search, and listed/removable via the admin API.
Catalog categories are hierarchical strings (e.g. "Outdoor > Footwear > Boots", "Outdoor > Hiking"). The storefront previously flattened every full path into one long top-level nav bar; it now groups by the first segment only, with a second bar of subcategories revealed underneath once a top-level category is selected.
| Aspect | Detail |
|---|---|
| Category tree | Built client-side from the loaded catalog: each hierarchical category string is split on >, grouped by its first segment, with the remainder mapped back to the original full path for exact filtering. |
| Top-level click | Filters/searches by the top segment (e.g. "Outdoor") and reveals its subcategory bar underneath, if it has any children. |
| Subcategory click | Filters/searches by the exact full category path (e.g. "Outdoor > Hiking"), keeping the subcategory bar visible with that entry highlighted. |
| Filter wiring | Category-nav clicks drive search through a dedicated override (state.navCategoryOverride) rather than the manual category <select>, since the dropdown only lists full leaf paths as options and can't represent a top-level-only selection; the override is cleared whenever the shopper interacts with the manual filters directly. |
Promotions/offers are indexed in-memory the same way products are: a per-tenant store plus a token-postings index built from each offer's title/subtitle/cta/category/brand/discount fields. This makes promotions a first-class searchable entity in WaveStore rather than only a client-side offer list.
| Endpoint | Purpose | Where visible |
|---|---|---|
| POST /search/ingest/promotions | Indexes a promotions list directly. | Fired automatically as part of Ingest from ERP. |
| POST /search/ingest/from-erp | Ingests the catalog and, best-effort, the ERP's /erp/offers in the same action. | Ingest from ERP button in Search Admin GUI. |
| POST /search/offers | Searches indexed promotions by query text; empty query returns all. | Proxied as POST /v2/offers/search and surfaced as the Search offers box on the WaveStore home page. |
Product-level linkage stays authoritative: each offer's productIds list (set in ERP) still drives which products actually receive the discount when a shopper clicks through; the search index only makes the promotion itself discoverable by text.
These figures were captured directly against the live wave-dev wavesearch-api pod (1 replica, 150m/1 vCPU request/limit, 384Mi/768Mi memory, catalog of 103 products), not modelled.
| Path | Result | Notes |
|---|---|---|
| Lexical/BM25 only (in-process, no HTTP) | ~61,000 req/s | Pure candidate retrieval + scoring cost; effectively free at this catalog size. |
| Full HTTP path, AI disabled | ~550 req/s sustained, sub-2ms avg latency | Scales cleanly through 50 concurrent requests with zero failures on a single pod. |
| Full HTTP path, AI reranking + intent classification enabled | ~1.0-1.8s avg latency/query | Down from ~4.7s after the reasoning/prompt optimization below; still bounded by the Azure OpenAI round-trip, not by the API itself. |
Current default: AI reranking and intent classification are switched OFF in wave-dev to keep query latency low (sub-2ms instead of ~1-2s). Both can be re-enabled live, per feature, from the AI harness panel in the Search Admin GUI without a redeploy (backed by POST /search/admin/ai-toggle), so the cost/latency-vs-relevance tradeoff is now an explicit operator decision rather than a fixed deployment setting.
AI-enabled latency optimization (3-4x faster, ~4.7s → ~1-1.8s): profiling the raw Azure OpenAI call in isolation found the model deployment, gpt-5-nano, is a reasoning model that silently burns hidden "reasoning tokens" (~256 tokens, roughly 1s) before ever emitting the JSON answer — wasted cost for a deterministic rerank/classify task with no chain-of-thought benefit. Three changes were made in wavesearch_api/_llm_chat_json and _rerank_results: (1) set reasoning_effort=minimal (env RETAIL_V2_LLM_REASONING_EFFORT), which drove measured reasoning tokens to 0; (2) capped max_completion_tokens (env RETAIL_V2_LLM_MAX_COMPLETION_TOKENS, default 300) to bound worst-case generation length; (3) trimmed the rerank candidate payload sent per product (dropped description/availability/price, truncated title) and shortened the requested rationale to ≤15 words, cutting prompt tokens roughly in half (~1,818 → ~850 for a 20-candidate rerank). All three are independent, measured levers — verified via an isolated raw-call profiling script before and after, not guessed.
Reliability bug found and fixed during this benchmarking: the AI rerank/intent HTTP calls were originally made synchronously inside async def handlers, which blocked the single shared event loop under concurrent load. At just 20 concurrent requests this made /healthz stop responding and Kubernetes killed and restarted the pod. Fixed by moving those calls onto worker threads via asyncio.to_thread, and by running rerank + intent classification concurrently (via asyncio.gather) instead of sequentially, which also roughly halved AI-enabled latency (from ~9-10s to ~4.7s worst case, before the reasoning/prompt optimization above brought it down further to ~1-1.8s).
Can you run more than one instance in parallel? Yes for stateless read traffic, but with an important caveat today: merchandising rules, search diagnostics, and the ingested catalog runtime are held in process-local memory in each wavesearch-api pod. Multiple replicas behind the same Kubernetes Service will each hold independent state — an ingest or rule change applied to one pod is invisible to the others until they are individually refreshed. Scaling replicas today is safe for pure query/recommend throughput, but not yet safe for consistent admin/ingest behavior.
Planned fix: move to a single-writer/multi-reader model — one writer replica owns ingest and rule mutation and publishes change notifications; reader replicas subscribe and refresh their local runtime, so read throughput can scale horizontally without state divergence.
Profiling the "in-memory search takes 1.4s?" question found the real cost wasn't the search itself (BM25 scoring is sub-millisecond) but the auth path wrapped around it: every single /v2/search call from WaveStore minted a brand-new token via a full STS /sts/login round-trip, including a 120,000-iteration PBKDF2-HMAC-SHA256 password verify, on every request.
| Change | Detail | Effect |
|---|---|---|
| Public search endpoints | POST /search/query, /search/recommend, and /search/offers on wavesearch-api no longer require a bearer token at all — they're read-only and already reachable by any anonymous shopper on the storefront, so requiring a signed JWT just to read the catalog added latency without adding real protection. Admin, ingest, and event-write endpoints are unchanged and still require an authenticated, scoped token. | WaveStore's do_search/do_recommend/do_search_offers proxy calls no longer call STS at all for these paths. |
| CORS enabled | CORSMiddleware added to wavesearch-api, allowing store.retail.demos.wavefunctionlabs.com (WaveStore) as an origin by default; configurable via WAVE_CORS_ORIGINS (comma-separated). | Lets a browser call wavesearch-api directly if needed, not only via the storefront's server-side proxy. |
| Token cache + longer TTL | For the auth paths that remain (e.g. ERP offers), WaveStore's issue_token() now caches tokens in-process per (audience, tenant, subject, scopes) and requests the maximum STS-allowed TTL of 3600s (up from a fresh 900s token every call), refreshing ~60s before expiry. | STS round-trips for those remaining calls drop from once-per-request to roughly once-per-hour per unique token key. |
Measured in-cluster after the fix: /v2/search repeat calls run in ~3-4ms end-to-end (down from ~1.4s), with the one-time STS /sts/login password verify itself confirmed at ~120ms in isolation — i.e. the fix was removing the round-trip from the hot path, not making the password hash faster.