Based in Chicago 芝加哥 Backend Software Engineer
Product Search
Product Search / Product Search   Product Search / Product Search    Product Search / Product Search   Product Search / Product Search   

Personal Project, 2026

A multi-stage hybrid search engine over a 423k-product catalog, extended with search by text or photo over 23.8k product images. Every design choice is backed by an offline eval, and the image search is deployed as an API on Fly.io.

Category Search & Machine Learning
Role Solo Engineer
Benchmarks Amazon ESCI (8,955 queries) + 2,000 LLM-written shopper queries
Catalog 423k products (23.8k with photos)
Product search results page

Overview

A four-stage text search pipeline: BM25 for exact brands and specs, dense retrieval for paraphrases, rank fusion to merge them, and a cross-encoder to rerank. On Amazon ESCI it lifts Hits@1 from 0.296 to 0.374 over BM25 alone.

Image search adds search by description, by uploaded photo, or by "more like this", using SigLIP 2 photo matches blended with MiniLM title matches. Vision-labeled color and category filters cover all 23.8k products.

Evals on 2,000+ LLM-written shopper queries drove the design: equal-weight fusion collapsed on queries that describe the photo, and a 70/30 score blend raised MRR@10 from 0.527 to 0.624. It runs as a FastAPI service on Fly.io.

Demo

Evals

Image search: text queries by type

MRR@10 [Recall@10] on held-out products. Queries were LLM-written from the product's brand, photo, or text, and fusion weights were tuned on the other half of the products.

MethodFrom brandFrom imageFrom textMean
Title only (MiniLM)0.772 [0.88]0.162 [0.29]0.438 [0.60]0.457
Photo only (SigLIP 2)0.594 [0.79]0.599 [0.79]0.457 [0.69]0.550
Equal-weight RRF (first design)0.759 [0.91]0.310 [0.52]0.511 [0.70]0.527
Score blend, w = 0.7 (shipped)0.822 [0.96]0.495 [0.70]0.554 [0.74]0.624
Score blend, w = 0.80.781 [0.94]0.536 [0.75]0.536 [0.73]0.618

Text search: Amazon ESCI

8,955 judged queries over 1.2M products. Each row adds or swaps one stage of the pipeline.

PipelineHits@1Hits@5Hits@10MRR@10
BM250.2960.5020.5810.385
Dense (MiniLM-L6)0.2490.4620.5600.340
Dense (BAAI bge-small)0.2860.4980.5940.378
RRF (dense + BM25)0.3150.5320.6300.413
RRF + cross-encoder rerank0.3740.6120.6850.474

Results

Hits@1: 0.296 → 0.374

The top result is an exact match for 26% more queries than with BM25 alone, measured on 8,955 judged ESCI queries.

MRR@10: 0.385 → 0.474

Relevant products rank higher on average. The cross-encoder rerank delivered the largest single-stage gain.

Image Search MRR@10: 0.527 → 0.624

Blending normalized photo and title scores replaced equal-weight rank fusion. The blend no longer collapses on queries that describe the photo, and it beats both sides on brand queries.

Measured, Then Not Shipped

Evals ruled out three plausible features: weighted rank fusion, boosting by the color named in the query, and 8-bit models, which dropped photo search MRR@10 from 0.971 to 0.087.

~110 ms per Query

End-to-end latency for all four stages over 423k products, using exact FAISS search on an RTX 5050 laptop GPU.

Controlled Experiments

The embedding model is a swappable component, so the MiniLM vs. BAAI bge-small comparison changed only the encoder.

Key Features

Hybrid Retrieval

BM25 and dense retrieval each pull 50 candidates, merged with Reciprocal Rank Fusion because their failure modes barely overlap.

Neural Reranking

A cross-encoder scores each (query, title) pair jointly, demoting look-alikes such as "AeroPress Movie" below actual coffee makers.

Stage-by-Stage Search UI

Switch between BM25, dense, RRF, and the full pipeline on the same query, with per-stage latency and badges showing which retriever found each result.

Pipeline Tracing

Each product's detail panel shows its rank at every stage, making it easy to see how fusion and reranking changed the order.

Search by Text or Photo

Describe a product, drop in your own photo, or click "more like this" to search by a product's image, using SigLIP 2 embeddings blended with MiniLM title matches.

Vision-Labeled Filters

A vision model labeled all 23.8k products with one of 16 colors and a Google-taxonomy category. A 300-product pilot showed that fixed-choice labels beat free-form ones.

LLM-Written Benchmarks

Shopper queries are generated through an LLM gateway, with one writer model so only the input varies. 1,000 requests take 2 minutes instead of over an hour on a laptop.

Hosted API

A FastAPI service on Fly.io with the models and index baked into the Docker image. Sizing torch threads to the container's CPU limit cut queries from 18 s to about 0.1 s.

Technologies

Python PyTorch Sentence Transformers FAISS BM25 (bm25s) Cross-Encoders Pandas SigLIP 2 / CLIP / DINOv2 Flask FastAPI Docker Fly.io Playwright JavaScript CUDA

Gallery

MORE PROJECTS EXPLORE
More Works   More Works   More Works   More Works    More Works   More Works   More Works   More Works   
GPU Animation
GPU Animation (01)
2389 Research
2389 Research (02)
Carbonbusters
Carbonbusters (03)