After 17 years on Solr, Etsy is migrating to Vespa. This is a field report from an ongoing migration: what we've learned, what surprised us, and where we're headed. We'll cover hybrid search, agentic developer tooling, embedding versioning, traffic replay, offline evaluation, and the infrastructure we built to make it all work.
This talk is a field report from that ongoing migration. Etsy's retrieval systems serve diverse use cases across search, ads, recommendations, and growth, so the lessons we've learned apply broadly to anyone building high-performance search and retrieval infrastructure.
On hybrid search, we'll describe how we combine traditional lexical retrieval with dense semantic embeddings natively in Vespa, and the tradeoffs we've navigated in balancing precision and recall across very different retrieval signals.
On agentic developer tooling, we'll show how we're using LLM-powered tools to help engineers not just to write code, but automate the full lifecycle of retrieval ML — hypothesis generation and testing, deployment, monitoring, evaluation, and hyperparameter tuning.
We'll also cover automation with PyVespa for dynamic schema management, cluster provisioning, and scaling policies; observability through native OpenTelemetry support we helped build with the Vespa team to integrate with Etsy's monitoring platform; embedding versioning powered by GitOps and automated deployments for models retrained nightly; offline ML evaluation and traffic replay for safe rollouts; ANN optimization and hyperparameter tuning; model distillation for latency-constrained ranking; streaming data ingestion; multi-currency search and more.
