Speaker Details
Daniele Antuzi
Software Engineer, Sease
Daniele is a software engineer passionate about high-performance data structures and algorithms.
He likes studying and experimenting with new technologies trying to improve the state of the art.
Talk Details
Evaluating Search in the LLM Era: Tools for Data & Vectors
This talk explores two neighbouring areas in modern search quality evaluation: preparing the ground-truth dataset of queries and ratings, and identifying the causes of the problems we see in our vector search implementation. Building datasets manually is expensive: queries can be extracted from production logs (when available), but relating queries to documents with a reasonable relevance rating is hard, requires a lot of time from human experts, and it’s extremely boring, so it’s likely that a human, after a while, will lose interest, and ratings become more approximate.
Alessandro Benedetti, Sease & Daniele Antuzi, Sease