Speaker Details
Alessandro Benedetti
R&D Director, Sease
Alessandro Benedetti is an Apache Lucene/Solr committer and Solr chair of the PMC, Director at Sease Ltd.
He believes in Open Source as a way to build a bridge between Academia and Industry and facilitate the progress of applied research.
Alessandro is a passionate R&D software engineer, continuously applying the latest trends in Information Retrieval and AI to solve search problems.
He’s been working on Learning To Rank for years and more recently he’s been exploring Generative AI techs like Large Language Models and Retrieval Augmented Generation.
When he isn't on clients' projects, he contributes to the open-source community and presents at meet-ups and conferences such as ECIR, Search Solutions, Community Over Code, Haystack and Berlin Buzzwords.
Talk Details
Evaluating Search in the LLM Era: Tools for Data & Vectors
This talk explores two neighbouring areas in modern search quality evaluation: preparing the ground-truth dataset of queries and ratings, and identifying the causes of the problems we see in our vector search implementation. Building datasets manually is expensive: queries can be extracted from production logs (when available), but relating queries to documents with a reasonable relevance rating is hard, requires a lot of time from human experts, and it’s extremely boring, so it’s likely that a human, after a while, will lose interest, and ratings become more approximate.
Alessandro Benedetti, Sease & Daniele Antuzi, Sease