SIRIP Keynotes & Panel

Toward AI-Powered Next Generation Personalized Experiences

Paul Bennett

Paul Bennett

Abstract: As the recent pace of AI innovation has increased, search and recommendation is being increasingly transformed in recent years. These transformations are fundamentally changing how people interact with search and recommendation experiences as well as how content is created. Understanding these trends is a key toward projecting the broader technology needs we anticipate in the future. After reflecting on these broader trends we dive into three key advances that have already changed the landscape of search and recommendation: dense retrieval, augmented LLMs, and GNNs. We discuss how these technologies have impacted content understanding, conversational recommendation, and user modeling. Finally, we conclude with speculations on the technical challenges surrounding the more general landscape of AI and its implications for supporting personalization for both creators and the audience in search and recommendation. This talk presents work with many collaborators both past and present at Microsoft, Spotify, and more broadly in academia.

Bio: Paul Bennett is Director of Research on Large Language Models at Spotify. His research focuses on how AI foundational models can improve current creator and audience experiences as well as the role they can play in the next generation of experiences. Previous to joining Spotify, Paul was Partner Research Manager for the Augmented Learning + Reasoning group at Microsoft Research. His published research has focused on a variety of topics surrounding the use of machine learning in information retrieval – including deep learning for ranking and retrieval, ensemble methods and the combination of information sources, calibration, consensus methods for noisy supervision labels, active learning and evaluation, supervised classification and ranking, crowdsourcing, behavioral modeling and analysis, and personalization. Some of his work has been recognized with awards at SIGIR, CHI, ECIR, and ACM UMAP including a SIGIR Test of Time Award in 2022.

Responsible Generative IR

Vanessa Murdock

Vanessa Murdock

Abstract: Responsible AI (RAI) seeks basic guarantees of fairness, safety, privacy, robustness, controllability, explainability, transparency, and governance for traditional ML systems and generative AI systems. With the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, the NIST ISO 42001 standard, and the EU AI Act, RAI has become a central focus of the AI/ML product development cycle. In this talk we provide an overview of the current practices for measuring and mitigating each RAI dimension and we discuss challenges in measuring RAI dimensions in generative IR systems.

Bio: Vanessa Murdock leads a research group in Amazon AWS AI/ML, whose focus is Responsible AI (bias/fairness, toxicity, privacy, robustness, transparency, safety). In addition to doing fundamental research in RAI topics, her team builds tools for assessing aspects of responsible AI, used in AWS Services such as Rekognition and Transcribe. Prior to joining AWS, she led a science team in Alexa Shopping focused on recommender systems, search and HCI. Her team provided the machine learning that backed Amazon’s Choice, and Alexa Shopping List, in addition to contributing content moderation for the generative AI system Rufus. She was previously at Microsoft, working on location inference and notifications at Bing and Cortana. Prior to Microsoft, Murdock led the Geographic Context and Experience Group at Yahoo! Research in Barcelona, which centered on geographic information retrieval and user-generated content. She has been awarded 19 patents, and has more than 20 patent applications pending, resulting in a Master Inventor Award from Yahoo! (2012). She received the OAA Award for Outstanding Achievement by a Young Alum from the University of Massachusetts in 2014. She is currently serving as the Chair of the ACM SIGIR Executive Committee. Murdock received a Ph.D. in Computer Science from the University of Massachusetts Amherst Center for Intelligent Information Retrieval.

Panel: Are Embeddings Enough? SIRIP Panel on the Future of Embeddings in Industry IR Systems

Moderator: Tracy Holloway King (Adobe)

  • Jon Degenhardt (eBay)
  • Rosie Jones (Spotify)
  • Donald Metzler (Google)
  • Chirag Shah (Univ. of Washington)

Abstract: The IR community as a whole is considering whether search and recommendations can move entirely to embedding-based technologies. This SIRIP panel discusses the future of embedding-based technologies in industry search given its broad range of document types, its specific query types, its performance requirements, and the features that accompany search. The panel comprises long-time industry experts and academics with industry ties. The panelists vary as to whether they believe that the industry in practice will move entirely to embeddings or will remain a hybrid domain.


Tracy Holloway King (Moderator)
Tracy Holloway King is a Senior Principal Scientist focusing on search science in Adobe’s Search and Discovery team. Her focus at Adobe is on multi-modal and text search and recommendation relevance, including evaluation and AI ethics for search models and features. After getting her Ph.D. from Stanford, she was an NLP researcher in PARC for over a decade. She then moved to applied science roles at Microsoft Bing, eBay Search Science, and Amazon Query Understanding before joining Adobe. She regularly reviews for NLP and search conferences and has co-organized workshops at ACL, COLING, and SIGIR including co-organizing SIGIR 2023’s industry track SIRIP.

Jon Degenhardt (Panelist)
Jon Degenhardt has been a senior member of the eBay search science team for almost 15 years and is currently a Principle Applied Researcher. He works on next generation relevance technology for e-commerce search, innovating on ranking and matching algorithms and the underlying technology platforms. He was the founder and coordinator of eBay’s Applied Sciences Forum and was co-chair of the 2017, 2018 and 2019 SIGIR workshops on e-commerce search. Before eBay, he was a search relevance architect and researcher at Yahoo, where he developed relevance signals used in ranking and components of the web search engine’s relevance architecture.

Rosie Jones (Panelist)
Rosie Jones is Director of Research at Spotify, where she leads a team of research scientists working on search and natural language understanding for music and podcasts. Before that, she worked at Microsoft, in the New England Research and Development Center (NERD), on con- versational understanding. Prior to this she was Director of Data Science at MediaMath, where she led a team conducting research on large-scale machine learned models for display advertising. She moved to MediaMath via an acquisition from Akamai. Prior to working at Akamai, she spent 8 years as a scientist at Yahoo! Labs, working on search and sponsored search. Her research interests in- clude online user behavior, conversational understanding systems, web search, natural language processing and computational adver- tising. She has a Ph.D. in Language and Information Technologies from Carnegie Mellon University and a B.Sc. from the University of Sydney, majoring in Computer Science. Her research at Microsoft, Akamai and Yahoo! has led to publications, product improvements and 14 issued patents, as well as academic service including senior PC member for COLING and SIGIR. She is also a Senior Member of the ACM.

Donald Metzler (Panelist)
Donald Metzler is a Senior Staff Research Scientist at Google Inc. Prior to that, he was a Research Assistant Professor at the University of Southern California (USC) and a Senior Research Scientist at Yahoo!. He has served as the Program Chair of the WSDM, ICTIR, and OAIR conferences and sat on the editorial boards of all the major journals in his field. He has published over 100 research papers, has been awarded 9 patents, and is a co-author of Search Engines: Information Retrieval in Practice. He currently leads a research group focused on a variety of problems related to machine learning, natural language processing, and information retrieval.

Chirag Shah (Panelist)
Chirag Shah is Professor of Information and Computer Science at University of Washington (UW) in Seattle. He is the Founding Director for InfoS- eeking Lab and Founding Co-Director of Center for Responsibility in AI Systems & Experiences (RAISE). His research focuses on build- ing, auditing, and correcting intelligent information access systems. In addition to creating AI-driven information access systems that provide more personalized reactive and proactive recommenda- tions, he is also focusing on making such systems transparent, fair, and free of biases. He works closely with industrial research labs on cutting-edge problems, typically as a visiting researcher. The most recent engagements included Amazon, Getty Images, Microsoft Research, and Spotify. Shah is a Distinguished Member of ACM.