Paul Bennett
Spotify
Toward AI-Powered Next Generation Personalized Experiences
Abstract: As the recent pace of AI innovation has increased, search and recommendation is being increasingly transformed in recent years. These transformations are fundamentally changing how people interact with search and recommendation experiences as well as how content is created. Understanding these trends is a key toward projecting the broader technology needs we anticipate in the future. After reflecting on these broader trends we dive into three key advances that have already changed the landscape of search and recommendation: dense retrieval, augmented LLMs, and GNNs. We discuss how these technologies have impacted content understanding, conversational recommendation, and user modeling. Finally, we conclude with speculations on the technical challenges surrounding the more general landscape of AI and its implications for supporting personalization for both creators and the audience in search and recommendation. This talk presents work with many collaborators both past and present at Microsoft, Spotify, and more broadly in academia.
Bio: Paul Bennett is Director of Research on Large Language Models at Spotify. His research focuses on how AI foundational models can improve current creator and audience experiences as well as the role they can play in the next generation of experiences. Previous to joining Spotify, Paul was Partner Research Manager for the Augmented Learning + Reasoning group at Microsoft Research. His published research has focused on a variety of topics surrounding the use of machine learning in information retrieval – including deep learning for ranking and retrieval, ensemble methods and the combination of information sources, calibration, consensus methods for noisy supervision labels, active learning and evaluation, supervised classification and ranking, crowdsourcing, behavioral modeling and analysis, and personalization. Some of his work has been recognized with awards at SIGIR, CHI, ECIR, and ACM UMAP including a SIGIR Test of Time Award in 2022.
Responsible Generative IR
Vanessa Murdock
Amazon
Abstract: Responsible AI (RAI) seeks basic guarantees of fairness, safety, privacy, robustness, controllability, explainability, transparency, and governance for traditional ML systems and generative AI systems. With the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, the NIST ISO 42001 standard, and the EU AI Act, RAI has become a central focus of the AI/ML product development cycle. In this talk we provide an overview of the current practices for measuring and mitigating each RAI dimension and we discuss challenges in measuring RAI dimensions in generative IR systems.
Bio: Vanessa Murdock leads a research group in Amazon AWS AI/ML, whose focus is Responsible AI (bias/fairness, toxicity, privacy, robustness, transparency, safety). In addition to doing fundamental research in RAI topics, her team builds tools for assessing aspects of responsible AI, used in AWS Services such as Rekognition and Transcribe. Prior to joining AWS, she led a science team in Alexa Shopping focused on recommender systems, search and HCI. Her team provided the machine learning that backed Amazon’s Choice, and Alexa Shopping List, in addition to contributing content moderation for the generative AI system Rufus. She was previously at Microsoft, working on location inference and notifications at Bing and Cortana. Prior to Microsoft, Murdock led the Geographic Context and Experience Group at Yahoo! Research in Barcelona, which centered on geographic information retrieval and user-generated content. She has been awarded 19 patents, and has more than 20 patent applications pending, resulting in a Master Inventor Award from Yahoo! (2012). She received the OAA Award for Outstanding Achievement by a Young Alum from the University of Massachusetts in 2014. She is currently serving as the Chair of the ACM SIGIR Executive Committee. Murdock received a Ph.D. in Computer Science from the University of Massachusetts Amherst Center for Intelligent Information Retrieval.
Panel: Are Embeddings Enough? SIRIP Panel on the Future of Embeddings in Industry IR Systems
Moderator: Tracy Holloway King (Adobe)
Panelists:
- Jon Degenhardt (eBay)
- Rosie Jones (Spotify)
- Donald Metzler (Google)
- Chirag Shah (Univ. of Washington)
Abstract: The IR community as a whole is considering whether search and recommendations can move entirely to embedding-based technologies. This SIRIP panel discusses the future of embedding-based technologies in industry search given its broad range of document types, its specific query types, its performance requirements, and the features that accompany search. The panel comprises long-time industry experts and academics with industry ties. The panelists vary as to whether they believe that the industry in practice will move entirely to embeddings or will remain a hybrid domain.
Bios:
Tracy Holloway King (Moderator)
Tracy Holloway King is a Senior Principal Scientist focusing on search
science in Adobe’s Search and Discovery team. Her focus at Adobe
is on multi-modal and text search and recommendation relevance,
including evaluation and AI ethics for search models and features.
After getting her Ph.D. from Stanford, she was an NLP researcher
in PARC for over a decade. She then moved to applied science
roles at Microsoft Bing, eBay Search Science, and Amazon Query
Understanding before joining Adobe. She regularly reviews for NLP
and search conferences and has co-organized workshops at ACL,
COLING, and SIGIR including co-organizing SIGIR 2023’s industry
track SIRIP.
Jon Degenhardt (Panelist)
Jon Degenhardt has been a senior member of the eBay search science team for almost
15 years and is currently a Principle Applied Researcher. He works
on next generation relevance technology for e-commerce search,
innovating on ranking and matching algorithms and the underlying
technology platforms. He was the founder and coordinator of eBay’s
Applied Sciences Forum and was co-chair of the 2017, 2018 and
2019 SIGIR workshops on e-commerce search. Before eBay, he was
a search relevance architect and researcher at Yahoo, where he
developed relevance signals used in ranking and components of
the web search engine’s relevance architecture.
Rosie Jones (Panelist)
Rosie Jones is Director of Research at Spotify, where she leads a team of research
scientists working on search and natural language understanding
for music and podcasts. Before that, she worked at Microsoft, in the
New England Research and Development Center (NERD), on con-
versational understanding. Prior to this she was Director of Data
Science at MediaMath, where she led a team conducting research
on large-scale machine learned models for display advertising. She
moved to MediaMath via an acquisition from Akamai. Prior to
working at Akamai, she spent 8 years as a scientist at Yahoo! Labs,
working on search and sponsored search. Her research interests in-
clude online user behavior, conversational understanding systems,
web search, natural language processing and computational adver-
tising. She has a Ph.D. in Language and Information Technologies
from Carnegie Mellon University and a B.Sc. from the University of
Sydney, majoring in Computer Science. Her research at Microsoft,
Akamai and Yahoo! has led to publications, product improvements
and 14 issued patents, as well as academic service including senior
PC member for COLING and SIGIR. She is also a Senior Member of
the ACM.
Donald Metzler (Panelist)
Donald Metzler is a Senior Staff Research Scientist at Google Inc. Prior to that, he
was a Research Assistant Professor at the University of Southern
California (USC) and a Senior Research Scientist at Yahoo!. He
has served as the Program Chair of the WSDM, ICTIR, and OAIR
conferences and sat on the editorial boards of all the major journals
in his field. He has published over 100 research papers, has been
awarded 9 patents, and is a co-author of Search Engines: Information
Retrieval in Practice. He currently leads a research group focused on
a variety of problems related to machine learning, natural language
processing, and information retrieval.
Chirag Shah (Panelist)
Chirag Shah is Professor of Information and Computer Science at University of
Washington (UW) in Seattle. He is the Founding Director for InfoS-
eeking Lab and Founding Co-Director of Center for Responsibility
in AI Systems & Experiences (RAISE). His research focuses on build-
ing, auditing, and correcting intelligent information access systems.
In addition to creating AI-driven information access systems that
provide more personalized reactive and proactive recommenda-
tions, he is also focusing on making such systems transparent, fair,
and free of biases. He works closely with industrial research labs on
cutting-edge problems, typically as a visiting researcher. The most
recent engagements included Amazon, Getty Images, Microsoft
Research, and Spotify. Shah is a Distinguished Member of ACM.