{"id":6940,"date":"2021-04-24T19:00:42","date_gmt":"2021-04-24T18:00:42","guid":{"rendered":"https:\/\/irsg.bcs.org\/informer\/?p=6940"},"modified":"2021-04-24T19:00:42","modified_gmt":"2021-04-24T18:00:42","slug":"fairness-and-bias-in-information-retrieval-for-industry-march-2021","status":"publish","type":"post","link":"https:\/\/archive-irsg.bcs.org\/informer\/?p=6940","title":{"rendered":"Fairness and Bias in Information Retrieval Glasgow 23 March 2021"},"content":{"rendered":"<p>2020 was a notable year in information retrieval research and development. First there was the realization that AI applications were going to need a level of transparency that would reassure customers that they were supporting, not controlling, the search process. Second there was the realization that the search community had a responsibility to think beyond the Holy Grail of perfect relevance and take into account issues around bias, fairness and reproducibility.<\/p>\n<p>Back in March 2020 I was looking forward to attending a workshop being run by Graham McDonald, Iadh Ounis and Craig Macdonald at the University of Glasgow on the subject of Fair Information Retrieval in Industry.\u00a0 It turned out to the first of many cancelled events last year! So I was delighted that the team were able to run a virtual version in March 2021 with a slightly different title.\u00a0 The papers are listed below, with video links where available.<\/p>\n<p><!--more--><\/p>\n<p>The keynote was presented by Professor Maarten de Rijke (<a href=\"https:\/\/youtu.be\/pmSXvPbm1lo\">https:\/\/youtu.be\/pmSXvPbm1lo<\/a>) who laid out what he described as the operating principles, which he then considered with great clarity and insights. It was a master class in terms of content and delivery. He encouraged us focus on robust AI, which he defined as involving<\/p>\n<ul>\n<li>Accuracy, including well-defined and explained contexts of usage<\/li>\n<li>Reliability, including exhibiting parity with respect to sensitive attributes<\/li>\n<li>Repeatable and reproducible results, including audit trails<\/li>\n<li>Resilience to adversarial examples<\/li>\n<li>Resilience to distributional shifts<\/li>\n<li>Safety, including privacy preservation<\/li>\n<\/ul>\n<p>The other presentations were<\/p>\n<p>Leif Azzopardi, University of Strathclyde, UK. &#8220;The Fairness Hypothesis: Is fairer, better?&#8221;. <a href=\"https:\/\/youtu.be\/36gsEYeop8g\">https:\/\/youtu.be\/36gsEYeop8g<\/a><\/p>\n<p>Tim Gollins, National Records of Scotland, UK. &#8220;The Challenges of Fair Retrieval in Archives and the Public Sector&#8221;. <a href=\"https:\/\/youtu.be\/Q34dCf7BtNs\">https:\/\/youtu.be\/Q34dCf7BtNs<\/a><\/p>\n<p>Rishabh Mehrotra, Spotify, London, UK. &#8220;Diversity &amp; Fairness in multi-stakeholder Marketplaces&#8221;. <a href=\"https:\/\/youtu.be\/1B0MP_jBezE\">https:\/\/youtu.be\/1B0MP_jBezE<\/a><\/p>\n<p>Claudia Hauff, Delft University of Technology (TU Delft), The Netherlands. &#8220;SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments&#8221;. <a href=\"https:\/\/youtu.be\/9F1Y0H8OjaQ\">https:\/\/youtu.be\/9F1Y0H8OjaQ<\/a><\/p>\n<p>Pablo Castells, Universidad Aut\u00f3nonoma de Madrid, Spain. &#8220;Bias in recommendation: avoid it or embrace it?&#8221;. <a href=\"https:\/\/youtu.be\/EGTSbQWr1EA\">https:\/\/youtu.be\/EGTSbQWr1EA<\/a><\/p>\n<p>Carlos Castillo, Universitat Pompeu Fabra, Spain &#8220;Fairness in Ranking&#8221;. <a href=\"https:\/\/youtu.be\/hxkXNql78IA\">https:\/\/youtu.be\/hxkXNql78IA<\/a><\/p>\n<p>Debasis Ganguly, University of Glasgow, UK. &#8220;Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning&#8221;. <a href=\"https:\/\/youtu.be\/05v-FDQc-Tw\">https:\/\/youtu.be\/05v-FDQc-Tw<\/a><\/p>\n<p>Frank Hopfgartner, University of Sheffield, UK. &#8220;Promoting Algorithmic Transparency in Information Access&#8221;. <a href=\"https:\/\/youtu.be\/IUHsnjAUlZo\">https:\/\/youtu.be\/IUHsnjAUlZo<\/a><\/p>\n<p>Jean-Michel Renders, Naver Labs Europe, France. &#8220;Delivering Fair and Useful Rankings: a Generic Geometrical Projection Framework &#8221;<\/p>\n<p>Graham McDonald, University of Glasgow, UK. &#8220;TREC 2021 Fair Ranking Track: Task Overview &#8221;<\/p>\n<p>Amifa Raj, Boise State University, USA. &#8220;Comparing Fair Ranking Metrics&#8221;. <a href=\"https:\/\/youtu.be\/vwwHIUotpQs\">https:\/\/youtu.be\/vwwHIUotpQs<\/a><\/p>\n<p>Graham McDonald tells me that there were 384 registrations. Attendance varied through the day but each session had at least 135 people. He was not yet sure of the peak session numbers but it was probably about 150. That is quite an outstanding achievement and a strong indication of the level of interest in this subject. The technology and timing worked to perfection, and there was no shortage of questions and comments.<\/p>\n<p>If this area is still new to you I can recommend a very good preprint<\/p>\n<p><a href=\"https:\/\/arxiv.org\/abs\/2103.16953\">[2103.16953] Mitigating Bias in Algorithmic Systems: A Fish-Eye View of Problems and Solutions Across Domains (arxiv.org)<\/a><\/p>\n<p>The lead authors are at <a href=\"http:\/\/www.cycat.io\/\">The Cyprus Center for Algorithmic Transparency (CyCAT)<\/a>which \u00a0is hosted at the Open University of Cyprus, the second public university in Cyprus and the only one dedicated to open and distance education. There is a link between the Centre and the Information School at the University of Sheffield, and the work of the Centre was the subject of Frank Hopfgartner\u2019s presentation.<\/p>\n<p>My take-away from the papers I was able to listen to (it conflicted with a major project meeting) was that the topic was so much more complex than I had imaged. Moreover we are only just beginning to identify the issues, much less come up with solutions and are still miles away from validating the solutions with search users.<\/p>\n<p>If I had to give a best paper award (accepting that Maarten de Rijke\u2019s was outstanding) I would give it to Tim Gollins from the National Records of Scotland, for reminding us that fairness and transparency are not just challenges to the management and discovery of digital records but to the collection of documents and artifacts from <a href=\"https:\/\/www.scotlandspeople.gov.uk\/what-records-are-in-the-site\">1127 to the present day<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>2020 was a notable year in information retrieval research and development. First there was the realization that AI applications were going to need a level of transparency that would reassure customers that they were supporting, not controlling, the search process. Second there was the realization that the search community had a responsibility to think beyond&hellip; <a class=\"more-link\" href=\"https:\/\/archive-irsg.bcs.org\/informer\/?p=6940\">Continue reading <span class=\"screen-reader-text\">Fairness and Bias in Information Retrieval Glasgow 23 March 2021<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[195,215],"tags":[],"class_list":["post-6940","post","type-post","status-publish","format-standard","hentry","category-conference-review","category-spring-2021","entry"],"_links":{"self":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts\/6940","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6940"}],"version-history":[{"count":0,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts\/6940\/revisions"}],"wp:attachment":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6940"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}