Already hinted at in one of my previous book reviews, this year finally saw the publication of the long-awaited book “Evaluating Information Retrieval and Access Tasks: NTCIR’s Legacy of Research Impact”. If this isn’t reason enough to stop everything you are doing right now and start searching for your reading glasses, there’s even more good news: Thanks to the financial contribution of the National Institute of Informatics (NII) of Japan, the book is an open access publication and can be downloaded directly from the publisher’s web portal.
But first things first… What is NTCIR and why should you take note of this book?
NTCIR (nowadays) stands for NII Testbeds and Community for Information access Research and is led by Prof. Noriko Kando of NII. Similar to TREC, the NTCIR project aims to promote research on information access by releasing shared test collections and by organising evaluation tasks. Results are presented every 18 months at the NTCIR conference, a set of evaluation workshops that are hosted at NII in Tokyo. This year, the 15th NTCIR conference will take place and thanks to COVID-19, it is very likely that it is going to be an online event.
Over the years, the numerous NTCIR evaluation tasks have led to several advances in the field of information access. Examples include contributions to graded relevance assessments, cross-language IR using Asian-language corpora, patent retrieval, or lifelogging. Given this, you can rest assured that a book that claims to cover ‘NTCIR’s legacy of research impact’ has a lot to offer for IR enthusiasts like you and me.
The book editors are Tetsuya Sakai, Douglas W. Oard, and Noriko Kando, three renowned academics who have played a significant role in turning NTCIR into the success story that it is today. For this book, they have opted for a transparent and engaging approach to guarantee well-written summaries of the main research contributions of NTCIR. After inviting the organisers of key evaluation tasks to contribute a chapter to the book, their first chapter drafts were made available for download. Attendees of the last NTCIR conference were then invited to give feedback and the drafts were revised accordingly. This has led to thirteen peer-reviewed chapters that provide a good overview of the main contributions of NTCIR tasks spanning a period of over two decades. In the final chapter, Doug Oard shares his thoughts on what he considers to be the future of IR evaluation.
Since the book is available open access, I am not going to provide a more detailed summary on the different chapters. Instead, I would like to motivate you to directly go to the source and read it yourself. Finally, I also would like to thank Tetsuya, Doug, and in particular Noriko for their devotion and enthusiasm in running NTCIR over all these years.
Frank Hopfgartner, Information School, University of Sheffield. Book Review Editor for Informer