The July issue of SIGIR Forum is now available, and you do not have to be a member of ACM to read it. SIGIR Forum is published twice a year and usually has good summaries of recent ACM meetings. This issue contains, amongst other items, a report on the CHIIR 2019 Second Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2019) which is well worth reading.
I was struck by comments in the closing section of the report.
“One of the perhaps surprising outcomes of the breakout discussions was that the groups spent much of their time developing a shared understanding among group members of the issues relevant to the topic under discussion in terms of personalisation. The groups all engaged energetically in their discussions and there is no doubt that all of them progressed in development of the topic under discussion. However, there was little detail reported by way of discussion on the actual topic of evaluation by each group, except for group actually tasked to look at evaluation issues. This is perhaps indicative that while many researchers are interested in personalisation in IR, there has been relatively little discussion of this topic by the broader community, and it makes sense that they should be clear on the relevant issues within personalisation as a topic, before they can think in detail about issues relating to evaluation. Vibrant and productive as they were, the main conclusion of these discussions in terms of evaluation, would appear to be that more discussion is needed.”