The opening presentation at Search Solutions 2021 in November given by Professor Katriina Byström (Oslo Metropolitan University) in which she summarised the outcomes of research she had carried out with Professor Marianne Lykke, Ann Bygholm and Louise Bak Søndergaard ( Department of Communications and Psychology at Aalborg University) on enterprise search use across an organisation. The paper was published in Journal of Documentation in late December under the title of ‘The role of historical and contextual knowledge in enterprise search’. The title is accurate but gives no hint of the scope, quality and implications of the outcomes of the project. There is an open access version on the Aalborg University site.
In my opinion it is one of the landmark papers in enterprise search, not just for the outcomes themselves but for the implications for enterprise search research and management. The extended analysis of the study results is outstanding but I would like to add some of my own comments from a practitioner/consultant perspective.
The case study
The study was carried out in 2016-2018 at an international biotechnology company with 7500 employees. Search log analysis was carried out on 5854 active users over a 4-month period, and this work was then followed up by a survey (gaining 98 user responses) and interviews with eight frequent users and a demonstration of 19 search tasks. The search application was SharePoint 2016 and my assumption is that it was running in Classic mode as the Modern mode was only just emerging at this time. The number and percentage of active searchers for five organisational areas (Business, R&D, Top Management, Production and Administration) is presented along with an analysis of three groups of searchers, Highly Frequent, Frequent and Infrequent. An analysis was also made of the types of search conducted by the three groups, with People Search, Quality, IT, HR, Product, Finance, Facility Services, Sourcing and Intranet as the most frequent search topics.
Primary outcomes
Summarising a 21 page paper is not sensible! I will however highlight some of the outcomes that I found especially interesting. To quote directly from the paper, the most frequent subject category was people search, covering 59.5% of the search queries. In the people search category the searchers searched with search terms such as initials, names, department, phone number, email, job description. Search with initials accounted for 83.7% of people searches and search for names 11.6%, department 2.1% and phone numbers 1.9%. The other types of searches related to people searches were all below 1%, i.e. job title, job description, or email. The second most frequent search topics of much less frequency included categories such as: Quality, IT, HR, Products, Finance, Facility Service, Sourcing, and Intranet. In Quality the searchers searched for i.e. quality standard names, GLP [good laboratory practice], in the subject category IT they searched for database names, IT concepts, security, in the HR category for benefits, insurance, employee handbook, in Product for product names and product codes, in Finance for payroll, credit card, SAP, VAT, and in Sourcing for vendor and supplier names.
Does it scale?
What is astounding about this research study is that it is the first ever to go into this level of detail across an entire organisation. The literature review is comprehensive and it is interesting to note that only 23 of the 46 papers cited have been published in the last 10 years, and with the exception of the work of Paul Cleverley and Simon Burnett none have looked across an entire organisation, focusing instead on specific roles. This highlights the academic invisibility of enterprise search even though there are perhaps 300 million users of enterprise search applications. The wealth of qualitative and quantitative results from the study is analysed in considerable depth around the context and working habits of the employees, in line with the title of the paper. One of the challenges of academic research is the extent to which outcomes scale beyond a specific case study. In my opinion the outcomes are reasonably scalable, if only because they match fairly closely the outcomes of 50+ search projects I have undertaken over the last 15 years! A biotechnology company is operating in a fast-moving technology sector in which compliance is a very important aspect of process management, and that is quite a common situation in 2022.
The importance of people search
The research very clearly indicates the importance of supporting people search in the enterprise. The outcomes are very similar to work carried out by Ido Guy and his colleagues at IBM in 2011 (https://doi.org/10.1145/2207676.2208310) which is not cited in the paper and again matches my own experience.
People search has two different purposes that sometimes coexist. Names are used to find specific persons, but also to mediate search as keywords to find other information and to decide paths to follow as part of the tracing search. The interviewees explained how they search for a person because this person can point to another person, a project, some possible (forgotten) search terms, or because they can remember that this person is part of a project report description and therefore the search engine will find the name – and the desired document.
It is however important to note how difficult it is to search for people by name, as is demonstrated by the technical specification for the Rosetta application developed by Basis Technology. Few commercial applications, and certainly not Microsoft O365, offer this level of functionality. It should also be noted that SharePoint/O365 does include employee profiles. In many enterprises the definitive list is held within the HR application, and only includes the ‘official’ name of the employee, a particular issue with names that are not written in European scripts. Looking at click logs is not helpful in recognizing the value of people search because each individual person is so far along the frequency axis that they become invisible.
The role of artificial intelligence in enterprise search
At present enterprise search vendors (over 70 of them) promote the benefits of using their AI/ML technology stack to improve search, highlighting the way in which the software can semantically mine the text of documents. But when you look at the uses being made of enterprise search only a minority would seem to benefit from textual analysis. There is certainly an important role for named entity extraction to be able to present product names and product codes (as two examples) but there is limited evidence for employees undertaking the type of large-scale document analysis where AI/ML could make a significant difference to search performance.
The value of enterprise search research
Enterprise search is different in so many ways from all other search applications. These challenges have not changed in principle since my book Enterprise Search was published in 2015, David Hawking’s review excellent 2005 analysis [download) and indeed the famous (or infamous, depending on your perspective!) assessment of the IBM STAIRS software in 1984 by Blair and Maron. [download].
- Enterprise content is not curated and there is no incentive for authors to write a document to be discoverable.
- Enterprise search is security trimmed and so the range of content available to an individual employee is unlikely to be all that is held in the search index.
- Employees have multiple roles and these roles can change at very short notice, so personalization based on prior search history is very suspect in its relevance.
- Many of the documents that appear in the first few search pages as being ranked highly relevant may not clicked and viewed because the employee already has these documents/information as a core component of their roles and responsibilities. This then distorts the analysis of query click frequency.
- Above all, when a search fails it can have a significant impact on the organisation and on the reputation and capability of the employee, who may end up making a poor decision just because of a failure of the search technology.
In my opinion if there was more research into enterprise search behaviours then search managers would be able to optimize the performance of their applications and drive corporate innovation at a time when the global economic growth needs to be stimulated.
The reasons for this lack of research given by academics are usually attributed to overcoming issues relating to GDPR and to business confidentiality. This paper shows that these barriers can be addressed, and I would point to an information management case study at Orange (the French telecommunications company) to show what can be achieved. I hope (but sadly not anticipate) that other research projects will now start to emerge.
I look forward to further insights from this very skillful and knowledgeable research group.
Martin White