Information retrieval for children: Search Behavior and Solutions

I have been awarded the degree of Phd (Cum Laude) after successfully defending my PhD thesis at the University of Twente on February 14, 2014.

The first contribution of this thesis provides a characterization, on a large scale, of the search behavior of young users. The problems they face when they search for information on the web, the topics they searched and the online activities that motivate search were explored in detail and contrasted against the search behavior of adult users. The results presented in this thesis have important implications for the development of search tools for young users and for the design of educational literacy. Two central problems were identified in the search process of young users: (1) difficulty representing the information needs with keyword queries, and (2) difficulty exploring the list of results.

We found that focused queries are often required to access high quality content for young user with modern search engines. However, young users were found to submit queries that lack the specificity needed to retrieve content that is suitable for them, which leads to frustration during the search process. This observation motivates the second contribution of this thesis. We propose novel query recommendation methods to improve the chances of young users to find content that is suitable and on topic. Concretely, we present an effective biased random walk based on informa- tion gain metrics. This method is combined with topical and specialized features designed for the information domain of young users. We show that our query suggestions outperform by a larger margin not only related query recommendation methods but also the query suggestions offered by the search services available today.

In respect to the second difficulty, it was found that young users have a strong click bias, in which results ranked at the bottom of the result list are rarely clicked. This behavior greatly hampers their navigational skills and exploration of results. It also reduces the chances of young users to find suitable information, since appropriate content for this audience is ranked, on average, at lower positions in the result list in comparison to the content aimed at the average web user.

The third contribution of this thesis aims at helping young users to im- prove their chances to find appropriate content and to ease the exploration of results. For this purpose, we envisage an aggregated search system in which parents, teachers and young users add search services with con- tent of interests for young audiences. We propose a test collection with a wide number of verticals with moderated content, a carefully selected set of search queries and vertical relevant judgments. We also provide novel methods of vertical selection in this information domain based on social media and based on the estimation of the amount of content that is appropriate for young users in each vertical. We show that our methods outperform state-of-the-art vertical selection methods in this information domain. We also show in a case study with children aged 9 to 10 years old that result pages derived from the collection proposed are preferred over the result pages provided by modern search engines. We provide evidence showing that the interaction and exploration of results are improved with result pages built using this collection, even if the users of this case study were unaware between the differences between the types of pages displayed to them.

This thesis is concluded by providing concrete follow-up research directions and by suggesting other information domains that can potentially benefit from the methods proposed in the thesis. My thesis is available online here.

Analysis of Search and Browsing Behavior of Young Users on the Web

by Sergio Duarte Torres, Ingmar Weber and Djoerd Hiemstra 

In this journal paper we expanded the study presented in paper What and How Children Search on the Web in two directions. Firstly, We provide a more detailed analysis of the topics that are searched by children on a state-of-the-art search engine by using novel classification based on fine-grained topics derived from the categories of the Yahoo! Answers service. The findings obtained through this analysis allow us to provide concrete recommendations for the development of modern IR systems for young users in specific age ranges.

Secondly, we employed toolbar logs from the Yahoo! search engine to characterize the browsing behavior of young users, particularly to understand the activities on the Internet that trigger search. We quantified the proportion of browsing and search activity in the toolbar sessions and we estimated the likelihood of a user to carry out search on the Web vertical and multimedia verticals (i.e.\ videos and images) given that the previous event is another search event or a browsing event. We found that certain group of young users are more likely to carried out multimedia search and that certain browsing events are more likely to trigger web search, such as knowledge related websites (e.g. Wikipedia).

Published at TWEB ACM, March 2014, Volume 8 Issue 2. Read the paper.

Query Recommendation in the Domain of Information for Children

by Sergio Duarte Torres, Djoerd Hiemstra, Ingmar Weber, Pavel Serdyukov. 

Children represent an increasing part of web users. One of the key problems that hamper their search experience is their limited vocabulary, their difficulty to use the right keywords, and the inappropriateness of general-purpose query suggestions. In this journal paper, we expanded the biased random walk introduced in our paper Query recommendation for Children by combining the score of the random walk with topical and language modeling features to emphasize even more the child-related aspects of the query suggestions.

We evaluate our methods using a large query log sample of queries submitted by children (from the Yahoo! Search logs). We show that our method outperforms by a large margin the query suggestions of modern search engines and state-of-the art query suggestions based on random walks.

Published at JASIST, February 2014. Read the paper.

Vertical Selection in the Information Domain of Children

by Sergio Duarte Torres, Djoerd Hiemstra and Theo Huibers 

In this paper we explore the vertical selection methods in aggregated search in the specific domain of topics for children between 7 and 12 years old. A test collection consisting of 25 verticals, 3.8K queries and relevant assessments for a large sample of these queries mapping relevant verticals to queries was built. We gather relevant assessment by envisaging two aggregated search systems: one in which the Web vertical is always displayed and in which each vertical is assessed independently from the web vertical. We show that both approaches lead to a di?erent set of relevant verticals and that the former is prone to bias of visually oriented verticals. In the second part of this paper we estimate the size of the verticals for the target domain. We show that employing the global size and domain specific size estimation of the verticals lead to significant improvements when using state-of-the art methods of vertical selection. We also introduce a novel vertical and query representation based on tags from social media and we show that its use lead to significant performance gains. Read the paper

This paper has been nominated for the best student paper award at JCDL 2013.

What and how children search on the web

by Sergio Duarte Torres and Ingmar Weber 

The Internet has become an important part of the daily life of children as a source of information and leisure activities. Nonetheless, given that most of the content available on the web is aimed at the general public, children are constantly exposed to inappropriate content, either because the language goes beyond their reading skills, their attention span differs from grown-ups or simple because the content is not targeted at children as is the case of ads and adult content. In this work we employed a large query log sample from a commercial web search engine to identify the struggles and search behavior of children of the age of 6 to young adults of the age of 18. Concretely we hypothesized that the large and complex volume of information to which children are exposed leads to ill-defined searches and to disorientation during the search process. For this purpose, we quantified their search difficulties based on query metrics (e.g. fraction of queries posed in natural language), session metrics (e.g. fraction of abandoned sessions) and click activity (e.g. fraction of ad clicks). We also used the search logs to retrace stages of child development. Concretely we looked for changes in the user interests (e.g. distribution of topics searched), language development (e.g. readability of the content accessed) and cognitive development (e.g. sentiment expressed in the queries) among children and adults. We observed that these metrics clearly demonstrate an increased level of confusion and unsuccessful search sessions among children. We also found a clear relation between the reading level of the clicked pages and the demographics characteristics of the users such as age and average educational attainment of the zone in which the user is located. Read the paper

Query recommendation for children

by Sergio Duarte Torres, Djoerd Hiemstra, Ingmar Weber, Pavel Serdyukov. 

One of the biggest problems that children experience while searching the web occurs during the query formulation process. Children have been found to struggle formulating queries based on keywords given their limited vocabulary and their difficulty to choose the right keywords.
In this work we propose a method that utilizes tags from social media to suggest queries related to children topics. Concretely we propose a simple yet effective approach to bias a random walk defined on a bipartite graph of web resources and tags through keywords that are more commonly used to describe resources for children. We evaluate our method using a large query log sample of queries aimed at retrieving information for children. We show that our method outperforms query suggestions of state-of-the-art search engines and state-of-the art query suggestions based on random walks.
Read the paper.

Cross-lingual alignment and completion of Wikipedia templates

by Gosse Bouma, Sergio Duarte Torres and Zahurul Islam. 

For many languages, the size of Wikipedia is an order of magnitude smaller than the English Wikipedia. We present a method for cross-lingual alignment of template and infobox attributes in Wikipedia. The alignment is used to add and complete templates and infoboxes in one language with information derived from Wikipedia in another language. We show that alignment between English and Dutch Wikipedia is accurate and that the result can be used to expand the number of template attribute-value pairs in Dutch Wikipedia by 50%. Furthermore, the alignment provides valuable information for normalization of template and attribute names and can be used to detect potential inconsistencies. Read the paper.

EmSe: initial evaluation of a child-friendly medical search system

by The PuppyIR team.

When undergoing medical treatment in combination with extended stays in hospitals, children have been frequently found to develop an interest in their condition and the course of treatment. A natural means of searching for related information would be to use a web search engine. The medical domain, however, imposes several key challenges on young and inexperienced searchers, such as difficult terminology, potentially frightening topics or non-objective information offered by lobbyists or pharmaceutical companies. To address these problems, we present the design and usability study of EmSe, a search service for children in a hospital environment. Read the paper.

Web search query assistance functionality for young audiences

by Carsten Eickhoff, Tamara Polajnar, Karl Gyllstrom, Sergio Duarte Torres and Richard Glassey.

The Internet plays an important role in people’s daily lives. This is not only true for adults, but also holds for children; however, current web search engines are designed with adult users and their cognitive abilities in mind. Consequently, children face considerable barriers when using these information systems. In this work, we demonstrate the use of query assistance and search moderation techniques as well as appropriate interface design to overcome or mitigate these challenges. Read the paper.