Searching as learning: A systematization based on literature

Search as Learning Special Issue Searching as learning: A systematization based on literature Journal of Information Science 2016, Vol. 42(1) 7 18 Ó The Author(s) 2016 Reprints and permissions: sagepub.co.uk/journalspermissions.nav DOI: 10.1177/0165551515615833 jis.sagepub.com Pertti Vakkari School of Information Sciences, University of Tampere, Finland Abstract The paper surveys empirical studies on the relations between information searching and learning, and presents some reflections about learning in a search process based on the findings. First, the meaning of the concepts learning and searching is briefly defined. Learning is conceptualized as changes in one s knowledge structures. Then it is described more in detail how learning occurs in the search process. The point of departure is to focus on tasks that require the restructuring of knowledge structures and to analyse how gradual stabilization of those structures is related to accessing and interacting with information sources. After that, empirical studies on searching and learning are categorized by identifying independent and dependent variables in those studies. In conclusion, some general remarks on the topic are presented. Keywords Criteria of success; information searching; information retrieval; knowledge structure; learning; research type; search outcome 1. Introduction Some research in information science either conceptualizes searching as learning or otherwise explores links between searching and learning. Most of the studies belong to the latter category. These studies do not explicitly use the word learning, although implicitly they deal with this phenomenon. The major framework in information science conceptualizing searching as learning is Kuhlthau s information search process (ISP) model [1]. Although she speaks about the information search process, the model deals with learning tasks, and it can be generalized to a certain extent to all tasks requiring learning. According to Kuhlthau [1], the bibliographic paradigm describes the task of information seeking primarily as the gathering and collecting of information, rather than a series of tasks within a constructive (i.e. learning) process. Kuhlthau [1] differentiates between two major aspects of information seeking, (a) access to sources and (b) guidance in the construction process. Both aspects are interrelated and users need intervention support for both of them. Source-related interventions assist with access to information, whereas process-related interventions assist with learning through the use of information. In information science most of the studies relating to searching and learning belong to the bibliographic paradigm, because they focus on learning when accessing, but not when using information in the sources found. However, there is a growing interest in exploring also how users select, structure, manipulate and combine information in documents accessed for creating the outcome of a search [2, 3]. Designing and evaluating tools for interacting with information accessed is also gaining popularity [4]. Despite this interest, there is a lack of overviews surveying studies relating to searching and leaning. The aim of this essay is to characterize the concepts learning and searching, and in the light of these notions describe how learning occurs in the search processes. Based on this analysis the major types of studies about the relation between searching and learning are introduced. The article begins by defining the notions of learning and searching. Based on these definitions, it is characterized how learning occurs in the search process. It is suggested how various types of learning are associated with differing Corresponding author: Pertti Vakkari, School of Information Sciences, University of Tampere, Kalevantie 4, Tampere, FIN-33014, Finland. Email: Pertti.Vakkari@uta.fi

Vakkari 8 stages in the search process. After that the major types of research on the relations between searching and learning are introduced by categorizing the independent and dependent variables in the studies. The conclusion includes implications for researchers wishing to analyse searching as learning. 2. Learning Learning is the act of gaining new or modifying and reinforcing existing knowledge (skill, behaviour, values), by studying, practising, being taught or experiencing something; or knowledge or skill gained from learning [5]. Learning theories can be categorized into three major groups, behavioural, cognitive and constructivist approaches. The respective conceptualizations are learning as response to strengthening, learning as information processing and learning as knowledge construction. More information about various notions of learning can be found in Ormrod [6]. The conception of learning selected for this study includes mainly gaining new or modifying and reinforcing existing knowledge. By knowledge is meant the totality what a person knows, that is, a personal knowledge or belief system. It may include both justified, true beliefs and less justified, not so true beliefs, which the person more or less thinks hold true [7]. Information is stored to some medium and communicated between two recipients. Information is transformed into knowledge in one s mind when it is processed and integrated into one s knowledge structures [7]. Studying information searching, treating learning as gaining knowledge, and perhaps as gaining skills, is more fertile than, for example, observing changes in values or behaviour in general. This idealization reduces the complexity of the phenomena of interest, and consequently, of analysis. As the definition above shows, learning occurs in many ways, not only by studying or by being taught, but also by experiencing something, like by doing. This definition includes both intentional and unintended learning. Learning occurs in all sectors of life, not only in education or schooling. All activities that are performed without sufficient knowledge (skills) require the acquisition of knowledge and learning to be performed. Naturally, the extent to which actors need new knowledge varies a lot depending on the task and their knowledge of the task. A novice securities analyst analysing how an industry is structured in a country needs to acquire much more information and process it to produce a report for a customer compared with an expert who is updating her earlier report on the same matter. Typically, persons seek to accomplish tasks as much as possible based on their prior knowledge to minimize the effort required. However, if actors are not sufficiently familiar with work tasks, hobbies or daily activities that they wish to perform, they have to learn what is necessary to realize these activities. This may happen by trial and error, by reading or by asking somebody, but all these means include learning as a component. The extent of this learning component varies from acquiring facts to restructuring one s knowledge concerning the task to be performed. After the performance, the person knows more about the object of the activity performed. In ordinary language use people do not typically speak about learning, when they accomplish work tasks or everyday activities. If a lawyer acquires information to familiarize herself with a case, or if a journalist surveys the background of an event for a feature article, they probably think that they do their job, not that they learn for accomplishing their job. Learning is embedded in the activity performed. In several learning theories human knowledge is described as knowledge structures, which consist of concepts and their relations representing some phenomenon [7]. These structures can be called, for example, mental models or schemas. If we use this definition of knowledge, then learning means gaining new or modifying and reinforcing existing knowledge structures. Naturally, one needs information inputs for changing or reinforcing current knowledge structures, that is, for learning. Typically, this implies working with information acquired to adjust it to existing or adapted knowledge structures. This means that the development of new knowledge (learning) is based on prior knowledge. We cannot pass what we know, when we seek to learn new things. The extent of learning varies from learning a fact to radically modifying one s knowledge about a phenomenon. In learning theories cognitive changes have been typically categorized as assimilation and accommodation [2]. Assimilation means addition of information into existing knowledge structures, whereas accommodation means modifying or changing existing knowledge structures. The latter refers to adding or removing concepts and their relations in the knowledge structure. The former means that the conceptual construct the knowledge structure consists of does not change, but merely new instances are added in to concepts. Some researchers have specified this distinction by distinguishing between three types of changes in knowledge structures [2]. Accretion refers to the gradual addition of factual information within existing knowledge structure without structural changes. Accretion does not change concepts and their relations in the structure, but may populate a concept with new instances or facts. Accretion is thus a synonym to assimilation. Accommodation is refined according to the degree of structural change [2]. Tuning or weak revision does not include replacing concepts or relations between concepts in the structure, but merely tuning of the scope and meaning of concepts

Vakkari 9 and their relations. This may include, for example, generalizing or specifying a concept. Restructuring means changing and replacing concepts and their relations in the knowledge structure. Understanding learning as conceptual change allows us to conceptualize knowledge structures as consisting of concepts and their relations. This notion matches the cognitive view of information retrieval, which conceptualizes information as knowledge structures both in humans and in documents [8, 9]. It matches also theoretical ideas concerning the dimensions of indexing and querying and their relations to retrieval effectiveness, for example, precision and recall [10], and with ideas and empirical results in the psychology of learning. This conception connects in a fertile way research traditions in information science and pedagogics. 3. Information searching There are two basic notions of information searching. The bibliographic paradigm restricts it to accessing information,that is, to querying and evaluating the search results produced by the query. The constructive, that is, learning notion extends it to include also the use of information in the documents accessed [1]. In the following, the latter is selected as the point of departure, because it represents more versatile learning. In describing the search process, it is typical to distinguish expressing information need, formulating query (selecting search terms and querying) and assessing search results (selecting sources). Belkin [11] and Ingwersen [9] sought in the 1990s to extend information searching to also include interacting with texts retrieved, that is, information use. Interaction with texts changes users understanding, and consequently the information need reflected in the search formulation. Currently, the use of information accessed is also more commonly included in the representation of the search process [12]. The latter can be divided into interacting with sources or information and synthesizing and presenting (Figure 1). Expressing the information need for search formulation and executing the query is the starting point of a search process [13]. Search formulation includes the articulation of search terms and the use of search operators if there are any. It also includes the search tactics used for reaching the search goal. The choice of search terms is a reflection of the searcher s understanding of her information need, what is insufficient in her knowledge structure. In practice, a searcher expresses her information need by selecting search terms and search tactics, that is, by formulating and reformulating queries. The learning process required for selecting terms and tactics depends at least on the searcher s level of knowledge on the search topic and system, and it is influenced also by the information provided in the search results [14] and the support tools incorporated in the search interface [15, 16]. The searcher probably reflects and reforms her understanding of the topic, and consequently on terms for querying. In selecting sources the searcher explores the document surrogates in the result list or full documents to assess the value (relevance, utility) of the documents found and to satisfy her information need, that is, to reduce the insufficiency of her knowledge. In particular, various document information elements are used for inferring the value of a source [17]. The aim of this interaction is to select sources, which probably contribute to the activity triggering the search so that they are available for immediate or later use. Interacting with sources is the phase when the searcher familiarizes herself with the sources to realize the task for which they were selected. It may include reading, making notes, extracting pieces of information, combining and organizing information and outlining [1]. The aim of this activity is to analyse, categorize and structure information in documents for preparing the final outcome of the task. The borderline to synthetizing and presenting information for task outcome is blurred, because it is impossible to determine exactly when, for example, combining and outlining information by writing for the task outcome becomes synthetizing or presenting. It is likely that, in a complex task that the actor is not familiar with, the characteristics of interacting with sources can be equated in Kuhltau s ISP model with the stages prior to information collection [1]. In any case, synthetizing and presenting include preparing the final outcome of the task. The form of the outcome depends naturally on the nature of the task. It can be a financing decision, a newspaper article or a greater understanding of some societal issue. Figure 1. Information search process.

Vakkari 10 4. Learning in the search process After explicating the notions of both oearning and information searching, it is possible to analyse when and how learning occurs in the search process. The aim is to analyse learning at various stages of the search process. As mentioned, the extent of learning varies from accommodation to assimilation, that is, from learning facts to the radical revision of knowledge structures [2]. The point of departure in this analysis is the revision of knowledge structures, which means that a person is not familiar with the topic the task generating searching deals with. We may also say that she has an anomalous state of knowledge concerning the task [18]. Accommodation is used as an example, because in the course of task performance the modified knowledge structure stabilizes and learning changes from revision to tuning and assimilation. This kind of learning is gradual and requires time to acquire and process information for accomplishing the task. Therefore information is sought over several search sessions. This process resembles Kuhlthau s ISP model [1]. The notion of exploratory search can also be applied here, because the aim of searching is supporting learning or investigative activities [12]. In the beginning of the search a person seeks to articulate her information need into search terms. Her understanding of the topic is vague. Her knowledge structure is insufficient for expressing exactly what she wants to know. It lacks sufficient concepts and their relations for representing the topic in order to proceed in the task. Therefore, she is searching for how to structure the topic, actually conceptual structures [19, 20]. In this kind of situation there are seldom readymade conceptual structures available, but the person has to create them incrementally. There is typically not a single right solution, but many options available for how to proceed and how to specify the structure. It is likely, that neither during the initial search formulation stage nor during the initial search session, a person s conceptual structure changes radically [21]. This would require exploration of the sources selected and work with the information found. The process of making sense of the search results found is typically so demanding that most of the searcher s intellectual capacity goes to identifying sources that seem useful, leaving hardly any capacity for revealing their contribution. It is likely that in the search formulation stage the person seeks to explore the conceptual space in documents related to her own conceptual structure of the topic. This usually starts by articulating first her own conceptual understanding as a query, and then extending it by additional concepts or specifying it by reformulating the query. The exploration of the conceptual space often occurs by using vary tactics, that is, replacing existing concepts with new ones in a query [14, 22]. One may be interested, for example, in the relations of searching and learning, and explore this conceptual space first by adding to the query searching, learning new concepts like support tools and then replacing this new concept by concept evaluation. Exploring by varying tactics may reveal new concepts and conceptual relations that facilitate restructuring the searcher s knowledge structure. New concepts are potentially found when the person explores document surrogates in the result list and full documents [23]. Titles, abstracts and text passages browsed include possible ideas for restructuring her conceptual understanding of the topic. Restructuring refers to changes in concepts and in relations between concepts, in particular. An essential condition for restructuring is changes in relations between concepts. Meaning typically depends on the relations between concepts [7]. Although concepts may be identical, that is, the structure is conceptually similar, differences in their relations, that is, in factual similarity, produce differences in the meaning [24]. Relating existing and new concepts in a new way, that is, restructuring one s knowledge on a topic, is mostly a laborious and time-consuming process, which probably does not occur during search sessions, although there may be sudden insights in front of search results. Therefore, it is more typical to recognize new concepts, that is, search terms, from search results than conceptual relations. These terms are used for reformulating queries. Studies also show that during learning tasks the number and specificity of search terms used tend to increase across the search sessions [21]. Growth of knowledge means growth in the number and specificity of concepts and their interrelations. When a person has revised her conceptual structure and formulated a focus for her topic according to the ISP model [1], it will include the major concepts and their relations, but probably require tuning to better fit with the aim of her task. Tuning typically includes limiting or broadening the scope of the concepts in the knowledge structure or its applicability area. In search formulation this means using either more specific or more general search terms to present the concepts in queries. It is likely that, in tuning, factual information is also sought to support the arguments developed. Tuning leads to assimilation, that is, to the population of concepts with new instances or facts. When the conceptual structure is stable, synonyms are typically used in query formulation to represent concepts. In source selection the searcher s behaviour probably changes by changing learning. When revising knowledge structure, a person s conceptual construct about the topic is vague. Therefore, she has difficulty in recognizing useful sources, leading to loose relevance criteria [1, 25 27]. Many of the sources may be useful, but she is not able to tell which ones, because she has as yet no firm ideas on how to shape her topic. In this phase, in addition to useful sources, it is typical to

Vakkari 11 Table 1. Characteristics of search process by the change in knowledge structures Search stage Modification of knowledge structures Restructuring Tuning Assimilation Search formulation Few general terms Increase in the number and specificity of terms Increase in number and specificity of terms Many new terms from results Increase in the number of terms Fewer new terms from results with associative relations (facets) Varying tactics Less reformulation Less reformulation Much reformulation A few synonyms Synonyms Synonyms Long search sessions Shorter search sessions Short search sessions Source selection Vague relevance criteria Clearer relevance criteria Clear relevance criteria Number of result pages viewed is large Number of result pages viewed decreases Number of result pages viewed is small Number of sources selected is large Number of sources selected decreases Number of sources selected is small Share of selected sources of consulted sources is large Share of selected sources of consulted sources decreases Share of selected sources of consulted sources is small Share of probably relevant sources great of all sources selected Share of probably relevant sources decreases, and that of relevant sources increases Share of probably relevant sources is small, and that of relevant sources is large Interacting with sources Proportion of general background information and theoretical information is large Most time is used for assessing probably relevant sources A small share of consulted sources is used in outcome Background and conceptual information are utilized from sources Notes are taken on themes and ideas Ideas are related and combined for a focus Proportion of procedural, specific and factual information increases Increasing share of consulted sources used in outcome Procedural and specific information utilized from sources Identification of information to support and refine focus Proportion of specific and factual information increases A large share of consulted sources is used in outcome Factual and specific information utilized from sources Refining the output with factual information Rechecking sources for information initially overlooked select also sources that may be of value, but the value is not known yet. This becomes known when the sources have been explored more in detail. When tuning a knowledge structure, a person s conceptual structure is stabilized, and she is able to select documents, applying strict relevance criteria [1, 27]. She seeks to select highly useful (pertinent) documents, which seem to make a clear contribution to her task. It is likely that the absolute number of documents selected is lower in tuning compared with restructuring, but the proportion of documents used out of documents selected is greater in tuning [28]. This trend is strengthened when moving from tuning to assimilation. It is likely that the type of information used also varies by the types of learning. In restructuring knowledge, general background information, definitions of concepts and models and theoretical ideas concerning the potential topic benefit users [29]. These information types will help to form an overview of the topic and provide various conceptualizations and categorizations of it. This will help in creating and revising the user s own conceptual structure of the topic. It is likely that in restructuring people also need procedural and method information on how to proceed in the knowledge construction process [29]. Facts and specific information are likely to be more useful when one has moved from restructuring to tuning the conceptual structure, and further to assimilation. These can be applied to support arguments related to the modified structure [29]. Interacting with sources is the most crucial phase for leaning [17, 30]. Users familiarize themselves with the sources selected first with the aim of revising their understanding concerning the topic. Some of the sources are weeded out as not contributing. The rest are read for making notes of the central themes and concepts, for summarizing major points and for comparing and combining central concepts and themes. Gradually users compare and relate concepts in sources with their own conceptual understanding for restructuring their knowledge of the topic. After restructuring they are able to tune the structure and support it by factual information, and finally to create a presentation of the outcome.

Vakkari 12 Table 2. Criteria of leaning across sessions for restructuring knowledge structures Search stage Search formulation Source selection Interaction with sources Criteria of learning (and search success) Increase in number and specificity of terms Increase in number of terms with associative relations (facets) Increase in number of synonyms Decrease in number of reformulated queries Decrease in variability of tactics Decreased time use per search sessions Increased clarity in relevance criteria = increased ability to distinguish between relevant and nonrelevant sources Decrease in number of sources viewed in result list The proportion of sources selected of sources viewed decreases (greater decrease in precision, increase in CG) The number of sources selected decreases The share of probably relevant sources decreases, and that of relevant sources increases for all sources selected The proportion of general background information and theoretical information decreases The proportion of specific and factual information increases The proportion of procedural information is inversely U-shaped in the search process Average time used for assessing a source decreases Increasing share of sources viewed and selected used in outcome Increase in number and specificity of concepts and their interrelations in the knowledge structure Decrease in the proportion of general background and theoretical information from sources used for outcome Increase in the proportion of specific and factual information utilized from sources for outcome The proportion of procedural information used for outcome is inversely U-shaped in the learning process Based on the argumentation above, Table 1 suggests how various types of learning are associated with the stages in the search process. During the learning process, search formulations change from simple queries including a few general terms to queries consisting of several, more specific terms and also synonyms. When actors learn, their ability to structure the topic increases, leading to fewer reformulations and varying tactics, and queries with more facets of the topic [2, 14, 21, 31]. During this process, actors relevance criteria change from vague to clear, reducing the time, effort and the number of items inspected in result pages. Among the sources selected, the proportion of pertinent documents increases and that of possible relevant documents decreases [17, 25 27]. Also the type of information needed and used changes during the information search process in task performance [29, 32]. An increasing share of consulted sources is used in the outcome [28, 30]. Table 2 suggests how changes in the various stages of searching can be used as criteria of learning during the search process. These criteria are inferred from the information in Table 1. 5. Learning in search tasks In the previous analysis, the point of departure for searching was learning for a task that requires restructuring of one s knowledge. However, learning may occur also during search tasks. The aim of search tasks is to find and identify information or information sources, not to create new knowledge based on the sources found. These types of search tasks are typical in interactive information retrieval (IR) experiments. In the following I discuss learning in search tasks. Search tasks consist of search formulation and source selection, because they do not require interaction with sources to produce new knowledge. Learning may occur in recognizing new search terms, discarding bad ones and perhaps identifying new aspects of the search topic. Typically, it is expected that search tasks are accomplished in a single search session. Also a search topic is typically given. Therefore, it is likely that learning occurs in the form of tuning knowledge structures and identifying facts. This may include broadening or narrowing search terms or identifying new ones. It is also possible that inspecting the result list and document surrogates increases searchers understanding of the scope of the topic, increasing their ability to distinguish between relevant and non-relevant sources. Krathwohl s [33] revised Bloom s taxonomy of learning objectives can be used for characterizing learning requirements in search tasks [cf. 34, 35]. The taxonomy describes what students are expected to learn as a result of instruction.

Vakkari 13 The taxonomy consists of two dimensions: knowledge dimension and cognitive process dimension. The knowledge dimension includes (a) factual knowledge (knowledge of terminology and of specific details and elements), (b) conceptual knowledge (of interrelationships between elements within a larger structure), (c) procedural knowledge (how to do something) and (d) metacognitive knowledge. The cognitive process dimension consists of six categories: (a) remember (retrieving, recognizing and recalling relevant knowledge from memory); (b) understand (determining the meaning of messages through interpreting, exemplifying, classifying, summarizing etc.); (c) apply (carrying out a procedure in a given situation); (d) analyse (breaking material into its constituent parts and detecting how the parts relate to one another and to overall structure); (e) evaluate (making judgments based on criteria); and (f) create (putting elements together to form a novel, coherent whole). The complexity of cognitive processes increases along the process dimension [33]. It is expected that remembering is less complex than applying, which is less complex than creating. It is possible to apply the knowledge dimension of taxonomy to typical search tasks, which require querying to identify topically relevant information or sources. They consist of search formulation and result evaluation (source selection). It is evident that searchers need factual knowledge (knowledge of terminology and of specific details and elements) in selecting search terms and identifying relevant sources from result lists. They also apply procedural knowledge to realize the search process. Cognitive processes required for this kind of search task mainly consist of remembering, that is, recognizing search terms from search task and search results, and recalling them from one s own memory, and also recognizing relevant items from the result list. The latter may also include some understanding in determining the meaning, that is, the relevance of a particular item, or identifying various aspects of the topic. Bloom s revised taxonomy has also been used in a few studies on interactive information retrieval to design search tasks and analyse search interactions. Jansen and his colleagues [34] inferred six types of search tasks from the cognitive process dimension of the taxonomy. The tasks represented each of the learning categories in the taxonomy. They studied whether these search tasks differentiate the characteristics of Web searching, for example, the numbers of queries, unique terms and result pages viewed. The findings seemed to show that the relation between the cognitive complexity of tasks and the quantitative value of search indicators used resembled an inverted curve. The realization of search tasks at the middle level of cognitive learning required more effort in querying and result exploration compared with lower and higher levels of learning. The results are very tentative, because the validity of some search task types may be questionable owing to operationalization. Wu and colleagues [36] used the cognitive process dimension in Bloom s taxonomy [33] for creating search tasks. Their tasks included five of the cognitive learning categories except apply. The results of an experimental study indicated that search interaction increased as the level of cognitive learning increased. The indicators of interaction consisted of time on task, the number of queries issued, the number of results clicked and the number of URLs visited. 6. Types of research on searching as learning Next are presented different types of studies on relations between searching and learning by observing dependent and independent variables in the studies. In the first group of studies, various aspects of learning are treated as dependent variables, while IR process variables are considered as independent variables. In the second group, the characteristics of learning are treated as independent variables and elements in the IR process as dependent variables. The third group consists of studies characterizing search outcome, typically text, as an indicator of learning. Most of the studies that deal with learning in the search process do not conceptualize the phenomenon as learning. They observe changes in the search process, which can be interpreted as learning. 6.1. From searching to learning The most common type of study includes search formulation or the use of a search support tool as an independent factor and source selection as a dependent factor. The typical aim of this study type is to find out how search formulation or result list inspection behaviour is associated with search effectiveness. The most common independent variable used is the indicator based on the number of relevant documents found, not learning. However, a byproduct of these studies may be results concerning identifying keywords from search results or new aspects of search topic [37]. These findings are not conceptualized as learning although they include learning processes remembering and understanding. However, in an exceptional study, Moraveji and his colleagues [15] analysed how tactical search feature tips were associated with search performance. These tips expanded user awareness of the task-relevant tools and features of the search application. The group exposed to the tips performed the search tasks in a shorter time compared with the control group. This difference

Vakkari 14 remained about one week later in a follow-up study with no tips shown. Also the number of correct answers was larger in the experimental group. Xie and Cool [16] have identified 15 different help-seeking situations during search sessions. There are also studies that extend the scope of analysis from search formulation and source selection to cover also either interacting with sources or the characteristics of the outcome (end product) of the search like a text. Hersh and his colleagues [38] were among the first to empirically evaluate search systems from the viewpoint of learning task performance. They assessed to what extent students were able to solve clinical problems by using two information retrieval systems. In these studies search formulation or source selection with possible search tools is used as an independent factor and the use of sources and search outcome as dependent variables. They typically seek to find out how the search process contributes to source use or to the end product of search. For example, Liu and Belkin [39] investigated whether task type in writing a feature article has an effect on search behaviour and writing task outcome over sessions. The results showed that neither task type nor task session affected outcome, but the actor s topic familiarity and task experience did have a positive effect on outcome. Irrespective of the task type, sub-chapters and the final article consisted of equal numbers of sentences and facts. The actor s search and writing behaviour affected outcome. More time proportionally used for writing instead of searching was associated with the increase in the number of sentences and facts in the report. However, a further analysis of the data showed that users perceived that their knowledge level changed after each of the three sessions [40]. Also studies by Bron and colleagues [41] and by Vakkari and Huuskonen [30] analysed how search process variables contributed to the outcome of the search. The outcome was in the former the quality of the research question created with the help of material retrieved, and in the latter the proportion of useful retrieved documents cited in the assigned essay and the quality of the essay. In both studies, the quality of outcome was assessed by experts. Diverging from the first study type, it is possible to conceptualize the evaluation of information resources found as a process of comprehension and learning, when one is studying how, for example, various interfaces are related to the evaluation of information resources retrieved. Thus, the dependent factor used does not represent search effectiveness, but the characteristics of cognitive processing in evaluating search results. Butcher and colleagues [42] studied how preservice teachers search educational material for classroom instructions using two different interfaces. The three search tasks in the study emulated online, daily instructional planning activities. The participants were asked to find digital materials for learners of various types (a) on the water cycle, (b) on cell biology and (c) to visualize how plate interactions relate to natural phenomena. Thus, the search tasks requested topical visual material. The study compared between the interfaces the cognitive processes in which participants engaged while they conducted searches and analysed lists of returned materials. The major data were search logs and participants think-alouds during the search process. The utterances concerning the evaluation of resources were, for example, classified into three categories that were informed by the comprehension integration model of comprehension [7]. The definition of these categories resembled some of those in Bloom s taxonomy. The categories reflected the information processing depth shown in evaluation. Also, other measures concerning the processing of information in utterances were used. The results showed that an interface representing the conceptual structure of a field supports deeper cognitive processing and evaluation of results during information search compared with an established keyword search system. The interface led actors to spend more time exploring domain ideas and relationships and less time exploring individual resources. This example shows that learning can be studied using a familiar research design in interactive IR with typical search tasks, if the search process is conceptualized and operationalized as learning or cognitive processes-like result evaluation in this case. Thus, if we wish to study information searching as learning, we have to conceptualize searching as learning and cognitive processes and use respective factors as dependent variables. 6.2. From learning to searching Some studies have explored how search formulation and source selection vary by the level (complexity) of cognitive learning processes based on Bloom s taxonomy [34 36]. There are also studies that categorize search tasks using characteristics that can be reduced at least to a certain extent to Bloom s taxonomy. For example, Liu et al. [40] characterized search tasks based on typical journalistic tasks like background information collection or copyediting. The type of search task was used as independent variables and search interactions as dependent ones. In these studies the complexity of learning processes is used as an independent (comparative) variable, and search interactions and the use of search support tool as dependent variables. To generalize, in these studies characteristics of learning like the complexity of learning or the type of knowledge required in learning are used as independent variables and search interactions as dependent ones. There is also a stream of studies mainly based on Kuhlthau s work [1], which explain the variation of search process variables by the stage in the ISP model. They treat learning as an independent variable and search interactions as

Vakkari 15 dependent variables. These studies suppose that participants gain and modify their knowledge of the task topic, that is, learn when they proceed in the ISP model. For example, Vakkari [14, 21] has demonstrated in longitudinal studies that users term selection, search tactics, relevance assessments and the type of information used vary systematically by the stages in ISP model, that is, by progressing learning. Also Wildemuth [43] has used in a longitudinal study participants progress in learning, that is, the growth of their domain knowledge, as an independent variable, and search tactics as dependent variables. The results show that search tactics changed over time when participants domain knowledge changed. 6.3. Search outcome as an indicator of learning An increasing interest in the outcome of searching has led to the development of measures of learning inferred from the outcome. A typical outcome analysed is a written text based on search results. Wilson and Wilson [44] used Bloom s revised taxonomy to create measures for the depth of learning in written summary texts based on documents retrieved. The first measure is the quality of facts recalled in the text. It reflects Bloom s category understanding. The second measure is the interpretation of facts into statements. It reflects to what extent facts were combined in statements to draw conclusions. This measure is based on Bloom s category analysis. The third measure reflects Bloom s category evaluation, by identifying statements that compared facts or used facts to raise questions about other statements. Thus, these measures indicate the depth of learning shown in the three levels of learning in Bloom s taxonomy. Sormunen and colleagues [45] developed a method for analysing information use in source-based writing. By information use is meant text transformations from sources of texts composed. The method identifies how the text from source documents is modified and composed to a new text. The text transformation variables are (a) the degree of paraphrasing, (b) the degree of synthesis, (c) credibility on building arguments, (d) accuracy of citing, (e) statement type and (f) source type. These variables could to some extent be translated to represent Bloom s cognitive processing categories. For example, the categories in the degree of paraphrasing stretch from copy paste and paraphrasing to own text. Copy paste is equal to remembering, paraphrasing to understanding and own text to analysing and evaluating, in particular. Some studies in this genre, like Saito and colleagues [46] have analysed how participants conceptual structures changed when they searched documents for writing magazine articles on certain topics. The participants were asked to draw a conceptual map on the topic of search before and after a 15 min search. From the concept maps the number of concept words (nodes), links between nodes and link words were identified. This approach seeks explicitly to uncover changes in participants conceptual structure, that is, learning. It also facilitates identification of the degree of structural change in knowledge, that is, to what extent the structure is reformulated or tuned. The studies presented so far are studies on interactive information retrieval. However, there is at least one query log study that analyses learning while searching. Eickhoff [47] investigated within session and cross-session developments of expertise. The study focused on how users query terms and search behaviour on a topic changed over time. They inferred from previous studies characteristics of learning while searching. They build indicators of learning using data from queries and results visited. The indicators of learning included, for example, focus, that is, the extent of the topical space explored by users, the display time of items viewed and query complexity as measured by a reading-level metrics. The results showed, for example, that the number of domains in a search result page increased as well as the complexity of keywords over the course of a session. The results suggest that visits in documents on result lists contributed to learning, that is, by inferring information that produced queries with more complex keywords and consequently results with increased focus. 7. Conclusions There is a growing trend in studies to expand and re-conceptualize the search process from querying and inspecting search results to cover also working with search results for creating the outcome of a search. This implies that, along with traditional independent variables, that is, relevance metrics, variables indicating the characteristics of source use and search outcome should be applied. This shift signifies understanding search not primarily as only to gather and collect information, but as a series of tasks within a constructive process, like Kuhlthau suggested [1]. Actors need both support in accessing sources and guidance in the construction process, that is, in learning. The early advocates of this view were also Belkin [11] and Marchionini [13]. A consequence of this trend has been the change in the role of relevance metrics from independent variable to process variable between search formulation and source use. In addition to extending the search process towards source use and search outcome, it is essential to conceptualize term selection and relevance assessments as thinking and learning processes, not just count the number of terms or relevant items. Knowing better the reasoning processes behind term selection and relevance assessments probably helps in

Vakkari 16 understanding searching as learning. There are studies that have sought to analyse cognitive processes associated with term selection and relevance assessments [17, 23, 27, 48]. The task-based notion of information searching has been perhaps the major driving force for expanding the framework of information searching [3]. Although it has produced some interesting empirical results [30, 39, 49], it seems to lack a coherent conceptual framework on which to build empirical research. One of the major conceptualizations in task-based studies has been Kuhlthau s ISP model, which is deeply rooted in pedagogy [1]. It actually models searching as learning. It hints that conceptualizing searching as a learning process may provide our research with richer tools for analysing how humans act and make sense of information accessed and encountered in various daily activities. Learning occurs in those activities when humans do not have sufficient information to accomplish them. It is a matter of learning, although in ordinary language use people do not typically speak about learning in this context. When exploring studies that use interaction with sources retrieved and task outcome as dependent variables, the nature of the findings was astonishing. They were surprising, but in some sense evident. Results seem to show that the more time and effort users spent at the end phases of the whole search process, the better the quality of the search outcome was. The more users were able to focus on results inspection instead of querying and on working with information in documents found instead of result inspection, the better this seemed to be to the process of construction for proceeding in the task [cf. 30, 39, 42]. Also a recent query log study [47] seems to hint at this. Thus, it is not enough to provide users with a high-quality result list; they also need tools that help in making sense of, structuring and manipulating search results and sources for restructuring and tuning their knowledge structures, that is, processing the expected outcome of the search. This matches Kuhlthau s idea of providing support both in accessing sources and in construction process [1]. These findings provide system design with new challenges. It would be important to create tools that support, for example, structuring the result list according to the potential conceptual structures in documents, and restructuring the list by clicking desired concepts to combine them. This would help users to explore the conceptual space in documents in relation to their insufficient conceptual structures. Also the type of information in documents (e.g. general background information, concepts, procedural information, facts, etc.) would be a useful categorization of search results when users proceed through their task according to Kuhltau s model [1, 29]. There is also an urgent need to develop tools that support users to interact with sources for constructing an end-product by structuring and manipulating information in those sources. There are already attempts in educational psychology to develop and evaluate cognitive personalization technologies to support students sense-making [50]. Conceptualizing searching as learning, and designing new tools to support learning while searching, evidently requires developing new evaluation indicators. Evaluation begins by articulating the goal of the object of evaluation. The goal provides ideas for inferring indicators and criteria of success in the use of the tool observed. Bloom s taxonomy of learning objectives [33] can be used for characterizing learning requirements in using search tools and also in designing search tasks [cf. 34, 35]. The taxonomy describes what students are expected to learn as a result of instruction. This can be extended to what users are expected to learn as a result of using support tools provided. Therefore, the taxonomy is a good point of departure in designing search tasks and evaluation goals and criteria. Conceptualizing learning as conceptual change connects research on information searching to both a cognitive view of information searching and major theoretical ideas in information retrieval, and also theoretical ideas and empirical results in the psychology of learning. This provides a fertile point of departure for those wishing to study searching as learning. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. References [1] Kuhlthau C. Seeking meaning. Norwood, NJ: Ablex, 1993. [2] Zhang P and Soergel D. Towards a comprehensive model of the cognitive process and mechanisms of individual sensemaking. Journal of the Association for Information Science and Technology 2014; 65(9): 1733 1758. [3] Vakkari P. Task based information searching. In: Annual review of information science and technology 37. Medford, NJ.: Information Today, 2003. [4] Blandford A and Attfield S. Interacting with Information. San Rafael, CA: Morgan and Claypool, 2010. [5] Schacter DL, Gilbert DT and Wegner DM. Psychology, 2nd edn. New York: Worth, 2011. [6] Ormrod J. Human learning, 6th edn. Boston, MA: Pearson, 2012. [7] Kintsch W. Comprehension. A paradigm for cognition. Cambridge University Press: Cambridge, 1998.