Extracting Searching as Learning Tasks Based on IBRT Approach
Abstract
:1. Introduction
- We successfully collect an SAL dataset of autonomous web learner trace data.
- To our best knowledge, we are the first study to employ specific learning styles in SAL task studies.
- We develop a novel efficient SAL task representation and extraction based on an extended Bayesian rose tree model.
2. Related Works
2.1. Searching as Learning
- (1)
- From the perspective of conceptualizing learning as a search process. Kuhlthau [17] first conceptualized learning as a search process and proposed the Information Search Process (ISP) model. This model emphasizes the stages of human cognition and the learning process. Gary and Marchionini [15] pointed out that searches for learning purposes involve multiple iterations, and learners must analyze and interpret search results to achieve the learning goal. Zarro [18] presented dual-process theory from the social psychology domain conceptualizing learning as searching behaviors and cognitive processes. On the basis of conceptualizing learning as a search process, Odijk and White [19] figured out that learners may appear to be struggling during the learning process. Rieh et al. [1] assessed learning from online searching behavior and develop a search system that supports SAL. Ghosh et al. [16] studied the relationship between search and learning by taking learning as the result of the information search process. Proao-Rí et al. [20] conceptualized searching as a learning path and focus on helping learners to plan their SAL path. Taibi et al. [21] analyzed search processes in web learning and proved the positive impact of SAL systems in stimulating students’ creativity and critical thinking.
- (2)
- From the aspect of exploring the relations between search and learning. Vakkari et al. [2] summarized existing studies and presented three types of relationships between searching and learning. Moraveji et al. [22] and Sun et al. [23] analyzed the relationship between search skills and learning outcomes and pointed out that search skills can help learners complete learning tasks in a shorter time. Liu and Belkin [24] pointed out that learners’ familiarity and experience with topics will affect search behaviors and further positively affect learning outcomes. Bron et al. [25] and Vakkari et al. [26] also focused on how variables in the search processes affect search results and learning outputs. Gimenez et al. pointed out that searching is the learning mode of learners in the learning process, which will affect the search process [27]. Marchionini [28] focused on the relationships among searching, sense-making, and learning. Yigit et al. [29] explored result diversification as a useful technique to support learning-oriented search and developed a new search engine for SAL. Liu et al. [30] determined that learning strategies will directly affect their learning outcomes, and they further presented two kinds of learning strategies; they found that learners with a task-adaptive strategy show better learning outcomes, e.g., knowledge points, facets, and scope.
2.2. Task Extraction
3. Data and Methods
3.1. The Definition of Tasks in SAL
- Hierarchical: Each process of SAL may cover one or more tasks, and each task may be divided into subtasks. In addition, subtasks can also be independent tasks. Therefore, an effective task extraction method should be able to accurately identify and represent this hierarchy.
- Learning style: In the process of SAL, learners divide learning goals depending on their learning style. Therefore, it is also important to develop a learning style-oriented SAL task extraction method.
3.2. Data Collection and Analysis
3.2.1. Data Collection
- (1)
- Assignments with clear targets. In the first week of the UWP course, students were asked to develop a multimedia player with file opening and playing functions. The full solution needed to work on the Win10 system and be able to play video and audio files. During this process, students struggled between searching and learning to finally complete the assignment. Similarly, we also assigned four other UWP programming assignments in the subsequent 4 weeks.
- (2)
- Assignments with open targets. In the sixth week of the UWP course, each student was asked to develop a UWP application with open targets. In order to understand the actual learning task, each student was asked to submit their study reports in text form.
3.2.2. SAL Learning Style Analysis
- (1)
- Learning according to the knowledge network (LKN). These learners prefer to learn by following authoritative knowledge points in the knowledge network and creating queries on the basis of these knowledge points. In our experiments, these students commonly searched and learned following Microsoft’s official UWP API reference.
- (2)
- Learning according to information needs (LIN). Learners who learn according to information needs prefer to learn according to the tasks partitioned by their own subjective. Compared with LKN, LIN learners exhibited more interleaved tasks. In the UWP course, these students usually explored the knowledge points needed to complete the learning task and learned to program sequences.
3.3. Modeling Progress
3.3.1. Improved Bayesian Rose Tree Model
- (1)
- Calculating the marginal probability for LKN
- (2)
- Calculating the marginal probability for LIN
- Queries: (i) Queries with identical or similar terms tend to have the same learning tasks. We, thus, used this factor to capture task relationships between a pair of search queries. To this end, we adopted cosine similarity, edit distance, and Jaccard distance to calculate these relationships. (ii) Queries with the same UPW-related terms tend to belong to the same tasks. In this paper, we adopted ‘percentage of identical UWP terms between queries’ to capture this feature. (iii) Pairs of search queries that are closer in the semantic space tend to belong to the same task. We, thus, used the ‘cosine distance between query semantics‘ to capture this factor.
- Search results. Search queries that belong to the same tasks tend to have similar search results. Therefore, search results are also an important factor for us to measure the relationships between a bunch of queries. Similarly, we used the ‘average cosine similarity between term sets of clicked on URLs after two queries’, ‘average edit distance between term sets of clicked on URLs after two queries’, etc. to calculate these relationships.
- Learning outcomes. We used snapshots to analyze how learners develop tasks since snapshots are the bridge linking queries and learning outcomes. In this paper, we used the ‘cosine similarity between class sets of snapshots after two queries’ and the ‘edit distance between class sets of snapshots after two queries’ to calculate these relationships.
3.3.2. Extraction of the Tasks in SAL Based on IBRT
4. Experiment Evaluation
4.1. Experimental Setup
4.2. Comparison with State-of-the-Art Methods on UWP Dataset
4.3. Subjective Metrics
- (1)
- The accuracy of SAL task extraction
“Does the task extraction result match the real situation when you are learning?”
- (2)
- The accuracy of the hierarchy
“Do you think the hierarchy of your learning process is valid?”
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Rieh, S.Y.; Collins-Thompson, K.; Hansen, P.; Lee, H.-J. Towards searching as a learning process: A review of current perspectives and future directions. J. Inf. Sci. 2016, 42, 19–34. [Google Scholar] [CrossRef]
- Vakkari, P. Searching as learning: A systematization based on literature. J. Inf. Sci. 2016, 42, 7–18. [Google Scholar] [CrossRef]
- Zhang, P.; Soergel, D. Process patterns and conceptual changes in knowledge representations during information seeking and sensemaking: A qualitative user study. J. Inf. Sci. 2016, 42, 59–78. [Google Scholar] [CrossRef] [Green Version]
- Liu, J. Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors. Inf. Process. Manag. 2021, 58, 102522. [Google Scholar] [CrossRef]
- Awadallah, A.H.; White, R.W.; Pantel, P.; Dumais, S.T.; Wang, Y.-M. Supporting complex search tasks. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, Shanghai, China, 3–7 November 2014; ACM: New York, NY, USA, 2014; pp. 829–838. [Google Scholar]
- Mehrotra, R.; Yilmaz, E. Terms, topics & tasks: Enhanced user modelling for better personalization. In Proceedings of the 2015 International Conference on the Theory of Information Retrieval, Northampton, MA, USA, 27–30 September 2015; ACM: New York, NY, USA, 2015; pp. 131–140. [Google Scholar]
- O’Brien, H.L.; Arguello, J.; Capra, R. An empirical study of interest, task complexity, and search behaviour on user engagement. Inf. Process. Manag. 2020, 57, 102226. [Google Scholar] [CrossRef]
- Wang, H.; Song, Y.; Chang, M.-W.; He, X.; Hassan, A.; White, R.W. Modeling action-level satisfaction for search task satisfaction prediction. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR ‘14), Gold Coast, Australia, 6–11 July 2014; ACM: New York, NY, USA, 2014; pp. 123–132. [Google Scholar] [CrossRef]
- Zhou, X.; Chen, J.; Wu, B.; Jin, Q. Discovery of Action Patterns and User Correlations in Task-Oriented Processes for Goal-Driven Learning Recommendation. IEEE Trans. Learn. Technol. 2014, 7, 231–245. [Google Scholar] [CrossRef]
- Shi, J.; Li, H.; Zhou, J.; Pang, Z.; Wang, C. Optimizing emotion–cause pair extraction task by using mutual assistance single-task model, clause position information and semantic features. J. Supercomput. 2021, 78, 4759–4778. [Google Scholar] [CrossRef]
- Aliannejadi, M.; Harvey, M.; Costa, L.; Pointon, M.; Crestani, F. Understanding Mobile Search Task Relevance and User Behaviour in Context. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval (CHIIR ‘19), Scotland, UK, 10–14 March 2019; ACM: New York, NY, USA, 2019; pp. 143–151. [Google Scholar] [CrossRef] [Green Version]
- Liu, J.; Sarkar, S.; Shah, C. Identifying and Predicting the States of Complex Search Tasks. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, Vancouver, BC, Canada, 14–18 March 2020; ACM: New York, NY, USA, 2020; pp. 193–202. [Google Scholar] [CrossRef] [Green Version]
- Mehrotra, R.; Yilmaz, E. Extracting Hierarchies of Search Tasks & Subtasks via a Bayesian Nonparametric Approach. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘17), Tokyo, Japan, 7–11 August 2017; ACM: New York, NY, USA, 2017; pp. 285–294. [Google Scholar] [CrossRef] [Green Version]
- Collins-Thompson, K.; Hansen, P.; Hauff, C. Search as Learning (Dagstuhl Seminar 17092). Dagstuhl. Rep. 2017, 7, 135–162. [Google Scholar]
- Marchionini, G. Exploratory search: From finding to understanding. Commun. ACM 2006, 49, 41–46. [Google Scholar] [CrossRef]
- Reynolds, R.; Meyers, E.; Ghosh, S.; Novin, A. Beyond Bloom’s Taxonomy: Integrating “Searching as Learning” and E-Learning Research Perspectives. Proc. Assoc. Inf. Sci. Technol. 2018, 55, 726–729. [Google Scholar] [CrossRef]
- Kuhlthau, C. Seeking Meaning; Libraries Unlimited: Westport, CT, USA, 2004. [Google Scholar]
- Zarro, M. Developing a dual-process information seeking model for exploratory search. In Proceedings of the HCIR 2012, Cambridge, MA, USA, 4–5 October 2012. [Google Scholar]
- Odijk, D.; White, R.W.; Awadallah, A.H.; Dumais, S.T. Struggling and Success in Web Search. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM ‘15), Melbourne, Australia, 18–23 October 2015; ACM: New York, NY, USA, 2015; pp. 1551–1560. [Google Scholar] [CrossRef] [Green Version]
- Proao-Ríos, V.; González-Ibáez, R. Dataset of Search Results Organized as Learning Paths Recommended by Experts to Support Search as Learning. Data 2020, 5, 92. [Google Scholar] [CrossRef]
- Taibi, D.; Fulantelli, G.; Marenzi, I.; Nejdl, W.; Rogers, R.; Ijaz, A. SaR-WEB: A Semantic Web Tool to Support Search as Learning Practices and Cross-Language Results on the Web. In Proceedings of the 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), Timisoara, Romania, 3–7 July 2017; pp. 522–524. [Google Scholar] [CrossRef]
- Moraveji, N.; Russell, D.; Bien, J.; Mease, D. Measuring improvement in user search performance resulting from optimal search tips. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘11), Beijing, China, 24–28 July 2011; ACM: New York, NY, USA, 2011; pp. 355–363. [Google Scholar]
- Sun, K.; Zhu, J. Searching and Learning Discriminative Regions for Fine-Grained Image Retrieval and Classification. IEICE Trans. Inf. Syst. 2022, E105.D, 141–149. [Google Scholar] [CrossRef]
- Liu, J.; Belkin, N.J. Searching vs. writing: Factors affecting information use task performance. Proc. Am. Soc. Inf. Sci. Technol. 2012, 49, 1–10. [Google Scholar] [CrossRef]
- Bron, M.; van Gorp, J.; Nack, F.; de Rijke, M.; Vishneuski, A.; de Leeuw, S. A subjunctive exploratory search interface to support media studies researchers. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘12), Portland, OR, USA, 12–16 August 2012; ACM: New York, NY, USA, 2012; pp. 425–434. [Google Scholar]
- Vakkari, P.; Huuskonen, S. Search effort degrades search output but improves task outcome. J. Assoc. Inf. Sci. Technol. 2012, 63, 657–670. [Google Scholar] [CrossRef]
- Margulieux, L.E.; Catrambone, R.; Schaeffer, L.M. Varying effects of subgoal labeled expository text in programming, chemistry, and statistics. Instr. Sci. 2018, 46, 707–722. [Google Scholar] [CrossRef] [Green Version]
- Marchionini, G. Search, sense making and learning: Closing gaps. Inf. Learn. Sci. 2019, 120, 74–86. [Google Scholar] [CrossRef]
- Yigit-Sert, S.; Altingovde, I.S.; Macdonald, C.; Ounis, I.; Ulusoy, Ö. Explicit diversification of search results across multiple dimensions for educational search. J. Assoc. Inf. Sci. Technol. 2020, 72, 315–330. [Google Scholar] [CrossRef]
- Liu, C.; Song, X. How do Information Source Selection Strategies Influence Users’ Learning Outcomes’. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR ‘18), New Brunswick, NJ, USA, 11–15 March 2018; ACM: New York, NY, USA, 2018; pp. 257–260. [Google Scholar] [CrossRef]
- Catrambone, R. Aiding subgoal learning: Effects on transfer. J. Educ. Psychol. 1995, 87, 5–17. [Google Scholar] [CrossRef]
- Liu, J.; Belkin, N.J. Personalizing information retrieval for multi-session tasks: Examining the roles of task stage, task type, and topic knowledge on the interpretation of dwell time as an indicator of document usefulness. J. Assoc. Inf. Sci. Technol. 2014, 66, 58–81. [Google Scholar] [CrossRef]
- Marulli, F.; Verde, L.; Marrone, S.; Barone, R.; De Biase, M.S. Evaluating Efficiency and Effectiveness of Federated Learning Approaches in Knowledge Extraction Tasks. In Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, 18–22 July 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Wang, T.X.; Lu, W.H. Constructing Complex Search Tasks with Coherent Subtask Search Goals. ACM Trans. Asian Lang. Inf. Process. 2016, 15, 6.1–6.29. [Google Scholar] [CrossRef]
- He, D.; Göker, A.; Harper, D.J. Combining evidence for automatic Web session identification. Inf. Process. Manag. 2002, 38, 727–742. [Google Scholar] [CrossRef]
- Kotov, A.; Bennett, P.N.; White, R.W.; Dumais, S.T.; Teevan, J. Modeling and analysis of cross-session search tasks. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘11), Beijing, China, 24–28 July 2011; ACM: New York, NY, USA, 2011; pp. 5–14. [Google Scholar] [CrossRef]
- Li, L.; Deng, H.; Dong, A.; Chang, Y.; Zha, H. Identifying and labeling search tasks via query-based hawkes processes. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, 23–27 August 2020; ACM: New York, NY, USA, 2014; pp. 731–740. [Google Scholar]
- Mittal, A.; Pagalthivarthi, K.V. Use of Relational and Conceptual Graphs in Supporting E-Learning Tasks. Int. J. E Learn. 2005, 4, 69–82. [Google Scholar]
- Jones, R.; Klinkner, K.L. Beyond the session timeout: Automatic hierarchical segmentation of search topics in query logs. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM ‘08), Napa Valley, CA, USA, 26–30 October 2008; ACM: New York, NY, USA, 2008; pp. 699–708. [Google Scholar] [CrossRef]
- Blundell, C.; The, Y.W. Bayesian hierarchical community discovery. In Proceedings of the 26th International Conference on Neural Information Processing Systems—Volume 1 (NIPS’13), Lake Tahoe, NV, USA, 5–10 December 2013; Curran Associates Inc.: Red Hook, NY, USA, 2013; pp. 1601–1609. [Google Scholar]
- Agosti, M.; Fuhr, N.; Toms, E.; Vakkari, P. Evaluation methodologies in information retrieval dagstuhl seminar 13441. ACM SIGIR Forum 2014, 48, 36–41. [Google Scholar] [CrossRef]
- Mohssine, B.; Mohammed, A.; Abdelwahed, N.; Mohammed, T. Adaptive Help System Based on Learners ‘Digital Traces’ and Learning Styles. Int. J. Emerg. Technol. Learn. (IJET) 2021, 16, 288–294. [Google Scholar] [CrossRef]
- Hassan, M.A.; Habiba, U.; Majeed, F.; Shoaib, M. Adaptive gamification in e-learning based on students’ learning styles. Interact. Learn. Environ. 2019, 29, 545–565. [Google Scholar] [CrossRef]
- Kolb, A.Y.; Kolb, D.A. Experiential learning theory: A dynamic, holistic approach to management learning, education and development. In The SAGE Handbook of Management Learning, Education and Development; Armstrong, S.J., Fukami, C.V., Eds.; SAGE Publications Ltd.: Southend Oaks, CA, USA, 2009; pp. 42–68. [Google Scholar] [CrossRef] [Green Version]
- Stander, J.; Grimmer, K.; Brink, Y. Learning styles of physiotherapists: A systematic scoping review. BMC Med. Educ. 2019, 19, 2. [Google Scholar] [CrossRef]
- Kolb, A.Y.; Kolb, D.A. Experiential learning theory as a guide for experiential educators in higher education. Exp. Learn. Teach. High. Educ. 2017, 1, 7–14. [Google Scholar]
- Li, C.; Yang, Y.; Jing, Y. Formulation of teaching strategies for graduation internship based on the experiential learning styles of nursing undergraduates: A non-randomized controlled trial. BMC Med. Educ. 2022, 22, 153. [Google Scholar] [CrossRef]
- Vizeshfar, F.; Torabizadeh, C. The effect of teaching based on dominant learning style on nursing students’ academic achievement. Nurse Educ. Pract. 2018, 28, 103–108. [Google Scholar] [CrossRef] [PubMed]
- Dinler, D.; Tural, M.K.; Ozdemirel, N.E. Centroid based Tree-Structured Data Clustering Using Vertex/Edge Overlap and Graph Edit Distance. Ann. Oper. Res. 2020, 289, 85–122. [Google Scholar] [CrossRef]
- Gotz, M.; Machanavajjhala, A.; Wang, G.; Xiao, X.; Gehrke, J. Publishing Search Logs—A Comparative Study of Privacy Guarantees. IEEE Trans. Knowl. Data Eng. 2012, 24, 520–532. [Google Scholar] [CrossRef]
- Blundell, C.; Teh, Y.W.; Heller, K.A. Bayesian Rose Trees. Comput. Ence 2012, 22, 217. [Google Scholar]
- Wang, H.; Song, Y.; Chang, M.-W.; He, X.; White, R.W.; Chu, W. Learning to extract cross-session search tasks. In Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, 13–17 May 2013; ACM: New York, NY, USA, 2013; pp. 1353–1364. [Google Scholar] [CrossRef]
- Finley, T.; Joachims, T. Supervised clustering with support vector machines. In Proceedings of the 22nd International Conference on Machine Learning (ICML ‘05), Bonn, Germany, 7–11 August 2005; ACM: New York, NY, USA, 2005; pp. 217–224. [Google Scholar] [CrossRef]
- Liao, Z.; Song, Y.; He, L.-W.; Huang, Y. Evaluating the effectiveness of search task trails. In Proceedings of the 21st International Conference on World Wide Web (WWW ‘12), Lyon, France, 16–20 April 2012; ACM: New York, NY, USA, 2012; pp. 489–498. [Google Scholar] [CrossRef]
Description | Factors |
---|---|
Query | |
Cosine similarity between term sets of two queries | |
Edit distance between term sets of two queries | |
Jaccard distance between term sets of two queries | |
Percentage of identical UWP terms between queries | |
Cosine semantics distance between queries | |
Search results | |
Average cosine similarity between term sets of clicked on URLs after two queries | |
Average edit distance between term sets of clicked on URLs after two queries | |
Average cosine semantics similarity between clicked on URLs after two queries | |
Cosine distance of UWP terms contained in clicked links after two queries | |
Cosine similarity between UWP class sets of two search results | |
Learning outcomes | |
Cosine similarity between class sets of snapshots after two queries | |
Edit distance between class sets of snapshots after two queries |
Score | Scoring Principles |
---|---|
0 | Mismatches the real situation |
1 | Only a small amount matches the real situation |
2 | Mostly matches the real situation |
3 | Completely matches the real situation |
Score | Scoring Principles |
---|---|
0 | Invalid |
1 | Valid in some parts; however, most of the extraction is invalid |
2 | Valid in some parts; however, some parts are invalid |
3 | Valid |
Score | |
---|---|
IBRT | 2.7 |
BRT | 2.5 |
BHC | 2.1 |
LDA-Hawkes | 2.5 |
BestLink-SVM | 2.4 |
Cluster-SVM | 2.3 |
QC-HTC | 1.9 |
QCWCC | 2.2 |
Score | |
---|---|
IBRT | 2.3 |
BRT | 2.0 |
BHC | 0.9 |
LDA-Hawkes | 1.6 |
BestLink-SVM | 1.4 |
Cluster-SVM | 1.4 |
QC-HTC | 1.5 |
QCWCC | 1.5 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, P.; Zhang, B.; Zhang, Y. Extracting Searching as Learning Tasks Based on IBRT Approach. Appl. Sci. 2022, 12, 5879. https://doi.org/10.3390/app12125879
Li P, Zhang B, Zhang Y. Extracting Searching as Learning Tasks Based on IBRT Approach. Applied Sciences. 2022; 12(12):5879. https://doi.org/10.3390/app12125879
Chicago/Turabian StyleLi, Pengfei, Bin Zhang, and Yin Zhang. 2022. "Extracting Searching as Learning Tasks Based on IBRT Approach" Applied Sciences 12, no. 12: 5879. https://doi.org/10.3390/app12125879
APA StyleLi, P., Zhang, B., & Zhang, Y. (2022). Extracting Searching as Learning Tasks Based on IBRT Approach. Applied Sciences, 12(12), 5879. https://doi.org/10.3390/app12125879