2012 EvaluatingtheEffectivenessofSea

From GM-RKB
Jump to navigation Jump to search

Subject Headings: Search Task Trail.

Notes

Cited By

Quotes

Abstract

In this paper, we introduce " task trail” as a new concept to understand user search behaviors. We define task to be an atomic user information need. Web search logs have been studied mainly at session or query level where users may submit several queries within one task and handle several tasks within one session. Although previous studies have addressed the problem of task identification, little is known about the advantage of using task over session and query for search applications. In this paper, we conduct extensive analyses and comparisons to evaluate the effectiveness of task trails in three search applications: determining user satisfaction, predicting user search interests, and query suggestion. Experiments are conducted on large scale datasets from a commercial search engine. Experimental results show that: (1) Sessions and queries are not as precise as tasks in determining user satisfaction. (2) Task trails provide higher web page utilities to users than other sources. (3) Tasks represent atomic user information needs, and therefore can preserve topic similarity between query pairs. (4) Task-based query suggestion can provide complementary results to other models. The findings in this paper verify the need to extract task trails from web search logs and suggest potential applications in search and recommendation systems.

References

  • 1. ODP:http://www.dmoz.org/.
  • 2. Paul N. Bennett, Krysta Svore, Susan T. Dumais, Classification-enhanced Ranking, Proceedings of the 19th International Conference on World Wide Web, April 26-30, 2010, Raleigh, North Carolina, USA doi:10.1145/1772690.1772703
  • 3. Paolo Boldi, Francesco Bonchi, Carlos Castillo, Debora Donato, Aristides Gionis, Sebastiano Vigna, The Query-flow Graph: Model and Applications, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458163
  • 4. Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, Hang Li, Context-aware Query Suggestion by Mining Click-through and Session Data, Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 24-27, 2008, Las Vegas, Nevada, USA doi:10.1145/1401890.1401995
  • 5. Lara D. Catledge, James E. Pitkow, Characterizing Browsing Strategies in the World-Wide Web, Computer Networks and ISDN Systems, v.27 n.6, p.1065-1073, April 1995 doi:10.1016/0169-7552(95)00043-7
  • 6. Nick Craswell, Martin Szummer, Random Walks on the Click Graph, Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 23-27, 2007, Amsterdam, The Netherlands doi:10.1145/1277741.1277784
  • 7. Hongbo Deng, Irwin King, Michael R. Lyu, Entropy-biased Models for Query Representation on the Click Graph, Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2009, Boston, MA, USA doi:10.1145/1571941.1572001
  • 8. Debora Donato, Francesco Bonchi, Tom Chi, Yoelle Maarek, Do You Want to Take Notes?: Identifying Research Missions in Yahoo! Search Pad, Proceedings of the 19th International Conference on World Wide Web, April 26-30, 2010, Raleigh, North Carolina, USA doi:10.1145/1772690.1772724
  • 9. Ted Dunning, Accurate Methods for the Statistics of Surprise and Coincidence, Computational Linguistics, v.19 n.1, March 1993
  • 10. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, Thomas White, Evaluating Implicit Measures to Improve Web Search, ACM Transactions on Information Systems (TOIS), v.23 n.2, p.147-168, April 2005 doi:10.1145/1059981.1059982
  • 11. Jianfeng Gao, Wei Yuan, Xiao Li, Kefeng Deng, Jian-Yun Nie, Smoothing Clickthrough Data for Web Search Ranking, Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2009, Boston, MA, USA doi:10.1145/1571941.1572003
  • 12. Ahmed Hassan, Rosie Jones, Kristina Lisa Klinkner, Beyond DCG: User Behavior As a Predictor of a Successful Search, Proceedings of the Third ACM International Conference on Web Search and Data Mining, February 04-06, 2010, New York, New York, USA doi:10.1145/1718487.1718515
  • 13. Ahmed Hassan, Yang Song, Li-wei He, A Task Level Metric for Measuring Web Search Satisfaction and Its Application on Improving Relevance Estimation, Proceedings of the 20th ACM International Conference on Information and Knowledge Management, October 24-28, 2011, Glasgow, Scotland, UK doi:10.1145/2063576.2063599
  • 14. Haveliwala T., Kamvar, S., and Jeh G. An Analytical Comparison of Approaches to Personalizing Page Rank. 2003.
  • 15. Daqing He, Ayse Göker, David J. Harper, Combining Evidence for Automatic Web Session Identification, Information Processing and Management: An International Journal, v.38 n.5, p.727-742, September 2002 doi:10.1016/S0306-4573(01)00060-7
  • 16. Chien-Kang Huang, Lee-Feng Chien, Yen-Jen Oyang, Relevant Term Suggestion in Interactive Web Search based on Contextual Information in Query Session Logs, Journal of the American Society for Information Science and Technology, v.54 n.7, p.638-649, May 2003 doi:10.1002/asi.10256
  • 17. Bernard J. Jansen, Amanda Spink, Vinish Kathuria, How to Define Searching Sessions on Web Search Engines, Proceedings of the 8th Knowledge Discovery on the Web International Conference on Advances in Web Mining and Web Usage Analysis, p.92-109, August 20, 2006, Philadelphia, PA, USA
  • 18. Thorsten Joachims, Optimizing Search Engines Using Clickthrough Data, Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 23-26, 2002, Edmonton, Alberta, Canada doi:10.1145/775047.775067
  • 19. Rosie Jones, Kristina Lisa Klinkner, Beyond the Session Timeout: Automatic Hierarchical Segmentation of Search Topics in Query Logs, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458176
  • 20. Rosie Jones, Benjamin Rey, Omid Madani, Wiley Greiner, Generating Query Substitutions, Proceedings of the 15th International Conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland doi:10.1145/1135777.1135835
  • 21. Alexander Kotov, Paul N. Bennett, Ryen W. White, Susan T. Dumais, Jaime Teevan, Modeling and Analysis of Cross-session Search Tasks, Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, July 24-28, 2011, Beijing, China doi:10.1145/2009916.2009922
  • 22. Zhen Liao, Daxin Jiang, Enhong Chen, Jian Pei, Huanhuan Cao, Hang Li, Mining Concept Sequences from Large-Scale Search Logs for Context-Aware Query Suggestion, ACM Transactions on Intelligent Systems and Technology (TIST), v.3 n.1, p.1-40, October 2011 doi:10.1145/2036264.2036281
  • 23. Yuting Liu, Bin Gao, Tie-Yan Liu, Ying Zhang, Zhiming Ma, Shuyuan He, Hang Li, BrowseRank: Letting Web Users Vote for Page Importance, Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 20-24, 2008, Singapore, Singapore doi:10.1145/1390334.1390412
  • 24. Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, Gabriele Tolomei, Identifying Task-based Sessions in Search Engine Query Logs, Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, February 09-12, 2011, Hong Kong, China doi:10.1145/1935826.1935875
  • 25. MacQueen, J. Some Methods for Classification and Analysis of Multivariate Observations. Pages 281--297, 1967.
  • 26. Martin E., Kriegel, H., Sander, J. and Xu, X. A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. KDD '96, Pages 226--231. AAAI Press, 1996.
  • 27. Qiaozhu Mei, Dengyong Zhou, Kenneth Church, Query Suggestion Using Hitting Time, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458145
  • 28. Christopher Olston, Ed H. Chi, ScentTrails: Integrating Browsing and Searching on the Web, ACM Transactions on Computer-Human Interaction (TOCHI), v.10 n.3, p.177-197, September 2003 doi:10.1145/937549.937550
  • 29. Dou Shen, Rong Pan, Jian-Tao Sun, Jeffrey Junfeng Pan, Kangheng Wu, Jie Yin, Qiang Yang, Q2C@UST: Our Winning Solution to Query Classification in KDDCUP 2005, ACM SIGKDD Explorations Newsletter, v.7 n.2, p.100-110, December 2005 doi:10.1145/1117454.1117467
  • 30. Xuehua Shen, Bin Tan, ChengXiang Zhai, Context-sensitive Information Retrieval Using Implicit Feedback, Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 15-19, 2005, Salvador, Brazil doi:10.1145/1076034.1076045
  • 31. Craig Silverstein, Hannes Marais, Monika Henzinger, Michael Moricz, Analysis of a Very Large Web Search Engine Query Log, ACM SIGIR Forum, v.33 n.1, p.6-12, Fall 1999 doi:10.1145/331403.331405
  • 32. Michael J. Swain, Dana H. Ballard, Color Indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 doi:10.1007/BF00130487
  • 33. Ryen W. White, Jeff Huang, Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs, Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2010, Geneva, Switzerland doi:10.1145/1835449.1835548
  • 34. Ryen W. White, Paul N. Bennett, Susan T. Dumais, Predicting Short-term Interests Using Activity-based Search Context, Proceedings of the 19th ACM International Conference on Information and Knowledge Management, October 26-30, 2010, Toronto, ON, Canada doi:10.1145/1871437.1871565
  • 35. Ryen W. White, Mikhail Bilenko, Silviu Cucerzan, Studying the Use of Popular Destinations to Enhance Web Search Interaction, Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 23-27, 2007, Amsterdam, The Netherlands doi:10.1145/1277741.1277771
  • 36. Biao Xiang, Daxin Jiang, Jian Pei, Xiaohui Sun, Enhong Chen, Hang Li, Context-aware Ranking in Web Search, Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2010, Geneva, Switzerland doi:10.1145/1835449.1835525

;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2012 EvaluatingtheEffectivenessofSeaZhen Liao
Yang Song
Li-wei He
Yalou Huang		Evaluating the Effectiveness of Search Task Trails10.1145/2187836.21879032012
Retrieved from "http://www.gabormelli.com/RKB/index.php?title=2012_EvaluatingtheEffectivenessofSea&oldid=863623"