Nowadays, IIR continues to increase in complexity: user tasks and needs are demanding; data and information systems are rapidly evolving and greatly heterogeneous; and the interaction between users and IR systems is much more articulated. For example, consider what Web search is today: highly diversified results are returned from Web pages, news, social media, image and video search, products and more, and all are merged through adaptive strategies driven by current and previous user-systems interactions. As a result, experimental evaluation needs to appropriately model these evolving tasks, needs, data sources and user interactions. An additional challenge pertains to the anticipated outcome of IIR research and application. It is no longer sufficient to focus solely on precision, recall and satisfaction: successful IIR systems must engage, inform, and relate to users, taking into account single session and more long-term use and re-use.
To progress current evaluation methodologies and ensure they are able to effectively support the development of next generation IIR systems, one of the most compelling prospects is to bridge system-oriented and user-oriented evaluation methods. Both methodological approaches have their advantages and drawbacks: while system-centered methods ensure greater internal validity, they may fail to take into account user and contextual factors that influence IIR; user-oriented methods may better approximate actual user behavior, affect and cognition, but provide less experimental control of independent variables.
The goal of this workshop is to unite system- and user-centered IIR researchers for the purposes of:
Submission deadline: January 22, 2016 (extended)
Notification of acceptance: February 12, 2016
Camera ready: February 26, 2016
Workshop day: March 17, 2016
Conference days: March 13-17, 2016
General areas of interests include, but are not limited to, the following topics:
Papers should be formatted according to the ACM SIG Proceedings Template.
Papers should be two-four pages (maximum) in length.
Papers will be peer-reviewed by members of the program committee through double-blind peer review, i.e. authors must be anonymized. Selection will be based on originality, clarity, and technical quality. Papers should be submitted in PDF format to the following address:
https://easychair.org/conferences/?conf=sauce2016
Accepted papers will be published online as a volume of the CEUR-WS proceeding series.
Heather L. O'Brien, University of British Columbia, Canada, h.obrienubc.ca
Nicola Ferro, University of Padua, Italy ferrodei.unipd.it
Hideo Joho, University of Tsukuba, Japan hideoslis.tsukuba.ac.jp
Dirk Lewandowski, Hamburg University of Applied Sciences, Germany dirk.lewandowskihaw-hamburg.de
Paul Thomas, CSIRO, Australia, paul.thomascsiro.au
Keith van Rijsbergen, University of Glasgow, UK, cornelis.vanrijsbergenglasgow.ac.uk
Eric Wiebe
Department of STEM Education at North Carolina State University, USA
Bio
Dr. Wiebe is a Professor in the Department of STEM Education at NC State University
and Senior Research Fellow at the Friday Institute for Educational Innovation.
Dr. Wiebe works on many different facets of STEM Education, including the design and evaluation of
innovative uses of computing technologies in STEM instructional settings, the use of multimedia tools
for teaching and learning, and student engagement and persistence in STEM career pathways.
Specific research programs include the use of intelligent agents to support science learning in
classrooms and basic research in the how instructional technologies
(including game-based learning environments) shape student engagement and learning.
Research contexts for this work range from individual student studies in the lab, to classroom studies,
to large scale remote data collection activities with MOOCs and other cloud-based learning environments.
Goals and Engagement: Learners in Information Environments
This talk will at the learner in increasingly complex educational environments. I will focus on psychological models of affect and cognition to look at how researchers in education, psychology, and computer science are analyzing behaviors of learners, including information seeking and retrieval, in digital environments.
Examples will be drawn from my project work over the last 15 years in building and studying cyberlearning tools in grades 4-16 science classrooms and in the laboratory. A particular focus will be on how students engage in goal-directed behavior as they construct knowledge, and what psychological models have been particularly useful for studying this. An array of methodologies will also be discussed as well as implications of emerging adaptive systems supporting learners.
Ioannis Arapakis
Eurecat, Barcelona, Spain
Bio
Dr. Arapakis received his M.Sc. degree in Information Technology from the Royal Institute of Technology (KTH), Sweden. He then pursued a Ph.D. degree at the SoCS of the University of Glasgow, in the area of Information Retrieval, under the supervision of Prof. Joemon M. Jose, and Prof. C. J. Keith van Rijsbergen. In the past four years, he worked as a full-time researcher at Yahoo Labs, Barcelona, where he conducted research in the areas of Data Mining, HCI, and IR. During this period he was an active member of the User Engagement, Web Retrieval, and Ad Processing and Retrieval groups and has been involved in several key internal projects. His groups have achieved many high quality publications in top-tier venues and journals, and have addressed research problems related to pattern recognition, predictive modelling, multimedia mining and search, social media analysis, distributed systems, and machine learning. Currently, he is a Senior Data Scientist at Eurecat (Barcelona) and member of the Data Mining group.
User Behaviour Modelling - Online and Offline Methods, Metrics, and Challenges
Network flows, social networking, smart devices, the Internet-of-Things. These innovations carry no deep value in themselves. Value invariably comes from understanding, obtaining accurate measurements, predicting, and controlling. This vision, however, rests on the application of machine intelligence and data mining techniques that can tackle large and diverse data collections, but also on our capacity to operationalise an interdisciplinary research in the intersection of many domains, such as statistics, signal processing, neuroscience, privacy and security, to name a few. In this talk, through the narration of my involvement in past and recent projects, I will share my experience in the domain of user behaviour analysis and predictive modelling. I will discuss offline and online experimental methods (and how they can be brought together), present current practices in measuring human behaviour in the online world, and highlight research challenges and opportunities that I have encountered.
Understanding user adjustment to slow search  
Challenges for Measuring Usefulness of Interactive IR Systems with Log-based Approaches  
Process-Learning as a factor in evaluation