posted by user: palantir || 2233 views || tracked by 3 users: [display]

UIIRS 2016 : Usefulness of Interactive IR Systems

FacebookTwitterLinkedInGoogle

Link: http://www.gesis.org/veranstaltungen/konferenzen/workshop-usefulness-of-interactive-ir-systems/
 
When Mar 17, 2016 - Mar 17, 2016
Where Chapel Hill, NC, USA
Submission Deadline Jan 8, 2016
Notification Due Jan 22, 2016
Final Version Due Feb 12, 2016
Categories    information retrieval   usefulness   user
 

Call For Papers

################################
1st Workshop on
Usefulness of Interactive IR Systems

CALL FOR PAPERS

In conjunction with the CHIIR 2016 in Chapel Hill, North Carolina, USA
13–17 March 2016

Workshop 2016: http://www.gesis.org/veranstaltungen/konferenzen/workshop-usefulness-of-interactive-ir-systems/

CHIIR 2016: http://sigir.org/chiir2016

################################

---------
Important dates
Submission deadline: January 8, 2016
Notification of acceptance: January 22, 2016
Camera ready: February 12, 2016
Workshop date: March 17, 2016
---------

Contact
Claus-Peter Klas
GESIS – Leibniz Institute for the Social Sciences, Cologne, Germany
Contact: claus-peter.klas@gesis.org

Motivation
Information Retrieval (IR) systems aim at helping a user to solve an information seeking problem. There is a long-standing
tradition in IR research to measure the quality of an IR system in terms of the relevance of the documents returned
(so-called Cranfield paradigm). However, due to the high interactivity of current IR systems it turned out that pure
relevance-based measures fail to capture many factors that should be taken into account for IR evaluations, such as
learning, task completion and effort. Current IR research therefore started to "take a broader perspective of the
information seeking process to explicitly include users, tasks, and contexts in a dynamic setting" (Cole et al. 2009).
This novel paradigm in IR evaluation expands the perspective to the entire search process instead of just evaluating
single search results (cp. Dumais 2012). In addition, modern digital libraries offer more functionality besides search
e.g. – browsing relations, recommender, storing and structuring information, sharing information -- broadening the
user-system interaction possibilities. These interactions lead to more valuable data for a better understanding of
user needs and contexts and what is useful or not.
The notion of usefulness was first introduced by Cole et al. (2009) as a general criterion evaluating "how well the
user is able to achieve their goal". However, there is still a lack of computational usefulness metrics that can be
taken to evaluate interactive IR systems.
The main goal of the workshop is to provide an international forum for discussing novel approaches that might contribute
to an approximation of usefulness in interactive information retrieval. The workshop aims at bringing together experts
from both 'user' and 'system' oriented information retrieval for a fruitful exchange of ideas and discussion how to
tackle the evaluation of interactive IR from the perspective of usefulness.
The long-term research goal is to develop and evaluate new approaches for measuring usefulness of interactive IR systems.
More specifically, we address questions such as:
* What is usefulness and how can it be measured?
* How can logging tools and frameworks look like to better capture usefulness?
* How can usefulness be evaluated?
* What can usefulness contribute to the improvement of interactive IR systems?

Workshop Topics
Contributions are solicited on, but not limited to, following topics:
* Evaluation of interactive information retrieval
* Information seeking behavior
* Task based user modelling, interaction and personalization
* Logging frameworks for sessions and tasks
* Analyzing user behavior

Format, Structure and Audience
The format of the workshop should reflect a form of grass root movement, meaning to go bottom up in order
to identify real measurements, to analyze existing logs and the gap in logs to measure usefulness along several dimensions.
The workshop will be a full-day workshop with a keynote speech and paper sessions laying the ground of understanding
in the morning. The paper sessions will focus on the following areas:
* Theoretical measures for usefulness
* How can usefulness be observed?
- Tools
- Case studies
- Evaluation methods and metrics

For the afternoon we propose a world cafe format to enable brainstorming of ideas and deeper discussions of the approaches presented in the morning.
We intend to have discussion rounds of 15-30 minutes at max. four tables which are moderated by the organizers and/or participants.
The participants should move from table to table after each round. In the final session the discussion result will be presented by the moderators.
It workshop is intended for around 20 researchers in the field of interactive information retrieval and digital libraries and librarians interested
in a better understanding of users and user needs and provide better system support to enhance the digital libraries systems.

Submissions
Short Papers (4 pages): Position papers or work in progress
Poster and Demonstrations (2 pages): Poster and presentation of systems or prototypes

Submissions have to follow the ACM template, see http://www.acm.org/sigs/publications/proceedings-templates
and should be submitted as PDF files to EasyChair: https://easychair.org/conferences/?conf=uiirs2016.
All submissions will be reviewed by at least two independent reviewers. At least one author per paper
needs to register for the workshop and attend the workshop to present the work. In case of no-show the
paper (even if accepted) will be deleted from the proceedings AND from the program.


Output
Workshop proceedings will be deposited online in the CEUR workshop proceedings publication service (ISSN 1613-0073).
This way the proceedings will be permanently available and citable (digital persistent identifiers and long term
preservation). All accepted workshop papers will be published in the workshop proceedings.

Program Committee
* Leif Azzopardi, University of Glasgow, UK
* Stefano Mizzaro, University of Udine, IT
* Norbert Fuhr, University of Duisburg-Essen, Germany
* Claus-Peter Klas, GESIS-Leibniz-Institute for the SocialSciences, Germany
* Peter Mutschke, GESIS-Leibniz-Institute for the Social Sciences, Germany
* Vivien Petras, Humboldt-Universität zu Berlin, Germany
* Chirag Shah, Rudgers University, USA

References
* Cole, M., Liu, J., Belkin, N. J., Bierig, R., Gwizdka, J., Liu, C., Zhjang, J., Zhang, X. (2009): Usefulness as the Criterion for Evaluation of Interactive Information Retrieval. Proc. HCIR, 1-4.
* Dumais, S. (2012): Whole-session evaluation of interactive information retrieval systems. Compilation of Homework, NII Shonan Workshop, Oct 8-12, 2012, http://research.microsoft.com/en-us/um/people/sdumais/niishonanworkshop-web/NII-Shonan-CompiledHomework_Final.pdf

Related Resources

PCDS 2024   The 1st International Symposium on Parallel Computing and Distributed Systems
JARES 2024   International Journal of Advance Robotics & Expert Systems
AVI 2024   International Conference on Advanced Visual Interfaces
ECNLPIR 2024   2024 European Conference on Natural Language Processing and Information Retrieval (ECNLPIR 2024)
ICISS 2024   The 20th International Conference on Information Systems Security (ICISS-2024)
AI & FL 2024   12th International Conference of Artificial Intelligence and Fuzzy Logic
CIIS 2024   ACM--2024 7th International Conference on Computational Intelligence and Intelligent Systems (CIIS 2024)
ANaNO 2024   International Conference on Advanced Nanoscience and Technology
ASPLOS 2025   The ACM International Conference on Architectural Support for Programming Languages and Operating Systems
DDECS 2024   27th International Symposium on Design and Diagnostics of Electronic Circuits and Systems