posted by user: grupocole || 3301 views || tracked by 8 users: [display]

EPS 2011 : Evaluating Personal Search

FacebookTwitterLinkedInGoogle

Link: http://www.cdvp.dcu.ie/iCLIPS/EPS2011/
 
When Apr 18, 2011 - Apr 18, 2011
Where Dublin, Ireland
Submission Deadline Feb 21, 2011
Notification Due Mar 18, 2011
Final Version Due Mar 25, 2011
Categories    information retrieval
 

Call For Papers


**CFP for ECIR workshop towards developing new TREC (or TREC-like) track for personal search evaluation**





**Apologies for cross postings. Please redistribute to other interested parties**







Call for Papers

Evaluating Personal Search. An ECIR 2011 [Half Day] Workshop. April 18, 2011. Dublin, Ireland.

http://www.cdvp.dcu.ie/iCLIPS/EPS2011/

Personal Search refers to the process of searching within one’s personal space of digital information, e.g., searching one’s desktop or mobile phone for required data items or information. While some recent advancements have been made in this domain, research acceleration is hindered by the lack of established or standardized baselines and evaluation metrics, and lack of commonly available test collections. There is a clear consensus within the research community of the need for standardized repeatable evaluation techniques in the PS space, perhaps in the form of a TREC track for example. However, there are a number of significant challenges associated with this, not the least of which is the fact that the data associated with this domain is personal to the individual, multimedia in nature, and different users will have different forms of collections, differing information needs and different memories of required information.



This workshop aims to bring together researchers interested in working towards standardized evaluation approaches for the personal search space. Due to the large space that this covers, as a first step towards overall standardized personal search evaluation this workshop will focus on evaluation for the textual elements within personal desktop collections and known item keyword queries for these elements. An interactive forum for researchers to share ideas and initiate collaborations will be provided, with the explicit goal of establishing a means to evaluate personal search algorithms and solutions in a comparable and repeatable way. It is also intended to form a consortium of people at the workshop interested in bringing this evaluation solution to fruition in the subsequent months.

The topics of the workshop will be evaluation focused and include but not be limited to:

* Understanding personal search evaluation
* Interesting target tasks and explanations of their importance
* Critiques or comparisons of existing evaluation methods
* How we can combine existing evaluation methods
* Improving on previously suggested methods
* Making evaluation more realistic
* Reducing the cost of evaluation
* Proposal of new evaluation methods
* Using human computation games for evaluation
* User study approaches
* Simulation approaches



Datasets



A number of resources are available prior to the workshop. We expect interested participants to be able to use these resources in various ways -- analyzing the characteristics of the data, evaluating various retrieval methods using the queries and result sets, suggesting a better data set for evaluating personal search, etc. Further information and download instructions are available at http://www.cdvp.dcu.ie/iCLIPS/EPS2011/datasets.html



Paper Submissions

The workshop is now accepting paper submissions. Short position papers (max. 2 pages) describing creative use of / modifications to provided datasets and approaches or ideas / challenges for the domain are invited. Submissions should be in ACM SIGIR format. LaTeX and Word templates are available at http://www.acm.org/sigs/publications/proceedings-templates (for LaTeX, use the "Option 2" style).

Papers should be anonymised and submitted in pdf format through the EasyChair system http://www.easychair.org/conferences/?conf=eps2011 no later than midnight Pacific Daylight Time on February 21, 2011. Submissions will be subject to double-blind reviewing. Accepted papers will be included in the Evaluating Personal Search Workshop 2011 proceedings.



Important Dates



February 21, 2011: Deadline for paper submission (midnight Pacific Daylight Time)
March 18, 2011: Notification to authors
March 25, 2011: Camera-ready copy due
April 18, 2011: Workshop



Further Information



Further information is available on the workshop website at http://www.cdvp.dcu.ie/iCLIPS/EPS2011/ or by emailing the workshop organizers.



Workshop Organizers



David Elsweiler – University of Erlangen, Germany (david@elsweiler.co.uk)
Liadh Kelly – Dublin City University, Ireland (lkelly@computing.dcu.ie)
Jinyoung Kim – UMass Amherst, USA (jykim@cs.umass.edu)



Program Committee


Leif Azzopardi – University of Glasgow, UK

Robert Capra – University of North Carolina, USA
Sergey Chernov – L3S, Germany
Bruce Croft – Univ. of Massachusetts, Amherst, USA
Ronald Fernandez – University of Santiago de Compostela, Spain
Karl Gyllstrom – Katholieke Universiteit Leuven, Beligium
Donna Harman – NIST, USA
David Hawking – Funnelback, Australia
Gareth Jones – Dublin City University, Ireland
Noriko Kando – National Institute of Informatics, Japan
Diane Kelly – University of North Carolina, USA
David Losada – University of Santiago de Compostela, Spain
Ian Ruthven – University of Strathclyde, Glasgow, UK
Alan Smeaton – Dublin City University, Ireland
Jaime Teevan – Microsoft Research, Redmond, USA
Paul Thomas – CSIRO ICT Centre, Australia



Related Resources

AutoML 2024   AutoML Conference 2024
JARES 2024   International Journal of Advance Robotics & Expert Systems
WSDM 2024   17th ACM International Conference on Web Search and Data Mining
ECNLPIR 2024   2024 European Conference on Natural Language Processing and Information Retrieval (ECNLPIR 2024)
BIAS 2024   International Workshop on Algorithmic Bias in Search and Recommendation
CG 2024   Computers and Games
AI & FL 2024   12th International Conference of Artificial Intelligence and Fuzzy Logic
KDIR 2024   16th International Conference on Knowledge Discovery and Information Retrieval
DeSeRe 2024   The 1st Workshop on Decentralised Search and Recommendation
BBuzz 2024   Berlin Buzzwords 2024