posted by user: castells || 2980 views || tracked by 9 users: [display]

RUE 2012 : International ACM RecSys Workshop on Recommendation Utility Evaluation: Beyond RMSE

FacebookTwitterLinkedInGoogle

Link: http://ir.ii.uam.es/rue2012
 
When Sep 9, 2012 - Sep 9, 2012
Where Dublin, Ireland
Submission Deadline Jun 18, 2012
Notification Due Jul 4, 2012
Final Version Due Jul 19, 2012
Categories    recommender systems   evaluation   utility   metrics
 

Call For Papers

=====================================================================
Call for Papers - RUE 2012
International ACM RecSys Workshop on
Recommendation Utility Evaluation: Beyond RMSE - RUE 2012
Dublin, Ireland, 9 September 2012
http://ir.ii.uam.es/rue2012
=====================================================================

* Submission deadline: 18 June 2012 *

== Scope ==

Measuring the error in rating value prediction has been by far the dominant evaluation methodology in the Recommender Systems literature. Yet there seems to be a general consensus that this criterion alone is far from being enough to assess the practical effectiveness of a recommender system in matching user needs. The end users of recommendations receive lists of items rather than rating values, whereby recommendation accuracy metrics -as surrogates of the evaluated task- should target the quality of the item selection, rather than the numeric system scores that determine this selection. Gaps in the adoption of ranking evaluation methodologies (e.g. IR metrics) result in methodological divergences though, which hinder the interpretation and comparability of empirical observations by different authors.

On the other hand, accuracy is only one among several relevant dimensions of recommendation effectiveness. Novelty and diversity, for instance, have been recognized as key aspects of recommendation utility in many application domains. From the business point of view, the value added by recommendation can be measured more directly in terms of clickthrough, conversion rate, order size, returning customers, increased revenue, etc. Furthermore, web portals and social networks commonly face multiple objective optimization problems related to user engagement, requiring appropriate evaluation methodologies for optimizing along the entire recommendation funnel. Other potentially relevant dimensions of effective recommendations for consumers and providers include confidence, coverage, risk, cost, robustness, ease of use, etc.

While the need for further extension, formalization, clarification and standardization of evaluation methodologies is recognized in the community, this need is still unmet for a large extent. When engaging in evaluation work, researchers and practitioners are still often faced with experimental design questions for which there are currently not always precise and consensual answers. RUE 2012 aims to gather researchers and practitioners interested in developing better, clearer, and/or more complete evaluation methodologies for recommender systems -or just seeking clear guidelines for their experimental needs. The workshop aims to provide an informal setting for exchanging and discussing ideas, sharing experiences and viewpoints, seeking to advance in the consolidation and convergence of experimental methods and practice.

== Topics ==

We invite the submission of papers reporting original research, studies, advances, experiences, or work in progress in the scope of recommender system utility evaluation. The topics the workshop seeks to address include –though need not be limited to– the following:

* Recommendation quality dimensions
- Effective accuracy, ranking quality
- Novelty, diversity, unexpectedness, serendipity
- Utility, gain, cost, risk, benefit
- Robustness, confidence, coverage, ease of use, etc.
* Matching metrics to tasks, needs, and goals
- User satisfaction, human factors
- Business-oriented evaluation
- Multiple objective optimization, user engagement
- Quality of service, quality of experience
* Evaluation methodology and experimental design
- Definition and evaluation of new metrics, studies of existing ones
- Adaptation of methodologies from related fields: IR, Machine Learning, HCI, etc.
- Evaluation theory
* Practical aspects of evaluation
- Offline and online experimental approaches
- Simulation-based evaluation
- Datasets and benchmarks
- Validation of metrics

== Submission ==

Two submission types are accepted: technical papers of up to 6 pages, and position papers up to 3 pages. Each paper will be evaluated by at least two reviewers from the Programme Committee. The papers will be evaluated for their originality, contribution significance, soundness, clarity, and overall quality. Within a required quality standard, position papers will be appreciated for presenting new perspectives and insights, and their potential for provoking thought and stimulating discussion.

All submissions shall adhere to the standard ACM SIG proceedings format: http://www.acm.org/sigs/publications/proceedings-templates. The accepted papers will be published in the CEUR Proceedings series.

Submissions shall be sent as a pdf file through the online submission system now open at: http://www.easychair.org/conferences/?conf=rue2012.

== Important dates ==

Paper submission deadline: 18 June 2012
Author notification: 4 July 2012
Camera ready version due: 19 July 2012
RUE 2012 workshop: 9 September 2012

== Programme Committee ==

Gediminas Adomavicius, University of Minnesota, USA
Alejandro Bellogín, Universidad Autónoma de Madrid, Spain
Iván Cantador, Universidad Autónoma de Madrid, Spain
Licia Capra, University College London, UK
Òscar Celma, Gracenote, USA
Charles Clarke, University of Waterloo, Canada
Paolo Cremonesi, Politecnico di Milano, Italy
Juan Manuel Fernández-Luna, Universidad de Granada, Spain
Pankaj Gupta, Twitter, USA
Juan F. Huete, Universidad de Granada, Spain
Dietmar Jannach, University of Dortmund, Germany
Jaap Kamps, University of Amsterdam, Netherlands
Neal Lathia, University College London, UK
Jérôme Picault, Bell Labs, Alcatel-Lucent, France
Filip Radlinski, Microsoft, Canada
Francesco Ricci, Free University of Bozen-Bolzano, Italy
Fabrizio Silvestri, Consiglio Nazionale delle Ricerche, Italy
Harald Steck, Netflix, USA
David Vallet, Universidad Autónoma de Madrid, Spain
Paulo Villegas, Telefónica R&D, Spain
Jun Wang, University College London, UK
Yi Zhang, University of California, Santa Cruz, USA

== Organizers ==

Xavier Amatriain, Netflix, USA
Pablo Castells, Universidad Autónoma de Madrid, Spain
Arjen de Vries, Centrum Wiskunde & Informatica, Netherlands
Christian Posse, LinkedIn, USA

Contact email: rue2012@easychair.org

More info at: http://ir.ii.uam.es/rue2012

Related Resources

UCC 2018   11th IEEE/ACM International Conference on Utility and Cloud Computing
UMAP 2019   User Modeling, Adaptation, and Personalization
RecSys 2019   13th ACM Conference on Recommender Systems
RecSys 2018   Conference on Recommender Systems
UMUAI FatRec 2019   UMUAI Special issue on: Fair, Accountable, and Transparent Recommender Systems
RSBD 2019   1st International Workshop on Recommender Systems with Big Data
SIGIR 2019   International ACM SIGIR Conference on Research and Development in Information Retrieval
KDD 2019   25TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING
ICMLC--ACM, Ei and Scopus 2019   2019 11th International Conference on Machine Learning and Computing (ICMLC 2019)--ACM, Ei Compendex and Scopus
ACM--ICCDA--Ei Compendex, Scopus 2019   ACM--2019 3rd International Conference on Compute and Data Analysis (ICCDA 2019)--Ei Compendex, Scopus