posted by user: bnaderi || 1340 views || tracked by 2 users: [display]

QoMEX-CS 2019 : QoMEX 2019 Special Session: Quality of Experience and User Experience Research with and for Crowdsourcing


When Jun 5, 2019 - Jun 7, 2019
Where Berlin, Germany
Abstract Registration Due Jan 18, 2019
Submission Deadline Jan 22, 2019
Notification Due Feb 26, 2019
Categories    quality of experince   crowdsourcing   user experince   crowd

Call For Papers

QoMEX 2019 - Berlin, Germany
Call for paper: submission for QoMEX 2019 Special Session on Quality of Experience and User Experience Research with and for Crowdsourcing

Abstracts submission deadline: January 18, 2019
Paper submission deadline: January 22, 2019
Special Session page:

Motivation and objectives

Crowdsourcing has become a valuable tool in Quality of Experience (QoE) and User Experience (UX) research. In contrast to lab-based studies, crowdsourcing enables researchers to easily and quickly collect a large number of subjective ratings from a diverse set of participants. This diversity further enables researching cultural effects, influence factors generated by different end-user devices, and effects of different surrounding environments. However, these opportunities come at costs. Experimenters have only limited control of the test settings and in which environmental conditions the study takes place. Additionally, the remote test procedure can be error-prone as the test participants are not under the direct supervision of the test conductor.

In this context, the aim of the special session is twofold. On the one hand, we want to foster contributions following the traditional way optimizing and designing crowdsourced subjective studies for Quality of Experience and User Experience research. This includes novel methodologies for quality assurance and replicability, new fields of application like assessing the QoE of IoT devices in a crowdsources fashion, and using new technologies like wearable to collect additional environmental and user signals. On the other hand, we also want to raise awareness and foster research in a new research direction, the Quality of Experience and User Experience of crowdsourcing workers. Crowdsourcing has become mature in the academic and business usage, and much effort is put into a (cost-) efficient and quality optimizing the design of the tasks. However, no or only very little efforts are made to improve the working experience of the workers.

Topics of interest
We invite you to submit original papers in any area related, including but not limited to:

Crowdsourcing for subjective studies
-- Novel application
-- Limitations of current crowdsourcing systems
-- Quality control mechanism / Monitoring of test and environmental conditions
-- Large scale crowdsourcing studie
Reproducibility of result
-- Cross platform studies / Repeated studies
-- Assessment of hidden influence factors / Impact of hidden influence factors
-- Bias estimation and bias reduction
-- Standardisation of crowdsourcing test methods
Usability and User Experience of crowdsourcing tasks
-- Optimization of task interfaces and task workflows
-- Relation to result quality and worker motivation
-- Enhancing workers’ UX (e.g., by means of gamification of tasks)
-- Quality of complex crowdsourcing workflows (e.g. combination of AI and Crowds)
Interconnection of crowdsourcing and lab-based tests, e.g.,
-- Studies comparing results from lab and crowdsourcing
-- Adaptations of established lab-test standards to the crowdsourcing environment

About Organizers
- Babak Naderi (Technical University of Berlin)
- Matthias Hirth (University of Wuerzburg)
- Katrien De Moor (Norwegian University of Science and Technology)

Related Resources

QoMEX 2021   Quality of Multimedia Experience
Crowdfunding, Springer 2021   Springer 2021: Call for Book Chapters Crowdfunding in Public Sector: Theory and Best Practice
QoMEX 2020   12th International Conference on Quality of Multimedia Experience
IHCI 2020   The 12th international conference on Intelligent Human-Computer Interaction (IHCI)
Session on HCI, HRI & UX at WMSCI 2020   Session on Human-Computer, Human-Robot and User Experience Technologies at WMSCI 2020 (SCOPUS Indexed)
HCI and Beyond 2021   Call for Chapter: HCI and Beyond: Advances Towards Smart and Interconnected Environments
IUI 2021   ACM Conference on Intelligent User Interfaces
IEEE CS: Special Issue on Explainable AI 2020   IEEE CS Computer: Special Issue on Explainable AI and Machine Learning
IUI 2021   ACM Intelligent User Interfaces 2021
U2BigData 2020   User Understanding from Big Data Workshop - in conjunction with IEEE Big Data 2020