posted by user: bnaderi || 1658 views || tracked by 2 users: [display]

QoMEX-CS 2019 : QoMEX 2019 Special Session: Quality of Experience and User Experience Research with and for Crowdsourcing


When Jun 5, 2019 - Jun 7, 2019
Where Berlin, Germany
Abstract Registration Due Jan 18, 2019
Submission Deadline Jan 22, 2019
Notification Due Feb 26, 2019
Categories    quality of experince   crowdsourcing   user experince   crowd

Call For Papers

QoMEX 2019 - Berlin, Germany
Call for paper: submission for QoMEX 2019 Special Session on Quality of Experience and User Experience Research with and for Crowdsourcing

Abstracts submission deadline: January 18, 2019
Paper submission deadline: January 22, 2019
Special Session page:

Motivation and objectives

Crowdsourcing has become a valuable tool in Quality of Experience (QoE) and User Experience (UX) research. In contrast to lab-based studies, crowdsourcing enables researchers to easily and quickly collect a large number of subjective ratings from a diverse set of participants. This diversity further enables researching cultural effects, influence factors generated by different end-user devices, and effects of different surrounding environments. However, these opportunities come at costs. Experimenters have only limited control of the test settings and in which environmental conditions the study takes place. Additionally, the remote test procedure can be error-prone as the test participants are not under the direct supervision of the test conductor.

In this context, the aim of the special session is twofold. On the one hand, we want to foster contributions following the traditional way optimizing and designing crowdsourced subjective studies for Quality of Experience and User Experience research. This includes novel methodologies for quality assurance and replicability, new fields of application like assessing the QoE of IoT devices in a crowdsources fashion, and using new technologies like wearable to collect additional environmental and user signals. On the other hand, we also want to raise awareness and foster research in a new research direction, the Quality of Experience and User Experience of crowdsourcing workers. Crowdsourcing has become mature in the academic and business usage, and much effort is put into a (cost-) efficient and quality optimizing the design of the tasks. However, no or only very little efforts are made to improve the working experience of the workers.

Topics of interest
We invite you to submit original papers in any area related, including but not limited to:

Crowdsourcing for subjective studies
-- Novel application
-- Limitations of current crowdsourcing systems
-- Quality control mechanism / Monitoring of test and environmental conditions
-- Large scale crowdsourcing studie
Reproducibility of result
-- Cross platform studies / Repeated studies
-- Assessment of hidden influence factors / Impact of hidden influence factors
-- Bias estimation and bias reduction
-- Standardisation of crowdsourcing test methods
Usability and User Experience of crowdsourcing tasks
-- Optimization of task interfaces and task workflows
-- Relation to result quality and worker motivation
-- Enhancing workers’ UX (e.g., by means of gamification of tasks)
-- Quality of complex crowdsourcing workflows (e.g. combination of AI and Crowds)
Interconnection of crowdsourcing and lab-based tests, e.g.,
-- Studies comparing results from lab and crowdsourcing
-- Adaptations of established lab-test standards to the crowdsourcing environment

About Organizers
- Babak Naderi (Technical University of Berlin)
- Matthias Hirth (University of Wuerzburg)
- Katrien De Moor (Norwegian University of Science and Technology)

Related Resources

Crowd@QoMEX 2021   Crowd-sourced and Remote User Studies for Quality of Experience and Usability Research
IEEE HMData 2021   IEEE HMData 2021: Fifth IEEE Workshop on Human-in-the-Loop Methods and Future of Work in BigData
QoMEX 2021   Quality of Multimedia Experience
ViSNext 2021   Design, Deployment, and Evaluation of Network-assisted Video Streaming
FUSION 2021   National Symposium on Human-Computer Interaction
LeSy 2021   Nutzendenzufriedenheit und Wohlbefinden in lernenden Systemen
FUSION 2021   National Symposium on Human-Computer Interaction
FUSION 2021   National Symposium on Human-Computer Interaction
DigEx 2021   3rd International Workshop on Transforming the Digital Customer Experience
IUI 2022   Intelligent User Interfaces