posted by user: bnaderi || 618 views || tracked by 3 users: [display]

QoMEX-CS 2019 : QoMEX 2019 Special Session: Quality of Experience and User Experience Research with and for Crowdsourcing


When Jun 5, 2019 - Jun 7, 2019
Where Berlin, Germany
Abstract Registration Due Jan 18, 2019
Submission Deadline Jan 22, 2019
Notification Due Feb 26, 2019
Categories    quality of experince   crowdsourcing   user experince   crowd

Call For Papers

QoMEX 2019 - Berlin, Germany
Call for paper: submission for QoMEX 2019 Special Session on Quality of Experience and User Experience Research with and for Crowdsourcing

Abstracts submission deadline: January 18, 2019
Paper submission deadline: January 22, 2019
Special Session page:

Motivation and objectives

Crowdsourcing has become a valuable tool in Quality of Experience (QoE) and User Experience (UX) research. In contrast to lab-based studies, crowdsourcing enables researchers to easily and quickly collect a large number of subjective ratings from a diverse set of participants. This diversity further enables researching cultural effects, influence factors generated by different end-user devices, and effects of different surrounding environments. However, these opportunities come at costs. Experimenters have only limited control of the test settings and in which environmental conditions the study takes place. Additionally, the remote test procedure can be error-prone as the test participants are not under the direct supervision of the test conductor.

In this context, the aim of the special session is twofold. On the one hand, we want to foster contributions following the traditional way optimizing and designing crowdsourced subjective studies for Quality of Experience and User Experience research. This includes novel methodologies for quality assurance and replicability, new fields of application like assessing the QoE of IoT devices in a crowdsources fashion, and using new technologies like wearable to collect additional environmental and user signals. On the other hand, we also want to raise awareness and foster research in a new research direction, the Quality of Experience and User Experience of crowdsourcing workers. Crowdsourcing has become mature in the academic and business usage, and much effort is put into a (cost-) efficient and quality optimizing the design of the tasks. However, no or only very little efforts are made to improve the working experience of the workers.

Topics of interest
We invite you to submit original papers in any area related, including but not limited to:

Crowdsourcing for subjective studies
-- Novel application
-- Limitations of current crowdsourcing systems
-- Quality control mechanism / Monitoring of test and environmental conditions
-- Large scale crowdsourcing studie
Reproducibility of result
-- Cross platform studies / Repeated studies
-- Assessment of hidden influence factors / Impact of hidden influence factors
-- Bias estimation and bias reduction
-- Standardisation of crowdsourcing test methods
Usability and User Experience of crowdsourcing tasks
-- Optimization of task interfaces and task workflows
-- Relation to result quality and worker motivation
-- Enhancing workers’ UX (e.g., by means of gamification of tasks)
-- Quality of complex crowdsourcing workflows (e.g. combination of AI and Crowds)
Interconnection of crowdsourcing and lab-based tests, e.g.,
-- Studies comparing results from lab and crowdsourcing
-- Adaptations of established lab-test standards to the crowdsourcing environment

About Organizers
- Babak Naderi (Technical University of Berlin)
- Matthias Hirth (University of Wuerzburg)
- Katrien De Moor (Norwegian University of Science and Technology)

Related Resources

IEEE HMData 2019   Third IEEE Workshop on Human-in-the-loop Methods and Human-Machine Collaboration in Big Data
DigEx 2019   International Workshop on Transforming the Digital Customer Experience
EPSS 2019   ECML/PKDD Summer School on Machine Learning and Data Mining for Geo-Spatial Data, Quality of Experience, and Human-Computer Interaction
Embodied@ILRN 2019   Special Track on Designing with Gesture and Interactivity for XR @ ILRN 2019
QoE-Management 2019   3rd International Workshop on Quality of Experience Management
IUI 2020   ACM International Conference on Intelligent User Interfaces
QRS 2019   The 19th IEEE International Conference on Software Quality, Reliability, and Security
ITiCSE 2020   25th annual conference on Innovation and Technology in Computer Science Education
QUATIC 2019   12th International Conference on the Quality of Information and Communications Technology
user2agent 2019   IUI 2019 Workshop on User-Aware Conversational Agents