posted by organizer: bnaderi || 1661 views || tracked by 1 users: [display]

Crowd@QoMEX 2021 : Crowd-sourced and Remote User Studies for Quality of Experience and Usability Research

FacebookTwitterLinkedInGoogle

Link: https://qomex2021.itec.aau.at/special-session-4
 
When Jun 14, 2021 - Jun 17, 2021
Where virtual
Submission Deadline Feb 16, 2021
Notification Due Mar 28, 2021
Categories    crowdsourcing   quality of experience   user experience   remote testing
 

Call For Papers


Special session on Crowd-sourced and Remote User Studies for Quality of Experience and Usability Research at QoMEX 2021 | 14 - 17 June 2021 | Virtual

13th International Conference on Quality of Multimedia Experiences
https://qomex2021.itec.aau.at/special-session-4
========================================================================================


Important Dates:
- 20th January 2021: Grant application (optional, see below)
- 21st January 2021: Grant acceptance notice
- 16th February 2021: Full paper submission
- 28th March 2021: Full paper notification

PAPERS SUBMISSION : Please check out the QoMEX 2021 timeline and guidance, as the papers should be submitted following the conference instructions.


Publication: IEEE

Grant: $ 200 financial support provided by Microworkers.com for paper ideas who want use the Microworkers.com for their experiment and their grant application is accepted. See below for details.


Dear Colleagues:
We organize a special session on Crowd-sourced and Remote User Studies for Quality of Experience and Usability Research at QoMEX 2021.
Laboratory studies are an established and essential tool for Quality of Experience (QoE) and User Experience (UX) research. However, they required well-equipped test rooms and personnel for the supervision of the test participants. Therefore, they are often cost- and time-intensive. Further, the number of test candidates is often limited due to the sparse laboratory space and need for the test taskers' physical presence in the test environment. In the past months, the pandemic situation made it even more challenging to conduct laboratory studies by increasing the organizational overhead and limiting potential participants. Two possibilities to overcome the current situation are crowdsourcing and remote user studies.
In this context, the special session aims to foster contributions following the traditional way of optimizing and designing crowdsourced subjective studies for Quality of Experience and User Experience research and additionally wants to raise awareness and foster research in a new research direction, the Quality of Experience and User Experience of crowdsourcing workers. The special session also seeks to encourage researchers to exchange their experiences in remote user studies, with non-anonymous test takers, how best practices from crowdsourcing studies can be applied in this context, and discuss which new challenges arise.

The topics of interest include but are not limited to the following:
• Crowdsourcing for subjective studies
• Novel applications
• Limitations of current crowdsourcing systems
• Quality control mechanism and reliability metrics
• Large scale crowdsourcing studies and diversity of participants
• Reproducibility of results
• Reproducibility and cross platform studies
• Assessment of hidden influence factors / Impact of hidden influence factors
• Bias estimation and bias reduction
• Automation and workflows
• Standardization of crowdsourcing test methods
• Usability and User Experience of crowdsourcing tasks
• Optimization of task interfaces and task workflows
• Relation to result quality and worker motivation
• Enhancing workers’ UX (e.g., by means of gamification of tasks)
• Quality of complex crowdsourcing workflows (e.g. combination of AI and Crowds)
• Interconnection of crowdsourcing and lab-based tests,
• Studies comparing results from lab and crowdsourcing
• Adaptations of established lab-test standards to the crowdsourcing environment
• Remote user studies
• Supervised remote user studies
• Remote studies with non-anonymous users
• (Crowdsourcing) best practices for remote user studies

Authors are kindly invited to submit their full papers by the deadline. All submissions are handled through the QoMEX2021 submission process: https://qomex2021.itec.aau.at/papersubmission/
For any query, Please write mail to babak.naderi@tu-berlin and matthias.hirth@tu-ilmenau.de.

Sincerely Yours,

Matthias Hirth, Institute of Media Technology, TU Ilmenau,
Babak Naderi, Quality and Usability Lab, Technical University of Berlin

Grant details:

In line to support research on the QoE using crowdsourcing, Microworkers.com offers $200 grants for selected ideas who want to conduct an experiment on their platform and submit a paper to this special session. The applicants have to send a research abstract and a rough cost plan how the grand will be spend to the to the session organizers. All applications will be forwarded to Microworkers.com where a company representative will make the final funding decision. Note, the funds cannot be withdrawn from Microworkers and can only be used for research purposes.
Please send your 1)research abstract, 2) cost plan by20th January (AOE) to matthias.hirth@tu-ilmenau.de and babak.naderi@tu-berlin.de. The selected candidates will be announced on 21th of January.

Related Resources

Security 2025   Special Issue on Recent Advances in Security, Privacy, and Trust
QoMEX 2025   17th International Conference on Quality of Multimedia Experience
CENTRIC 2025   The Eighteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services
EMERGING 2025   The Seventeenth International Conference on Emerging Networks and Systems Intelligence
GISTAM 2025   11th International Conference on Geographical Information Systems Theory, Applications and Management
CSI Personification of Evil 2025   Croatica et Slavica Iadertina Special issue 'From the Antichrist to Woland. Personification of Evil in the Slavic Languages, Literature, Theatre, and Film'
The Mississippi River: A Cultural Artery 2025   Call for Papers: The Mississippi: Soundings on America’s Arterial River
QRS 2025   The 25th International Conference on Software Quality, Reliability, and Security
JSA 2024   Japan Studies Association Conference
IGARSS 2025   International Geoscience and Remote Sensing Symposium