CALL FOR PAPERS

Workshop on the Question Generation Task and Evaluation Challenge
Sponsored by National Science Foundation

September 25-26, 2008
National Science Foundation, Arlington, VA
www.questiongeneration.org

Goal

The natural language generation community has recently identified shared task evaluation campaigns as a potential avenue to provide a focus of research in natural language generation and to increase the visibility of natural language generation in the wider natural language processing/computational linguistics community.

We would like to contribute to this effort by first properly defining and then offering a shared task on Question Generation. As a first approximation, we define Question Generation as the automatic generation of questions (Factual questions, Yes/No-questions, Why-questions, etc.) from inputs such as text (in particular, declarative sentences), raw data, and knowledge bases. The role of this workshop is to reach a community consensus with respect to the Question Generation shared task. The discussions will be guided by the Desiderata for Evaluation of Natural Language Generation (Paris et al., 2007 in Shared Tasks and Comparative Evaluation in Natural Language Generation: Workshop Report, Michael White and Robert Dale, Eds.) outlined at the NSF/SIGGEN Workshop on Shared Tasks and Comparative Evaluation in Natural Language Generation in 2007.

Question Generation is an essential component of learning environments, help systems, information seeking systems, multi-modal conversations between virtual agents, and a myriad of other applications. However, systematic research on Question Generation has been conspicuously absent in the computational linguistics community at large and in the natural language generation community in particular, in spite of substantial attention to the complementary question answering processes. The time is ripe to address the important topic of automated Question Generation.

We invite submissions of short papers (2 pages including references) that address topics relevant to Question Generation as a shared task, as provided below:

-cognitive and computational models of Question Generation that could inform the definition of Question Generation shared task(s)
-question taxonomies
-Question Generation main task(s)
-Question Generation sub-task(s)
-data collection and preparation
-representation language(s) for the input and output data
-annotation schemes and processes
-forms of evaluation (human-based vs. automatic vs. semi-automatic)
-evaluation metrics
-others

Submission Guidelines

Papers should follow the ACL paper format and should not exceed 2 pages including references. Templates are available at http://www.ling.ohio-state.edu/acl08/stylefiles.html. Submissions should include information about authors (no anonymous submissions). Papers must be submitted via email to vrus@memphis.edu before or on the submission deadline (see next). The papers must be submitted in PDF format.

Important Dates

Submission Deadline: August 1, 2008
Notification Deadline: August 15, 2008
Camera-Ready Papers Due: September 1, 2008

Organizing Committee

Workshop Co-chairs:

Vasile Rus, The University of Memphis
Art Graesser, The University of Memphis

Steering Committee:

James Lester, North Carolina State University
Jose Otero, University of Alcala
Paul Piwek, Open University
Amanda Stent, Stony Brook University