posted by organizer: nandana || 4046 views || tracked by 2 users: [display]

SMART 2020 : SeMantic AnsweR Type prediction (SMART) - ISWC 2020 Challenge

FacebookTwitterLinkedInGoogle

Link: https://smart-task.github.io/
 
When Nov 2, 2020 - Nov 6, 2020
Where Online
Submission Deadline Sep 21, 2020
Notification Due Sep 26, 2020
Final Version Due Oct 7, 2020
Categories    semantic web   wikidata   dbpedia   linked data
 

Call For Papers

ISWC 2020 Challenge - SeMantic AnsweR Type prediction (SMART) task.
in conjunction with ISWC 2020 [https://iswc2020.semanticweb.org/]


*********************************************************************************

Challenge website: https://smart-task.github.io/
Conference date and location: 2-6 November 2020, (Online - Virtual)
Submission deadline: September 21, 2020

*********************************************************************************


UPDATES:

* Extended deadlines. Please refer to the website for the updated timeline.
* Test data and evaluation scripts are now available, please refer to the website for details.
* A Slack channel is available for any questions/discussions - please send an email to get an invitation.


Brief Background

SMART task is focused on answer type prediction. Question or answer type classification plays a key role in question answering. The questions can be generally classified based on Wh-terms (Who, What, When, Where, Which, Whom, Whose, Why). Similarly, the answer type classification is determining the type of the expected answer based on the query. Such answer type classifications in literature have been performed as a short-text classification task using a set of coarse-grained types, for instance, either 6 or 50 types within the TREC QA task. A granular answer type classification is possible with popular Semantic Web ontologies such as DBpedia (~760 classes) and Wikidata (~50K classes).


Task Description

In this challenge, given a question in natural language, the task is to predict the type of the answer using a set of candidates from a target ontology. e.g.,


Who is the heaviest player of the Chicago Bulls? -) dbo:BasketballPlayer
How many employees does IBM have? -) number
When did Margaret Mead marry Gregory Bateson? -) date


Datasets

We have created two datasets; one using the DBpedia ontology and the other using the Wikidata ontology each containing 21,964 questions and 22,822 questions respectively. Systems can participate using only one dataset or both.


Submission Details

Participants are requested to submit the system output for the test data. The format is as same as the training data. In addition, the participants are requested to submit a system description that will be included in a joint ISWC challenge proceedings volume in CEUR. System descriptions must be in English either in PDF or HTML, formatted in the style of LNCS, and no longer than 8 pages. Submissions can be sent via email. The accepted systems will get the opportunity to show their results during the ISWC 2020 poster and demo session.


Organizers

Mohnish Dubey, University of Bonn / Fraunhofer IAIS Dresden
Alfio Gliozzo, IBM Research AI
Jens Lehmann, University of Bonn / Fraunhofer IAIS Dresden
Nandana Mihindukulasooriya, IBM Research AI
Axel-Cyrille Ngonga Ngomo, Universit├Ąt Paderborn
Ricardo Usbeck, Fraunhofer IAIS Dresden

Related Resources

ISWC 2023   The International Semantic Web Conference
SOFTPA 2024   3rd International Conference on Emerging Practices in Software Process & Architecture
ICSC 2024   IEEE International Conference on Semantic Computing
BES 2024   8th International Conference on Biomedical Engineering and Science
ISEEIE 2024   2024 4th International Symposium on Electrical, Electronics and Information Engineering (ISEEIE 2024)
CiViE 2024   8th International Conference On Civil Engineering
CCESG 2024   2024 International Joint Conference on Clean Energy and Smart Grid (CCESG 2024)
IJWesT 2023   International Journal of Web & Semantic Technology
CSTFM 2024   2024 International Conference on Smart Transportation and Future Mobility (CSTFM 2024)
ICoTSC 2024   2024 International Conference on Transport and Smart Cities (ICoTSC 2024)