posted by organizer: gcheng || 407 views || tracked by 1 users: [display]

SharedTasks@EYRE 2018 : Shared Tasks on Entity Search and Summarization

FacebookTwitterLinkedInGoogle

Link: https://sites.google.com/view/eyre18/sharedtasks
 
When Oct 22, 2018 - Oct 22, 2018
Where Turin, Italy
Submission Deadline Aug 6, 2018
Notification Due Aug 13, 2018
Final Version Due Aug 27, 2018
Categories    entity search   entity summarization   knowledge graph   semantic web
 

Call For Papers

Call for Participation
------------------------------
Shared Tasks at the 1st International Workshop on EntitY REtrieval (EYRE '18)
co-located with CIKM 2018

https://sites.google.com/view/eyre18/sharedtasks


As part of the EYRE '18 workshop co-located with CIKM 2018, two shared tasks related to entity retrieval are organized, in order to assess strengths and weaknesses of existing systems, compare performance of techniques, and enhance communication among researchers and developers.


Task 1: Entity Summarization
------------------------------

An entity in a knowledge base is involved in a set of triple-structured facts that describe the entity. When there are many facts, entity summarization is the task of selecting a size-constrained subset of triples that best represent the entity.

For this shared task, we use the Entity Summarization BenchMark (ESBM, https://w3id.org/esbm). The benchmark has provided a Java tool for evaluating system-generated entity summaries based on a crowdsourced gold standard. Please note that the benchmark is based on two datasets: DBpedia and LinkedMDB. Participants can report their results for either or both datasets, but are encouraged to report both.

We would be happy to add your results to the website of ESBM, assuming you report F-measure for both DBpedia and LinkedMDB under both k=5 and k=10.


Task 2: Entity Search
------------------------------

Entity search is the task of returning a ranked list of entities in a knowledge base in response to a free text user query.

For this shared task, we use the DBpedia-Entity v2 benchmark (http://tiny.cc/dbpedia-entity). The benchmark has provided a crowdsourced gold standard for evaluating system-generated ranked entities. Please note that the benchmark categorizes queries into four groups: named entity queries, IR-style keyword queries, natural language questions, and queries that seek a particular list of entities. Participants can report their results for any subset of categories, but are encouraged to report at least two.

We would be happy to add your results to the website of DBpedia-Entity v2, assuming you follow the provided train/test split.


Submission Guidelines
------------------------------

Each participating system should submit a short paper (up to 4 pages) describing the system and reporting the results. Participants are encouraged to publish the outputs and/or source codes of their systems, and include a link to those resources in their papers.

Manuscripts should be submitted electronically, as PDF formatted using the ACM camera-ready templates; please use sample-sigconf.tex as the template. Papers should be submitted through the EYRE '18 online submission system: https://easychair.org/conferences/?conf=eyre18

Papers will be reviewed by the organizers to ensure the presentation quality. Accepted papers will be included in the CIKM 2018 workshops proceedings. At least one author of each accepted paper is expected to attend the workshop and present.


Important Dates
------------------------------
* Submission: 6 August 2018 (Anywhere on Earth)
* Notification: 13 August 2018
* Camera-ready: 27 August 2018

Related Resources

IJAB 2018   International Journal of Advances in Biology
WSDM 2019   WSDM 2019: The 12th ACM International Conference on Web Search and Data Mining
IEEE SITIS - IWeCa 2018   IEEE SITIS 2018 Conf. - Special Track on Intelligent Web Computing and Applications (I-WeCA)
WSDM 2018   The 11th ACM International Conference on Web Search and Data Mining
KGSWC 2019   1st Iberoamerican Knowledge Graph and Semantic Web Conference
CL-SciSumm 2018   [CfP] CL-SciSumm Shared Task 2018 @SIGIR’2018: The Scientific Summarization Shared Task
JIST 2018   The 8th Joint International Semantic Technology Conference
EYRE 2018   1st International Workshop on EntitY REtrieval, co-located with CIKM 2018
RevOpiD 2018   Opinion Mining, Summarization and Diversification
IEEE SITIS 2018   The 14th International Conference on SIGNAL IMAGE TECHNOLOGY & INTERNET BASED SYSTEMS