posted by user: grupocole || 1422 views || tracked by 2 users: [display]

Insights 2023 : The Forth Workshop on Insights from Negative Results in NLP

FacebookTwitterLinkedInGoogle

Link: https://insights-workshop.github.io/
 
When Jun 2, 2023 - Jun 6, 2023
Where Dubrovnik, Croatia
Submission Deadline Feb 13, 2023
Notification Due Mar 13, 2023
Final Version Due Mar 27, 2023
Categories    NLP   computational linguistics   artificial intelligene
 

Call For Papers



Dear colleagues,

The Fourth Workshop on Insights from Negative Results in NLP Co-located with EACL, May 2 or 6, 2023

First Call for Participation

TO-BE-UPDATED: (https://insights-workshop.github.io/)

Contact email: insights-workshop-organizers@googlegroups.com


*Overview

Publication of negative results is difficult in most fields, but in NLP the problem is exacerbated by the near-universal focus on improvements in benchmarks. This situation implicitly discourages hypothesis-driven research, and it turns creation and fine-tuning of NLP models into art rather than science. Furthermore, it increases the time, effort, and carbon emissions spent on developing and tuning models, as the researchers have no opportunity to learn what has already been tried and failed.

This workshop invites both practical and theoretical unexpected or negative results that have important implications for future research, highlight methodological issues with existing approaches, and/or points out pervasive misunderstandings or bad practices. In particular, the most successful NLP models currently rely on different kinds of pretrained meaning representations (from word embeddings to Transformer-based models like BERT and GPT-3). To complement all the success stories, it would be insightful to see where and possibly why they fail. Any NLP tasks are welcome: sequence labeling, question answering, inference, dialogue, machine translation - you name it.

A successful negative results paper would contribute one of the following:

** broadly applicable recommendations for training/fine-tuning, especially if X that didn’t work is something that many practitioners would think reasonable to try, and if the demonstration of X’s failure is accompanied by some explanation/hypothesis;

** ablation studies of components in previously proposed models, showing that their contributions are different from what was initially reported;

** datasets or probing tasks showing that previous approaches do not generalize to other domains or language phenomena;

** trivial baselines that work suspiciously well for a given task/dataset;

** cross-lingual studies showing that a technique X is only successful for a certain language or language family;

** experiments on (in)stability of the previously published results due to hardware, random initializations, preprocessing pipeline components, etc;

** theoretical arguments and/or proofs for why X should not be expected to work;

** demonstration of issues with data processing/collection/annotation pipelines, especially if they are widely used;

** demonstration of issues with evaluation metrics (e.g. accuracy, F1, or BLEU), which prevent their usage for fair comparison of methods.


* Important Dates

** Submission due: February 13, 2023

** Submission due for papers reviewed through ACL Rolling Review: March 17, 2023

** Notification of acceptance: March 13, 2023

** Camera-ready papers due: March 27, 2023

** Workshop: May 2 or 6, 2023

* Submission

Submission is electronic, using the Softconf START conference management system.

Submission link: (https://softconf.com/eacl2023/insights2023/)

The workshop will accept short papers (up to 4 pages, excluding references), as well as 1-2 page non-archival abstract submissions for papers published elsewhere (e.g. in one of the main conferences or in non-NLP venues). The goal of this event is to stimulate a meaningful community-wide discussion of the deep issues in NLP methodology, and the authors of both types of submissions will be welcome to take part in our get-togethers.

The workshop will run its own review process, and papers can be submitted directly to the workshop by March 13, 2023. It is also possible to submit a paper accompanied by reviews from the ACL Rolling Review system by March 17, 2023. The submission deadline for ARR papers follows the ACL RR calendar. Both research papers and abstracts must follow the ACL two-column format. Official style sheets:

(https://www.overleaf.com/read/crtcwgxzjskr)

(https://github.com/acl-org/ACLPUB/tree/master/templates)

Please do not modify these style files, nor should you use templates designed for other conferences. Submissions that do not conform to the required styles, including paper size, margin width, and font size restrictions, will be rejected without review.


* Multiple Submission Policy

The workshop cannot accept work for publication or presentation that will be (or has been) published elsewhere and that has been or will be submitted to other meetings or publications whose review periods overlap with that of Insights. Any questions regarding submissions can be sent to insights-workshop-organizers@googlegroups.com.

If the paper has been rejected from another venue, the authors will have the option to provide the original reviews and the author's response. The new reviewers will not have access to this information, but the organizers will be able to take into account the fact that the paper has already been revised and improved.

* Anonymity Period

We are not enforcing any anonymity period.

* Presentation

All accepted papers must be presented at the workshop to appear in the proceedings. Authors of accepted papers must notify the program chairs by the camera-ready deadline if they wish to withdraw the paper. At least one author of each accepted paper must register for the workshop.

Previous presentations of the work (e.g. preprints on arXiv.org) should be noted in a footnote in the camera-ready version (but not in the anonymized version of the paper).

The workshop will take place on May 2 or 6 2023. The workshop will be hybrid with both in-person and virtual presentations.

* Organization Committee

** Shabnam Tafreshi, University of Maryland: ARLIS

** Arjun Reddy Akula, Google

** João Sedoc, New York University

** Anna Rogers, University of Copenhagen

** Aleksandr Drozd, RIKEN

** Anna Rumshisky, University of Massachusetts Lowell / Amazon Alexa

* Contact info

Any questions regarding the workshop can be sent to insights-workshop-organizers@googlegroups.com.


Please continue reading about: Authorship, Citation and Comparison, Ethics Policy, Reproducibility, Anonymity Period, and Presentation in the call for the paper page on our website: https://insights-workshop.github.io/2023/cfp/


Regards,

Insights 2023 Organizers

Related Resources

Insights 2024   Fifth Workshop on Insights from Negative Results in NLP
IEEE-Ei/Scopus-ITCC 2025   2025 5th International Conference on Information Technology and Cloud Computing (ITCC 2025)-EI Compendex
LIMO2024@KONVENS 2024   2nd workshop on Linguistic Insights from and for Multimodal Language Processing @KONVENS 2024
ACL 2025   The 63rd Annual Meeting of the Association for Computational Linguistics
Exploring Data Ecosystems and Markets 2024   Special Issue on Exploring Data Ecosystems and Markets: Trends and Insights in VLDB Journal
LDK 2025   Fifth Conference on Language, Data and Knowledge
DMML 2025   6th International Conference on Data Mining & Machine Learning
ERROR 2024   4th Workshop on Escience ReseaRch leading tO negative Results
CyFRP 2024   Advances in Interdisciplinary Cybersecurity: Insights from Funded Research Projects
Big Data in Healthcare 2024   Special Session (SS10) on Harnessing the Power of Big Data in Healthcare: Transforming Insights into Action