The GEM (Generation, Evaluation, Metrics) workshop at ACL, 2021 is inviting transformation submissions to NL-Augmenter 🦎 → 🐍.

The NL-Augmenter is a collaborative effort intended to add transformations of datasets dealing with natural language. Transformations augment text datasets in diverse ways, including: introducing spelling errors, translating to a different language, randomizing names and numbers, paraphrasing, etc. and whatever creative augmentation you contribute to the benchmark. We invite submissions of transformations to this framework by way of GitHub pull request, through September 1, 2021. All submitters of accepted transformations (and filters) will be included as co-authors on a paper announcing this framework by the end of 2021.

Note that the call is related to the GEM workshop at ACL'21, but only a preliminary list of transformations will be presented (by the NL-Augmenter organizers) at the workshop . We encourage submitters to attend the workshop (not mandatory).

Project: https://github.com/GEM-benchmark/NL-Augmenter

We strongly believe that the benefits of open science should reach everyone and hence we are making this effort to reach you. We also encourage you to share this with other researchers in your department who would benefit from this open collaboration. To know more about the framework, check our motivation and review criteria and some of our recent work.

GEM: https://gem-benchmark.com/
Motivation and review criteria: https://github.com/GEM-benchmark/NL-Augmenter/blob/main/docs/doc.md
More information (paper): https://arxiv.org/pdf/2106.09069.pdf


Organizers:
Kaustubh Dhole (Amelia R&D)
Sebastian Gehrmann (Google AI Language)
Varun Gangal (LTI, Carnegie Mellon University)
Jascha Sohl-Dickstein (Google Brain)
Tonghuang Wu (University of Washington)
Simon Mille (Universitat Pompeu Fabra)
Zhenhao Li (Imperial College, London)
Saad Mahmood (Trivago R&D)
Aadesh Gupta (Amelia R&D)
Samson Tan (Salesforce Research)
Jinho Choi (Emory University)