posted by organizer: lefterav || 778 views || tracked by 2 users: [display]

WMT-Testsuites 2023 : Call for Participation: Test suite sub-task of the Eighth Conference on Machine Translation (WMT23)

FacebookTwitterLinkedInGoogle

Link: http://www2.statmt.org/wmt23/testsuite-subtask.html
 
When Dec 6, 2023 - Dec 7, 2023
Where Singapore
Submission Deadline Jun 19, 2023
Notification Due Jul 27, 2023
Final Version Due Sep 1, 2023
Categories    machine translation
 

Call For Papers

the “Test suites” sub-task will be included for the sixth time in the General MT Shared Task of the Conference on Machine Translation (WMT23).


*OVERVIEW*

Test suites are custom extensions to the test sets of the General MT Shared Task, constructed so that they can focus on particular aspects of the MT output. They cοnsist of a source-side test-set and a customized evaluation service. As opposed to the standard evaluation process which produces generic quality scores, test suites often produce separate fine-grained results for each phenomenon.

Given the massive improvement of MT and the emergence of LLMs recently, test suites can be useful at revealing serious flaws of otherwise highly scoring systems.


*IMPORTANT DATES*

19th June: Test suite source texts must reach us


27th July: Translated test suites shipped back to test suites authors:


TBC - September: Test suite description and analysis paper


Potential participants are kindly requested to fill in this form https://forms.gle/s4JEJt9WSqAzP74o6


*MORE INFORMATION*

The sub-task is co-ordinated by Ondrej Bojar and Eleftherios Avramidis.

Further information can be found in the dedicated page of the WMT website
http://www2.statmt.org/wmt23/testsuite-subtask.html


Related Resources

WMT-Testsuites 2024   'Help us break LLMs' - Test suite sub-task of the Ninth Conference on Machine Translation (WMT24)
CCCIoT 2025   6th International Conference on Cloud Computing and IoT
Débâcle 2025   Call for papers 'Débâcle', Suite française Journal, Issue 8/2025
IJHAS 2024   International Journal of Humanities, Art and Social Studies
WMT-metrics 2024   WMT24 Metrics Task: Call for Participation
NeCoM 2025   17th International Conference on Networks & Communications
OP 2025   Call for Papers - Kant's Concept of Spontaneity and Its Legacy in Later Theories of Subjectivity
Carpathian culture and heritage 2025   Call for Papers: The cultures and heritage of the communities along the Carpathian Arc
Call For Papers Special Issue 2024   Smart Cities, innovating in the Transformation of Urban Environments
IJFMA Vol. 10 No. 3 - Dossier II 2025   What Future for the Cinema of Small European Countries? - Open Call for Papers IJFMA Vol. 10 No. 3 Dossier II