posted by organizer: gfursin || 2109 views || tracked by 2 users: [display]

ReQuEST-DNN 2018 : 1st Reproducible Quality-Efficient Systems Tournament: Pareto efficient deep learning


When Mar 24, 2018 - Mar 24, 2018
Where Williamsburg, VA, USA
Abstract Registration Due Feb 5, 2018
Submission Deadline Feb 12, 2018
Notification Due Feb 22, 2018
Final Version Due Mar 9, 2018
Categories    artificial intelligence   computer science   machine learning

Call For Papers



ReQuEST: 1st Reproducible Quality-Efficient Systems Tournament
Call for Pareto efficient deep learning

Intent to submit: February 5, 2018 AoE

Associated ReQuEST workshop co-located with ASPLOS 2018
March 24th, 2018 (afternoon), Williamsburg, VA, USA


Co-designing emerging workloads across the hardware/software
stack to optimize for speed, accuracy, costs and other metrics
is extremely complex and time consuming. The lack of
a rigorous methodology and common tools for open, reproducible
and multi-objective optimization makes it challenging or even
impossible to evaluate and compare different published works
across numerous heterogeneous hardware platforms, software
frameworks, compilers, libraries, algorithms, data sets, and

The 1st ReQuEST workshop aims to bring together
multidisciplinary researchers in systems, compilers,
architecture and machine learning to optimize the quality vs.
efficiency Pareto optimality of deep learning systems
on complete hardware/software platforms in a standardized,
reproducible and comparable fashion. The target application
for the first incarnation of ReQuEST will be the ImageNet
Large Scale Visual Recognition Challenge (ILSVRC) and will
focus solely on optimizing inference on real systems
. Restricting the competition to a single application domain
will allow us to test an open-source tournament
infrastructure, validate it across multiple platforms and
environments, and prepare a dedicated live scoreboard with
the "winning" solutions. For future incarnations of ReQuEST,
we will provide broader application coverage.

Unlike the classical ILSVRC where submissions are ranked based
on accuracy, ReQuEST submissions will be evaluated across
multiple metrics and trade-offs (exposed by authors):
accuracy, speed, throughput, energy, cost of usage, etc.
Furthermore, in contrast with other deep learning benchmarking
challenges ReQuEST participants will be asked to submit
a complete workflow artifact (see submission procedures) which
encompasses toolchains, frameworks, algorithm, libraries, and
target hardware platform; any of which can be fine-tuned,
or customized at will by the participant to implement their
optimization technique.

We strongly encourage artifact submissions for already
published techniques since one of the ReQuEST goals is to
prepare an open set of reference and optimized implementations
of popular deep learning algorithms as portable and
customizable workflows which can be easily reused, improved
and build upon!

A ReQuEST artifact evaluation committee (AEC) will be tasked
to independently reproduce and evaluate workflow submissions
on compliant hardware platforms to reproduce results and
aggregate them in a multi-objective public leaderboard. Due to
the multi-faceted nature of the competition, submissions won't
be ranked according to a single metric, but instead the AEC
will assess their Pareto optimality across two or more metrics
exposed by authors. There won't be a single ranking
of submissions since this competition is multi-objective:
it accounts for classification accuracy, inference latency,
energy, ownership/usage cost and so on. As such, there won't
be a single winner, but better and worse designs based
on their relative Pareto optimality (up to 3 design points
allowed per submission).

The workshop co-located with ASPLOS 2018 will be the
opportunity for the participants to share their research and
implementation insights with the research community. A common
academic and industrial panel will be held at the end of the
workshop to discuss how to improve common SW/HW co-design
methodology for deep learning and other real-world
Further details about deadlines, submission procedures
and artifact evaluation:


Looking forward to your participation and submissions!

Related Resources

Recommender Systems 2020   Data Science for Next-Generation Recommender Systems
ICDMML 2020   【EI SCOPUS】2020 International Conference on Data Mining and Machine Learning
Machine Learning Computer Networks@ESANN 2020   Machine Learning applied to Computer Networks - Special Session @ ESANN 2020
ISBDAI 2020   【Ei Compendex Scopus】2020 International Symposium on Big Data and Artificial Intelligence
ADC 2020   The 31st Australasian Database Conference
IEEE-CVIV 2020   2020 2nd International Conference on Advances in Computer Vision, Image and Virtualization (CVIV 2020)
SIUSAI 2021   2021 International Symposium on Intelligent Unmanned Systems and Artificial Intelligence(SIUSAI 2021)
ICPR 2020   International Conference on Pattern Recognition 2020
RQD 2020   26th ISSAT International Conference on Reliability and Quality in Design
FSPC 2020   2020 2nd International Conference on Frontiers of Systems, Process and Control (FSPC 2020)