posted by user: srl_cfps || 383 views || tracked by 1 users: [display]

NRE 2016 : Numerical Reproducibility at Exascale


When Nov 18, 2016 - Nov 18, 2016
Where Salt Lake City, UT
Submission Deadline Aug 29, 2016
Notification Due Sep 19, 2016
Final Version Due Dec 19, 2016

Call For Papers


Where: Part of SC16, Salt Lake City, UT
When: Friday morning, Nov 18, 2016
Deadline: Monday, August 29, 2016
Notifications: Monday, September 19, 2016
Full Papers: Monday, December 19, 2016
Organized by: Walid Keyrouz (NIST) and Michael Mascagni (FSU & NIST)
Registration: handled by SC16 (

Motivation and Previous Offerings

This is the second offering of Numerical Reproducibility at Exascale, the first edition was at SC15, and it's webpage can be found here (

A cornerstone of the scientific method is experimental reproducibility. As computation has grown into a powerful tool for scientific inquiry, the assumption of computational reproducibility has been at the heart of numerical analysis in support of scientific computing. With ordinary CPUs, supporting a single, serial, computation, the ability to document a numerical result has been a straight-forward process. However, as computer hardware continues to develop, it is becoming harder to ensure computational reproducibility, or to even completely document a given computation. This workshop will explore the current state of computational reproducibility in HPC, and will seek to organize solutions at different levels. The workshop will conclude with a panel discussion aimed at defining the current state of computational reproducibility for the Exascale.

Call for Participation

Experimental reproducibility is a cornerstone of the scientific method. As computing has grown into a powerful tool for scientific inquiry, computational reproducibility has been one of the core assumptions underlying scientific computing. With "traditional" single-core CPUs, documenting a numerical result was relatively straightforward. However, hardware developments over the past several decades have made it almost impossible to ensure computational reproducibility or to even fully document a computation without incurring a severe loss of performance. This loss of reproducibility started with CPUs that used out-of-order execution to improve performance. It has accelerated with recent architectural trends towards platforms with increasingly large numbers of processing elements, namely multicore CPUs and compute accelerators (GPUs, Intel Xeon Phi, FPGAs).

Programmers targeting these platforms rely on tools and libraries to produce codes or execute them efficiently. As a result, codes can run efficiently, but have execution details that can be impossible to predict and are often very difficult to understand after execution. Furthermore, parallel implementations often result in code with varying execution orders between runs, leading to nonreproducible computations. The underlying reasons are that (1) the hardware and system software allocate parallel work in ways that are not always specifiable at compile time and (2) the execution often proceeds in an opportunistic manner with the execution order changing between runs. As such, floating-point computations, which are non-commutative, can have different execution orders and execute on different processing elements between runs, leading to runs with varying results as a matter of fact. The predictability of systems is further complicated by two issues that are becoming more critical as systems grow in scale: (1) interconnect systems with latencies that are often outside the control of programmers and (2) reliability as the mean time between failure (MTBF) is now measured in hours on large systems. This situation seriously affects the ability to rely on scientific computations as a metrological substitute for experimentation!

Workshop Scope

The workshop is meant to address the scope of the problems of numerical reproducibility in HPC in general and those anticipated as we scale to Exascale machines in the next decade. We initially seek contributions of extended abstracts (two pages) in the areas of computational reproducibility in HPC from academic, government, and industry stakeholders. Areas of interest include, but are not limited to:

- Case studies of reproducibility or the lack of it
- Reproducibility issues in current HPC
- System-level solutions
- Algorithmic solutions
- Software solutions
- Uncertainty quantification in computational reproducibility
- Fundamental numerical analysis of reproducibility
- Future prospects

Workshop Format

The workshop will have:

two plenary talks with authors TBD
a morning session of contributed talks
a panel discussion to summarize the problem, current research, and prospects on long-term solutions
Papers submitted to the workshop will be refereed. The referees will select the papers that will be presented in the workshop. In addition, a group of papers will be published in a special issue of Mathematics and Computers in Simulation (MATCOM) devoted to Numerical Reproducibility.


Submissions of two page extended abstracts are sought. The format for the abstracts is not specified, but full papers that are accepted will be published in the MATCOM special issue. The MATCOM instructions for authors can be found here.

The abstracts are to submitted as a PDF document using Easychair at

Travel Support

Some limited travel support may be available via NIST.

Important Dates (all are Mondays)

August 29, 2016: submission deadline for two page abstracts via
September 19, 2016: notification of to authors about their submissions based on: rejection, acceptance as a paper, acceptance as a paper and presentation
December 19, 2016: submission deadline for full papers for refereeing via the MATCOM site, the papers must be in MATCOM format

Organizers and Co-Editors of the MATCOM Special Issue

Walid Keyrouz, National Institute of Standards and Technology (NIST), USA
Michael Mascagni, National Institute of Standards and Technology (NIST) and Florida State University, USA

Scientific Committee Committee

Dong H. Ahn, Lawrence Livermore National Lab, USA
David Bailey, UC Davis, USA
Mike Heroux, Sandia National Laboratory, USA
David R. C. Hill, Université Blaise Pascal, Clermont-Ferrand, France
Torsten Hofler, ETH-Zurich, Switzerland
Walid Keyrouz (co-organizer), NIST, USA
Miriam Leeser, Northeastern University, USA
Xiaoye Sherry Li, Lawrence Berkeley National Laboratory, USA
Yaohang Li, Old Dominon University, USA
Michael Mascagni (co-organizer), FSU/NIST, USA
Junji Nagano, Insitute of Statistical Mathematics, Japan
Nathalie Revol, INRIA/ENS-Lyon, France
Siegfried Rump, University of Hamburg, Germany
Michela Taufer, University of Delaware


E-mail: (replace ".at." by "@")

Related Resources

AsHES 2016   The Sixth International Workshop on Accelerators and Hybrid Exascale Systems
HiPINEB 2017   The 3rd IEEE International Workshop on High-Performance Interconnection Networks in the Exascale and Big-Data Era
CANA 2016   9th Workshop on Computer Aspects of Numerical Algorithms
NAMDAC 2016   International Workshop on Numerical Algorithms and Methods for Data Analysis and Classification
NSAD 2016   6th Workshop on Numerical and Symbolic Abstract Domains
SDATA 2016   Scientific Data Special Article Collection (Replication Data) - Nature Publishing Group
RepScience 2016   First International Workshop on Reproducible Open Science
RRoBIn 2016   Reproducibility and Robustness in Biological Data Analysis and Integration
EFM 2016   Experimental Fluid Mechanics 2016
REPPAR 2016   International Workshop on Reproducibility in Parallel Computing