ETP4HPC SRA, Multi-dimensional HPC vision: Since October 2014, ECMWF has assumed full membership status of the ETP4HPC as the first NWP centre as an effort to generate a concerted European approach to produce sustainable excellence in HPC for NWP. ESCAPE is the first project aiming to address specific aspects of the ETP4HPC research agenda and their realisation in numerical fine-scale modelling of the atmosphere. Regarding the ETP4HPC’s multi-dimensional HPC-vision these are:
ETP4HPC SRA, Technical research priorities: At the system architecture level, ESCAPE investigates selected options for specialized compute units but does not contribute directly to hardware development. Based on existing technology, the offloading of compute-intensive tasks, e.g. microphysics parameterizations and radiation, to low-energy-cost NVIDIA GPUs (GPU ensembles) and Intel MICs will be established in partnership with HPC centres and vendors. Different intrusion levels of code adaptation and their effect on performance will be benchmarked. An innovative and entirely new approach to perform spectral transforms and large matrix operations, which are major contributors to computational cost in NWP, will be exploited through low-energy optical processing of such operations at the speed of light in partnership with partner OSYS. ESCAPE will perform the interfacing of such processors with existing hardware and the programming environment, not hardware design. This ESCAPE model for employing specialized hardware is applicable to the majority of codes used in weather and climate prediction and thus has community wide impact.
ESCAPE only contributes indirectly to the programming environment by assessing the effectiveness of compiler directives and domain-specific languages for enhancing computing and energy efficiency in key algorithmic components. The collaboration with vendors (and NVIDIA in particular) introduces an interface to compiler design in support of hardware abstraction towards a higher management level of data locality and concurrency.
Enhancing energy efficiency in weather and climate prediction is essential when approaching kilometre-scale simulations at global scale and under operational time constraints. ESCAPE combines a paradigm change for NWP relevant algorithms (Weather & Climate Dwarfs) with a concept for employing specialized hardware in a heterogeneous environment for dedicated tasks dealing with the resolved flow (model dynamics) and unresolved processes (physical parameterizations). New approaches to enhancing energy efficiency represent the core of ESCAPE. ESCAPE proposes the development of novel techniques for minimizing communication, enhancing data locality and for overlapping of computation and communication. Key is to accommodate the diverse set of numerical operations performed, the large dimension of prognostic variables integrated per model time step and the complex dependencies in the internal workflow of NWP models. At the same time, code portability and readability requirements are addressed. In ESCAPE, performance models that simulate the most cost-intensive components in terms of memory usage, computing and communication will be developed for both global as well as regional modelling. These performance models will achieve community-wide applicability by providing a detailed quantification of efficiency gains obtained from algorithmic changes as well as compiler directive and domain-specific language implementations.
Accelerator usage will target the identified compute-intensive tasks, with optical processors specifically targeting spectral transforms and matrix-matrix multiplications, thus offloading the most compute-intensive and costly elements to low-energy hardware. The optimal integration of these units into the overall architecture and workflow will be assessed based on the above described performance models.
ETP4HPC SRA, Completing the value chain: ECMWF combines advanced research and operational application which benefits both the application and service layers spanned by ECMWF (including Copernicus services), its member states and ESCAPE project partners as they represent a significant portion of the European weather forecasting community. The push-through of the envisaged ESCAPE developments follows the same impact route. While the ETP4HPC SRA focuses its recommendations on the industrial impact, a similar value-chain template applies to environmental application and service provision.