Revision as of 22:52, 22 January 2020

SW4lite

In high-performance computing (HPC), proxy applications (“proxy apps”) are small, simplified codes that allow application developers to share important features of large applications without forcing collaborators to assimilate large and complex code bases. The Exascale Proxy Applications Project, part of the Exascale Computing Project (ECP), is a project where developers can share explore and share proxy apps.

Proxy apps are often used as models for performance-critical computations, but proxy apps can do more than just represent algorithms or computational characteristics of apps. They also capture programming methods and styles that drive requirements for compilers and other elements of the toolchain. Within ECP, application teams, co-design centers, software technology projects, and vendors all plan to use proxy apps as a major mechanism to drive collaborations and co-design solutions for exascale challenges.

A major goal of the Exascale Proxy Applications Project is to improve the quality of proxies created by ECP and maximize the benefit received from their use. To accomplish this goal, an ECP proxy app suite composed of proxies developed by ECP projects that represent the most important features (especially performance) of exascale applications has been created. To ensure high quality of ECP proxy apps, the Exascale Proxy Applications Project has defined standards for documentation, build and test systems, performance models and evaluations, etc.

Figure 1. The communication pattern of SW4lite proxy app.

One such proxy application is the SW4lite app. Sw4lite is a bare bone version of SW4 intended for testing performance optimizations in a few important numerical kernels of SW4. SW4 implements substantial capabilities for 3-D seismic modeling, with a free surface condition on the top boundary, absorbing super-grid conditions on the far-field boundaries, and an arbitrary number of point force and/or point moment tensor source terms. Each source time function can have one of many predefined analytical time dependencies, or interpolate a user-defined discrete time series.

The proxy starts by decomposing the original 2D grid into a number of partitions equal to the number of available processes. Each processor is assigned one equally sized partition of the grid, and thus the processors can be thought to be logically positioned in a 2D grid. Next, some preprocessing takes place on these partitions that includes some data sharing between them. Once this step completes an iterative process starts consisting of computations intercepted by two cycles of neighbor-to-neighbor communication per iteration. The communication pattern of each cycle is as follows: each processor sends a portion of the partiton it holds to its left neighbor and waits to receive the respective data from its right neighbor. Once the respective data are received, the same pairs of processors share data in the opposite direction. Next, the same process takes place for the y-axis, each processor sends another portion of its data to its bottom neighbor and waits to receive the respective data from its top neighbor, and then they communicate in the opposite direction. Processors located in the edges of the grid only send/receive data to/from their remaining neighbors. Figure 1 shows the communication pattern schematically.

MPI Times

PREMA Times

Allocation: 300 cores in different configurations

Pure MPI Time: 182.32

Cluster: Wahab

Nodes:

     d4-w6420b-[07-12],
     e1-w6420b-20,
     e2-w6420b-[02-04,06,08,17],
     e3-w6420b-[09-12,17,20]

Allocation: 1000 cores without dedicated thread for MPI. 1 MPI rank per socket -> 4 MPI ranks/node. Pure MPI Time: 69.8s

Cluster: Turing

Pure MPI Time: 268.56

Allocation: 310 cores (due to issues with the tools on Turing)

496 cores