Scheduler

To make efficient use of MSA, the scheduling system has to take the system-level heterogeneity into account. This is not limited to the distinction between accelerated and non-accelerated computing nodes, but extends to different accelerator types, heterogeneous memory hierarchies, and hierarchical storage systems. This variety of resources, even including global, non-compute resources (e.g. To make efficient use of MSA, the scheduling system has gateway resources or I/O nodes) has to be made available to take the system-level heterogeneity into account to application scenarios ranging from coupled cosimulations to application workflows executing each step on the most suitable platform one after another. As Slurm–the most widely used open source scheduling system in HPC as of today–lacks support for the management of global resources, ParaStation Management takes over the task of managing the gateway nodes in MSA systems with network federation.

To address the applications’ need for more flexibility in terms of required and available resources, the resource management and scheduling facilities will have to provide support for malleability in the future. This requires the negotiation of resource adjustments between applications and/or workflow engines and the resource manager. That way, large-scale application workflows can be started without the necessity for all resources being available from the beginning. This kind of customisation, in which the ratio and the type of resources can be defined during runtime to match each application, is inherent to MSA. It is realised with a dynamic scheduling and resource management, which has to be modular itself, to match the demands of system operators for standardised interfaces between the different components of the system software. The Process Management Interface PMIx is such an open standard designed to act as the interface between runtime systems and resource managers and is (in its current form) already supported by several MPI implementations (e.g., ParaStation MPI) and several resource managers (e.g., Slurm).