E-CAM Final Report
Page 30
5 Conclusions E-CAM has sought to prepare the extended E-CAM community for the demands of upcoming exascale systems. We have addressed core cross-cutting topics such as • Load balancing - through the development of the ALL load-balancing library, • Intelligent High Throughput Computing - through the development of the jobqueue_features library • Accelerator portability - through the promotion of libraries such as Kokkos, and implementations in E-CAM codes, • Portability of entire software stacks - through contributions to EasyBuild and, now, EESSI E-CAM has paved the way for the portability of entire application stacks and workflows to all the architectures that will be available through EuroHPC. These alone are major contributions to preparing the E-CAM community, providing them with some of the tools that they will need to address issues that are likely to occur at scale: hardware diversity, portability of workflows, load imbalance, resiliency, coordinating ensemble petascale calculations. . . E-CAM has gone far beyond cross-cutting work, however, and has repeatedly worked with individual applications to evaluate and improve their performance, collaborating with other projects in the EuroHPC eco-system along the way. A primary example of this is given by the DL_MESO application which, as a result of E-CAM, has been shown to be scalable to 4096 GPUs (the largest GPU partition available in Europe at the time). In addition, E-CAM has raised awareness of best practices in scientific computing, attempting to encourage people to consistently use • version control, • documentation, • continuous integration, • modular software development, • configure/build/install installation processes and the tools that can support these, • open source software licences. E-CAM has trained hundreds of scientists and exposed them to High Performance Computing (HPC) resources and tools that they might never otherwise have known. It has raised awareness of the EuroHPC eco-system among the thousands of scientists that are associated with CECAM, and trained the next-generation of those scientists so that HPC is a go-to tool in their computational workflows. The scientific output from E-CAM is remarkable, both in terms of the development of new algorithms and methods and the related scientific publications with clear benefits for academic and industrial researchers. We produced 44 publications during E-CAM, and our articles have more than 1980 citations so far. E-CAM has injected about 5.5 M€ funding into the community, to fund developments that are freely available on our software library, on the training portal and through open access scientific publications. Our software library is composed of 220 certified software modules, and ten more modules are work in progress. A set of actions (software development, new formats for workshops) has been initiated that has clearly proved its strategic interest for the community and will be maintained and further developed. Specific tools and initiatives will be continued and have funding at least for the next 48 months. A list is below. Workshops • Extended Software Development Workshops: – Improving bundle libraries, 11-22 October 2021, CECAM-HQ (EPFL). Workshop website.
