Software & Betrieb

Software provision is one of the main tasks of HPC.NRW. This includes not only the appropriate simulation programs, but also components such as optimized numerical libraries, software development tools like sequential and parallel debuggers, tools for performance analysis, and general tools for the efficient use of HPC resources.

To this end, the needs of the data centers, their users, and the HPC support are regularly assessed. This is followed by an evaluation of possible alternatives.

HPC.NRW then acquires the required software and/or its alternatives and makes it available to the users. Particular attention is paid to the use of freely-available packages and, in the case of commercial software, at least a dual-vendor strategy is pursued in order to avoid a vendor lock-in. In addition, HPC.NRW supports other initiatives for state-wide software acquisition such as the state license for NAG compilers and libraries.

Statewide licenses for the following products have already been successfully negotiated by HPC.NRW. They are available via HPC.NRW and include support for all locations:

If a state license only allows a limited number of licenses they are managed by shared and redundantly operated license servers so that the maximum number of license tokens is available at each facility. This allows the largest possible HPC jobs to be run with the programs or tools.

In addition, individual sites or smaller groups thereof have already successfully carried out further acquisitions through HPC.NRW.

The acquisition of simulation programs poses an additional challenge. On the one hand, licensing models are often designed for workgroups rather than data centers, and the software vendor’s client model does not incorporate all data center users (both local and external). On the other hand, simulation programs are usually specifically tailored to individual research directions. HPC.NRW circumvents these problems in different ways:

In addition to the hardware, the software environment of an HPC cluster is one of the most important aspects of an HPC system. In this respect, harmonization of the software environment is almost certainly an essential prerequisite. HPC.NRW has improved the mobility of users between systems and, in particular, between different levels of the HPC supply pyramid (from workgroup clusters, through tier-3 and tier-2 systems to tier-0/1 systems). Users are sufficiently qualified to efficiently utilize HPC systems of higher tiers.

The HPC.NRW competence network pursues an implicit path with regard to harmonization: Provision of best practice guides and practical assistance on HPC operation aim to achieve a gradual harmonization, which still offers sufficient flexibility to incorporate local specifications such as special hardware or the needs of individual user groups.

The HPC security incident in the spring of 2020, which affected systems worldwide, including some in NRW, has shown that best practice guides are essential, especially in the area of HPC system security, and that they are also gratefully received in institutions outside the state of NRW.

The best practice guides are first developed within HPC.NRW and, as soon as they have reached a certain level of maturity, they are published on the HPC Wiki, which is publicly available to a worldwide audience. This accessible approach allows a worldwide collaboration to support the ongoing development of the guides.

Joining forces: a prerequisite for the bundling of software requirements or harmonization is to evaluate the HPC software ecosystem on offer across all sites. Several aspects are relevant here:

A competence network to support the participating sites in operational issues was established through the following activities: