Moscow State University High Performance Computing

MOSCOW STATE UNIVERSIT Y H I G H

P E R F O R M A N C E

C O M P U T I N G

FACILITIES APPLIC ATIONS AC TIVITIES

Contents Supercomputing Facilities

4

Computers and computing in Moscow State University

5

MSU Supercomputers: T-Platforms “Lomonosov”

9

MSU Supercomputers: SKIF MSU “Chebyshev”

11

MSU Supercomputers: Hewlett-Packard “GraphIt!”

12

MSU Supercomputers: IBM Blue Gene/P

13

Perspective supercomputing technology: RVS-5 reconfigurable supercomputer

14

Supercomputing Applications Investigation of the heliospheric boundary

16 17

Turbulent astrophysical convection and magneto-convection in compressible fast rotating spherical shells 18 Development of mesoscale distributed-memory atmospheric model

19

Development of a new generation mathematical model of the dynamics of the oceans. Investigation of the sensitivity of interannual variability of the circulation of ocean waters to climate changes

20

Prognosis of perspective variants of color proteins by the results of calculations with the quantum mechanical molecular mechanical methods 21 Modelling mechanisms of hydrolysis reactions of cyclic nucleoside phosphates

22

Theoretical and computational studies of nano-size single-molecule devices

23

Investigation of point mutation influence on the bacterial potassium channel KcsA conduction and selectivity

24 Simulation the seasonal flooding of the Volga-Akhtuba floodplain

25

Specialized software to optimize and control of ventilation systems of large industrial plants on the basis of gas dynamic simulations using parallel technologies 26 Propagation of nonlinear acoustic signals in a randomly inhomogeneous turbulent medium with different types of spectra

27

Computational modelling of open rotor noise

28

Development and application of new methods in computational aerodynamics and rarefied gas dynamics

29

Numerical modelling of initiation and propagation of detonation waves in tubes with complex shape

30

High-performance computing for the problems of multiphase dynamics and explosion hazards

31

Investigation of development of hydrodynamic properties in problems with anomalous properties of liquids 32 Supercomputer simulation of detonation in chambers and channels of complicated geometry

33

Study of the mechanism of ion transfer in biological membranes by molecular dynamics

34

Supercomputer-aided drug design

35

Mechanisms of saturation of shock wave parameters in high power focused beams of pulsed and periodic signals

36

Improving the efficiency of heating biological tissue irradiated by high intensity ultrasound through the rib cage using nonlinear acoustics effects 37 Three-dimensional nonlinear fields of therapeutic ultrasound arrays

38

Alzheimer disease treatment drug design

39

Computer modelling of acetylcholine /acetylcholinesterase hydrolysis

40

Heat exchanger flow modelling using CABARET method

41

Modelling high speed transient dynamic processes in mechanics of deformable solids using supercomputer technologies 42 Formation of quantum low-dimensional nanostructures on a semiconductor surfaces and its combined study by means of LT STM/STS and DFT techniques 43 Supercomputer-aided design of bioinspired functional nano-structures based on silk-forming peptides and conductive thiophene blocks 44 Polyelectrolyte complexes consisting of macromolecules with varied stiffness: computer simulation

45

Numerical simulations of laser radiation fields in highly anisotropic scattering media

46

Interaction of femtosecond laser filaments

47

Supercomputer modelling of sub-cycle mid-infrared conical emission from filamentation in air

48

Nonlinear optics of ultrashort laser pulses in the multiple filamentation regime

49

Supercomputer simulation of polyamphiphiles

50

Supercomputer simulation of membranes for fuel cells

51

Development of a numerical algorithm for solving the self-consistent mean-field equations for systems of copolymers with rigid and flexible blocks

52

Supercomputing Activities

54

Supercomputing Consortium of Russian Universities

55

Top50 rating: the most powerful supercomputers of CIS

56

Joint MSU-Intel center on High-Performance Computing

58

CUDA Center of Excellence

59

Supercomputing conferences

60

The Project "Supercomputing Education"

61

HOPSA: Holistic Performance System Analysis

64

Parallel programs efficiency and scalability evaluation

66

Cleo: cluster batch system

67

X-Com: distributed computing software

68

New methods of efficient use of GPU-based high-performance computers

70

NUDA: using extensible languages for GPGPU programming

71

SIGMA: collective bank of tests on parallel computing

72

MSU supercomputing workshops

73

Trainings on application development and optimization

74

Summer Supercomputing Academy

75

Excursions to supercomputing centers

76

Supercomputing Facilities

4

Computers and computing in Moscow State University The history of computers at Moscow State University goes back to the mid-fifties of the 20th century when Research Computing Center of Moscow State University (RCC MSU) was founded in 1955 and equipped with up-to-date computing hardware. This made it possible for university researchers to solve many challenging problems in meteorology, satellite and manned space flights, aerodynamics, structural analysis, mathematical economy, and other fields of science.

In 1956, RCC MSU received its first computer “Strela”. It was the first serially manufactured mainframe in the USSR. “Strela” mainframe functioned with a three-address instruction set capable of implementing approximately 2000 operations per second. It had a clock cycle of 500 microseconds, RAM of 2048 words with 43 bits each, energy consumption of 150 KW. The computer occupied up to 300 square meters.

In 1961, M-20 computer was installed in RCC MSU. Mainframe M-20 provided 20000 operations per second. It had ferrite core-based RAM with capacity of 4096 words, with external memory stored on drums and magnetic tapes. These common and efficient mainframes had essential influence on the development of computational mathematics in the former Soviet Union.

BESM-6 computer was and is still considered to be of great importance to Russian history of computer development. Designing of BESM-6 was completed in 1967 and its serial production was started in 1968. Same year RCC MSU received its first BESM-6 computer, and despite its serial number 13 it proved to be lucky for the Center. As a result RCC MSU installed its second BESM-6 computer in 1975, and then the third and the forth ones in 1979. BESM-6 had RAM on ferrite cores capable of storing 32 000 of 50-bit words. This number was later increased

Computer “Setun” was originally designed in RCC MSU. In 1959, RCC launched “Setun” prototype and in 1961 “Setun” started to be manufactured serially. It was an impressive and extraordinary computer, being the first one in the world that was based on ternary, not binary, logic. Trit, having capacity superior to that of a bit, can exist not in two, but in three states: 1,0,-1. The “Setun” computer took up to 25-30 square meters, and required no special cooling.

Its frequency was 200 kHz.

5

a 3.2 GB HDD. The system peak performance was 18 GFlops. The SCI network with a high data transfer rate (80 MB/s) and low latency (5.5 ns) made this system very effective for solving a wide range of problems. Research groups formed around the first cluster started using a new type of technology – parallel computers with distribRCC MSU has also used mainframes from uted memory in order to boost their research. other series. In 1981, along with four BESM-6 In 2002, the second cluster with a standard mainframes RCC was equipped with two ES1022, two MIR-2 and MINSK-32 computers. low-cost and effective Fast Ethernet technology In 1984, two-processor ES-1045 was installed. for communication and control was installed.

Since 1986, RCC MSU has used a series of mini- This cluster contained 20 nodes of one type (2 computers: SM-3, SM-4 and SM-1420. Between x Intel Pentium III/850 MHz, 1 GB, 2 x HDD 15 1955 and the early 1990s, more than 25 main- GB) along with 24 nodes of another type (2 x Inframe computers of various architecture and tel Pentium III/1 GHz, 1 GB, HDD 20 GB). With performance were installed and actively used at a total number of 88 processors, it had peak performance of 82 GFlops.

Moscow State University. to 128 000 words. The BESM-6 peak performance was one million instructions per second.

The computer had about 60000 transistors and three times more diodes. It had a frequency of 10 MHz, occupied up to 150-200 square meters and consumed 30 KW of energy supply.

Since 1999, Research Computing Center has decided to focus its main attention on cluster supercomputers. The result of this decision wasn’t obvious at that time, but later it has proved to be the right one. The first cluster consisted of 18 compute nodes connected via a high-speed SCI network. Each node contained two Intel Pentium III/500 MHz processors, 1 GB of RAM and

6

In 2004 a new Hewlett-Packard cluster with 160 AMD Opteron 2.2 GHz processors and a new InfiniBand network technology was launched in the supercomputing center. This cluster peak performance exceeded 700 GFlops. By that time more than 50 research groups from MSU, Russian Academy of Sciences and other Russian

universities had become active users of MSU supercomputing facilities.

Now Moscow State University Supercomputing Center exploits “Lomonosov”, SKIF MSU “Chebyshev”, “GraphIT!”, IBM Blue Gene/P supercomputers and several small HPC clusters, with a peak performance of the “Lomonosov” flagship at 1.7 PFlops. Taking the supercomputing road more than ten years ago Moscow State University Supercomputing Center is planning to move forward to exaflops and further in the future.

■

7

“Lomonosov”

8

MSU Supercomputers: T-Platforms “Lomonosov” “Lomonosov” supercomputer was installed at the Moscow State University in 2009. This supercomputer was created by Russian company “T-Platforms.” The system launch ceremony was

attended by D.A. Medvedev, President of Russian Federation, who proposed to name the supercomputer after the great Russian XVIII century scientist.

Peak performance 1.7 PFlops Linpack performance 872.5 TFlops

X86 compute nodes GPU compute nodes PowerXCell compute nodes X86 processors X86 cores GPU CUDA cores Total RAM Main processor types System / Service / Management Network Storage system Operating system Total area (supercomputer) Power consumption Total equipment weight

5 104 8 840 30 12 346 52 168 954 240 92 TB Intel® Xeon X5570 / X5670, NVIDIA X2070 QDR Infiniband 4x / 10G Ethernet / Gigabit Ethernet Lustre parallel file system, NFS, StorNext hierarchical file system, backup and archiving system Clustrx T-Platforms Edition 252 m2 2.6 MW More than 75 tons

9

“Lomonosov” Supercomputer Timeline 2009 – First stage: designing, installation and commissioning of the base section of “Lomonosov”. The main computational supercomputer’s section consisted of 4160 dual-processor diskless compute nodes based on 4-core Intel Xeon 5570 processors, the second section included 260 dual-processor compute nodes with 4-core Intel Xeon X5570 processors and local hard drives. The total number of x86 processor cores was 35,360. In addition to x86 compute nodes, the supercomputer included 26 nodes based on PowerXCell8i accelerators. The total amount of memory was 56.5 TB, storage – 0.35 PB, and backup system capacity – 234 TB uncompressed. Supercomputer’s power consumption was 1.5 MW. At the time, its peak performance was estimated at 420 Tflops, and Linpack performance – 350 Tflops which resulted in a very good efficiency index of 83%. This performance level allowed “Lomonosov” to lead the list of the most powerful computers in CIS and Eastern Europe. It was ranked 12th in the global Top500 list in November, 2009.

2010 – The second stage in the supercomputer creation process. The system was supplemented with 640 diskless compute nodes based on TB2-XN computing platform and 40 compute nodes equipped with local HDD storage. Each of new compute nodes was equipped with 6-core Intel Xeon X5670 processor as CPU. The total amount of RAM increased to 79.92 TB, storage - to 1.75 PB. Supercomputer’s peak performance increased to 510 Tflops, and its Linpack performance was 397.1 Tflops. The efficiency was 77.8%. Its decline from preceding year’s level was caused by system’s heterogeneity, as compute nodes with different CPUs were used in the test.

2011 – Third stage: system expansion. Following the trend in the supercomputer industry, “Lomonosov” was additionally supplemented

10

with 777 compute nodes equipped with GPUbased computing accelerators. As a hardware platform for nodes, TB2-TL solution was used where each node has two Intel Xeon E5630 CPUs and two NVIDIA X2070 computing accelerators.

A total peak performance of the computer system was 1.37 Pflops, and Linpack performance – 674 Tflops. “Lomonosov” was ranked 13th in the June 2011 edition of Top500 list.

In June 2011, “Lomonosov” was included in the Graph500 list. According to tests results, the system ranked third (positions were allocated depending on the workload), but showed the best performance among all systems in the list. During tests, a result of 43,471,500,000 TEPS (Traversed Edges Per Second) was obtained using 8192 cores/4096 nodes based on Intel Xeon 5570 processors. Lately the system was ranked 2nd in the November 2011 list edition with 103,251,000,000 TEPS using 32,768 cores / 4,096 nodes based on Intel Xeon 5570 processors.

2012 – The fourth stage and yet another system expansion round. The supercomputer has been additionally equipped with 288 compute nodes with Intel® Xeon X5570/X5670 processors and GPU-based computing accelerators. Its total amount of memory has increased to 92 TB, and now computer consumes 2.6 MW. As a result of modernization, a peak performance of the computing system has been increased to 1.7 Pflops, and Linpack performance reached 872.5 Tflops.

■

MSU Supercomputers: SKIF MSU “Chebyshev” On March 19, 2008 Moscow State University, T-Platforms, PSI RAS and Intel announced the deployment of the most powerful supercomputer in Russia, CIS and Eastern Europe SKIF MSU “Chebyshev” that was built in the framework of the joint Russian-Belorussian supercomputing program “SKIF-GRID”. The peak performance of the supercomputer based on 1 250 Intel Xeon E5472 (3.0 GHz, 45 nm) quad-core processors, is 60 TFlops. The Linpack performance of 47.17 TFlops (78.6% of peak performance) had become the best efficiency result among all quadcore Xeon-based systems in the top hundred of

the June 2008 edition of the Top500 list where SKIF MSU “Chebyshev” was ranked #36. It was ranked #11 in the recent (March 2012) edition of Top50 rating list of the most powerful CIS supercomputers. The supercomputer is based on T-Blade modules which incorporates up to 20 CPUs in a 5U enclosure. The system network is based on the DDR InfiniBand technology with Mellanox 4th generation microchips. SKIF MSU “Chebyshev” data storage capacity provided by T-Platforms ReadyStorage ActiveScale Cluster is 60 TB.

■

Peak performance

60 TFlops

Linpack performance

47 TFlops

Linpack efficiency

78.6%

Compute racks / total racks

14 / 42

Blade enclosure / blade nodes Number of CPUS / cores Processor type Total RAM Primary / secondary interconnect Power consumption Top500 position

63 / 625 1 250 / 5 000 4-core Intel Хеоn 5472 3.0 GHz 5.5 TB DDR Infiniband / Gigabit Ethernet 330 KW 36 (2008.VI)

11

MSU Supercomputers: Hewlett-Packard “GraphIt!” “GraphIT!” is the first cluster of MSU Supercomputing Center based on GPU, an innovative supercomputing architecture. GPUs, originally designed for real-time 3D graphics acceleration, are now widely used to accelerate HPC. Compared to traditional CPUs, GPUs provide higher parallelism, higher Flops and memory bandwidth per chip, and also have higher cost- and energy-efficiency.

tions. As a result, configuration based on 4 HP S6500 4U chassis, occupying a total of 2 racks was chosen. Each chassis has 4 nodes, and each node has 3 NVidia “Fermi” Tesla M2050 CUDAenabled GPUs, for a total of 16 compute nodes and 48 PUs in the cluster. All compute nodes are connected by a high-speed 4x QDR InfiniBand network. This provides a total performance of 26.76 TFlops, of which 24.72 TFlops, or more than 92%, are due to GPU. It achieves “GraphIT!” was originally envisioned as a piLinpack performance of 11.98 TFlops, with 44% lot GPU-based cluster which can be used as efficiency. a testbed for practicing with hybrid programming technologies. It was required to be small “GraphIT!” cluster is used to solve problems enough to fit into existing server room but pow- on molecular dynamics, cryptoanalysis, quanerful enough to be used for real-world applica- tum physics, climate modelling, as well as other computationally intensive problems which benefit from GPU usage. It is used by researchers from various MSU departments as well as other research institutions.

■

Peak performance (CPU / GPU / CPU+GPU) Linpack performance Racks / compute nodes Node type Number of 6-cores Intel Xeon X5650 CPUs CPUs per node Number of GPUs GPU type Total CPU RAM / GPU RAM Per node CPU RAM / GPU RAM Data storage capacity Primary / secondary interconnect Power consumption

12

2.04 / 24.72 /26.76 TFlops 11.98 TFlops 2 / 16 DL380G6 32 2 48 Nvidia "Fermi" Tesla M2050 768 GB / 144 GB 48 GB / 9 GB 12 ТB QDR Infiniband 4x / Gigabit Ethernet 22 KW

MSU Supercomputers: IBM Blue Gene/P Since 2008 the IBM Blue Gene/P supercomputer has been operating at the Faculty of Computational Mathematics and Cybernetics of MSU. The MSU Blue Gene/P computer was one of the first systems of this series in the world.

Blue Gene architecture has been developed by IBM in the framework of the project seeking for new solutions in highperformance computing. MSU Blue Gene/P was at the 128-th place in the Top500 issued in November 2008. It was ranked #15 in the March 2011 Top50 list of the CIS most powerful supercomputers.

The IBM Blue Gene/P system is a representative of a supercomputer family providing high performance, scalability, and facility to process large datasets and at the same time consuming significantly less energy and space in comparison with the earlier systems.

The configuration of MSU Blue Gene/P includes two racks, containing totally 2 048 compute nodes, each consisting of 4 PowerPC 450 cores, working at 850 MHz frequency.

Peak performance of the system is 27.9 TFlops.

Peak performance

27.9 TFlops

Linpack performance

23.9 TFlops

Number of racks Number of compute nodes / I/O nodes CPU model Number of CPUs / cores Total RAM Programming technologies Performance per watt Top500 position

2 2 048 / 64 4-core PowerPC 850 MHz 2 048 / 8 192 4 TB MPI, OpenMP/pthreads, POSIX I/O 372 MFlops/W 128 (2008.XI)

The Blue Gene/P architecture has been developed for programs that scale well up to hundreds and thousands of processes. Individual cores work at a relatively low frequency, but applications being able to effectively use large numbers of processor units demonstrate higher performance as compared to many others supercomputers.

â–

13

Perspective supercomputing technology: RVS-5 reconfigurable supercomputer Reconfigurable supercomputer RVS-5 installed in Research Computing Center of MSU is one of the most powerful reconfigurable computing systems in the world. This system was designed in Research Institute of Multiprocessor Computing Systems, Southern Federal University (Taganrog, Russia). The heads of the design team were Prof. I. Kaliaev and Dr. I. Levin.

base modules form a computational block, four blocks per each rack.

Reconfigurable computing system RVS-5 outperforms all known general purpose FPGAbased computing systems. Most programs for this supercomputer are written in the high-level Colamo language, which has been created by developers of RVS-5. The main features of this lanThe main computational element of the RVS- guage are high efficiency of programs written in 5 computer is a base module Alkor. Each Alkor Colamo and possibility of using a large number module contains 16 FPGA Xilinx Virtex-5 chips. of FPGAs for any program (all FPGAs of a rack).

Base modules are connected together via LVDS Various scientific applications have been succhannels which allow several base modules cessfully implemented on RVS-5. Among them to be effectively assigned to a program. Four are: • Tomographic researches of near-surface layers of the Earth using acoustic and electromagnetic waves; • Modelling and forecasting the hydrophysical and biogeochemical processes in the Sea of Azov; • Modelling natural objects and processes in the functioning area of the Rostov atomic power station; • Modelling astrophysical processes and adjustment of instrumental distortion of optical images; • Creation of fundamentally new drugs and new generation materials.

■

14

15

Supercomputing Applications In 2012, MSU supercomputer complex based on “Lomonosov” had over 550 users from MSU, Russian Academy of Sciences and other organizations. Areas of research requiring the use of supercomputer processing power are hydro- and magnetohydrodynamics (MHD), hydro- and aerodynamics, quantum chemistry, seismic surveys, computer modelling of drugs, geology and materials science, fundamentals of nanotechnology, cryptography, ecology, astrophysics, engineering calculations, new materials design, and more.

Environmental science and management Weapons and special machines

Transport and aerospace systems Energy efficiency, nuclear energy

Security and terrorism counteraction

Life sciences

Information and telecommunication systems

Nanosystems industry

Distribution of MSU Supercomputing Center applications by areas.

16

Investigation of the heliospheric boundary V. V. Izmodenov 1,2 , E. A. Provornikova 1, 2 , D. B. Alexashov 2, 3, Yu. G. Malama 2, 3 1

Moscow State University, Mechanical and Mathematical Dept.

2

Moscow State University, Space Research Institute

3

Institute for Problems in Mechanics, Russian Academy of Sciences

izmod@ipmnet.ru

AREA

Numerical methods in space gas dynamic.

DRIVER

USAGE

Interpretation and prediction of new experimental results.

Numerical modelling of the physical processes at the heliospheric boundary.

STRATEGY

Development of the 3D selfconsistent kinetic-MHD model of the heliospheric interface.

OBJEC TIVE

The goal of the project is to study fundamental properties of the stellar wind — interstellar medium interaction regions by exploring the region of the solar wind interaction with the local interstellar medium.

IMPAC T

Effective model which can be used for analysis of the different experimental data, progress in the heliospheric research.

Astrophysics

17

Turbulent astrophysical convection and magnetoconvection in compressible fast rotating spherical shells K. Kuzanyan, V. Obridko kuzanyan@izmiran.ru, kuzanyan@gmail.com

AREA

DRIVER

3D Direct Numerical Simulations of

netic field generation in fast rotating

Astrophysical Convection.

stratified media with scale separa-

Modelling turbulent convection and

tion, multi-scale approach.

magneto-convection and structure

STRATEGY

OBJEC TIVE

tion of coherent structures in fast

planets and convective zones of the

rotating highly turbulent convective

Sun and stars.

media.

Numerical solution of MHD equa-

IMPAC T

tions leading convection and mag-

Theoretical understanding of origin of helicity in solar magnetic fields.

USAGE

18

Understanding condition of forma-

formation in atmospheres of giant

Improving analysis of space weather.

Astrophysics

Development of mesoscale distributed-memory atmospheric model V. M. Stepanenko Moscow State University, Research Computing Center; Moscow State University, Geographical Faculty vstepanenkomeister@gmail.com

AREA

DRIVER

Numerical modelling of mesoscale

IMPAC T

Increasing of quality of reproducing

atmospheric processes.

the regional specifics of climate sys-

Numerical simulation of atmospheric

tem processes in numerical weather prediction and climate projection

dynamics using multicomputers

models.

(clusters).

STRATEGY

Numerical modelling.

OBJEC TIVE

Development of parameterizations of processes that are subgrid for large-scale atmospheric models.

Atmospheric and ocean dynamics

USAGE

Using the new physical (subgridscale) processes parameterizations in models of large scale atmospheric dynamics and other mesoscale meteorological models.

19

Development of a new generation mathematical model of the dynamics of the oceans. Investigation of the sensitivity of interannual variability of the circulation of ocean waters to climate changes R. A. Ibrayev 1, R. N. Khabeev 2 , V. V. Kalmykov 2 1

Institute of Computational Mathematics, Russian Academy of Sciences

2

Moscow State University

ibrayev@inm.ras.ru, kh.renat@gmail.com, vvk88@mail.ru

AREA

DRIVER

STRATEGY

OBJEC TIVE

Thermohydrodynamic processes of

the physics of ocean processes and

the oceans and the Caspian Sea.

effectively work on high-perfor-

Investigation of intra- and inter-

mance computing systems.

annual variability of water circulation IMPAC T

New opportunities in estimating the

of the oceans and seas.

state of the oceans.

Mathematical modelling of the

USAGE

Prediction of small-scale and large-

dynamics of the oceans and the

scale, intra- and inter-annual process-

Caspian Sea.

es in the oceans and the Caspian Sea.

Creation in Russia of modern mathematical model of the ocean dynamics which could describe much of

20

Atmospheric and ocean dynamics

Prognosis of perspective variants of color proteins by the results of calculations with the quantum mechanical - molecular mechanical methods B. L. Grigorenko, A. V. Nemukhin, I. V. Polyakov, D. I. Morozov Moscow State University, Chemistry Dept. bell_grig@yahoo.com

AREA

Fluorescent proteins.

DRIVER

Design of new fluorescent proteins

OBJEC TIVE

cent proteins with high quantum yields for further applications in cell

by molecular modelling.

STRATEGY

Quantum mechanics/molecular mechanics simulations.

Biology

Predictions of the effective fluores-

biology.

IMPAC T

Effective tools for molecular design.

USAGE

Applications in cell biology.

21

Modelling mechanisms of hydrolysis reactions of cyclic nucleoside phosphates B. L. Grigorenko, A. V. Nemukhin Moscow State University, Chemistry Dept. bell_grig@yahoo.com

AREA

Enzymatic reactions, cyclic nucleo-

OBJEC TIVE

side phosphates.

DRIVER

cyclic nucleoside phosphates.

Solving puzzles in biochemistry of

IMPAC T

Effective tools for molecular design

cyclic nucleoside phosphates by

USAGE

Applications in cell biology.

molecular modelling.

STRATEGY

The role of the key aminoacids in

Quantum mechanics/molecular mechanics simulations.

22

Biology

Theoretical and computational studies of nano-size single-molecule devices B. L. Grigorenko, A. V. Nemukhin, A. A. Moskovsky Moscow State University, Chemistry Dept. bell_grig@yahoo.com

AREA DRIVER

“Nanodevices”.

novel structures for nanocars, that

“Nanocar” molecules have been stud-

should exhibit better ability for directional motion.

ied by using theoretical methods.

STRATEGY

OBJEC TIVE

Quantum mechanics/molecular

IMPAC T

Effective tools for molecular design.

mechanics simulations.

USAGE

Applications in nanotechnology.

Mathematical models were developed for nanocars, which allowed us to describe dynamics of molecules adsorbed on a surface, and design

Biology

23

Investigation of point mutation influence on the bacterial potassium channel KcsA conduction and selectivity M. Kasimova, A. K. Shaytan, K. V. Shaitan marina.kasimova@molsim.org, shaytan@polly.phys.msu.org, shaitan49@yandex.ru

AREA

Biology, Fundamental Science.

DRIVER

Voltage-gated ion channels are

OBJEC TIVE

from first principles. Investigate the impact of the mutation on channel’s

responsible for production and

selectivity and high-rate conduction.

propagation of nerve impulses along neuronal structures and across syn-

STRATEGY

Study ion conduction mechanism

IMPAC T

Simulations of physical mechanisms

apses. They present both fundamen-

underlying the conductivity and

tal and practical interest as potential

selectivity of ion channels will con-

drug targets.

tribute to fundamental understand-

Molecular dynamics simulations of

ing of the cellular processes with

ion transfer through native bacte-

possible biomedical applications.

rial potassium channel KcsA and its mutant at atomistic resolution.

24

Biology

Simulation the seasonal flooding of the VolgaAkhtuba floodplain A. V. Khoperskov, S. S. Khrapov, N. M. Kuzmin, A. V. Pisarev, I. A. Kobelev, M. A. Butenko, T. A. Dyakonova Volgograd State University khoperskov@gmail.com, khoperskov@volsu.ru weather conditions and to develop AREA

Computational fluid dynamics.

DRIVER

The study of seasonal floods in the Volga-Akhtuba floodplain and opti-

an optimal hydrological regime of the various hydro facilities.

IMPAC T

STRATEGY

natural landscape of the Volga-Akh-

The construction of numerical mod-

tuba floodplain, taking into account

els of the dynamics of surface water

a reasonable balance of environmen-

on a given terrain.

OBJEC TIVE

Multi-dimensional time-dependent numerical experiments allow us to predict the state of the VolgaAkhtuba floodplain, depending on

Computational f luid dynamics

Construction of optimal hydrographs will preserve the unique

mization of hydraulic regime.

tal, energy and nature using tasks.

USAGE

The results of the calculations should be used to establish the federal commission planned for spring hydrographs cascade on the Volga.

25

Specialized software to optimize and control of ventilation systems of large industrial plants on the basis of gas dynamic simulations using parallel technologies A. V. Khoperskov 1, A. A. Voronin 1, M. A. Butenko 1, D. V. Burnos 1, A. G. Zhumaliev 1, V.S. Kholodkov 2 1

Volgograd State University

2

Volgograd State University of Architecture and Civil Engineering

khoperskov@gmail.com, khoperskov@volsu.ru

AREA DRIVER

Industrial ventilation.

IMPAC T

The study of the physical conditions

ing costs.

within a large industrial plant, depending on the process and the work of aspiration devices.

STRATEGY

Improving working conditions and increase productivity while optimiz-

USAGE

Creating a software tool for modelling of an optimal ventilation system.

The construction of numerical models of the dynamics of air.

OBJEC TIVE

26

Optimization of the ventilation system.

Computational f luid dynamics

Propagation of nonlinear acoustic signals in aโ ฏrandomly inhomogeneous turbulent medium with different types of spectra O. A. Sapozhnikov, P. V. Yuldashev, M. V. Averiyanov Moscow State University oleg@acs366.phys.msu.ru, petr@acs366.phys.msu.ru, misha@acs366.phys.msu.ru

AREA

Aeroacoustics, generation and

approximation. Construction of

propagation sonic boom from

algorithms both in time and in the

supersonic aircraft, the problem of

spectral representation.

environmental noise.

DRIVER

OBJEC TIVE

The use of numerical simulation of

schemes for modelling the effects of

nonlinear wave equation for the

nonlinearity, diffraction, scattering,

description of N-waves propaga-

and absorption in the propagation

tion in randomly inhomogeneous

of N-waves in randomly inhomoge-

turbulent flows and identification

neous medium.

of differences in the statistics of the shock front amplitude and width for

IMPAC T

The developed approach allows to simulate the propagation of waves

the spectra of various types.

STRATEGY

Development of original numerical

not only with a plane initial front but

Construction of algorithms for

also to approximate the field of a

solving Khokhlov-Zabolotskaya

point source in a cone of [-50 ยบ +50 ยบ].

and Westervelt type equations in a randomly inhomogeneous medium, calculation of statistical characteristics of the field from the numerical solutions. Accounting for diffraction within the wide-angle high order

USAGE

The use of an ensemble of numerical results obtained with different implementations of a random medium to calculate statistics of the amplitude and the width of the N-wave shock front in thermal and kinematic turbulence.

Computational f luid dynamics

27

Computational modelling of open rotor noise V.A. Titarev 1 , V.F. Kopiev 2 , I.V. Belyaev 2 1

Dorodnicyn Computing Centre, Russian Academy of Sciences

2

TsAGI

titarev@ccas.ru, kopiev@mktsagi.ru, aeroacoustics@mktsagi.ru

AREA

DRIVER

STRATEGY

Computational aeroacoustics,

curacy and good parallel efficiency

computational physics, high-perfor-

of the method and software will

mance computing.

shorten the required analysis time.

Design of new open rotor systems in

Time efficient and accurate computational predictions of aeroacoustics

Computational modelling using

properties of open rotor.

state-of-the art methods in computational physics and aeroacoustics.

OBJEC TIVE

IMPAC T

aerospace.

USAGE

In design and evaluation of new engines.

Create novel computer modelling tools for studying aerodynamics and aeroacoustics of open- rotor systems with ability to model real-life configurations. High integration with modern CAD and mesh generation tools will speed up the preparation of the model whereas the high ac-

28

Computational f luid dynamics

Development and application of new methods in computational aerodynamics and rarefied gas dynamics V. A. Titarev Dorodnicyn Computing Centre, Russian Academy of Sciences titarev@ccas.ru

AREA

DRIVER

Computational physics, high-perfor-

It will become possible to model

mance computing.

accurately rarefied gas flows in real

Computer modelling of rarefied

geometries.

flows.

STRATEGY

IMPAC T

Computer experiment using the

USAGE

In modelling of microscale and high altitude flows.

state-of-the-art methods of computational physics.

OBJEC TIVE

Further development of a novel computer modelling package for computational rarefied gas dynamics, based on the original conservative methods and use of high-performance computing.

Computational f luid dynamics

29

Numerical modelling of initiation and propagation of detonation waves in tubes with complex shape I. V. Semenov, P. S. Utkin, I. F. Akhmedyanov, A. Yu. Lebedeva Institute for Computer Aided Design, Russian Academy of Sciences icad.org.ru, utkin@icad.org.ru, aildarf@gmail.com, lebedeva-zp@yandex.ru

AREA

DRIVER

STRATEGY

Numerical modelling of multidimen-

and finding the tube geometry which

sional problems of continuum me-

provides detonation initiation for

dium mechanics taking into account

incident shock wave Mach number

combustion processes.

about 3.0.

Shock-to-detonation transition in

The new way of detonation initiation in fuel – air mixtures with minimal

mixture.

energy demands.

The mathematical model is based on Euler equations coupled with single-

OBJEC TIVE

IMPAC T

profiled tube filled with methane-air

USAGE

Detonation of natural gas-air mixture is very important and promising

stage chemical kinetics model. The

problem from the sense of energy

numerical procedure is based on

plants effectiveness increasing. Be-

finite volume approach, explicit time

sides the importance of investigation

integration scheme and Godunov’s

of detonation initiation critical condi-

method for fluxes calculation.

tions in natural gas-air mixtures are

The goal of the project is multidi-

connected with explosion hazards in

mensional modelling of shock-todetonation transition in methane-air

coal-mining industry and natural gas production.

mixture under normal conditions

30

Computational f luid dynamics

High-performance computing for the problems of multiphase dynamics and explosion hazards I. V. Semenov, P. S. Utkin, I. F. Akhmedyanov, A. Yu. Lebedeva Institute for Computer Aided Design, Russian Academy of Sciences semenov@icad.org.ru, utkin@icad.org.ru, aildarf@gmail.com, lebedeva-zp@yandex.ru

AREA

DRIVER

Numerical modelling of multidi-

dynamics including the problems in

mensional problems of multiphase

areas with complex shapes using high-

medium mechanics taking into

performance computers with distributed

account interphase interaction and

memory. The model problem of dust

combustion processes.

lifting from the layer behind the propa-

The problem of dust lifting and

gating shock wave in two- and threedimensional statements is investigated.

combustion behind the propagating shock wave in multidimensional

STRATEGY

The mechanisms of dust dispersion and

statement.

combustion behind the propagating

The mathematical model is based

shock wave.

on interpenetrating continuums ap-

OBJEC TIVE

IMPAC T

USAGE

Dust explosions are of great danger

proach taking into account the phas-

for the industries connected with fi ne

es volume fractions. The numerical

organic dust formation. In case when the

procedure is based on finite volume

dust is deposited in the layer on the fl oor

approach, explicit time integration

of the technical room or on the surfaces

scheme and Godunov’s method for

of the equipment even small explosion

fluxes calculation in gaseous and

can provide particles dispersion from

dispersed phases.

the layer with the formation of explosion

Within the framework of the project

heterogeneous mixture in the volume

the software is developed for the numerical modelling of multidimensional

which is signifi cantly larger than the area of the initial explosion.

problems of multiphase reactive fl ows

Computational f luid dynamics

31

Investigation of development of hydrodynamic properties in problems with anomalous properties of liquids I. N. Sibgatullin, D. V. Kuznetsova, M. I. Gritsevich Moscow State University sibgat@imec.msu.ru, morven9@yandex.ru, gritsevich@list.ru

AREA

DRIVER

Theory of hydrodynamic stability

OBJEC TIVE

development. Estimation of impact of

Descriptions and prognosis of

boundary condition. Investigation of types of bifurcations. Comparison of

thermodynamical behavior of liquids

anomalous and classical fluids.

with anomalous dependence of density on temperature.

STRATEGY

Description of flow structure and

and computational fluid dynamics.

IMPAC T

types and instabilities. Efficient de-

Application of theory of bifurcations

sign of water reservoirs and progno-

and qualitative theory of dynamical

sis of heat exchange and fluid flow

system to computer-modeled fluid

structure.

flows. Application of spectral methods. Study of influence of different types of boundary conditions.

Fundamental classification of flow

USAGE

Efficient design of water reservoirs and prognosis of heat exchange and fluid flow structure.

32

Computational f luid dynamics

Supercomputer simulation of detonation in chambers and channels of complicated geometry I. S. Manuylovich, V. V. Markov Institute of Mechanics, Moscow State University ivan.manuylovich@gmail.com

AREA

DRIVER

Computational fluid dynamics of

chambers and channels of compli-

multicomponent reactive flows.

cated geometry, capable of solving

Simulation of 1D, 2D and 3D

various problems with combustion and detonation. Scalability to several

multicomponent reactive flows in

tens of billions of computational cells

chambers and channels of compli-

with parallelization on tens of thou-

cated geometry with applications

sands of processor cores. Local part

for development of new types of

of the software complex linked to

engines and for problems related to

supercomputer, with graphical user

explosion safety at various facilities.

STRATEGY

Fast parallelized simulation of reactive multicomponent flows in

interface, pre- and postprocessors.

IMPAC T

cated 1D, 2D and 3D multicompo-

domains of complicated geometry,

nent reactive flows with combustion

based on finite-volume numerical

and detonation.

schemes with various combustible mixtures and chemical kinetics.

OBJEC TIVE

Create original software complex intended to simulate 1D, 2D and 3D

Fast and easy simulation of compli-

USAGE

Development of new types of engines, solving problems related to explosion safety at various facilities.

multicomponent reactive flows in

Computational f luid dynamics

33

Study of the mechanism of ion transfer in biological membranes by molecular dynamics D. A. Cherepanov 1, A.Y.Mulkidjanyan 2 , V.P.Skulachev 2 1

A. N. Frumkin Institute of physical chemistry and electrochemistry, Russian Academy of Sciences

2

Moscow State University

cherepanov@biologie.Uni-Osnabrueck.DE, mulkid@googlemail.com, skulach@genebee.msu.su

AREA

Supercomputer-aided drug design.

DRIVER

Study of the effect of ion chemical

OBJEC TIVE

ficiency of different ionic mitochondria-targeted antioxidants.

nature on the mechanism of their diffusional penetration into biologi-

IMPAC T

cal membranes based on molecular modelling.

STRATEGY

34

Study of the pharmacological ef-

Efficient, quick and cheap rational development of new drugs.

USAGE

New drugs development for a mi-

Usage of the potential of mean force

tochondria-targeted antioxidant

in molecular-dynamics simulations.

protection.

Drug design and therapeutic methods

Supercomputer-aided drug design P. V. Vrzesch, I. S. Filimonov, N. A. Trushkin1, E. E. Hrameeva, S. I. Mitrofanov, G. D. Lapshin, O. Zolotareva Moscow State University biocentr@list.ru, fis82@yandex.ru, hydrodiction@gmail.com, ekhrameeva@gmail.com, mitroser04@mail.ru, sadsaviour@gmail.com, olya_zol@inbox.ru

AREA

DRIVER

Bioinformatics: molecular dynamics,

OBJEC TIVE

proach) of new inhibitors search with

Supercomputer-aided drug design:

high ability to predict efficient inhibitors for a given complicated target

design of new inhibitors for a given

protein. To find new COX-inhibitors.

target-protein based on molecular dynamics and docking.

STRATEGY

To develop a detailed protocol (ap-

molecular docking.

IMPAC T

new drugs.

Virtual screening of databases of ligands by means of docking,

Efficient, quick and cheap search of

USAGE

Comparison of SOL and AutoDock us-

quantum chemistry and molecular

ing complicated protein PGHS (COX).

dynamics methods for estimation of

New inhibitors search in ACB Blocks

protein-ligand binding energies.

databank for a given target protein.

Five new cyclooxygenase inhibitors revealed.

Drug design and therapeutic methods

35

Mechanisms of saturation of shock wave parameters in high power focused beams of pulsed and periodic signals V. A. Khokhlova, M. M. Karzova, M. V. Averiyanov Moscow State University vera@acs366.phys.msu.ru, masha@acs366.phys.msu.ru, misha@acs366.phys.msu.ru

AREA

DRIVER

Therapeutic applications of ultra-

tion. Combined nonlinear, diffrac-

sound.

tion, and absorption effects determine limiting values of shock wave

Characterization of spatial structure

parameters in the focus. Knowing

and evaluation of limiting values of

the field is necessary for certification

acoustic parameters in nonlinear

of medical devices and for planning

focused beams radiated from thera-

therapeutic effect of the treatment.

peutic ultrasound transducers.

STRATEGY

Numerical modelling of ultrasound

IMPAC T

fields within a wide range of param-

sound fields for calibration purposes,

eters typical for modern therapeutic

quality assurance, and treatment

systems.

OBJEC TIVE

Development therapeutic transducers and characterization of their ultra-

protocols.

Develop original modelling tools

USAGE

to simulate nonlinear, diffraction, and absorption effect in high power focused beams of either pulsed or continuous signals. Predict focal waveforms and spatial distribution of acoustic parameters such as peak

Transducer design and clinical application of high power ultrasound waves in, high intensity focused. .ultrasound for tumor treatment, extracorporeal shock wave lithotripsy and other shock wave therapies.

pressures, intensity, and heat deposi-

36

Drug design and therapeutic methods

Improving the efficiency of heating biological tissue irradiated by high intensity ultrasound through the rib cage using nonlinear acoustics effects V. A. Khokhlova, S. M. Bobkova, S. A. Ilyin, P. V. Yuldashev Moscow State University vera@acs366.phys.msu.ru, sveta@acs366.phys.msu.ru, sergey_ilyin@acs366.phys.msu.ru, petr@acs366.phys.msu.ru

AREA

DRIVER

Therapeutic applications of ultra-

ribs. Modelling of nonlinear effects

sound.

associated with the propagation of

Improving spatial localization and

the acoustic wave behind the ribs.

Modelling of the thermal field by

enhancing thermal heating of tumor

solving the heat transfer equation

tissues behind the rib cage.

STRATEGY

and evaluation of the influence of

Design of special configuration of

nonlinear effects on the efficiency of

operating the elements of multi element therapeutic array and numerical modelling of the ultrasound field

heating the target area.

IMPAC T

ficient ultrasound irradiation of tumor

generated by the array to achieve

tissue behind the ribs.

the desired results.

OBJEC TIVE

Development of a simulation

Development of new methods for ef-

USAGE

scheme of turning off the elements of the array to irradiate through intercostal spaces between the

Drug design and therapeutic methods

Destruction of tumor tissue behind the ribs of the chest using high intensity focused ultrasound without overheating ribs.

37

Three-dimensional nonlinear fields of therapeutic ultrasound arrays O. A. Sapozhnikov, P. V. Yuldashev, S. A. Ilyin Moscow State University oleg@acs366.phys.msu.ru, petr@acs366.phys.msu.ru, sergey_ilyin@acs366.phys.msu.ru

AREA

DRIVER

Therapeutic applications of ultra-

the effectiveness of therapeutic

sound.

effect. Define the parameters of the

Development of a new numerical

field devices required for the certification of ultrasonic surgery devices

model to simulate three-dimension-

and planning the therapeutic effect

al nonlinear fields produced by mod-

of irradiation.

ern therapeutic ultrasound arrays.

STRATEGY

Development of an algorithm for

IMPAC T

element transducers for ultrasound

solving the Westervelt equation to

surgery systems, calculation of the

enable calculation of three-dimen-

characteristics of their fields, as-

sional nonlinear ultrasound fields in

sessment of the impact of nonlinear

the presence of shock fronts, which

effects in instrument calibration,

are localized near the focus.

OBJEC TIVE

Development of high power multi-

quality control and planning of treat-

Develop original numerical al-

ment protocols.

gorithms to simulate the effects

USAGE

of nonlinearity, diffraction, and absorption in the three-dimensional focused fields produced by multielement arrays used in noninvasive ultrasound surgery. Collect data on the distortion of the wave profile at the focus and spatial distributions of

Development of transducers for clinical application of high intensity focused ultrasound waves to destroy tumors, the use of non-linear regimes of irradiation to improve the efficiency of the thermal bioeffect in tissue.

the field parameters that determine

38

Drug design and therapeutic methods

Alzheimer disease treatment drug design S. V. Lushchekina, G. F. Makhaeva, K. A. Petrov, V. S.Reznik, E. E. Nikolsky, S. D. Varfolomeev Institute of biochemical physics, Russian Academy of Sciences Institute of physiologically active compouds, Russian Academy of Sciences Institute of organic and physical chemistry Kazan SC, Russian Academy of Sciences sof ya.lushchekina@gmail.com

AREA

computer-aided drug-design.

DRIVER

Explanation of experimentally

OBJEC TIVE

drugs and enzymes and development of new drugs.

observed hysteresis in catalytic reactions.

STRATEGY

IMPAC T

New effective anti-AD drugs with lesser side-effects.

Study and development of ChEs inhibitors as anti-Alzheimer drugs

Description of interactions between

USAGE

Alzheimer disease therapy.

by means of molecular docking and molecular dynamics methods.

Drug design and therapeutic methods

39

Computer modelling of acetylcholine / acetylcholinesterase hydrolysis S. V. Lushchekina, M. S. Kochetov, I. A. Kaliman, B. L. Grigorenko, A. V. Nemukhin Institute of biochemical physics, Russian Academy of Sciences Moscow State University, Chemistry department sof ya.lushchekina@gmail.com

AREA

DRIVER

Computer modelling of enzymatic

OBJEC TIVE

matic reaction and influence of

Detailed information about mecha-

surrounding amionacids. Rational strategy of QM subsystem for QM/

nism of the hydrolysis reaction and

MM modelling to reduce computa-

development of the method of

tionl time and maximize modelling

computer modelling of enzymatic

reliability.

reactions.

STRATEGY

Detailed description of the enzy-

reactions.

IMPAC T

Combined quantum mechanics/

modelling; detailed description of

molecular mechanics study of the

biochemistry processes.

mechanism of hydrolysis reaction with varying QM subsystem.

Standard method of computer

USAGE

Study of biochemical processes and drug design.

40

Drug design and therapeutic methods

Heat exchanger flow modelling using CABARET method М. А. Zaitsev Nuclear Safety Institute, Russian Academy of Science ztsv@ibrae.ac.ru

AREA

Energy equipment.

DRIVER

Design of next generation energy equipment.

STRATEGY

Full scale mathematical modelling of turbulence flows with heat transfer using Lomonosov supercomputer facility and CABARET method software.

OBJEC TIVE

Design of complex technical facility, such as heat exchanger of energy equipment, demands exact evaluation of rows turbulence parameters, for example – hydraulic resistance. In addition to expensive experimental evaluation, numerical theoretical estimation is also possible.

IMPAC T

Implemented high resolution mathematical modelling method on Lomonosov supercomputer facility can valuably decrease amounts of experimental jobs.

USAGE

New energy equipment structures.

Energy efficiency

41

Modelling high speed transient dynamic processes in mechanics of deformable solids using supercomputer technologies P.A. Mossakovsky, F.K. Antonov, L.A. Kostyreva, A.V. Inykhin antonof@gmail.com, shurik@sectorb.msk.ru

AREA

DRIVER

Supercomputer modelling of highly

IMPAC T

nonlinear dynamic processes.

of complex problems associated

Problems, interconnected with high

with a substantially non-linear dynamic processes.

speed transient dynamic processes (impact, penetration, blast).

STRATEGY

The possibility of an exact solution

Computational method for solving essentially nonlinear dynamic strength problems which is implemented as

USAGE

The solution of specific problems associated with the modelling of various emergency situations that can lead to disastrous consequences.

an iteration procedure of verification modelling is implemented.

OBJEC TIVE

Development of experimental-computational procedure for modelling of highly nonlinear dynamic processes, which allows to obtain a solution with a controlled precision.

42

Mechanics of solids

Formation of quantum low-dimensional nanostructures on a semiconductor surfaces and its combined study by means of LT STM/STS and DFT techniques D. A. Muzychenko Moscow State University, Faculty of Physics mda@spmlab.ru, dima@spmlab.phys.msu.ru AREA

Energy equipment.

addition to expensive experimental

DRIVER

Design of next generation energy

evaluation, numerical theoretical estimation is also possible.

equipment.

STRATEGY

Full scale mathematical modelling of

IMPAC T

mathematical modelling method on

turbulence flows with heat transfer

Lomonosov supercomputer facility

using Lomonosov supercomputer

can valuable decrease amounts of

facility and CABARET method soft-

experimental jobs.

ware.

OBJEC TIVE

Design of complex technical facility,

Implemented high resolution

USAGE

New energy equipment structures.

such as heat exchanger of energy equipment, demands exact evaluation of rows turbulence parameters, for example – hydraulic resistance. In

Nanoscience and nanotechnology

43

Supercomputer-aided design of bioinspired functional nano-structures based on silk-forming peptides and conductive thiophene blocks A. K. Shaytan, P. G. Khalatur, A. R. Khokhlov Moscow State University shaytan@polly.phys.msu.ru

AREA DRIVER

Nanotechnology.

Design of advanced organic-semiconducting devices.

STRATEGY

Molecular dynamics and quantum chemistry simulations.

OBJEC TIVE

Design hybrid molecular compounds that could self-assemble into conducive nanowires.

IMPAC T

Organic electronics.

USAGE

Nanowires for organic electronic applications.

44

Nanoscience and nanotechnology

Polyelectrolyte complexes consisting of macromolecules with varied stiffness: computer simulation A. A. Lazutin 1, V. V. Vasilevskaya 1, A. N. Semenov 2 1

Institute of Organoelement Compounds, Russian Academy of Sciences

2

Institut Charles-Sadron, Strasbourg

lazutin@polly.phys.msu.ru

AREA

DRIVER

Computer simulation of polyelectro-

IMPAC T

gimes of polyelectrolyte complexes

Investigation of polyelectrolyte

consisting of semiflexible polymeric chains.

complexes formed by macromolecules with varied stiffness.

STRATEGY

Knowledge of conformational re-

lytes.

Computer simulation by Monte Carlo

USAGE

Development of polyelectrolyte theory.

method. Off-latice model.

OBJEC TIVE

Determination of conformational behaviors of polyelectrolyte complexes on stiffness of composing macromolecules.

• Lazutin A. A., Semenov A. N., Vasilevskaya V. V. Polyelectrolyte Complexes Consisting of Macromolecules With Varied Stiffness: Computer Simulation, Macromol. Theory & Simul., 2012, mats.201100097

Nanoscience and nanotechnology

45

Numerical simulations of laser radiation fields in highly anisotropic scattering media Ya. A. Ilyushin, V. G. Oshlakov, Ya. A. Ilyushin, V. G. Oshlakov ilyushin@physics.msu.ru, oshlakov@iao.ru

AREA

Atmospheric and oceanic optics,

DRIVER

OBJEC TIVE

simulations of performance of laser

Numerical simulations of the radia-

navigational systems, lidar measurement analysis and validation etc.

tive field of laser beam in fog.

STRATEGY

Develop robust numerical model for

radiative transfer theory.

New effective numerical techniques

IMPAC T

prediction and analysis of laser sys-

of radiative transfer calculations are

tem capabilities will be elaborated.

developed and practically implemented.

Reliable numerical techniques for

USAGE

New optical aircraft landing system, naval beacons, lidars.

R = 0.05 τ=2 Λ=1 g=0.9

R = 0.20 τ=2 Λ=1 g=0.9

R = 0.35 τ=2 Λ=1 g=0.9

R = 0.50 τ=2 Λ=1 g=0.9

46

R = 0.05 τ=6 Λ=1 g=0.9

R = 0.20 τ=6 Λ=1 g=0.9

R = 0.35 τ=6 Λ=1 g=0.9

R = 0.50 τ=6 Λ=1 g=0.9

R = 0.05 τ=10 Λ=1 g=0.9

R = 0.20 τ=10 Λ=1 g=0.9

R = 0.35 τ=10 Λ=1 g=0.9

R = 0.50 τ=10 Λ=1 g=0.9

R = 0.05 τ=14 Λ=1 g=0.9

R = 0.20 τ=14 Λ=1 g=0.9

R = 0.35 τ=14 Λ=1 g=0.9

R = 0.50 τ=14 Λ=1 g=0.9

Optics

Interaction of femtosecond laser filaments A. A. Dergachev, S. A. Shlenov Moscow State University, Physics Dept. and International Laser Center dergachev88@yandex.ru

AREA

DRIVER

Optics of femtosecond laser fila-

OBJEC TIVE

laser plasma distributions. Optimiza-

Numerical simulation of femto-

tion of parameters of laser systems to obtain high fluence and plasma

second laser pulses interaction via

concentration.

filamentation.

STRATEGY

Obtain patterns of 3D fluence and

ments.

Numerical solving of the nonlinear

IMPAC T

filamentation.

Schrรถdinger equation for slow varying amplitude of laser field to obtain

Control laser pulse parameters via

USAGE

Femtosecond laser technology.

3D distributions of the fluence and laser plasma concentration.

Optics

47

Supercomputer modelling of sub-cycle mid-infrared conical emission from filamentation in air A. A. Voronin 1, A. M. Zheltikov 1 1

Moscow State University, International laser center

aa.voronin@physics.msu.ru

AREA

Supercomputer-aided development

OBJEC TIVE

of new efficient mid infrared ultra-

DRIVER

short pulse sources.

able to simulate complicated 3-D

Development of new techniques of

pulse dynamics in the extreme conditions of broadband spectrum,

coherent broadband mid infrared

covered terahertz, infrared and vis-

ultrashort pulse generation through

ible spectral range, and strong laser-

the multicolor filamentation.

STRATEGY

Develop new ultrashort laser pulse propagation modelling software

matter interaction with ionization of

Numerical solution of nonlinear integro-differential equations describing propagation of high-power

the media.

IMPAC T

mid infrared ultrashort pulse sources.

ultrashort laser pulses through ionizing media with cubic and quintic nonlinearities.

Development of the unique efficient

USAGE

New ultrashort laser pulse sources for mid-infrared nonlinear spectroscopy and microscopy.

48

Optics

Nonlinear optics of ultrashort laser pulses in the multiple filamentation regime A. A. Voronin 1, A. M. Zheltikov 1, P. A. Zhokhov 2 1

International laser center, Moscow State University

2

Texas A&M University

aa.voronin@physics.msu.ru, peterzhokhof f@gmail.com

AREA

DRIVER

Supercomputer-aided development

ditions of laser-matter interaction,

of new ultra-high power laser tech-

such as multiple level ionization and

nologies.

pulse compression down to one

Development of new techniques of

cycle of electromagnetic field.

coherent supercontinuum genera-

IMPAC T

tion and ultrashort pulse com-

STRATEGY

tools for wide range of tasks span-

pression in multiple filamentation

ning from observation of ultrafast

regime.

processes in molecules, atoms and

Numerical solution of nonlinear

nuclei to the remote atmosphere sensing.

integro-differential equations describing propagation of high-power

OBJEC TIVE

Development of the unique laser

USAGE

Atmosphere pollution control sys-

ultrashort laser pulses through ion-

tems, remote spectroscopy applica-

izing media with cubic and quintic

tions, atomic and nuclear physics

nonlinearities.

applications.

Develop new ultrashort laser pulse propagation modelling software able to simulate complicated 3-D pulse dynamics in the extreme con-

Optics

49

Supercomputer simulation of polyamphiphiles P. G. Khalatur A. N. Nesmeyanov Institute of Organoelement Compounds, Russian Academy of Sciences khalatur@polly.phys.msu.ru

AREA

Polymer science.

DRIVER

Design of new advanced nano-

OBJEC TIVE

self-assembly mechanism of copolymers.

materials using high performance

STRATEGY

50

Fundamental understanding of the

computing.

IMPAC T

New functional polymer nanomaterials.

Using multiscale simulation methods

USAGE

Advanced nanomaterials, photovol-

for studying the properties of self-

taics, solar cells, plasma panels and

organizing smart polymers.

flexible displays .

Polymer science

Supercomputer simulation of membranes for fuel cells P. G. Khalatur 1, A. R. Khokhlov 2 , P. V. Komarov 1, 3, I. V. Neratova 3 1

A. N. Nesmeyanov Institute of Organoelement Compounds, Russian Academy of Sciences

2

Moscow State University

3

Tver State University

khalatur@polly.phys.msu.ru

AREA

DRIVER

Polymer ion exchange membranes

OBJEC TIVE

for fuel cells.

predicting of the structural and the

Design of new materials based on

transport properties of ion exchange membranes.

multiscale modelling.

STRATEGY

Development of approaches for

Using multiscale simulation methods

IMPAC T

New functional materials.

to study the structural and the

USAGE

Hydrogen fuel cells.

transport properties of ion exchange membranes.

Polymer science

51

Development of a numerical algorithm for solving the self-consistent mean-field equations for systems of copolymers with rigid and flexible blocks P. G. Khalatur 1, 2 , Y. A. Kriksin 3 1

A. N. Nesmeyanov Institute of Organoelement Compounds, Russian Academy of Sciences

2

University of Ulm, Department of Polymer Science

3

Keldysh Institute of Applied Mathematics, Russian Academy of Sciences

khalatur@polly.phys.msu.ru, kriksin@nm.ru

AREA

DRIVER

Self-assembly of rod-coil block

It is expected that a solid-state

copolymers.

morphology of block copolymers

Design of nanomaterials for energy

helps to perform an efficient charge collection and transferring to elec-

production, storage and use of

trodes.

energy.

STRATEGY

Computer simulations of copolymer

IMPAC T

energy-related nanomaterials.

nanomaterials by means of mesoscopic methods.

OBJEC TIVE

Efficient and cheap development of

USAGE

New nanomaterials for solar cells.

Organic photovoltaic technology of a production of an inexpensive, clean and renewable energy sources.

52

Polymer science

53

Supercomputing Activities

54

Supercomputing Consortium of Russian Universities The agreement about the formation of the Supercomputing Consortium of Russian Universities was signed by Rectors of Lomonosov Moscow State University, Lobachevsky Nizhny Novgorod State University, Tomsk State University and South Ural State University at the session of the Russian Union of Rectors on the 23d of December, 2008. The main objective of the Consortium is to use the School of Higher Education’s powerful potential to develop and implement Supercomputing Technologies in Russian education, science and industry.

Currently, the Consortium comprises over 50 regular and associate members, among whom there are leading Russian universities, the institutes of the Russian Academy of Sciences and commercial companies, actively working in the Supercomputing Technologies. On the Consortium’s initiative, a number of scientific confer-

ences, All-Russia Youth Schools and workshops were held, books are published, and analytical reports on Supercomputers are issued.

The Supercomputing Consortium of Russian Universities initiated and organized a successful implementation of the Russian Federation Presidential Commission Project on the modernization and technological development of Russian economy – "Supercomputing Education".

http://hpc-russia.ru (in Russian)

Number of Consortium organizations in cities > 5 organizations 2 - 5 organizations 1 organization

Arhangelsk Petrozavodsk St. Peterburg Kirov Kostroma Izhevsk Perm Nizhny Novgorod Ekaterinburg Vladimir Moscow Kazan Chelyabinsk Kaliningrad Saransk Ufa Penza Minsk Tambov Samara Saratov Voronezh Belgorod Tver

Chisinau

Tomsk

Omsk

Novosibirsk

Krasnoyarsk Irkutsk

Rostov-na-Donu Vladivostok

55

Top50 rating: the most powerful supercomputers of CIS Beginning from the very first computers, researchers constantly encountered the tasks, solution of which demanded much more computing power than it was available at that time. Such tasks appeared and continue appearing everywhere: aerodynamics and oil production, weather forecast and microelectronics, pharmaceutics and design of new materials, cryptography and bioengineering. It is just a small list of the application areas, where computers with the actually beyond the limits productivity will be required for the successful advance.

In May 2004 in order to help being oriented correctly in the world of High-Performance Computing systems and to have the possibility to operationally track the development trends in this area, Joint Supercomputer Center of the Russian Academy of Sciences and Research Computing Center of Moscow State University

started a joint project “Top50 list of the most powerful supercomputers of the CIS”.

The rating includes 50 computing systems, running at the territory of the CIS, which show the highest performance on the Linpack test up to the moment. The list renews twice a year (March and September). The rules of information submitting for participating in the rating are represented in the section of regulation rules for Top50 on the website of the rating.

Despite the comparatively short history of the project, published by this time editions of Top50 already form a rich content, which can be used as a basis for the substantial analysis of the dynamics of HPC area evaluation both in Russia and in the CIS. The website of the project offers a number of information blocks and services: the current and all previous editions, the detailed information on some systems, statistics, news, archived history, resource for creating a special view on the rating with the users’ criteria in order to perform advanced analysis. http://top50.supercomputers.ru (in Russian)

Top500 November list

XI

Top50 March list

III

IX VI

Top500 June list

56

Top50 September list

Total performance of top systems, TFlops

Top50 list systems performance, aggregated by cities

57

Joint MSU-Intel center on High-Performance Computing Joint MSU-Intel center on high-performance panies are invited. Such contests are annual and computing was opened in Research Computing help to reveal perspective projects and young Center of Moscow State University in May 2007. specialists or working groups.

Objectives of this center are: The website of the Center regularly provides • Active collaboration and coordination bethe latest news, research results, translated techtween scientific, industrial and educational nical materials with useful comments based on organizations; the research activities. A number of events are • Dissemination of the latest achievements in being held: scientific seminars, trainings, master HPC within science and education commuclasses and so on, resulting in sharing of accunity of Russia; mulated experience between teams and groups • Research on parallel computing systems, of Russian science and education community new methods of parallel programming, approaches and tools for software develop- engaged in high performance computing. ment, perspective computing technologies.

A significant number of research results are obtained, including both software and hardware testing. For example, the research on compilerlevel optimization effectiveness, running on different platforms with various sets of Joint MSUIntel center on high-performance computing custom compilation settings. The experiments aimed at revealing specific program behavior in different conditions are run, providing the basis for the generation of recommendations on what tools could be used and how user programs could be tuned.

Educational activies find support of Center of Competence as major activities of high importance. The most recent one is the certification project. The certification is aimed at improving of the knowledge quality, attracting more students to HPC and producing more professionals as a result. The Certification is to start during Summer Academy in early July 2012.

The support for leading HPC contests is one of the major activities of the Center. Participants from the leading academical and industrial com-

58

http://msu-intel.parallel.ru/ (in Russian)

CUDA Center of Excellence Moscow State University was awarded an NVIDIA CUDA Center of Excellence (CCOE) in the end of 2011 for being one of the first Russian universities to recognize the importance of emerging GPU technologies and first to embrace CUDA. MSU faculties integrate the CUDA programming model into their curriculum, MSU students eagerly adopt CUDA-capable parallel processing architectures in their Masters and PhD research, MSU scientists leverage the parallel architecture of the GPU to accelerate their research in a number of scientific domains.

NVIDIA software engineers and MSU research groups actively collaborate on a number of projects.

Coordination: Alexei R. Khokhlov – MSU CCOE PI, ViceRector of MSU, Head of Department for Innovations, Informatization and International Scientific Cooperation, Member of Russian Academy of Sciences Vladimir V. Voevodin – Vice-Director of RCC MSU, Corresponding Member of Russian Academy of Sciences http://ccoe.msu.ru (mostly in Russian)

Sample of the CUDA related projects using GPU Computing being developed at MSU: • Self-consistent field theory modelling of complex polymer systems (Prof. Alexei R. Khokhlov, Chair of Polymer and Crystal Physics, Faculty of Physics) • GPU-accelerated molecular dynamics simulations of complex biological systems (Prof. Konstantin V. Shaitan, Department of Bioengineering, Faculty of Biology) • FIDESYS package (Prof. Vladimir A. Levin, Faculty of Mechanics and Mathematics) • Extensible Programming for HPC Applications (Dr. Sci. Vladimir V. Voevodin, Andrew A. Adinetz, RCC MSU) • Hash Collision Search using GPU (Andrew A. Adinetz, RCC MSU) • CUDA calculations of strongly correlated and disordered systems (Dr. Sci. Alexey N. Rubtsov, Chair of Quantum Electronics, Faculty of Physics) • Photorealistic rendering (Prof. Bayakovski, Media Lab, CMC MSU) • Fluid Simulation (Prof. Paskonov, Departement of Mathematical Physics)

59

Supercomputing conferences Moscow State University (together with the Russian Academy of Sciences and Supercomputing Consortium of Russian Universities) is one of the most active organizers of the international supercomputing conferences series in Russia.

The chairman of the conferences series is rector of MSU Viktor Sadovnichy.

mercial products. The conference is held annually in March-April in different Russian cities where the largest supercomputing centers are located.

This year the conference was held in Novosibirsk (2 800+ km from Moscow), and in 2013 Chelyabinsk (1 500+ km from Moscow) welcomes the conference again.

“Scientific Service & Internet” conference has quite a long history. It is held annually in September in Novorossiysk (1 200+ km from Moscow), on the Black Sea coast. The conference is devoted to using supercomputer resources in science and education, discussing of new parallel programming technologies, developing of Internet projects on supercomputing activities.

Both of these conferences are coordinated with other noticeable HPC events in Russia. In particular, on the 1st conference day the Top50 rating of the most powerful CIS computers is announced. Important part of both conferences is PhD showcase for students, postgraduates and young scientists. Conferences are sponsored by leading HPC companies.

Another conference, “Parallel Computing http://agora.guru.ru/pavt/eng (in English and Technologies”, is much younger but it is already Russian) considered to be a large-scale event oriented to in- http://agora.guru.ru/abrau (in Russian) dustrial usage of supercomputing facilities. Traditional part of this conference is an exhibition which presents ready-to-use industrial and com-

60

The Project "Supercomputing Education" The leader of the project – Rector of Lomono- academic institutions. A total of 62 Russian unisov Moscow State University academician Sa- versities were involved in the project. dovnichy V. A. Major project participants include LomonoThe second year of “Supercomputing Educa- sov Moscow State University, Lobachevsky State tion” project in Russia has completed. The idea University of Nizhny Novgorod, Tomsk State for the project was presented to the President University, South Ural State University, St. Peof Russia, Dmitry Medvedev, back in 2009. The tersburg National Research University of Inforwork was immediately approved and scheduled mation Technologies, Mechanics and Optics, for the 2010–2012 timeframe, with the imple- Southern Federal University, Far Eastern Fedmentation assigned to Lomonosov Moscow State eral University, Moscow Institute of Physics and University, the university that hosts the largest Technology and other members of Supercomsupercomputing center of Russia. Victor Sa- puting Consortium of Russian Universities. In dovnichy, rector of Lomonosov Moscow State aggregate, more than 600 people participated in University, was named the head of the project. this endeavor in 2011.

The strategic goal of the “Supercomputer Education” project is to create a national system for training of highly skilled supercomputing professionals. The initial effort in 2011 centered around developing and implementing the basic elements of such a system in Russia’s leading

The national System of Research and Education Centers for Supercomputing Technologies (REC SCT), covering all federal districts of Russia, became the basis for the success of the project. The main goal of REC SCT is efficient organization of training and retraining of su-

61

percomputing professionals in universities, in- the Body of Knowledge were developed and disstitutes, industry and business. seminated among universities.

A large-scale training of entry-level specialists on supercomputing technologies was initiated in 2011. This was an activity which involved all federal districts and encompassed 45 Russian universities. Educational programs at most universities were aimed at basic training and getting knowledge of the most widely needed parallel programming technologies required to start using supercomputers and parallel computing systems. A total of more than 1,800 specialists were trained.

Intensive training of special groups aimed at deep studying of specific supercomputing technologies, one of the most difficult but important activity of the project, has been completed successfully. 18 special groups have been organized which were trained by 14 different educational programs. Special groups have been organized in all REC SCT federal districts, and more than 400 people have completed the education successfully.

Distance learning was actively used throughout the project. A total of 251 students from 100 Russian towns and cities have completed training in supercomputing technologies using the Internet University of Supercomputing Technologies (http://hpcu.ru, in Russian).

One of the most important aspects of the project was to develop the “Body of Knowledge� on parallel computing and supercomputing technologies. Based on this, proposals were composed to extend the third-generation national educational standards for Applied MathematA national system of summer youth schools in ics and Computer Science and Mathematics for all the major regions of Russian Federation has more in-depth education on supercomputing also been organized. A number of educational, technologies. Next, proposals will be put forward for academic curricula and retraining programs on supercomputing technologies.

An extensive program for development and reviewing of educational literature on supercomputing technologies for bachelors, masters and postdocs is in the progress. More than 20 textbooks by Russian and foreign scientists are recommended to be included in Supercomputing Education series. All the books will be sent to 43 Russian university free of charge.

In 2011, retraining programs for professors and faculty members were implemented in all federal districts of Russia. More than 160 teachers from 40-plus Russian universities have successfully completed the program.

A lot of work has been done in development of supercomputing educational courses. In all, a total of 37 courses covering the main chapters of

62

analytic and popular papers, materials and Major results of the project are available at the books have been developed in close cooperation Internet Center on Supercomputing Education with leading scientists as well as representatives (http://hpc-education.ru, in Russian), which is of industry and business. gradually becoming an unofficial national center for coordination of supercomputing educaInternational cooperation is one of the key astional activities for the country. pects of the project. Three joint educational programs were developed by Russian and foreign It is extremely important that all activities, universities. Under the project, many foreign both planned and implemented, are targeted scientists participated in scientific and educa- to form an integrated national educational intional activities. In additions, many partnership frastructure, which is required to grow highly agreements on supercomputing technologies qualified supercomputing professionals. An exwere made between Russian and foreign uni- tensive program of activities planned for 2012 versities, institutes, supercomputing centers and envisions wider dissemination of ideas about the companies.

Supercomputing Education project in Russian academic and industrial community.

A special system of public relations activities aimed at promoting supercomputing applica- http://hpc-education.ru tions in different areas of industry, science and society has been developed and implemented. Its scope is extremely broad and includes TV lectures, articles in IT-related online journals, a series of publications in federal and regional mass media, participation in conferences, scientific festivals, exhibitions, excursions to supercomputing facilities, as well as many other activities.

63

HOPSA: Holistic Performance System Analysis To maximize the scientific output of an HPC opers are trying to shorten the time of solution system, different stakeholders pursue different by optimizing their codes, system administrastrategies. While individual application devel- tors are tuning the configuration of the overall system to increase its throughput. Yet, the comRussian partners:

EU partners:

•

Research Computing Center, Moscow State Univer-

•

Forschungszentrum Jülich GmbH (EU coordinator);

sity (Russian coordinator);

•

Rogue Wave Software AB;

•

T-Platforms;

•

Barcelona Supercomputing Center;

•

Joint Supercomputer Center of the Russian Acad-

•

German Research School for Simulation Sciences;

emy of Sciences;

•

Technical University Dresden.

•

Scientific Research Institute of Multiprocessor Computer Systems, Southern Federal University.

64

plexity of today’s machines with their strong interrelationship between application and system performance presents serious challenges to achieve these goals.

the former being under EU and the latter being under Russian responsibility. Starting from system-wide basic performance screening of individual jobs, an automated workflow will route findings on potential bottlenecks either to appliThe HOPSA project (HOlistic Performance cation developers or system administrators with System Analysis) therefore sets out to create an recommendations on how to identify their root integrated diagnostic infrastructure for comcause using more powerful diagnostic tools. bined application and system tuning – with

â–

65

Parallel programs efficiency and scalability evaluation Predict parallel program efficiency, scalability The work has been supported by the Ministry and bottlenecks using hardware and software of Education and Science of the Russian Federamonitoring data. tion, contract 07.514.11.4017.

Strategy

Analyze monitoring data with frequency about 1Hz (CPU load, interconnect traffic, processor performance counters etc.) on series of parallel program runs to identify possible bottlenecks and predict program scalability boundaries.

Objective

Create a tool to identify possible bottlenecks and scalability boundaries with little or without source code modification, and with minimal overhead.

Usage

66

Parallel program development.

Cleo: cluster batch system One of the most important components of supercomputers is a batch system. Developed in RCC MSU, Cleo system is intended for task control on clusters with different configurations and requirements on computing resources usage. All major parallel programming environments are supported, such as mpich, mvapich, Intel MPI and others.

The system state is available in XML format and can be used by any external application. Statistics gathering tools provide both integrated and detailed reports on users’ activities on a cluster.

Cleo is used on SKIF MSU “Chebyshev” (5 000 CPU cores) and “GraphIT!” (192 CPU cores + 48 GPUs) supercomputers of Moscow State UniverCleo allows users to run applications with- sity and in several other organizations. out knowledge of MPI-implementation. Just http://sf.net/projects/cleo-bs.html run mpirun - your application will be queued and then will be started in an appropriate way.

If a user wants to use special running modes or schemes he can use standard POSIX-scripts to run application via Cleo.

The system is portable between majority of UNIX platforms, so it can be used on almost any system. It is able to work with several clusters simultaneously as well as with particular partitions of clusters. Flexible tuning of system parameters, resource usage policies and independent scheduler make Cleo more convenient for administrating and management of different cluster modes. Automatic and manual blocking of nodes and tasks simplifies cluster management without necessity to stop all users’ activities. Priority system and load prediction help managing cluster usage effectively with minimal efforts.

Sample Cleo status web-page for “Chebyshev” supercomputer.

Cleo is easily extendable. Modules’ interface is documented and provided with examples, which helps to create quickly new schedulers or complement system capabilities with new functionality. All Cleo modules work in protected environment, therefore security of the entire system increases.

67

X-Com: distributed computing software Distributed computing, or metacomputing, implies using of available heterogeneous computing resources interconnected with general purpose networks (e.g. Internet) for time-consuming data processing. Imagine you have a task and a supercomputer access, but you are limited in processing time because of supercomputer overload, so you cannot solve your task in appropriate time or with specified precision. But you could use several supercomputers with limited accounts joined together to work on your task and solve it as you like. Next, imagine you don’t have access to a supercomputer at all but you have a number of workstations in office, 1-2 PCs at home, notebook etc. Joining these computers may solve your problem as well. This is what distributed computing is aimed at. Of course you need special software to be installed on your computers to make them work together as a united computer (metacomputer). You may treat this software as a parallel programming technology which helps you to make your application parallel.

In RCC MSU we are working on such software called X-Com Metacomputing System. Distinctive features of X-Com are: quick and easy installation and running without need of super-user privileges, large number of modern hardware and software platforms support, different modes of resource utilization, simple application programming interfaces. X-Com implements multilevel client-server model, it is written in Perl, so it can be used on supercomputers as well as on almost any computer type.

X-Com was successfully used to solve a number of real problems from different scientific areas (electrodynamics, molecular simulation and drug design, network traffic analysis…). For

68

these applications we simultaneously used highperformance computing resources from Top50 list located in Moscow, Saint-Petersburg, Novosibirsk, Chelyabinsk, Ufa, Tomsk, etc.

This time X-Com project is concentrated on efficiency metrics of distributed systems. It is easy to measure efficiency of classical supercomputer – just run LINPACK (or any other benchmark) and get Rmax, take calculator and get Rpeak… But what exactly is Rpeak in highly dynamic environment, how to get Rmax which could be repeated from run to run? In general, how can we present, compare and highlight all specific points of distributed computations series? Now we are working on metrics set and appropriate benchmarking suite which analyzes X-Com log files to solve all this questions.

X-Com is free software available at http://x-com.parallel.ru (mostly in Russian).

Brief English X-Com tutorial is available at http://hpc.msu.ru/?q=node/55

69

New methods of efficient use of GPU-based high-performance computers Joint project of RCC MSU (Russia) and CHPC (South Africa) Graphics processing units (GPU) are widely Development of tests for measuring GPU perused to solve computationally intensive prob- formance characteristics has finished recently. lems. However, efficient algorithm implemen- Characteristics being measured include: tation on GPU still remains a challenge. Imag• bandwidth and latency for different GPU ine a developer who ported his/her application memories (global, local, constant, texture); to GPU, but failed to obtain the 100x speedup • performance of atomics (both global and promised. This could happen to a variety of realocal); sons. The cause of low application performance • host-device bandwidth; need to be determined, bottlenecks must be pin• performance of barrier synchronization; pointed, and application performance needs to • cache line size; be improved.

• number of global memory channels and banks.

The project described here is intended for resolving such issues. There are two main directions of this work. The first direction involves developing a sophisticated set of tests to measure various performance characteristics of various GPUs. It also includes developing samples which demonstrate various GPU optimization techniques. The second direction involves development of tools for monitoring and performance analysis of GPU applications, as well as integrating such tools into existing performance analysis frameworks. Results of both directions will Special memory test results for Tesla C1060. In case of many threads reading the same memory element, they hit the same bank, and performance drops considerably. be tested on real-world applications.

Using local memory is a good optimization that gives significant performance boost, but loop

Special memory test results for Tesla C2050 - Fermi. In case of many threads reading the same

unroll is much better in this case.

memory element, cache comes into play, and performance does not drop.

70

NUDA: using extensible languages for GPGPU programming Graphics processing units are widely used for solving computationally intensive problems.

However, they are mostly programmed with low-level tools, such as CUDA or OpenCL. In order to port an application to GPU, a number of transformations and optimizations must be performed by hand. This makes the resulting program hard to read and maintain. And even if such a program is portable across various GPU architectures, performance is often lost. A higher-level GPGPU programming system is therefore desired.

in runtime. Code for marshaling parameters, copying data arrays to and from GPU memory, and invoking the kernel is automatically generated by NUDA. As GPU kernels in real world are derived from loops, NUDA provides a macro which transforms a parallel loop into a kernel and executes it on a GPU. A number of loop-transforming annotations are provided, which make it possible to perform optimizations without compromising code size or readability. Overall, macros and annotations provided by NUDA increase productivity of porting applications to GPU. NUDA itself is also an example of how extensible languages increase productivity: its size is merely 18000 lines of code.

NUDA (Nemerle Unified Device Architecture) is a project aimed at creating such a system based on Nemerle, an extensible programming language. Extensible languages have flexible syntax and semantics which can be extended to NUDA has been tested on a number of modprovide a better tool for solving specific probel kernels, such as matrix multiplication, Nlems. NUDA adds support for writing GPU kerbody,2D and 3D convolution and some others. In nels using Nemerle. Kernels are translated into each case, it took just several macro applications OpenCL at compile time, and executed on a GPU to make host-only code execute on GPU. Further 2x-7x increase in performance was obtained using loop-transforming annotations, without affecting code size and readability. Resulting code works both on AMD and NVidia GPUs. Fig. 1. This simple annotated DGEMM implementation in Nemerle + NUDA achieves 41% efficiency (213 GFlop/s) on NVidia Tesla C2050 “Fermi�.

NUDA is freely available under LGPL from its website at SourceForge: http://nuda.sf.net.

Fig. 2. Acceleration provided by annotations (inline, dmine) of simple NUDA GPU kernels

Fig. 3. Hypothetical code growth if transformations of simple NUDA kernels were performed

compared with unannotated GPU implementation.

by hands. When annotations are used, size of code does not grow (see fig. 1).

71

SIGMA: collective bank of tests on parallel computing SIGMA is an Internet service that helps to test Short statistics on using “SIGMA” in 2011 can knowledge of students. The main feature of this be presented as follows: system is possibility to combine work of special• Teachers registered: 44 ists to create a distributed mode of high quality • Tests made up: 57 set of tests, questions and exercises for testing • Students groups created: 85 students’ knowledge level. Every specialist can • Students registered in groups: 2756 offer questions to be included into the shared • Number of tests held: 162 bank. Every teacher can make up his own set of • Number of answers to tests: 2530 tests using the collective data bank or using his • Total number of questions answered: 66986 own questions. After the test has been created, the only thing needed for students to start the testing is Internet access.

The main area of use is parallel computing, but SIGMA can bу easily adapted for use in other subject areas.

sigma.parallel.ru (in Russian)

72

MSU supercomputing workshops From the end of 2011 Research and Education Center for Supercomputing Technologies of MSU holds “Supercomputing Technologies in Science, Education and Industry” seminar.

Twice a month REC MSU invites well-known scientists, industry experts and outstanding MSU Supercomputing Center users with the reports about their real HPC tasks and experience.

Reports subjects covers such areas as nanosystems, life sciences, environmental science and management, transport and aerospace systems, energy efficiency and nuclear energy.

More information about the http://agora.guru.ru/sct (in Russian)

- Thomas Sterling (Louisiana State University, USA) with the report “Challenge and Opportunities towards Exascale Computing”; - Felix Wolf (German Research School for Simulation Sciences, RWTH Aachen University) with the report “Application Performance Analysis on Petascale Systems”; - Fayé A. Briggs (Intel Fellow, Intel Architecture Group Director) with the report “Intel multicore processors: scalability and efficiency”;

- Jack Dongarra (University of Tennessee, seminar: USA) with the report “The current state, trends, and future of supercomputing”.

RCC MSU also holds “Parallel.Ru” scientific More information about the workshop: workshop devoted to the world of parallel com- http://agora.guru.ru/parallel (in Russian) puting. A good tradition of the workshop is inviting of leading HPC experts from all over the world. In last three years the guests of our workshop were:

73

Trainings on application development and optimization The understanding of program run-time behavior such as memory usage, logical structure, I/O influence and so on is necessary for most optimization types and the effective utilization of the resources as a result. Specialists of Intel together with MSU-Intel center conducted training series devoted to efficient programming with Intel HPC software tools. The key feature of this event was that Intel specialists helped to understand and tune user’s own code during the training, significantly increasing the effectiveness of the training. Target programming languages included C and Fortran with MPI, OpenMP or combination of the both. 25 MSU Supercomputing Center users took part in this 2-days event.

In November, 2011, the 2-days training on RogueWave TotalView debugger was arranged for MSU Supercomputing Center users. Training was performed by invited RogueWave specialists. 20 users form MSU and other organizations took part in this training. In March, 2012, another RogueWave training was conducted in MSU. Devoted to ThreadSpotter system for analyzing efficiency of application memory usage, this training gathered 13 participants interested in fine tuning of their software.

After installing the Accelrys Material Studio on “Lomonosov” supercomputer, specialists from Accelrys were invited to conduct the training devoted to this toolkit, a modelling and simulation application that is designed for scientists in chemicals and materials R&D as well as pharmaceuticals development. Over 60 people that exploit HPC facilities of Moscow State University took part in the 4-days seminar.

74

Trainings and schools series devoted to various aspects of GPU programming were held in MSU Research and Education Center in the end of 2011 - beginning of 2012. Trainings covered CUDA and OpenCL programming basics as well as OpenACC technology intro. 155 people of 59 organizations took part in four trainings.

Youth supercomputing school “Development of super-scalable applications” was held in Moscow State University in summer of 2011. 23 students, post-graduates and young scientists took part in this 8-days event. Youth schools series will be continued this year as Summer Supercomputing Academy in Moscow (more info included in this book) just after ISC’2012.

Other supercomputing education activities include retraining programs for professors and faculty members (two 10-days trainings with total of 26 people in 2011) and special group trainings for students and post-graduates (10-days training with 55 people in 2011).

Summer Supercomputing Academy Lomonosov Moscow State University and Su- cessfully been trained at the Academy will be percomputing Consortium of Russian Universi- awarded certificates. ties will hold Summer Supercomputing AcadAdvanced retraining courses of faculty staff emy from 25 June to 7 July, 2012. on the programs “Scalable Application DevelSummer Supercomputing Academy is a unique opment” and “Supercomputing System Manpossibility to be trained on a wide range of spe- agement”, when the graduates are subsequently cializations in supercomputing technologies and awarded the State Diplomas, will be held simulparallel computing with practice on the MSU taneously with the work of the Academy. Apart supercomputers: “Lomonosov”, “Chebyshev”, from intensive training, advanced retraining Blue Gene/P. program participants will have a unique opportunity to be involved in all the Academy’s activiUndergraduates, postgraduates, junior scienties. tists, faculty staff, and school teachers can participate in Academy.

The President of Summer Supercomputing Academy: Rector of Moscow State University, The Academy educational program covers a academician V.A.Sadovnichy. wide area of HPC topics. The first week program consists of general lectures and master-classes The leader of the Academy educational proon the most topical issues of exaflops computa- gram: The Corresponding Member of the RAS, tions, prospects for the supercomputing tech- professor V.V.Voevodin. nology development, modern technologies of An official language of the Academy is Russuper-scalable applications development, comsian. putationally complex problems and methods of their solution, overview of supercomputing ht tp://academy.hpc-russia.ru/en technologies from leading IT companies.

The second training week comprises independent training tracks, i.e. profound courses on specific themes. The tracks define the training specialization individually selected by each student of the Academy: graphical and multicore processor programming, advanced MPI/ OpenMP parallel programming, FPGA programming, supercomputing modelling using OpenFOAM, Accelrys Material Studio, FlowVision and others. A lot of related training activities are planned: master-classes of Intel, Mellanox, NVIDIA, RogueWave; excursions to MSU Supercomputing Center; popular science lectures and others. The students who have suc-

75

Excursions to supercomputing centers Research and Education Centers for Supercomputing Technologies carry out an active work with school children and students.

In particular, research workers of REC “SCT– Center” organize excursions for schoolchildren to Lomonosov Moscow State University Supercomputing Center, and REC “SCT–North–West” carries out excursions for pupils of schools – participants of the Olympiads, held by St. Petersburg State Research University of IT, Mechanics and Optics.

sport shoes and why for these purposes a supercomputer is needed?

• Why can a modern computer occupy the whole hall, weigh 20 tons and why is it considered to be normal?

• Why is the Internet the world biggest computer?

• How is it possible to make a supercomputer system from school or home personal computers?

During these excursions a lot of other problems are discussed and the real supercomputer During the excursions the schoolchildren get systems are shown. information about different problems in supercomputing area: • Why has our world become computerized, and the computerized world – parallel?

• Why is it not enough to use common computers to create a good automobile and even

76

■

2012 Lomonosov Moscow State University Research Computing Center 119991 Russia, Moscow Leninskie Gory 1, bld. 4

FACILITIES APPLIC ATIONS AC TIVITIES