Issue 70, autumn 2011
New horizons New management at EPCC and a new era in HPC
In this issue... 2
EPCC becomes NVIDIA CUDA Research Center 14 Data-intensive engineering at EPCC
3
We are under new management!
15 OGSA-DAI Visual Workbench
4
C ray’s Exascale Research Initiative in Europe
16 Software Sustainability Institute
European Exascale Software Initiative 5
Asynchronous algorithms PlanetHPC: research and innovation strategy
6
Nu-FuSE: Nuclear fusion simulations at Exascale
7
CRESTA and the Exascale Challenge
8
HPC-Europa research visitor programme
10 BonFIRE: building Service test beds on FIRE 11 SPRINT: data analysis on HPC systems 12 DEISA: virtual European supercomputer
17 Understanding the brain 18 Simulating rare events 19 EDIM1: Data Intensive Research machine 20 The MSc in HPC: ten years on! 21 Empowering research in the humanities 22 APOS-EU: Application Performance Optimisation and Scalability 23 Overcoming digital exclusion 24 HECToR: the third and final phase
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh
Editorial Mario Antonioletti
It is always interesting to find out what my colleagues at EPCC have been up to, and EPCC News offers an opportunity to do just that. We are currently involved in a number of Exascale activities that will help determine how future HPC applications will reach 1018 flops and will also develop the types of systems and tooling required to make this possible. For example Nu-FuSE (page 6), which will model the impact of the fusion process on the materials used to build fusion reactors, will require Exascale performance. Cloud computing activities are discussed in the BonFIRE article (page 10). Several data projects are also currently active, with an OGSA-DAI visual workbench (page 15) being developed to facilitate the construction of OGSA-DAI workflows. The ADMIRE project, which developed a data mining platform, has recently come to an end and some of its work is being taken forward by EUDAT (both page 14). Finally, EPCC in collaboration with the School of Informatics here in Edinburgh has designed and built a new machine called EDIM1 to investigate how this type of hardware may be used to deal with data-intensive research problems (page 19).
There are some anniversaries: our MSc in HPC (page 20) has now been in running for ten years, while the Software Sustainability Institute (page 16), headquartered at EPCC, has reached its first year. Articles for both of these activities discuss how productive they have been. You will find many other topics covered in this issue and I hope you will dip in to find out what is happening at EPCC. As a final note, we have come under new leadership. Mark Parsons and Alison Kennedy have jointly taken on the directorship role - you can read about how they see the centre evolving in their article on the opposite page. Arthur Trew, who has ably directed EPCC for the last 16 years, has recently become Head of the School of Physics & Astronomy (of which EPCC is a part) here at Edinburgh. We thank him for his leadership and wish him well in his new role. The one abiding image that I will take away from my editorial role is that of a million chickens pulling a plough as a metaphor of Exascale computing (see Cray Exascale research story on page 4). I hope you enjoy reading this issue of EPCC News and find it informative.
EPCC awarded NVIDIA CUDA Research Center status EPCC’s pioneering work in the area of GPU computing has gained it the status of NVIDIA CUDA Research Center [1]. Graphics Processing Units (GPUs) were originally designed to display computer graphics, but they have developed into extremely powerful chips capable of handling demanding, general-purpose calculations. However, the GPU architecture is somewhat different to that of the CPU, and cannot be programmed with traditional computing languages. In 2007 NVIDIA made GPUs widely accessible with the release of the CUDA hardware and software environment, and since then EPCC has been working hard to help researchers and commercial partners overcome the challenges of using this novel architecture to exploit its performance capabilities. For example, EPCC’s expertise in large-scale parallel computing has been combined with the powerful CUDA environment to create an application capable of simulating extremely complex fluid dynamic models on the largest of NVIDIA GPU-accelerated supercomputers, allowing cutting-edge research into condensed matter physics [2]. Furthermore, EPCC is actively involved in the development of new programming techniques which promise to increase future productivity, in particular the expansion of the OpenMP directive-based model to support accelerators such as GPUs [3]. 2
Dr Alan Gray, who leads EPCC’s GPU activities, said, “We are very excited by this new partnership with NVIDIA. We believe that GPU accelerated architectures are becoming increasingly important computational resources, and we aim to work closely with NVIDIA to benefit as wide a range of research areas as possible.” [1] NVIDIA CUDA Research Center Program: http://research. nvidia.com/content/cuda-research-center-crc-program [2] A. Gray, A. Hart, A. Richardson and K. Stratford, Lattice Boltzmann for Large-Scale GPU Systems, to appear in the proceedings of ParCo 2011 (2011): www2.epcc.ed.ac. uk/~alang/publications/GRAYParCo2011.pdf [3] J. Beyer, A. Gray, A. Hart, L.M. Levesque, D. Oemhke, H. Richardson, ‘Programming hybrid architectures with OpenMP accelerator directives’, talk presented at Many Core and Reconfigurable Supercomputing 2011 Conference (2011), slides available at www.mrsc2011.eu/Programme Further examples of EPCC’s GPU activities are described in the Spring 2011 issue of EPCC News: www.epcc.ed.ac.uk/library/newsletters/epcc-news
EPCC is under new management!
Clockwise from right: Arthur Trew, Mark Parsons and Alison Kennedy.
Alison Kennedy and Mark Parsons
Alison Kennedy and Mark Parsons assumed responsibility for the management of EPCC on 1 January 2011. As Executive Directors, they take the helm at a very exciting time for EPCC. When EPCC was founded in 1990, computational simulation was in its infancy and parallel computing was a novel niche technology. Today, simulation is accepted as the third methodology of science, complementing theory and experiment. It is used widely in academic research and by industry and commerce for improving products and reducing time to market. With the advent of multicore processors, parallel computing has entered the mainstream, with all of the challenges that this entails. EPCC has been through a number of distinct phases in its history. The early days of of the centre were enabled through the Government’s Parallel Applications Programme, which was designed to stimulate the use of parallel computing by industry and commerce. By the mid-to-late 1990’s our focus had shifted to the European Commission’s Technology Transfer Node network and subsequent projects that sought to cascade the benefits of HPC down to smaller companies using more readily available computers. Our third phase focused on the UK e-Science Programme and the explosion of Grid computing research that occurred around the start of the new millennium and through its first decade. Throughout all of this time EPCC has followed a strategy of working with multiple partners from the public and private sectors to deliver innovative projects whilst also delivering National HPC services on behalf of the UK Research Councils. Today EPCC is entering a new phase, not just with a new management team, but also in terms of where current computing technologies are taking us. In 1994 the Cray T3D
service had 512 processors. In 2010, the end of the IBM HPCx service had 2,500 processors. Current systems such as HECToR have tens if not hundreds of thousands of processor cores. This is having profound consequences on how we program and use modern high performance computers. The multicore era is also affecting desktop computing users. As the number of cores in processors has increased on laptop and desktop computers, the speed of each core has slowed. This means everyone has parallel computing challenges. EPCC’s skills have never been in greater demand both from the scientific and the industrial domains. In the last year our Exascale Technology Centre with Cray has worked on some of the key technologies required for the next generation of massively parallel computers. These computers are expected to contain millions of cores and this challenges all we know about numerical simulation and modelling. The challenges are significant, the potential benefits are enormous, and the opportunities for the future are greater than ever. As EPCC’s directors, we bring complementary skills in business management, project management, research management and commercialisation and are ably supported by a small team of Directors, with specific responsibilities for key EPCC activities. We thank our former Director, Professor Arthur Trew, who, having led EPCC for sixteen years, has taken up a new position as Head of the School of Physics and Astronomy at the University of Edinburgh. We wish him well in his new job. 3
Cray’s Exascale Research Initiative in Europe Alistair Hart, Cray
Asynchronous algorithms
For the past two years, Cray’s Exascale Research Initiative Europe has been working closely with EPCC’s Exascale Technology Centre (ETC) to understand and overcome the barriers to programming the next generation of supercomputers. With the launch of the EU CRESTA project (see page 7), this work will continue to drive the development of new programming models and tools to push scientific application performance to the next level.
Mark Bull
Every ten years since 1988, supercomputers have got a thousand times faster. Today, computers like the HECToR Cray XE6 run real scientific programs at a speed measured in petaflops. Continuing this trend, by 2018 we expect another thousand-fold leap forward in performance, when sustained application performance will be measured in exaflops. Put another way, this is one hundred million sums for every person on the planet, every second. But the Road to the Exascale is a hard one. Seymour Cray, the pioneer of supercomputing, famously once asked if you would rather plough a field with two strong oxen or five-hundred-andtwelve chickens. Since then, the question has answered itself: power restrictions have driven CPU manufacturers away from “oxen” (powerful single-core devices) towards multi- and manycore “chickens”. An exascale supercomputer will take this a step further, connecting tens of thousands of many-core nodes. Application programmers face the challenge of harnessing the power of tens of millions of threads.
Cray’s Exascale Research Initiative Europe and EPCC’s ETC are finding ways to overcome these challenges. New programming models like Fortran Coarrays simplify and accelerate the communication of data between processors. The Exascale collaboration has pioneered work in this, presenting tutorials at the Supercomputing Conferences in 2010 and 2011, and developing the first benchmark performance suite. The CPUs in future Exascale systems may have more in common with today’s GPUs (Graphics Processing Units, beloved of high-end gamers) than the Opterons used in HECToR. The collaboration’s work on new programming models for these is contributing to a new, extended OpenMP international standard and has resulted in invited presentations at international conferences. Network and I/O performance are already important for applications and will be crucial at the Exascale. Nevertheless, they are poorly understood and hard to quantify. By developing tools to monitor the performance of these subsystems, Cray and EPCC can now present the information needed to tune the hardware and applications for maximum performance. The challenge and importance of Exascale scientific computing has now been recognised in the EU by the funding of research networks focusing on system and application software. Cray is the Industrial Partner in the CRESTA project, with EPCC as lead and many other Cray customers are partners. By the time CRESTA reaches its final milestone in late 2014, Cray’s Exascale Research Initiative Europe will already be halfway down the Road to the Exascale.
One of the problems we face is that the algorithms are very synchronous: processors frequently have to wait for data from other processes to arrive before they can proceed. It turns out that some algorithms will still work, albeit with slower convergence, if we relax some of the synchronisation, and compute using the most recent data which happens to have arrived. These asynchronous (or chaotic) algorithms were studied theoretically back in the 1980s, but with HPC machines typically having a few hundred processors, they didn’t have much practical application at the time. However, as we head towards exascale systems with more than a million cores, revisiting these ideas may prove beneficial. EPCC is a partner in a project which aims to do just this, together with collaborators from the universities of Manchester, Leeds, Hull and Strathclyde. The project will re-analyse some existing algorithms, and try to develop new ones, looking at theoretical properties, practical implementations on modern
One of the applications used by ASYNC calculates the maximum principal strain in a human mandible undergoing incisor biting, using about half a million elements. For more detail see: F Gröning et al. 2011 “Why do humans have chins? Testing the mechanical significance of modern human symphyseal morphology with finite element analysis,” American Journal of Physical Anthropology, 144: 593-606.Thanks to Michael Fagan for providing this image. architectures, and the potential of these algorithms to tolerate certain types of faults. The project also has two demonstrator applications – one in finite element modelling of stress in bones, and another in the optimisation of electrical power grids – to which these alternative algorithms will be applied. Together we hope to find new ways of solving this type of problem on future generations of very parallel, but possibly unreliable, hardware. This project is funded by the Engineering and Physical Sciences Research Council.
PlanetHPC: a strategy for HPC research and innovation Mark Sawyer
European Exascale Software Initiative Lorna Smith The European Exascale Software Initiative (EESI) is building a European vision and roadmap to address the challenges of developing and utilising future large-scale HPC systems. As part of this, EPCC and our colleagues at CSC in Finland have been working on a review of the current state of HPC, providing a baseline and guide for this road mapping activity.
PFLOPS. China, whose Tianhe-1A supercomputer was ranked top in November 2010, remains the fastest developing country in Asia and the world. China has invested heavily in supercomputing technology and now holds second position in the TOP500 list. The Chinese government has actively promoted independent innovation to avoid reliance on foreign technology.
The report provides an overview of the funding structures, HPC centres and large scale HPC initiatives across Europe, the US and Asia. A brief summary is provided here to whet your appetite, the full report is available on the EESI website.
However, following significant investment over the last few years, the US is still dominant in the supercomputing sector, with five systems performing at the petaflop level and with two 20 PFLOPS systems anticipated by 2012.
Asia has been investing heavily in supercomputing technology and now hosts the two fastest supercomputers and four of the five fastest in the world. The two dominant countries in Asia are Japan and China. In 2011, Japan reclaimed the top spot in the TOP500 (the list of the world’s 500 fastest supercomputers) with the “K Computer” at the RIKEN Advanced Institute for Computational Science in Kobe. This has a performance of 8.77
Europe has two petascale systems, in France and Germany. The Bull system (9th in the world) installed at the French CEA is the most powerful in Europe, with a peak performance of 1.254 PFLOPS. Significant funding has been made available in Europe through the Framework 7 programme, both in Petascale (eg PRACE) and Exascale terms (see CRESTA article on p7).
4
Many large scale application codes rely on algorithms for solving sparse systems of linear equations. But as core counts in HPC machines seem set to increase over the next few years, will this type of algorithm scale up?
EESI website: www.eesi-project.eu
The PlanetHPC team has been hard at work over the summer months producing a report for the European Commission to advise it on its strategy for HPC research and development. PlanetHPC is creating a roadmap for HPC research, looking forward to 2020 and with the exascale era in mind. This is likely to be a period of disruption in HPC with many challenges, including energy consumption (an issue dominating much discussion across ICT as a whole) and how to program the new generations of HPC systems that will arise over the next decade. PlanetHPC held a workshop in May 2011 which was attended by leading industrialists and HPC experts. The delegates took part in brain-storming sessions to discuss requirements for future HPC applications, new programming models and techniques and how to engage industry in using HPC. A follow-up workshop was held in July 2011, at which EPCC presented a draft strategy paper based on the findings of the first workshop. Delegates also heard presentations about the
www.planethpc.eu
HPC requirements of life sciences presented by Dr Peter Maccallum of Cancer Research UK and Phil Butcher of the Wellcome Trust Sanger Institute, together with experiences of commercialising research presented by Dr Guy Lonsdale of scapos AG. PlanetHPC has now completed its strategy paper. Entitled A Strategy for Research and Innovation through High Performance Computing, it will shortly be released for public comment and will be available on the PlanetHPC website. Central to the strategy is the recommendation to set up regionally based HPC pilot networks to stimulate the HPC market and prepare the foundation for future longer term R&D. The aim is to encourage more industries (in particular SMEs) to try out HPC solutions, and at the same time look towards the requirements of HPC in 2020. PlanetHPC website: www.planethpc.eu PlanetHPC is funded under Framework Programme 7. It receives funding from the European Commission under contract no 248749. 5
Nu-FuSE Nu-FuSE: Nuclear fusion simulations at Exascale Adrian Jackson
Nuclear fusion promises a low pollution route to generate a large fraction of the world’s energy needs in a sustainable manner. This is especially important as global energy consumption has grown by more than 20 times over the last century with the majority of current energy generation reliant on fossil fuels. But the scientific and engineering challenges in designing a fusion reactor are formidable and commercial power plants are not expected before 2050. To be economically sustainable, fusion energy reactors will need a lifetime of many years. Therefore, the materials that the reactor is built from must be able to withstand the likely impacts and damage that fusion plasmas will inflict on them. Reproducing the actual conditions of extended high neutron flux (such as materials would encounter in a fusion reactor) is currently impossible in the laboratory, so materials tests will require extrapolations from experiments conducted under alternate irradiation conditions. Since radiation damage is a nonequilibrium process, there is no simple analytic way to carry out such extrapolations. Thus a detailed understanding of the fundamental processes involved is required, and this can only be obtained from simulation and a full understanding of the processes involved in irradiation damage at atomic levels. At its most ambitious, this will enable the design of radiation-resistant materials, but even at more pragmatic levels it will enable us to build models based on low-dose, short timescale experiments for reliable exploration of longer term effects. At present, stainless steels are by far the most likely materials to be used in the construction of fusion reactors. However there is some evidence that ferritic (bcc) steel has better self-healing properties, albeit with a reduction in the structural behaviour at high temperatures. The main causes of radiation damage in steels are corrosion, swelling, creep, precipitation and grain boundary segregation. All of these are relatively slow processes and so performing accurate calculations requires simulation of years of material time - an average 250-atom quantum calculation of impurity interaction in steel might take a day of supercomputer time. Typical commercial steel might have ten elements, each of which interacts with the other elements present. Vacancies, interstitials helium and other defects also 6
need to be considered. It is evident that very large amounts of computational resource will be needed to model the physics required to understand the radiation damage sustained by metals over the large time scales necessary for the damage to become apparent. Nu-FuSE is an international project (funded through the G8 Research Councils Initiative on Multilateral Research Funding) looking to significantly improve computational modelling capabilities to the level required by the new generation of fusion reactors. The project’s focus is on three specific scientific areas: fusion plasma; the materials from which fusion reactors are built; and the physics of the plasma edge. This will require computing at the exascale level across a range of simulation codes, collaborating to work towards fully-integrated fusion tokamak modelling. The project is led by Professor Graeme Ackland at The University of Edinburgh, and includes research teams in: Caderache, France (also the location of ITER, the next generation of fusion reactors); Edinburgh; Princeton, USA; Garching and Jülich, Germany; Tsukuba, Japan and Keldysh, Russia. The largest supercomputers today can perform over a petaflop of calculations per second on real scientific applications. But exascale systems are planned for 2016-2018, which means a million million million calculations per second: a thousand times faster than today’s systems. Major challenges in scaling, resiliency, result validation and programmability must be addressed before exascale systems can be exploited effectively for fusion modelling. The Nu-FuSE project is focusing on meeting these challenges by improving the performance and scaling of community modelling codes to enable simulations far larger than are currently undertaken. The materials simulation work, where EPCC is working closely with Prof. Graeme Ackland and his group of researchers, is modelling the type of active particle impacts that a fusion reactor wall will encounter during the fusion process. NuFuSE website www.nu-fuse.com
A mountain to climb: CRESTA and the Exascale Challenge Lorna Smith
The Exascale Challenge is to build a supercomputer that can deliver an exaflop – that is, a million million million calculations per second. Meeting the challenge will require research and development on a global scale and so a number of exascale research programmes have been created. The European Union, for example, announced the start of its exascale research programme in October 2011, with three complementary Framework 7 projects with a combined funding of €25 million. CRESTA (Collaborative Research into Exascale Systemware, Tools and Applications) is one of these projects. Having demonstrated a small number of scientific applications running at the petascale (1015 flop/s), the HPC community, particularly the hardware community, is looking to the next challenge: to reach the milestone of an exaflop (1018 flop/s). Formidable challenges are involved, including scale (such systems could have over a million cores), reliability, programmability, power consumption and usability. The timescale for demonstrating the world’s first exascale system is estimated to be 2018. From a hardware point of view we can speculate that such systems will consist of:
fusion energy, the virtual physiological human, numerical weather prediction and engineering. Over the duration of the project, the aim is to deliver key, exploitable technologies that will allow the co-design applications to successfully execute on multi-petaflop systems in preparation for the first exascale systems towards the end of this decade.
At the heart of the proposal is the co-design process. The co-design vehicles are closely integrated to the systemware development. The applications will exploit the output of the • large numbers of low-power, many-core microprocessors systeware tasks as well as providing guidance and feedback to (possibly millions of cores) direct future systemware work. A key feature of CRESTA is its • numerical accelerators with direct access to the same memory use of dual pathways to exascale solutions. Many problems in as the microprocessors (almost certainly based on evolved HPC hardware and software have been solved over the years GPGPU designs) using an incremental approach. Most of today’s systems have also • high-bandwidth, low-latency novel topology networks (almost been developed incrementally, growing larger and more powerful with each product release. However, we know that certainly custom-designed) some issues at the exascale, particularly on the software side, • faster, larger, lower-powered memory modules (perhaps with will require a completely new, disruptive approach. CRESTA evolved memory access interfaces). will therefore employ incremental and disruptive approaches to technical innovation, sometimes following both paths for a Only a small number of companies will be able to build such particular problem to compare and contrast the challenges systems. But it is crucial to note that software and applications associated with each approach. are also part of the exascale computing challenge: this is where the EPCC-led CRESTA project comes in. CRESTA brings together four of Europe’s leading HPC centres (EPCC together with HLRS, CSC and PDC), a world-leading supplier of HPC CRESTA will focus on the use of six applications with exascale systems (Cray), seven application and problem owners from science and potential and use them as co-design vehicles to develop an industry (DLR, KTH, ABO, JYU, UCL, ECMWF and CRSA), Europe’s integrated suite of technologies required to support the leading HPC tool company (Allinea) and Europe’s leading performance execution of applications at the exascale. The application set in CRESTA has been chosen as a representative sample from across analysis organisation (TUD). CRESTA project website: www.cresta-project.eu the supercomputing domain including: biomolecular systems, 7
HPC-Europa: short research visits, long-term benefits Catherine Inglis
Regular readers of EPCC News will know that HPC-Europa is an EC-funded programme which allows European computational scientists to undertake short collaborative research visits in a host group with similar interests, while gaining access to some of the most powerful HPC facilities in Europe. Most visits produce successful results, with many
leading to long-lasting collaborations, often resulting in scientific publications, joint research proposals and job offers. We are always pleased to hear how past visitors’ careers have developed following their visits. Here, two of our recent visitors share their experience in their own words with us. We will publish further examples on the EPCC website. Left to right: Anastasia Romanova, Maxim Fedorov, Andrey Frolov.
Dr Rosa Filgueira Edinburgh Data Intensive Research Group, School of Informatics University of Edinburgh
Left side: Our simulations reveal that – due to the ion depletion from the carbon nanotube surface – the nanotubes should precipitate in organic dispersions upon addition of salts and form bundles.
Right side: Experiments confirm the simulation results: NaI salt addition leads to precipitation of nanotubes and formation of bundles that are well recognizable by eye.
Maxim V. Fedorov, Andrey I. Frolov, Anastasia O. Romanova Max Planck Institute for Mathematics in the Sciences
Project: Basic mechanisms of salt effects on carbon nanotube dispersions in organic solvents: insights from large-scale Molecular Dynamics simulations. I started as a Research Scholar in the Computer Science Department at Carlos III University in Madrid in 2004. In July 2010 I finished my PhD thesis: Dynamic optimization techniques to enhance scalability and performance of MPI based application. Since then, I have been working in the same university as a Teaching Assistant. My research has focused on improving the scalability and performance of applications executed on multicore clusters applying several run-time techniques such as adaptive compression and data locality aware methods. Between February and May 2011, I had the opportunity to spend a research visit at the Data Intensive Research Group of the University of Edinburgh funded by the HPC-Europa2 program. This group is led by Professor Malcolm Atkinson who was also my host during my visit. During my visit I improved one of the techniques that I developed during my doctoral work. Moreover, I learned about new methods to exploit the rapidly growing wealth of data. Being exposed to the extensive experience of the Data Intensive Research Group in this area has been very beneficial to my understanding of the research problems associated with the challenge of dealing with vast amounts of data. One of the most important outcomes of my research visit is that I have joined this group as a Postdoctoral Research Associate. Without the contacts made through the support of the HPC Europa-2 program and the collaboration with my future research group this opportunity would certainly not have presented itself to me, so the visiting fellowship program has been extremely useful for my future career development. 8
Fig. 1: Overall Speedup by using ACN and AVN aggregation pattern. As I mentioned before, my research proposal for this visit was to improve one technique developed in my doctoral work. This means improving scalability and performance of MPI-based applications executed in multi-core architectures, reducing the overhead of the I/O subsystem by improving the performance of Two-Phase I/O technique. I developed two new different aggregator patterns for this I/O technique called aggregation-by-communication-number (ACN) and aggregation-by-volume-number (ACV). Further, I studied the performance of these aggregator patterns using an MPI application called BISP3D, that uses Two-Phase I/O technique to write the results to a file. I have executed this application with different input data, with different numbers of processes (ranging from 8 to 128) and by using the original Two-Phase I/O and the modified Two-Phase I/O with the ACN and AVN aggregator patterns. The evaluation (Figure 1) shows that in most of the cases the modified Two-Phase I/O brings a speedup between 1.2 and 1.3 with one pattern that we have proposed. This means that with the appropriate aggregator pattern, the overall execution time is reduced. In the worst cases, our technique can lead to an additional overhead of up to 5% in computation time. For these cases, the default aggregator pattern seems to be good enough.
When myself and my junior colleagues (Mr Andrey Frolov and Ms Anastasia Romanova) received our HPC-Europa grant in the beginning of 2011, I was working as a Group Leader (Group of Computational Physical Chemistry and Biophysics) at Max Planck Institute for Mathematics in the Sciences in Leipzig, Germany. My research interests include chemical physics and molecular biophysics with the main focus on modelling solvent and ion effects on biomolecules and nanoparticles. From February 2011 till May 2011 we worked together with our host, Dr Alex Rozhin (School of Engineering and Applied Science) at Aston University in Birmingham on ion effects on carbon nanotube dispersions in organic solvents. During our visit we fully exploited the power of the HECToR supercomputer by doing atomistic simulations of large solvention-nanotube systems (hundred thousands of atoms). In parallel, our host (a well-known expert in experimental nanoscience) and his colleagues ran experiments with the same systems. This synergetic combination of simulations and experimental techniques helped us to reveal the main molecular-scale mechanisms of ion effects on nanotube dispersion stability. We have shown that it is possible to regulate the nanotube dispersion stability and bundle formation by changing the type and concentration of ions. Because our findings provide an inexpensive and efficient route to bundle engineering of carbon nanotubes, they are expected to have a high impact on several areas of nanoscience. During our visit we generated a bulk of interesting results on molecular-scale interfacial phenomena in nanotube dispersions in organic electrolytes. Some of the results we have already published in two papers (Fedorov et al, PCCP, (2011), 13, 12399; Frolov et al, Faraday Discussions, accepted, doi:10.1039/ c1fd00080b) and several more manuscripts are under review and in preparation for publication. The importance of the results
have also been appreciated by the poster evaluation panel (“Colloid chemistry” division) on the 242nd ACS National meeting in Denver where Andrey Frolov was awarded a prize (sponsored by BASF) for our joint poster presentation “Effects of salts on carbon nanotubes dispersions: understanding molecular mechanisms” during the session “Fundamental research in colloid and surface science”. This HPC-Europa visit also helped me to strengthen my existing collaborations in the UK and establish some new contacts there, particularly in Scotland (my very first visit to Scotland in 2005 was also supported by the HPC-Europa program). Therefore, I cannot say that it is just a coincidence that shortly after my 2011 HPC-Europa visit I received a professorship offer from the University of Strathclyde in Glasgow where I am moving in October 2011 as a Professor in Physics and Life Sciences in the Department of Physics. I would like to emphasize that an interdisciplinary project such as this would be almost impossible to perform without the HPC-Europa support that gave us: (i) access to the high-end supercomputer facilities; (ii) a great opportunity to communicate on a daily basis with our experimental colleagues in Aston. That was very important for the project and brought much ‘synergy’ to our collaborative work. We very much appreciate the highly qualified level of support from the HPCEuropa staff in Edinburgh. I am also very grateful to them for organising my seminar there in May 2011 where for the first time I presented our new results in public. We are particularly thankful to Catherine Inglis (administration) and Chris Johnson (high-performance computing) for their day-to-day support. More information about HPC-Europa, including the application closing dates and details about how to apply, can be found at: www.hpc-europa.eu 9
BonFIRE – building service test beds on FIRE Kostas Kavoussanakis The BonFIRE project is building a Future Internet Research and Experimentation (FIRE) facility for the large-scale testing of distributed applications, services and systems. To this end, BonFIRE offers a multi-site Cloud test bed with heterogeneous compute, storage and networking resources. Users can run experiments on the BonFIRE facility to determine how their application will work with different types of Cloud infrastructure, thus allowing them to understand and improve their application’s performance in these types of environments. The BonFIRE facility has been operational since June 2011. Currently access is only available through Open Calls, although the facility will open up in Quarter 3 of 2012 to researchers outside the BonFIRE consortium. BonFIRE aims to support three experimentation scenarios, each offering the user a different level of network control: • Basic experimentation involves a geographically dispersed, multi-site Cloud facility whose heterogeneous resources are interconnected through the public Internet. Dedicated resources are offered at four BonFIRE sites: EPCC, INRIARennes, HLRS in Stuttgart and HP Labs in Bristol. • The second scenario allows users to experiment with their own virtualized, controlled emulated networks. It involves the “Virtual Wall” Cloud at IBBT in Ghent, which is based on the Emulab software. • The third type of scenario, for which support is planned for May 2012, will allow experimentation with controlled physical networks. The pragmatic evolution of BonFIRE is important and three embedded experiments have driven its development since its inception. The first is investigating the requirements arising from the composition of services at the different layers of the Future Internet ecosystem. This includes Cloud and services composition. The second experiment is working to classify virtualized resources in terms of their performance when running high-level benchmarks, with a view to generating predictive models of how these resources will fit the needs of user applications. The third experiment is investigating the elasticity requirements of Cloud-based web applications. Twenty-seven use cases were captured from these experiments, and from the experience and expertise of the BonFIRE partners; from these, 101 functional and non-functional requirements were elicited. These have underpinned the BonFIRE architecture in the first twelve months of the project. Four more experiments joined BonFIRE in September, as a result of the first Open Call. The first new experiment will use the Virtual Wall in Ghent to validate the hypothesis that 10
SPRINT: R for HPC
The BonFIRE infrastructure. Picture courtesy HLRS, University of Stuttgart.
Muriel Mewissen, Division of Pathway Medicine
proprietary desktop virtualization and dynamic virtual network path technologies can enhance user experience, especially regarding multimedia applications. The second new experiment will investigate the impact of using a distributed infrastructure on the high-throughput problem of calculating the dose of radiotherapy treatment. The third new experiment will exploit the various hypervisors and Cloud managers on BonFIRE to test at a large scale a proprietary, continuous security-monitoring framework. Finally, the fourth experiment will deploy and test two service optimization models across the distributed BonFIRE infrastructure. At present, BonFIRE provides the following features to help users understand how their applications work in Cloud-type distributed environments: • Advanced monitoring tools give access to performance metrics for the virtualized resources and to specific experiment monitoring results. • Both real and emulated networking resources are available on interconnected test beds. Controlled networks are expected to be available from June 2012. • In-depth user documentation and support for experimenters facilitate uptake of the facility and increase the quality of the research. Access to BonFIRE resources is through the standard, Open Cloud Computing Interface (OCCI) interface [1]. BonFIRE additionally supports experimentation that can be done using the RESTfully scripting language [2]. The BonFIRE Portal provides a higher level mechanism for experiment specification and also visualises resource deployment. In total, 250 cores with 460GB of RAM and 12TB of storage are available permanently. This will increase to 350 cores, with 700GB of RAM and 30TB of storage by December 2012. On-request access to about 2,300 additional, multi-core nodes is also possible. Single sign-on to all BonFIRE services, root access to compute resources and support for cross-site elasticity come as standard. The BonFIRE facility is planned to evolve in four major phases, each adding functionality based on the needs of its users. The first release introduced the features and capabilities that are currently available in the system. The first upgrade to the platform software is due in October 2011. Of particular interest is the planned support for experiment descriptors based on the Open Virtualization Format standard, as well as increased
Continues opposite.
Following two years of development, the SPRINT (Simple Parallel R INTerface) framework is nearing the release of its first full version, SPRINT v1.0.0, due in October 2011. SPRINT, an R package specifically designed for use in high performance computing (HPC), provides R users with the parallel tool kit needed to perform computationally challenging statistical analyses easily on HPC. SPRINT lifts the limitations on data size and analysis time commonly found in many disciplines like bioinformatics where large volumes of highly parallel, high throughput post-genomic data are generated. SPRINT’s unique strengths are its HPC platform independence, its focus on complex, hard to parallelise, data dependent functions and its user friendliness. SPRINT has two main components: a library of parallelised R functions and an intelligent harness handling all aspects of using HPC. The SPRINT harness is implemented using the MPI standard, a parallel processing technology supported by a wide range of platforms. This ensures SPRINT can run and provide performance benefits on multi-core desktop PCs, shared memory platforms, clusters, clouds and supercomputers. Use of SPRINT requires no HPC knowledge and allows the researcher to concentrate on biological questions rather than the technicalities of IT infrastructure. Existing R scripts only need minimal changes to exploit the SPRINT library. This enables the biologist to use tried and tested scripts on larger datasets and on HPC without further programming. The functions implemented in SPRINT were selected in order to address analysis bottlenecks as identified by a survey of the R user community. Previous releases of SPRINT have included parallel implementations of Pearson correlation (pcor), permutation testing (pmaxT) and a clustering function for partitioning around medoid (ppam). These SPRINT functions have shown extremely good performance and scalability. SPRINT version 1.0 includes three new parallel functions. pApply is an implementation of the R apply function. This allows a user to perform the same operation over all the support for the experiment lifecycle. A second upgrade to the platform software in May 2012 will be shaped by the experiences and results of the users gained though the first Open Call for experiments. Additionally, it will incorporate the ability to federate BonFIRE systems with external facilities, like FEDERICA [3], while focusing on the deployment of a Cloudto-network interface, which will enable the control of different types of network infrastructures. To this end a sixth facility, at PSNC, will be added to BonFIRE. With EPCC leading the activity that builds BonFIRE, these are very exciting times. The final phase, due in May 2013, will use results from the second Open Call for experiments, together with any non-funded experiments, to construct the final version of the facility.
Photo: Mark Sadowski elements of data objects such as matrices or lists. The other two functions are implementations of the machine learning methods, random forest and rank product, respectively prandomForest and pRP. The random forest algorithm is an ensemble tree classifier that constructs a forest of classification trees from bootstrap samples of a dataset. The random forest algorithm can be used to classify both categorical and continuous variables. Rank products are a method of identifying differentially regulated genes in replicated microarray experiments. Planned future additions to SPRINT include the R bootstrap and SVM functions as well as an investigation of algorithms for the processing of next generation sequencing. Recent paper publications and presentations at international conferences have increased SPRINT’s visibility. The figures on the Comprehensive R Archive Network (CRAN) show an increased and sustained interest in the SPRINT package with downloads currently running at over 150 per week (cran.r-project.org/report_cran.html). Clearly the field of application and usefulness of SPRINT will grow even further with the addition of new functionality. The release of SPRINT v1.0.0 is therefore expected to add to the popularity of the framework. In addition the SPRINT team are currently organising their first training courses in Edinburgh and Oxford. Further details can be found on the SPRINT website. SPRINT is an open-source project and is open to contributions. All sources are available from R-Forge: r-forge.r-project.org Packaged releases are available from CRAN: cran.r-project.org SPRINT project: www.r-sprint.org Contact the SPRINT team: sprint@ed.ac.uk
If you are interested in joining BonFIRE, the second Open Call for funded experiments will open on 1 February 2012 and close at 17:00 CET on 7 March 2012. During the third quarter of 2012, the facility is expected to open for experimenters outside the BonFIRE consortium. Announcements will be made on BonFIRE’s website: bonfire-project.eu [1] Open Cloud Computing Interface (OCCI): www.occi-wg.org [2] Restfully, developed by Cyril ROHR at INRIA: https://github.com/ crohr/restfully [3] FEDERICA: www.fp7-federica.eu
11
Illustration of density fluctuations in Ohmic Textor simulations (narrow simulation region is zoomed in radial direction for visualization purposes). Taken from DECI-6 project fullfgk: Numerical simulation of tokamak fusion.
DEISA: homogenous access to heterogeneous supercomputing.
Successful redocking of heparin with antithrombin.Taken from DECI-6 project BLOODINH: High throughput in-silico screening in HPC architectures for new inhibitors for treatment of blood diseases.
DEISA ultimately consisted of the 11 project partners and 4 associated partners shown above. It successfully integrated a number of existing national high-end systems using Grid technologies to form the DEISA environment. DEISA then enabled scientific and technological discovery which would not have been possible otherwise.
Gavin J. Pringle
The European collaborative project DEISA (Distributed European Infrastructure for Supercomputing Applications) came to an end in April this year after successfully deploying and operating a virtual supercomputer spanning the European continent over a 7-year period. To be deemed successful such an infrastructure has to operate as a transparent and non-disruptive layer on top of existing national services; to hide the complexity of Grid technologies from the scientific end users; and to guarantee persistence and portability of scientific applications, since these are essential to the activities of research organizations. Not only this, but the platform has to be managed across a number of heterogeneous and autonomous HPC systems, run by operators speaking different languages and observing different polices at their sites. These requirements were met by integrating a number of existing national high-end systems using Grid computing. DEISA did not attempt to solve all the problems in Grid computing, but rather applied the most efficient pragmatic solutions in order to enable the DEISA environment. Linking the core infrastructure to other infrastructures or virtual organizations was a fundamental priority for DEISA. Once achieved, this integration of national resources contributed significantly to the enhancement of HPC capability and capacity in Europe and thus to European scientific achievements. The HPC centres involved were also tightly coupled using a dedicated 10 Gb/s network bandwidth provided by the European research network GEANT and the national research networks to reduce intra machine communication overheads. By using the DEISA middleware the DEISA services hid the heterogeneous nature of the 12
infrastructure, allowing developers and users to manage their applications, compute jobs, I/O, etc on the distributed infrastructure in almost the same way as they would have done at a single national HPC site. There were two main routes for accessing the DEISA infrastructure: the DEISA Extreme Computing Initiative (DECI) and the Virtual Community access. DECI-targeted applications were complex, demanding and innovative simulations with a multi-national dimension. Under DECI, users received not only cycles on one or more HPC systems, but also access to Europewide, coordinated expert teams for operation, technology developments, and application enabling and support. Virtual Community access was similar to DECI, but each project ran for more than a year. DEISA supported Virtual Communities in four important grand challenge areas: life sciences; fusion/energy research; space science/cosmology and climate research. DEISA thus operated as a virtual European supercomputing centre, and was exploited by many scientists across a range of subjects such as cosmology, earth science, engineering, material science, particle physics, plasma physics and life sciences. These groups used DEISA to run simulations which would simply have not been possible otherwise. In total, DEISA was used by more than 200 research institutes and universities from over 25 European countries with collaborators from four other continents. By the end of DEISA, a large number of HPC systems had been integrated into its infrastructure. These included: CRAY X2/ XT4/XT5/SE6, IBM Power5/Power6, IBM Blue Gene/P, IBM PowerPC, SGI ALTIX 4700, and NEC SX8/9 vector systems.
The final DEISA environment gave an aggregated peak performance of over 2.82PF/s, with a fixed fraction of resources dedicated to DEISA usage. This contributed a significant enhancement to the capabilities of European scientific research that needed high performance computing (HPC) capabilities. DEISA also successfully demonstrated how a multiterascale European computing platform could be deployed and operated through the use of Grid technologies that interfaced the DEISA research infrastructure with other European and US IT infrastructures, over existing national supercomputers as a persistent, production quality virtual European supercomputing centre that provided global, reliable, non-disruptive, and transparent services to its community. Although DEISA has come to an end, its work is carried forward by the PRACE (Partnership for Advanced Computing in Europe) project, which will provide European researchers with access to so-called Tier-0 and Tier-1 systems, funded by national agencies (with EC support for leadership-class computers). Indeed, DEISA worked closely with PRACE and it has been agreed, in principle, that the vast majority of the DEISA infrastructure will be employed by PRACE. The two DEISA user access programs – DECI (under PRACE’s Project Access) and Virtual Communities (under PRACE’s Programme Access) – will also continue, where DECI now stands for Distributed European Computing Initiative. DEISA was judged a great success by the European Commission and scientists alike, and we expect PRACE to be even more successful. DEISA was financed by the EU’s FP6 and FP7 programmes.
Simulation of fusion plasma undertaken using DEISA resources through EUFORIA’s modelling platform. Modelling nuclear fusion: EUFORIA project Adrian Jackson
EUFORIA is working on the design and construction of ITER, the world’s biggest fusion reactor. Modelling the physical processes involved in nuclear fusion requires exceptional computational resources, and EUFORIA drew on Grid and HPC technology to build a computer modelling platform that meets the needs of European fusion scientists. DEISA enabled the EUFORIA platform to access vital HPC resources through a comprehensive software infrastructure. In particular, the remote access and job submission infrastructure that DEISA provided, combined with access to multiple computational resources from the same software, enabled EUFORIA and European fusion scientists to easily integrate HPC resources with fusion simulations. www.euforia-project.eu 13
OGSA-DAI Visual Workbench
Never mind the flops, where’s my baud? Data-intensive engineering at EPCC
Adrian Mouat
Rob Baxter
The landscape of computationally-assisted research has changed over the last few years. Where computational simulation once defined a complementary third paradigm to scientific theory and experiment, the use of computers to distil new results from ever-increasing volumes of digital data has given rise to a fourth paradigm: data-intensive research. Characterised by the work of Microsoft’s Jim Gray and collaborators, and captured in the seminal book The Fourth Paradigm: Data-Intensive Scientific Discovery [1], data-intensive research looks different in approach and applicability to any of the other scientific models. In their 2010 report Riding the Wave, the European High-Level Experts Group on data (HLEG) used the phrase “data scientist” to describe a new kind of researcher, one who uses the ever-growing wealth of digital data as their research infrastructure, using computers and software tools to examine huge corpora of data, searching for links and patterns hidden among the bytes. Because of the sheer volume of digital data, “data science” and data-intensive methods revolve not so much around maximising calculations-per-second but around moving bytes through systems intelligently and efficiently – or perhaps, trying not to move them unless absolutely necessary. Streaming data, incremental processing, long-running database queries, “move the computation to the data” – building efficient systems in this environment is the preserve of the data-intensive engineer. EPCC has been working in data-intensive engineering since the earliest days of the UK’s e-Science Core Programme, and most recently through the ADMIRE project. ADMIRE saw EPCC and the University’s School of Informatics lead a European team of researchers and engineers in exploring new methods to tame these large-scale data-intensive problems. It finished this summer with an “excellent” final review by its funders, the European Commission.
ADMIRE investigated how data sources could be brought together with the rich diversity of tools and applications used by data scientists in a way that could be controlled and engineered sensibly. The result was an hourglass, a system which introduces a common, canonical description of data-intensive workflow processes which both tools and systems can understand. ADMIRE introduces DISPEL, the data-intensive systems process engineering language, to provide the necessary canonical description for data-intensive processes. DISPEL documents describing complex data combination and analysis tasks can be sent to a “Gateway” service, compiled and executed on a distributed enactment platform built on top of the OGSA-DAI data workflow engine. Users can interact with the Gateway with a wide variety of tools, from simple web-based portals to a fully-fledged development environment derived from the popular Eclipse workbench. ADMIRE’s ideas and approaches were successfully demonstrated by addressing data-intensive problems in fields including customer relationship management, astronomy, flood prediction and gene annotation. A full account of the development of ADMIRE’s architecture and language, and its use to address real-life data-intensive challenges, will be published early next year in a major book. ADMIRE’s technology will be further developed by upcoming projects. For example, the JISC-funded An Enhanced Visual Workbench for OGSA-DAI is improving the usability of ADMIRE’s Eclipse-based workbench (see opposite). On a larger scale, the pan-European seismology project VERCE is one of a number of so-called “research data infrastructure” projects funded recently by the European Commission. With so much digital data now being produced by researchers across the world, what tools and systems are needed to allow it to be exploited to create new science or deliver genuine sociological impact? VERCE addresses these questions in the context of
Continues opposite.
14
The OGSA-DAI Visual Workbench project, which started in August, aims to create a powerful, highly usable and intuitive development environment for OGSA-DAI. The basis for this environment will be the Workbench developed for the ADMIRE project as seen in the figure. If you are unfamiliar with OGSA-DAI, it is a mature platform for integrating distributed data by building workflows from composable functions (a full explanation can be found at the project website [1]). A common request from OGSA-DAI users has been for some form of visual editor to aid the creation of simple workflows, similar to that provided by Taverna [2]. In contrast to OGSA-DAI, the ADMIRE project [3] had broader aims. It was a research project that developed a distributed, streaming platform for knowledge discovery. The project created a new language, DISPEL, designed to succinctly describe dataintensive workflows. To support users of the platform, the project also created the ADMIRE Workbench, essentially a large set of co-operating plug-ins for the Eclipse IDE [4]. The ADMIRE Workbench contains both a graphical workflow editor and a text editor for the DISPEL language, as well as various plug-ins for monitoring running processes and visualising their results. In the ADMIRE project, DISPEL workflows were sent to “Gateways” which converted DISPEL into OGSA-DAI workflows for execution on OGSA-DAI installations (in the ADMIRE parlance, OGSA-DAI is an “Enactment Platform”). Because of this, it is reasonably straightforward to integrate the Gateway with the Workbench to seamlessly provide OGSA-DAI developers with the ADMIRE tools: an OGSA-DAI Workbench.
seismology; the EUDAT project takes a broader look and asks: what should a common data infrastructure for European research look like? EUDAT seeks to build on results from a number of key “vertical”, discipline-specific data projects and create prototypes of the first genuine “horizontal” research data infrastructure. Its mission is to create a concrete example of the vision laid out in the HLEG report, a common infrastructure platform that will allow researchers to work with data from a wide variety of existing and future archives in a standard way, using common services and their own preferred tools. While a lot of the
ADMIRE workbench in action. The project work will concentrate on addressing various usability and robustness issues as well as hiding ADMIRE architectural details. Part of the focus is on improving the maintainability of the ADMIRE code, which will lower the barrier for future projects and developers to continue work on the Workbench. The project will deliver a virtual machine image (or “Workbench-in-a-box”) that will allow users to get started straight away without the need to waste time installing various dependencies. The virtual image will include everything needed to run a simple but complete example including documentation, sample data and code. Development started with an evaluation of the usability of the software and the quality of the underlying code. These evaluations were used to identify the areas in greatest need of improvement. You can follow the OGSA-DAI Workbench project on the OGSA-DAI blog [5], which touches on general software development and usability engineering issues as well as providing progress updates. The OGSA-DAI Visual Workbench is a four-and-a-half-month project funded by JISC. [1] www.ogsadai.org.uk [2] www.taverna.org.uk [3] www.admire-project.eu [4] www.eclipse.org [5] sourceforge.net/apps/wordpress/ogsa-dai (Look at posts with the “workbench” tag).
technology to create such a platform already exists, integrating the policies, systems and protocols to provide a seamless user experience is EUDAT’s main challenge. EPCC has a leading role in EUDAT, our particular remit being to create a strategy for the long-term sustainability of research data infrastructure. The ideas, approaches and experiences of ADMIRE will stand us in good stead for EUDAT as we continue to engineer solutions to the problems being washed up by the rising tides of digital data. [1] research.microsoft.com/en-us/collaboration/fourthparadigm EUDAT website: www.eudat.eu 15
The Software Sustainability Institute: a year of progress Neil Chue Hong, SSI Director The Software Sustainability Institute (SSI), which is headquartered at EPCC, was set up to identify and develop the software considered to be important to researchers, working with them to make it easier to maintain. It recently celebrated its first birthday. In the last 12 months we have worked with software from domains including fusion energy, climate policy, chemistry, psychology, social simulation and distributed computing. There’s a strong emphasis at the SSI on software engineering and collaborative development, two principles which have guided EPCC over the last 20 years. An example of this approach is illustrated in Mike Jackson’s case study of a project we’ve recently completed with the Brain Research Imaging Centre [2] – see page opposite for more information.
Looking to the future, the SSI is always seeking to expand its domain-specific knowledge through networks such as the Product/Area Liaisons (PALs) and SeIUCCR Community Champions [3]. Simon Hettrick, the SSI Dissemination and Outreach Officer, explains below why we’ve just recruited a set of Agents [4], and how they’ll gathering information that might just mean the difference between life and death… for software. [1] www.software.ac.uk [2] www.bric.ed.ac.uk [3]www.software.ac.uk/what-do-we-do/seiuccr-project [4] www.software.ac.uk/agents
Not so secret Agents Simon Hettrick, Software Sustainability Institute
If getting good information is difficult, getting good information from across disciplines is near impossible. Huge organisations – the Googles and Microsofts of this world – get round the problem by employing lots of experts from different backgrounds. We don’t have those kinds of resources at the Institute, so we came up with a different solution. The Institute helps researchers (from all disciplines) build better software. To do this, we need to know who’s using software and who needs our help. The easiest way to learn about these developments is to attend conferences – after all, that’s the way the researchers do it. But what are the right conferences to attend and, when you get there, who are the right people to talk to? Unless you’re already an expert in the field, these questions are difficult to answer. To solve our information problem, the Institute needed experts. We don’t have the resources to employ them, so we sent them packing instead. We came up with the idea of a swap: the Institute needs information and researchers want to travel. We agreed to pay researchers up to £3000 a year in travelling expenses in exchange for the benefit of their expertise. It’s a fair deal: we get information that has been sifted and verified by experts in the field, and the researchers can go to conferences that they would not otherwise have been able to attend. In July this year, the hunt for our Agents began. We based the application process around the two skills that an Agent must possess: they have to learn new information quickly, and they have to effectively communicate that information to nonspecialists (ie us). In the first round of judging, we asked the applicants to describe their research, and the way they use 16
MCMxxxVI’s automatic lesion extractor, showing an MRI scan of a brain (left), tissue of interest (right) and a brain mask that extracts the tissue from the brain scan (centre).
Software Sustainability Institute
software, in a way that a non-expert audience could understand. The second round was more difficult: we asked the applicants to write a blog about someone else’s research. There were other factors to consider. We needed a good coverage of the research community, so we wanted researchers from a wide range of organisations and as wide a range of backgrounds as possible. We also gave preference to early career researchers, because we have found that they tend to be active in emerging technologies and are more likely to write the reports we need. The application process was a great success. We were inundated with applications in the first round – over 130 of them for the 10 places we had on offer. This was whittled down to 30 applications in the first round of judging and then, with the help of the NGS’s Claire Devereux as our independent adjudicator, we chose our 10 Agents. Now that we’ve recruited our Agents, getting good information is a lot easier. We have access to experts in fields as diverse as climate change and sports science. This is an incredible resource. We have massively increased the breadth of research that we can deal with and have exposure to, and we have a team to call on when we face problems that are outside of our field of expertise. Details about our Agents are available on the Software Sustainability Institute’s website. If you would like to meet the Agents, the best opportunity will take place in the UK next March at the Collaborations Workshop 2012. www.software.ac.uk/agents
Understanding the brain in sickness and in health
Mike Jackson
To diagnose and treat severe brain conditions, we need an understanding of the structure and operation of the brain. However, there is still much that we don’t understand about the brain’s function, not only when we are ill, but also when we are healthy. The Brain Research Imaging Centre (BRIC), based at the Division of Clinical Neurosciences at Edinburgh’s Western General Hospital, develops and uses a wide variety of medical imaging techniques to better understand neurological disorders, such as multiple sclerosis, stroke and prion diseases including BSE/CJD. BRIC is a significant contributor to research into brain disease and is a member of SINAPSE, the Scottish Imaging Network – A Platform for Scientific Excellence, an initiative to promote collaboration in brain imaging across Scotland. BRIC has developed many applications for medical image processing and analysis. In our role as members of the Software Sustainability Institute (SSI), EPCC worked with BRIC to improve the sustainability of two of its applications. DICOM Confidential takes sensitive medical data – DICOM images – and allows users to remove any information that could be used to identify individual patients. We assessed how easy it is to use, extend and modify DICOM Confidential, and reviewed its code. We also looked at effective ways to support DICOM Confidential and ensure its long-term availability. The result of this is that DICOM Confidential is now hosted by the open source project repository SourceForge, and there is on-going development to address the matters our review raised.
MCMxxxVI (Multispectral Colour Modulation and Variance Identification) is a suite of innovative image manipulation and analysis tools. They colourise and fuse together medical images of a brain generated by MRI (magnetic resonance imaging) scanners. Tissues of interest can then be identified and extracted from these images. MCMxxxVI tools are currently used by students and are also of interest to clinicians. One tool, the Automatic Lesion Extractor, runs under MATLAB. We ported this tool to be a standalone Windows executable, written in C++ and using the open source VTK image visualisation toolkit and wxWidgets GUI toolkit. Distributing ALE as a stand-alone application makes it far easier for users, for example, students, researchers and clinicians, to download and use. We also created a new SourceForge project, BRIC1936, to host ALE. Our collaboration benefited BRIC, in contributing to their goal of promoting the wider adoption of these applications within the medical imaging community. The SSI also benefited as the collaboration allowed project processes to be exercised and refined, and our knowledge of using open source packages and setting up projects on SourceForge to be enhanced. The Software Sustainability Institute: www.software.ac.uk Brain Research Imaging Centre: www.bric.ed.ac.uk SINAPSE: www.sinapse.ac.uk DICOM Confidential project: sourceforge.net/projects/privacyguard BRIC1936 project: sourceforge.net/projects/bric1936
17
Image: Homeless World Cup Official
Scotland wins World Cup! Kevin Stratford
The above headline was recently spotted by your correspondent on the newspaper billboards in the streets of Edinburgh. At first sight the event would seem rather unlikely, particularly for the long-suffering followers of the Scottish national team. But how likely or unlikely is it really? Such questions are very important to those considering events which may be extremely rare, but potentially have a very wide-ranging impact. (For interested readers, at the time of writing a bookmaker was offering 300-1 for Scotland to win the next FIFA World Cup in 2014.) A more serious example [1] of a rare event is the credit crunch of 2008, a global event of a size not seen probably since the 1930s. A part of the systematic failure to predict this event is, arguably, attributable to the failure of the banks’ computer models to account properly for these events which, while having very low probability, cannot be ignored. From the scientific perspective, such a failure is understandable. Simulating rare events is extremely hard. This is for the simple, but fundamental, reason that in a ‘brute-force’ simulation, the vast majority of the computational effort is spent simulating the uninteresting waiting time between the events that do matter. So few, if any, occurrences of the rare event itself are observed in any one simulation. So most (if not all) of your computational time and effort is wasted. While high-performance computing has led to impressive advances in our ability to simulate larger systems for longer, it is clear that these advances need to be combined with more intelligent methods than brute-force simulation to make it possible to simulate rare events effectively. In the area of microscopic simulation, a number of such methods have been developed. Broadly, these are based on statistical sampling methods which try to help the simulation toward the final goal. However, these methods tend to suffer from limitations, 18
including the inability to deal with systems subject to some external driving. A new project based in Edinburgh, funded by the UK Engineering and Physical Sciences Research Council, will attempt to improve the situation. To do this, we will combine a powerful new technique for capturing rare events with the use of large supercomputers. The goal of the work is to provide software which will make it easier for scientists to apply this technique (known as “Forward Flux Sampling” [2]). The method works by generating many different simulations, and discarding those which do not make progress against the final goal. In this way, sampling is concentrated on the simulations which do make progress, allowing the whole system to be ratcheted forward. As forward flux sampling does not care about the details of the underlying simulation algorithm (provided it has a random component), the software should be relevant to many different types of application. As an example, the method can be used in conjunction with molecular dynamics to study conformational (shape) changes in proteins; rare mistakes in the folding of proteins are thought to play an important role in neurodegenerative diseases. It is hoped our new software will become available to help study such problems in the near future. However, it will almost certainly not predict the next financial crisis; as for Scotland winning the next World Cup, that remains in the lap of the footballing gods. Oh yes, and that headline referred to the Homeless World Cup. [1] Diehard Scotland supporters may beg to differ. [2] RJ Allen et al., Journal of Physics: Condensed Matter, 21, 463102 (2009).
EDIM1: a new machine for Data Intensive Research Adam Carter
The use of computers in science is now ubiquitous, and the amount of simulation data is increasing at a staggering pace. The advent of more automated sensing and screening techniques means ever more digital output is being produced by devices such as telescopes, microscopes and atmospheric sensors, as well as the growing data collected from experiments. There is now so much data that it’s not possible to process and analyse it without computers. The sheer quantity however, is allowing new methodologies to emerge; by joining data together, by automating analysis and by identifying patterns in the data we are discovering new ways to understand the world around us.
described three quantities (all ratios) which should be close to one in order to make the best use of a computer:
We believe that Data Intensive Research will become increasingly important in coming years and so, building on the success of EPCC’s various data-related projects and its HPC expertise, we have installed a new machine known as EDIM1, designed specifically for Data Intensive Research.
Building such a machine from commodity parts had its challenges. One of these was trying to interpret how the manufacturers’ part specifications actually translate into machine performance. It’s quite common for different manufacturers to quote their own set of incomparable specifications designed to show their hardware in its best light. IOops can be reads and writes, sequential and random, and of different sizes. The I/O bandwidth can also be quoted in various ways, some of which take into account a disk’s onboard cache, for example. It has also been a challenge to ensure that all the parts are compatible: not just that they work together, but that they actually perform well together. To give one example, we had to track down (with the help of our suppliers) why our disks were negotiating transfer rates down to SATA1 when they were actually capable of communicating at twice that speed. Fixing this involved not only flashing chips in the hardware but actually replacing nearly all of the 480 SATA cables in the machine.
EDIM1 was built from commodity components to a custom specification. The 120-node system has over 6TB of fast storage directly connected inside every node, and by using low-cost, low-power Atom processors it has been possible to create a machine which is more balanced in favour of I/O-intensive applications. As well as commissioning this system, we have been working with research groups in fields such as astronomy and geosciences to evaluate how it can be applied to existing and future research problems. While the processing power of computers continues to double every couple of years, not all components have kept up. In particular, hard disk technologies continue to improve, both for spinning disks and the more recent solid state disks, but the speed at which data can be read from and written to disk has not increased nearly so quickly as the rate at which it can be processed by CPUs (and now GPUs). Computers are becoming increasingly unbalanced, and in the case of data-intensive problems, the processor can spend a larger proportion of its time sitting idle, waiting for input and output (I/O) to disk. EDIM1 attempts to address this issue by concentrating on IOops rather than FLOPs. In particular, we aimed to make the machine “Amdahl-balanced”. Gene Amdahl, famous for noting that the performance of a parallel program is ultimately limited by the serial part of the code, also made several other observations about the balance between the rate at which instructions are performed by a computer and its memory and I/O characteristics. Referred to by some as “Amdahl’s other law”, he
• Amdahl Number: bits of sequential I/O per second per instruction per second,
EDIM1: built and hosted by EPCC.
• Amdahl Memory Ratio: megabytes of main memory per million instructions per second • Amdahl IOops ratio: IOops per 50,000 instructions.
EDIM1 is now up and running and one of our MSc students has been putting it through its paces. Measurements [1] have shown that its empirical Amdahl number is around 1.2, close to the theoretical value of 1.33 deduced from the manufacturers’ specifications and in excess of unity (the point where a machine becomes balanced in favour of I/O). These numbers are far higher than would be expected from a PC with a high-end processor and a single hard disk. These results suggest that for problems that are too big to fit in a computer’s main memory, EDIM1 could out-perform many machines and clusters that are considerably more costly and more power-hungry. EDIM1 was jointly funded by EPCC and the School of Informatics at the University of Edinburgh. [1] Kulkarni, O., MSc Dissertation Report. Available soon from: www.epcc.ed.ac.uk/msc 19
The MSc in HPC: ten years on!
Empowering research in the humanities
Judy Hardy, EPCC
Mark Hedges, Centre for e-Research, King’s College London
The MSc in High Performance Computing is celebrating its tenth anniversary this year! In 2001 we welcomed our first intake of six students. Since then the programme has grown year on year, and we will see more than thirty students graduate in November. The MSc also continues to have a broad-based and international appeal. Over the years we have had students from across Europe, Asia, Central and North America, the Middle East and Africa. This year’s class is possibly one of the most diverse, with 15 nationalities represented. We recently decided to take a step back and review the curriculum and structure of the whole MSc. This review has now been completed. As a result, space has been created for new content, reflecting new trends and developments in HPC, while existing material has been consolidated and rationalised. Although many of the changes are evolutionary, most course names have been changed and in some cases there has been a change of focus and the inclusion of a significant quantity of new material. At the same time, we have moved away from our previous model of delivering each course in intensive half-day blocks taught over a semester. Although each course still runs for a Taught Courses
• Advanced Parallel Programming • HPC Architectures • HPC Ecosystem • Message Passing Programming • Parallel Design Patterns • Performance Programming
• Parallel Numerical Algorithms • Project Preparation • Parallel Programming Languages • Programming Skills • Software Development • Threaded Programming
Dynamic Loop Nesting in Shared Memory Programming: Orestis Agathokleous
Regions of nested loops are a common feature of HPC codes. In shared memory programming models such as OpenMP, these structures are the most common source of parallelism. This work investigated the possibility of applying code transformations before compilation to enable a dynamic choice to be made at runtime on how parallelisation should be applied on the nested loops. A source-to-source compiler was developed to perform the transformations automatically using a directive based approach similar to OpenMP.
Recent years have seen several collaborations in humanities e-research between EPCC and the Centre for e-Research (CeRch) at King’s, as personified by CeRch researchers Mark Hedges and Tobias Blanke. Collaboration began back in 2009, with the Linking and Querying Ancient Texts (LaQuAT) project, funded as part of the JISC ENGAGE project at EPCC. LaQuAT looked at the thorny problem of integrating the very diverse and fuzzy datasets created by researchers into classical antiquity, such as documents written on stone or papyrus. LaQuAT raised more questions than it answered, but these questions led in turn to a more recent collaboration – the Supporting Productive Queries for Research (SPQR) project, funded by the JISC Managing Research Data programme. semester it is now structured into separate weekly lecture and practical sessions. This gives students more time to consolidate and reflect on new material after the lectures and to get to grips with programming and other practical work before tutorials. The new structure has also enabled us to integrate our timetable more closely with related MSc courses, for example in Computer Science, Informatics and Mathematics. As a result, students can now choose from a wide range of optional courses to complement the core subjects taught in the MSc. One of the highlights of the MSc for most students – and staff – is the dissertation project. These cover a wide range of topics, reflecting the extensive range of staff skills and interests at EPCC. Below and opposite we highlight a selection of this year’s dissertations. Dissertation reports of these should be available from November on the MSc website. Past dissertation reports – and videos of recent MSc graduates talking about living and studying in Edinburgh – can be downloaded from the MSc website: www.epcc.ed.ac.uk/msc Krylov Subspace Solvers in OP2 on GPUs: Lesleis Nagy
An extensible framework for the execution of Krylov Subspace solvers is presented which implements the Conjugate Gradient method using OP2 and CUSPARSE & CUBLAS libraries. Timings were taken for OP2, CUSPARSE & CUBLAS and serial versions of the implemented Conjugate Gradient method. OP2 was found to be sensitive to the structure of the input matrix for the linear system being solved. A metric called the ‘structure effect factor’ was derived to quantify the performance of OP2 with respect to the structure of the input matrix. Continues opposite.
20
Inscription on the Archive Wall in the Theatre in Aphrodisias. Transcriptions of such inscriptions into electronic forms are helping classicists undertake research into the ancient world. Photograph (c) 1977, Mossman Roueché. to be done. The SPQR triples were also incorporated by the PELAGOIS project into a wider set of geographically annotated linked data, which can be explored at: pelagios.dme.ait.ac.at/graph-explore.
The EPCC team of David Scott and Bartosz Dobrzelecki, led by Mike Jackson, developed scripts to convert the source XML and databases into over fifty-six thousand RDF triples, incorporating links to geographically linked datasets provided by Pleiades and GeoNames. An off-the-shelf linked data browser was used as the basis for evaluation by researchers, who concluded that this approach offers an intuitive and usable means of exploring and understanding the data, although of course more work remains
Our other success was the JISC-funded TEXTvre (pronounced “texture”) project, which was an international collaboration between King’s, EPCC and the University of Sheffield in the UK, and the State and University Library at Göttingen in Germany. TEXTvre builds upon the successful e-humanities project TextGrid, which developed a collaborative environment for creating critical editions of literary texts (think big XML files and annotations) that used the German grid infrastructure. EPCC’s contribution was to replace this underlying infrastructure with a repository based on Fedora, which means that TEXTvre can be set up more easily as an institutional environment, initially for the digital humanists at King’s, but also potentially elsewhere. This work involved many complexities, notably including the mapping of the semantic structure within TextGrid onto Fedora’s data model, and was carried out by EPCC’s Malcolm Illingworth, under the supervision of Kostas Kavoussanakis. TEXTvre recently finished, with the successful installation of the software at CeRch. But we have been awarded continuation money from the Software Sustainability Institute (SSI) at Edinburgh, which will make the TEXTvre software more robust, modular and easier to set up, and which will thus increase uptake.
Benchmarking an Amdahl-balanced Cluster for Data Intensive Computing: Omkar Kulkarni
MPI Visualisation: Thomas Richard
This project evaluated the performance of an Amdahl-balanced cluster (see EDIM1 on page 19) designed for data-intensive research. It used numerous benchmarks to characterise individual components of a node and the cluster as a whole. Special attention was paid to the performance of the secondary storage. A custom benchmark was written to measure the balance of the cluster – the extent of difference in performance between the processor and I/O. The tests also included distributed computing benchmarks based on the popular MapReduce programming model using the Hadoop framework. The results of these tests were very encouraging and showed the practicality of this configuration.
This project aimed to generate a real-time visualisation tool for MPI codes. It used the MPI Profiling interface to wrap MPI functions, collect the data and display it in a real-time display as the program is executed. The tool aimed to be easy to use and accessible to MPI learners. It could handle most of the MPI point-to-point communications functions. The tool also proposes a memory registration system; the code registers arrays and the tool then displays which parts are accessed during communications. Some derived MPI data types are also supported, such as Vectors and Contiguous, in order to be useful for the first steps of programming with MPI.
While LaQuAT used OMII-UK’s OGSA-DAI software to attempt a relational database model of integrating various database and XML resources, SPQR instead used a linked data approach, based on semantic web technologies. The relational database model had proved too rigid to deal with the “fuzzy”, incomplete and inconsistent data that we possess about antiquity, and it was hoped that this model, which used RDF as the fundamental form for representing information, and ontologies to provide the semantic “glue” between datasets and their divergent representations, would be more successful.
21
Product Purchasing
Digital exclusion: building consumer relations with the invisible Ashley D. Lloyd University of Edinburgh Business School
Initial water saturation of an oil-bearing reservoir where blue indicates 100% water. Image courtesy of Total Exploration Production.
Branches
APOS-EU, in collaboration with a peer project funded by the Ministry of Education and Science of the Russian Federation, will target popular codes from the strategically important application areas of seismic modelling, oil- and gas-reservoir simulation, computational fluid dynamics, fusion energy, and nanotechnology. Through a programme of research activities and workshops, the joint EU-Russian team will enable representative 22
APOS-EU is a two-year project, which runs from February 2011 until January 2013.The consortium has five partners: EPCC (project coordinator, representing the University of Edinburgh), CAPS entreprise, Uniwersytet Warszawski,TOTAL S.A., and Höchstleistungsrechenzentrum Stuttgart. For more information see the project website: apos-project.eu Or follow the group on Twitter: @aposeu.
Direct Channels (Internet, ATM, Mobile)
3
43
100
1
77
5 16 100
Mortgages
80
11 4 4100
38
3
72
58
100
13 5 10 100
SOURCE: Efma online survey across 150+ European Banks Face-to-Face: A €15-20BN Multichannel Opportunity, McKinsey & Co., April 2011.
Figure 1: Channel use in 150+ European Banks. Adapted from McKinsey & Co report “Face to face: A €15-20Bn Multichannel Opportunity”, April 2011.
EPCC has been working with the Edinburgh University Business School for almost 20 years, applying high-performance computing to business problems and building a better understanding of consumer behaviour.
In scientific computing, faster and more powerful computers do not immediately lead to better results. Incompatibility between the requirements of existing software and the capabilities of new supercomputers is a growing problem that will be addressed by a pioneering new Russian-European collaboration. The European branch of this collaboration is called APOS-EU. The future of high-performance computing means two things: massive parallelism and heterogeneity. Processor manufacturers are squeezing ever more computational cores onto their CPUs, and HPC vendors are looking to augment these many-core chips with GPU-like accelerators to deliver the next push forward for computing capacity. Although such developments give scientists the potential to run ever bigger, faster, or more detailed simulations, there are significant challenges to be overcome before today’s most important application codes can exploit these hardware advances efficiently.
53
4 9 100 2
Current funding from the EPSRC has enabled a link with an industry partner in China and creation of a secure collaborative environment allowing exploration of consumer behaviour in response to the introduction of new channels – the methods by which consumers find, select and purchase products and services. These channels range from shops, established technologies such as phone, mail and fax, and newer technologies such as the Internet.
The suite of codes that we select will be exemplars for the techniques that we advocate – for example, illustrating how to properly exploit the memory hierarchy of a NUMA platform or how to structure computations for maximum acceleration by GPGPUs. Furthermore, case studies will be developed using representative distributed-memory applications to showcase the innovation in our tool chain. Looking beyond the project, our work will make an important contribution to the wider field of scientific computing, streamlining and simplifying the upgrade path for many other application codes.
84
Savings accounts
George Beckett
codes from the five application areas to efficiently exploit the capabilities of tomorrow’s supercomputers. These activities will drive the creation of new tools for code sustainability, enabling next-generation science on next-generation supercomputers.
Call Centre
Current sales breakdown by channel Expected dominant Channel 2010 % of sales 2015, % of respondents 1 3
Current accounts
With the advent of Grid computing and globally distributed infrastructures for secure communication and computation, the possibility of collaborations around shared data views of global markets became a technical possibility, leading to the creation of the ESRC-funded INWA Grid connecting EPCC to Perth in Western Australia and the Chinese Academy of Sciences in Beijing [1] [2].
APOS-EU: Application Performance Optimisation and Scalability
Agents/Brokers
From a corporate perspective there is a real value in migrating consumers from expensive channels such as shops to more efficient channels such as the Internet, but each stage in that migration has the potential to exclude significant parts of society – for example in 2010, 9.2 million adults in the UK had never used the Internet. In the same year Martha Lane Fox, the UK Digital Champion, published a report entitled Directgov 2010 and Beyond: Revolution Not Evolution. The tension that a step change to a ‘Digital by Default’ assumption implies for parts of society that have not yet chosen, or had the opportunity, to engage are severe – companies cannot easily design for consumers they do not know about, and if the populations are seen as small then there is no economic incentive to do so. These parts of society stand the risk of becoming permanently invisible to the designers of future channels, and hence permanently excluded.
In a global village, the problem can get worse as companies have the possibility of using Internet channels to find more customers that are of the same type as their existing customers – growing larger, more homogeneous markets rather than addressing the differences in society from which exclusion stems. Martha Lane Fox’s focus on ‘revolution’ aptly describes just how rapid the changes in industry structure are expected to be. This is shown in Figure 1 in a study of over 150 banks across Europe. Despite the presence of direct channel technologies such as telephone banking, electronic banking and ATMs for many years, their share of total transactions has grown very slowly and is still relatively small. This contrasts with the planned restructuring of the banking industry where, for example, the 16% of savings accounts sold across direct channels in 2010 is expected to rise to 58% in only 5 years, while all the channels that involve contact with a person, including branches and call centres, are expected to decline. If this ‘supply-side’ revolution is not able to include new types of customer, then those that it excludes may well become permanently invisible. The BRIDGE (Building Relations with the Invisible in the Digital Global Economy) project, a collaboration between the University of Edinburgh together with Yvonne Barnard from the University of Leeds and Mike Bradley from Middlesex University, is exploring this. Leeds and Middlesex are investigating the impact of interface design on adoption while Edinburgh is performing adoption studies at a scale that allows even small segments within the UK to be related to potential demand in much larger export markets. Establishing this potential would make it economically attractive for UK business to invest in inclusion, and hence contribute to a smoother migration to being digitally engaged for those caught up in the ‘revolution’. More details can be found at: www.bridge-relate.org [1] www.epcc.ed.ac.uk/news/announcements/inwa-grid [2] www.tein3.net/upload/pdf/INWA_final_web.pdf 23
Somewhere over the rainbow...
Liz Sim, EPCC User Support
“... skies are blue and the dreams that you dare to dream really do come true.” The HECToR project started back in 2007, and in late 2011 it will enter its third and final phase, which is due to run until 2013. Back at the start of the HECToR project we could only have dreamed about what the third phase of HECToR would comprise, but now those dreams are about to become reality. An upgrade to the HECToR system was announced in May. In late 2011, the existing 20-cabinet Cray XE6 will be expanded and upgraded to a 30-cabinet system, utilizing the next generation AMD Interlagos processor and Cray’s Gemini interconnect. The machine will have a projected theoretical peak of 820TF and as such represents a greater than two-fold increase in capability over the current Phase 2b system. To mark the unveiling of the third phase, the UK Research Councils launched a competition to produce a design for the front of HECToR. The competition was open to children aged 11–16 years old. The response to the competition was very positive, and over 70 entries were received from school children across the UK. The winning designs were chosen by a panel of judges from the Research Councils and the University of Edinburgh. The prizes were awarded to the designs that best demonstrated: • an understanding of the breadth of science undertaken on HECToR • that science can be inspiring and benefit society as a whole • originality and creativity. Congratulations to the overall winner: Lily Johnson, of Heathersett Old School in Norwich, whose winning design appears at the top of this page.
Lily’s design has been converted by a professional designer to fit the cabinet layout. As the winner, Lily will receive a contribution towards travel costs to attend a launch ceremony in Edinburgh in 2012 when the finalised artwork will be unveiled. She will also receive a wall plaque for her school to mark her contribution to the understanding of science in the UK. Congratulations also to Harriet Wheatley and Alice Salt (who took second place with a joint entry), and Katie Knox (third place). Last but not least, a huge thanks to all of those who entered the competition. We were delighted by the number and quality of entries we received. PRACE and HECToR
As one of the “Tier 1” sites within PRACE, HECToR will be playing an important role. Back in June our call for applications for DECI (Distributed European Computing Initiative) resources attracted over 50 proposals within Europe with a good response from within the UK. Each of the 17 participating Tier 1 sites is providing at least 5% of its annual resources to DECI. Successful projects will start running in November, utilising a whole range of architectures distributed across Europe including IBM BlueGene/P and Power 6, a number of clusters (some with GPGPUs) and of course Cray XT/XE machines such as HECToR. The next call for applications opens on 2nd November and information on applying can be found on the PRACE website: www.prace-ri.eu The HECToR system is funded by the UK Research Councils. For more information on the HECToR service, please see: www.hector.ac.uk
EPCC is a European centre of expertise in developing high performance, novel computing solutions; managing advanced systems and providing HPC training. Our clients and partners include local and global industry, government institutions and academia. EPCC’s combination of advanced computing resources and expertise is unique and unmatched by any European university. www.epcc.ed.ac.uk/info@epcc.ed.ac.uk