MSc in HPC: Cluster Building Challenge

The MSc in High Performance Computing (HPC) teaches students the tools and techniques they require to program modern parallel computers. As part of the first University of Edinburgh Innovative Learning Week, we offered them a "Cluster Building Challenge".

The MSc focuses on using modern HPC facilities for computational science applications in both research and industry. System administration is not taught, as it is assumed that students will use computers that are maintained and supported centrally. However, as HPC is at the leading edge of computing technology, a certain level of practical experience of "under-the-hood" system administration can be quite useful in understanding how machines actually work and may be helpful in diagnosing the causes of any unexpected behaviour.

To support this, the MSc in HPC decided to develop a ‘scrapheap challenge’ exercise for students to put together small-scale parallel machines completely from scratch, using Open Source software and a range of otherwise unwanted computing and networking hardware. The Innovative Learning Week comes at an ideal time in the MSc calendar as students have already learned the basics of parallel programming in the first semester.

Sixteen students signed up for the challenge. They were split into four teams and asked to choose a team name and design a logo for their customised team T-shirts! Each team was challenged to build a demonstration scale cluster from one of several different collections of spare hardware and undertake a short investigation tailored to their particular hardware. Three different challenges were devised, depending on the hardware available:

  • "The Expendables" and "The Jedi Knights" undertook the Heterogeneous Challenge. They were given a mixed bag of desktops and laptops, and asked to put them together into a single computer cluster and then run an image-processing application as fast as possible.
  • "Big Data" was given a rack of low-power servers and asked to run a datamining application as efficiently as possible.
  • "-O3" took a stack of old computer servers and tried to get the highest performance possible using the standard HPC Linpack performance benchmark.

An open “show and tell” session was held on the last day of the challenge. Each team demonstrated its cluster and talked about the successes and failures of their challenge to many interested staff and undergraduate students.

"I was very impressed that all the teams produced working parallel clusters from a heap of old hardware, very basic instructions and a pile of blank DVDs!” Dr David Henty, MSc in HPC Programme Director

Overall the Cluster Challenge was a great success, with our students enjoying building and working with their systems -despite the inevitable setbacks and frustrations! We definitely plan to repeat the challenge in future years.

Current student and challenge team member Ioan Hadade said:

“The Cluster Building Challenge has been an excellent opportunity to put into practice the knowledge gathered through our MSc in High Performance Computing course. It was very interesting to witness first hand the importance of finding the right synergy between the hardware and software components in order to squeeze every possible ounce of performance from any computing system.

"Our task was to build, configure and fine tune a computer cluster using spare computing machines and benchmark their performance on a suite of applications including High Performance Linpack, a benchmarking tool that is used to classify the world's most powerful supercomputers. After an entire week of hard work, our efforts have been rewarded by obtaining 40 MFlops per Watt performance which positioned our system within the Top 500 Green list!

"I am very grateful to EPCC and Dr David Henty for all the effort that was put into making this exercise possible. I would truly encourage that this exercise be continued in the following years and perhaps even culminating with a presence at the ISC's Cluster Building Challenge.”

The MSc in HPC would like to thank Kym Eden-Jones and Gareth Francis for their help in putting the challenge together and the Principal's Teaching Awards Scheme and the School of Physics ILW fund for financial support.