High performance Computing

Question 1

Week 1

Question 2

What new science is a major user of high performance computing?

Answer 2

Life sciences for applications such as genome processing

Question 3

How can we determine the performance of a high performance computer using floating point mathematics

Answer 3

• Linpack is a performance benchmark which measures floating point operations per
  second (flops) using a dense linear algebra workload
• A widely used performance benchmark for HPC systems is a parallel version of Linpack
  called HPL (High-Performance Linpack)

Question 4

Why is parallelization important in HPC

Answer 4

High requirement floating point calculations can be run in parallel to make use of multiple cores

Question 5

What is dennard scaling?

Answer 5

Dennard scaling is a recipe for keeping power per unit area (power density) constant as transistors were scaled to smaller sizes

As transistors became smaller they also became faster (delay reduction) and more
energy efficient (reduce threshold voltage)

With very small features limits associated with the physics of the device (e.g. leakage
current) are reached

Dennard scaling has broken down and processor clock speeds are no longer
increasing

Question 6

What is the current most common supercomputer architecture

Answer 6

Current systems are all
based on integrating many
multi-core processors

• The dominant architecture is
  now the “commodity cluster”
• Commodity clusters
  integrate off-the-shelf (OTS)
  components to make an
  HPC system (cluster)

Question 7

Give the proper definition for a Commodity cluster

Answer 7

A commodity cluster is a cluster in which both the network and the compute nodes are commercial products available for procurement and independent application by organisations (end users or separate vendors) other than the original equipment manufacturer.

Question 8

Give four components of a cluster algorithm

Answer 8

Compute nodes: provide the processor cores and memory required to run the workload
* Interconnect: cluster internal network enabling compute nodes to communicate and access storage
* Mass storage: disk arrays and storage nodes which provide user filesystems
* Login nodes: provide access (e.g. ssh) for users and administrators via external network

Question 9

Why do High performance Computers use compiled languages

Answer 9

Maximizes performance.
Compilers parse code and generate executables with optimizations.
Optimizations at compile-time are less costly than at runtime.

Question 10

What are common langauges for High performance computers?

Answer 10

C, C++ and Fortran

Question 11

Why must parellisation be done manually

Answer 11

Parallelization is too complex for compilers to handle automatically.
Programmers add parallel features.

Question 12

Week 2

Question 13

What can we use OpenMP for?

Answer 13

OpenMP provides extensions to C, C++ and Fortran
* These extensions enable the programmer to specify where parallelism should be added and how to add it
* The extensions provided by OpenMP are:
- Compiler directives
- Environment variables
- Runtime library routines

Question 14

What does it mean to say that OpenMP uses a fork join execution model?

Answer 14

Execution starts with a single thread (master thread)
- Worker threads start (fork) on entry to a parallel region
- Worker threads exit (join) at the end of the parallel region

Question 15

What can we use the OMP_NUM_THREADS header for?

Answer 15

We can use the OMP_NUM_THREADS environment variable to control the number of threads forked in a parallel region e.g.

• export OMP_NUM_THREADS=4
• OMP_NUM_THREADS is one of the environment variables defined in the standard
• If you don’t specify the number of threads the default value is implementation defined
  (i.e. the standard doesn’t say what it has to be)

Question 16

What does openMP provide that we can call directly from our functions?

Answer 16

OpenMP provides compiler directives, environment variables and a runtime library with functions we can call directly from our programs

Question 17

What header file must be included to use the open.mp library

Answer 17

<omp.h>
</omp.h>

Question 18

Why is conditional compilation useful in programs that use OpenMP?

Answer 18

It ensures that the program can compile and run as a serial version when OpenMP is not enabled, avoiding errors caused by missing OpenMP compiler flags.

Question 19

What is the role of the C pre-processor in the compilation process?

Answer 19

The C pre-processor processes source code before it is passed to the compiler, handling directives such as #include and #ifdef.

Question 20

What does the _OPENMP macro indicate when it is defined?

Answer 20

It indicates that OpenMP is enabled and supported by the compiler.

Question 21

hat is the syntax of the #ifdef directive used for conditional compilation?

Answer 21

ifdef MACRO

// Code included if MACRO is defined #else
// Code included if MACRO is not defined #endif

Question 22

What is the main benefit of using conditional compilation with OpenMP programs?

Answer 22

It allows the same source code to support both serial and parallel execution by enabling or disabling OpenMP-related code.

Question 23

What is one good way to distribute the workload when working in parallel

Answer 23

One way to do this in OpenMP is to parallelise loops

• Different threads carry out different iterations of the loop
• We can parallelise a for loop from inside a parallel region:
  #pragma omp for
• We can start a parallel region and parallelise a for loop:
  #pragma omp parallel for

Question 24

What changes about the order of loop iterations when they are executed in parallel?

Answer 24

When the loop is parallelised the iterations will not take place in the order specified by the loop iterator
* We can’t rely on the loop iterations taking place in any particular order

Question 25

How do we solve the issue of loops not iterating in order when executed parallelly

Answer 25

The results are stored in arrays which do hold the results in the order we require
* If we want to print out the results of our calculation in order we will need a second sequential (not parallelised) loop

Question 26

What is the correct way to define variable scope for OpenMP loops

Answer 26

In the examples in the previous unit different threads accessed the same copies of arrays x and y (shared)
* The loop index i was different for each thread (private)
The correct functioning of an OpenMP loop requires correct variable scoping
* In this case the default scoping rules did the right thing

Question 27

Why might we want to define the variable scope before running parallelised code?

Answer 27

Explicitly declaring variable scope makes the code much easier to understand and more likely to be correct

• We can specify the default scope by adding a clause to the directive which starts the parallel region:
• default (shared)
• default (none)
• The default clause can be followed by a list of private and/or shared variables