MPI collectives

Question 1

what kinds of collective operations are ther e

Answer 1

synchs
commnication
reduction

Question 2

propertries of MPI collectives

Answer 2

done by all processes
in same sequence

Question 3

barier

Answer 3

int MPI_Barrier(MPI_Comm communicator);

synch operation

Question 4

broadcast

Answer 4

int MPI_Bcast(void* buffer,
int count,
MPI_Datatype datatype,
int emitter_rank,
MPI_Comm communicator);

MPI_Bcast(&buffer, 1, MPI_INT, broadcast_root, MPI_COMM_WORLD);

not synced

Question 5

gather

Answer 5

int MPI_Gather(const void* buffer_send,
int count_send,
MPI_Datatype datatype_send,
void* buffer_recv,
int count_recv,
MPI_Datatype datatype_recv,
int root,
MPI_Comm communicator);

MPI_Gather(&my_value, 1, MPI_INT, buffer, 1, MPI_INT, root_rank, MPI_COMM_WORLD);

stores data ordered by ranks

Question 6

scatter

Answer 6

int MPI_Scatter(const void* buffer_send,
int count_send,
MPI_Datatype datatype_send,
void* buffer_recv,
int count_recv,
MPI_Datatype datatype_recv,
int root,
MPI_Comm communicator);

MPI_Scatter(buffer, 1, MPI_INT, &my_value, 1, MPI_INT, root_rank, MPI_COMM_WORLD);

Question 7

gather v

Answer 7

like gather but each proc sends different number of elements

int MPI_Gatherv(const void* buffer_send,
int count_send,
MPI_Datatype datatype_send,
void* buffer_recv,
const int* counts_recv,
const int* displacements,
MPI_Datatype datatype_recv,
int root,
MPI_Comm communicator);

MPI_Gatherv(&my_value, 1, MPI_INT, buffer, counts, displacements, MPI_INT, root_rank, MPI_COMM_WORLD);

Question 8

alltoall

Answer 8

int MPI_Alltoall(const void* buffer_send,
int count_send,
MPI_Datatype datatype_send,
void* buffer_recv,
int count_recv,
MPI_Datatype datatype_recv,
MPI_Comm communicator);

MPI_Alltoall(&my_values, 1, MPI_INT, buffer_recv, 1, MPI_INT, MPI_COMM_WORLD);

sends data from and to each process

scaling bottleneck, expensive

Question 9

other gather and scatter and alltoall types

Answer 9

allgather

allgatherv

scatterv

alltoallv

alltoallw: also different types!! buffer+counts[]+displ[]+types[]

int MPI_Alltoallw(const void* buffer_send,
const int counts_send[],
const int displacements_send[],
const MPI_Datatype datatypes_send[],
void* buffer_recv,
const int counts_recv[],
const int displacements_recv[],
const MPI_Datatype datatypes_recv[],
MPI_Comm communicator);

now displacements are by bytes and not n of types

Question 10

reduction

Answer 10

int MPI_Reduce(const void* send_buffer,
void* receive_buffer,
int count,
MPI_Datatype datatype,
MPI_Op operation,
int root,
MPI_Comm communicator);

MPI_Reduce(&my_rank, &reduction_result, 1, MPI_INT, MPI_SUM, root_rank, MPI_COMM_WORLD);

example operations
MPI_MAX, MPI_PROD, MPI_SUM

Question 11

other reduce variants

Answer 11

allreduce: reduce + bcast

reducescatter_block: reduce + scatter (equal chunks)

reducescatter: unequal

MPI_Scan: reduction with prefix??

Question 12

communicator creation

Answer 12

either duplicate
int MPI_Comm_dup(MPI_Comm old_comm,
MPI_Comm* new_comm);

MPI_Comm_free(comm)

or split
int MPI_Comm_split(MPI_Comm old_communicator,
int colour,
int key,
MPI_Comm* new_communicator);