CUDA C API - Advanced

Question 1

What is CUDA Stream

Answer 1

• In CUDA programming, a stream is a series of commands that execute in order.
• Operations within a given stream occur in order.
• Operations in different non-default streams are not guaranteed to operate in any specific order relative to each other.

Question 2

What is the default Stream?

Answer 2

• Kernels runs on default stream if no manually created one is provided. This stream is special.
• The default stream is blocking and will both wait for all other streams to complete before running, and, will block other streams from running until it completes

Question 3

How to add additional streams, what are they called? How to destroy them?

Answer 3

• non-default CUDA streams

```c
cudaStream_t stream;
cudaStreamCreate(&stream);

someKernel«<number_of_blocks, threads_per_block, 0, stream»>();

// cudaStreamDestroy will return immediately (is non-blocking), but will not actually destroy stream until
// all stream operations are complete.
cudaStreamDestroy(stream);
~~~

Question 4

Asynchronously copy pinned host memory to device over non-default stream.

Answer 4

```c
cudaMemcpyAsync(&device_array, // Take care to access correct location in array.
&host_array, // Take care to access correct location in array.
segmentSize, // Only copy a segment’s worth of memory.
cudaMemcpyHostToDevice,
stream); // Provide optional argument for non-default stream.
~~~