Chapter 3 : Architecture In Parallel Processing Flashcards
1
Q
What is the technology before Parallel Processing?
A
- Serial Computation
2
Q
What is the definition of serial computation?
A
- To be run on a single computer having a single Central Processing Unit ( CPU )
- A problem is broken into a discrete series of instructions
- Instructions are executed one after another
- Only one instruction may execute at any moment in time
3
Q
What is Parallel Processing?
A
- Is the sumultaneous use of multiple compute resources to solve a computational problem
4
Q
What are required in Parallel Processing?
A
- To be run using multiple CPUs
- A problem that is broken into discrete parts that can be solved concurrently
- Each part is further broken down to a series of instructions
5
Q
How can the task be divided into?
A
- Problem
- Discreate Parts
- Instructions
6
Q
What event is in 1940s - 1950s Conceptual Beginnings ( 2 )
A
- John von Neumann’s Contributions ( Architecture )
- IBM 7030 Stretch
7
Q
What event occurs in 1960s : The Advent of Supercomputing?
A
- CDC 6600 by Seymour Cray
8
Q
What event occurs in 1970s : The Rise of Multiprocessing
A
- Cray-1 and vector processing
- Introduction of SIMD
- ILLIAC IV and array processing
9
Q
What event occur in 1980s: Commercialization and Expansion?
A
- Connection Machine by Thinking Machines Corporation
- Introduction of MIMD Architectures
- Development of MPI
10
Q
What event occurs in 1990s: Growth of Multi-core and Many-core Processors?
A
- Introduction of multi-core processors by Intel and AMD
- Use of GPUs in parallel processing
11
Q
What event occurs in 2000s- Present : Ubiquity and Advanced Architectures
A
- Parallel Processing in consumer-grade CPUs and GPUs
- Cloud computing and distributed systems
- Advanced in AI and machine learning.
- Exploration of quantum and neuromorphic computing
12
Q
When is Conceptual Beginnings happen?
A
- 1940s - 1950s
13
Q
When is The Advent of Supercomputing happen?
A
- 1960s
14
Q
When is The Rise of Multiprocessing occurs?
A
- 1970s
15
Q
When is Commercialization and Expansion occurs?
A
- 1980s
16
Q
When is Growth of Multi-core and Many-core processors occurs?
A
- 1990s
17
Q
When is Ubiquity and Advanced Architectures occurs?
A
- 2000s - Present
18
Q
What is the types of Parallelism? ( 2 )
A
- Data Parallelism
- Task Parallelism
19
Q
What does Data Parallelism involves?
A
- Involved distributing subsets of the same data across multiple processors and performing the same operation on each subset simultaneously
20
Q
What is the advantages of Data Parallelism? ( 2 )
A
- Simplifies the parallelization process as the same operation is applied to all data
- Suitable for problems with a high degree of regularity
21
Q
What are the applications for Parallel Processing?
A
- Image Processing
- Scientific Modelling
- Linear Algebra
22
Q
Why image processing requires parallel processing?
A
- There is multiple human poses are being detected and tracked simultaneously
23
Q
What does task parallelism involves?
A
- Distributing different tasks or threads across multiple processors where each task can execute independently
24
Q
What is the applications for Task Parallelism? ( 2 )
A
- Web Servers
- Realtime Data Processing
25
What does granularity means?
1. The size of the tasks ( Units of work )
26
List out 2 types of Granularity
1. Fine-Grained Parallelism
2. Coarse-Grained Parallelism
27
What does Fine-Grained Parallelism involves?
1. Dividing tasks into small, easily manageable units that can be executed in parallel
28
What does Frame Decomposition refer to in Fine-Grained Parallelism in Video Decoding? ( 2 )
1. A video frame is divided into smaller blocks (macroblocks or tiles)
29
How does Parallel Task Assignment enhance Fine-Grained Parallelism in Video Decoding?
1. The decoding of individual blocks within a frame can be assigned to different cores of a processor or threads on a GPU
30
Why is fine-grained parallelism beneficial for video decoding?
1. By breaking a video frame into smaller blocks ( macroblocks or titles) and decode independently of other
31
What is the significance of "concurrent execution" in fine-grained parallelism?
1. Concurrent execution means that while some cores are decoding one part of the frame, others work on different parts
2. Which can ensure that the entire frame is decoded as quickly as possible
32
What is the advantages of Fine-Grained Parallelism? ( 2 )
1. Achieve high performance for problems with significant opportunities for parallelism
2. Suitable for applications requiring detailed and frequent coordination
33
What is the challenges of Fine-Grained Parallelism?
1. High overhead from task management and synchronization
2. Increases complexity in programming and debugging
34
What does Coarse-Grained Parallelism involves?
1. Involves dividing tasks into larger units of work that are processed in parallel
35
What is the characteristics of Coarse-Grained Parallelism?
1. Lower communication overhead compared to fine-grained parallelism
2. Task are larger and less frequent communication is needed
* Dividing simulation into large, independent segmenst processed by different processors
36
What is the advantages of Coarse-Grained Parallelism?
1. Reduced synchronization and communication overhead
2. Easier to implement and manage compared to fine-grained parallelism
37
What is the challenges of Coarse-Grained Parallelism?
1. Less flexibility and granularity in task distribution
2. May not fully utilize all processing resources if tasks are unevenly distributed
38
What is the widely used classifications to classify parallel computers?
1. Flynn's Taxonomy ( 1966 )
39
How does Flynn's Taxonomy distinguishes multi-processor computer architectures?
1. According to how they can be classified along the 2 independent dimensions of **Instruction and Data**
2. Each 2 dimensions can have only 2 possible states **Single or Multiple**
40
List out 4 possible classifications for Flynn's Taxonomy
1. SISD
- Single Instruction
- Single Data
2. SIMD
- Single Insctruction
- Multiple Data
3. MISD
- Multiple Instruction
- Single Data
4. MIMD
- Multiple Instruction
- Multiple Data
41
What is a SISD also known as?
1. A serial ( non-parallel ) computer
42
What doe SISD do?
1. Single Instruction :
- Only 1 instruction stream is being acted on by the CPU during any more clock cycle
2. Single Data
- Only 1 data stream is being used as input during any one clock cycle
43
What does SIMD do?
1. Single Instruction
- All processing units execute the same instruction at any given clock cycle
2. Multiple Data
- Each processing unit can operate on a different data element
* It is a type of parallel computer
44
What does MISD do?
1. Multiple Instruction
- Each processing unit operates on the data independently via independent instruction sreams
2. Single Data
- Single data stream is fed into multiple processing units
45
What does MIMD do? ( 2 )
1. Multiple Instruction
- Every processor may be executing a different instruction stream
2. Multiple Data
- Every processor may be working with a different data stream
46
What is the most common type of parallel computer?
1. MIMD ( Multiple Instruction, Multiple Data )
47
What category ( Flynn's Taxonomy ) that most modern computers fall into?
1. MIMD ( Multiple Instruction, Multiple Data )
48
What is a multi-core processors?
1. Containes 2 or more independent cores within a single computing component
2. Each core can execute its own thread simulteneously, sharing the same memory and I/O
* Common configurations include dual-core, quad-core, hexa-core and octa-core
49
List out examples for SIMD
1. Modern Graphics Processing Unit ( GPUs )
- In video games, image processing, and scientific simulations, GPUs execute the same operation simultaneously on multiple data points (such as pixels or vertices). This enables highly efficient parallel processing for tasks like rendering and deep learning computations.
50
List out examples for SISD
1. Simple Microcontrollers and Legacy Systems
- Many embedded systems, such as those in basic appliances or older computing devices, use SISD architectures where a single processor executes one instruction on one piece of data at a time. This model is sufficient for applications with limited processing requirements.
51
List out examples for MMD
1. Multi-core CPUs and Distributed Computing Systems
- In desktop computers, servers, and supercomputers, each core or node in a multi-core processor or cluster can execute its own set of instructions on separate data streams. This design is common in operating systems managing multiple applications or in large-scale scientific computations where different tasks run concurrently
52
List out examples for MISD
1. Niche fault-tolerant systems
- Same data is processed by different algorithms to ensure reliability
