P2L5 Thread Performance Considerations Flashcards
Which threading model is better? Boss/Worker or Pipeline?
It depends …
It depends on which metric is most important.
Total time or average time per order …
Review: Give 4 ways threads are useful
- parallelization (speed up)
- specialization (hot cache)
- efficiency (lower memory usage, cheaper syncronization - read/write shared variables vs. IPC)
- In single CPU, threads hide latency of I/O
What is useful for:
- Matrix multiply application?
- Web service application?
- Hardware?
- execution time
- requests/sec and response time
- CPU utilization (time CPU is working)
Give 3 metrics important for toy shop and OS. Give examples of the metrics for toy shop vs. OS
- Throughput
- toys/hour
- process completion rate
- Response time
- avg. time to respond to order
- avg. time to respond to input (e.g. mouseclick)
- Utilization
- busy workbenches
- % CPU
Which of the following are performance metrics?
- performance/$
- performance/W (per watt)
- percentage of SLA violations
- client perceived performance
- aggregate performance
- platform efficiency
- throughput
- wait time
all of the above!
Define a ‘test bed’
Ideally, we will obtain metrics running real software on real machines with real workloads.
Often, this is not feasible for many different reasons. In these cases, we may have to settle on “toy” experiments that are representative of realistic situations that we may encounter.
We refer to these experimental settings as a testbed. The testbed tells us where/how the experiments were carried out and what were the relevant metrics being gathered.
What is a metric?
a metric is some measurable quantity we can use to reason about the behavior of a system.
- Is ‘it depends’ a correct answer to “Are threads useful?”
- Is ‘it depends’ an accepted answer to “Are threads useful?”
- Yes
- No
For example, some graph traversal algorithms work best on sparse graphs, while others work best on dense graphs. Some filesystems are optimized for read access, while others might be optimized for a write-heavy system.
The answer is: It depends! While, this answer is almost always correct, it is rarely accepted. What is more important perhaps is to modify the question, extending it to include the context you wish to examine and the metrics you wish to obtain.
Refer to the diagram
- Which steps are computationally expensive?
- Which steps involve interaction with the network?
- Which steps involve interaction with the disk?
- parser step, header creation
- accepting a connection, sending data
- read/write file
State 1 advantage and 4 disadvantages of performance speedup by making the web server multi-process
Advantage: simple programming - spawn multiple processes - because we have a working debugged process already. what could be better?
Disadvantages:
- higher memory footprint, which can hurt performance.
- high cost of a contest switch whenever we want to run a different process.
- hard/costly to maintain shared state across processes due to IPC constraints.
- difficult to have multiple processes listening on a specific port.
State 2 advantages and 2 disadvantages of improving web server performance by multi-threading
Advantages:
- Cheaper context switch because shared address space.
- Lighter memory requirements because of shared information across all threads in the process.
Disadvantages:
- Software complexity: Multithreaded requires explicit application level synchronization code
- Depends on underlying operating system level support for threads, although this is less of an issue now that it was in the past. (true for Solaris paper)
Describe the event-driven model for an application
An event driven application is implemented in a single address space, with a single thread of control. The main part of the process is an event dispatcher which in a loop looks for incoming events and then based on those events invokes one or more of the registered handlers.
How does the event-driven model support concurrency?
In the event driven model, the processing of multiple requests are interleaved within a single execution context.
Why does the event-driven model with one thread work for concurrency?
because there is no idle time
context switching just wastes cycles that could have been used for request processing.
In the event driven model, a request will be processed exactly until a wait is necessary, at which point the execution context will switch to servicing another request.
What about if we have multiple CPUs?
Note the gotcha
If we have multiple CPUs, the event driven model still makes sense, especially when we have to service more concurrent requests than we have CPUs. Each CPU could host a single event-driven process, and all multiple requests to be processed concurrently within that process.
This can be done with less overhead than if each CPU had to context switch among multiple processes or multiple threads.
Gotcha: It is important to have mechanisms that will steer the right set of events to the right CPU.
Describe helpers
If the kernel is not multithreaded - it wasn’t back in the day - the helpers need to be processes. The model was called the Asymmetric Multi-Process Event-Driven Model or AMPED. The multithreaded equivalent acronym is AMTED.
Asynchronous I/O calls are not available on all platforms or for all types of devices.
Handler passes the call to the operation to a helper, and returns to the event dispatcher.
The helper will block, but the main event loop will not!
The helper will be the one that handles the blocking I/O operation and interact with the dispatcher as necessary.
In doing this, the synchronous I/O call is handled by the helper.
At about 100MB Flash becomes better than SPED. Why?
- Because Flash can handle I/O without blocking
- Because at 100MB, the workload becomes I/O bound. So the system that performs better on I/O wins.
