Lecture 2 - Fundamentals of algorithms and problem solving Flashcards
Process of designing algorithms:
- Understanding the problem
- Ascertaining the capabilities of the device
- Choosing between exact and approximate problem
- Deciding an appropriate language and data structure
- Implement any solution
- Improve on your initial solution (generate more solutions)
Factors that need to be taken into consideration when thinking of a solution:
Very important factors:
• How often are you going to need the algorithm?
• What is the typical problem size you are trying to solve?
Less important factors:
• What language are you going use to implement your algorithm?
• What is the cost-benefit of an efficient algorithm?
How do you analyze an algorithm?
By in investigating an algorithm’s efficiency in respect of two resources:
• Running time
• Memory space
Why is it not always possible to study algorithm’s efficiency in terms of its input size?
There are algorithms that require more than one parameter (e.g. Graph algorithms)
The input size may not be well-defined as one would wish (eg. Matrix multiplication)
What are the problems of using a standard unit for time measurement?
We’d face serious drawbacks:
• Dependence on the speed of a particular computer
• Dependence on the quality of the program implementing the algorithm
• Dependence on the quality of the compiler used to generate the executable
• Difficulty of clocking the time precisely
How is time efficiency measured in an algorithm?
The standard approach is to identify the basic operation(s) and count how many times it is executed.
What are the basic operations for matrix multiplication ?
multiplications and additions
What is the basic operation for sorting ?
comparisons
What is the established framework for counting operations?
Count the number of times the algorithm’s basic operation is executed for inputs of size n (Where n is clearly defined)
Machine-independent algorithm design depend on what hypothetical computer?
The Random Access Machine or RAM
What are the characteristics of the RAM?
- Each simple operation (+, *, -, /, =, memory call) takes 1 time step;
- The count of operations of the algorithm is equal to n 1
- Loops and subroutines are the compositions of many single-step operations.
- Takes no notice of whether an item is in cache or on the disk.
Algorithm Design Manual Steve Skiena : RAM advantages
- Under the RAM model, we measure the run time of an algorithm by counting up the number of steps it takes on a given problem instance.
- By assuming that our RAM executes a given number of steps per second, the operation count converts easily to the actual run time.
RAM Disadvantages:
- A common complaint is that it is too simple, that these assumptions make the conclusions and analysis too coarse to believe in practice.
- Multiplying two numbers takes more time than adding two numbers on most processors, which violates the first assumption of the model.
- Memory access times differ greatly depending on whether data sits in cache or on the disk, thus violating the third assumption.
What type of functions are 2n and n! ?
Exponential. Solutions with these running times are only practical for very small input sizes. Problems having solutions in this class are called “intractable”.
Common growth functions:
- ~1: Constant Time.
- ~log n: Logarithmic.
- ~n: Linear.
- ~nk log n: Polylogarithmic.
- n2: Quadratic.
- n3: Cubic.
- nk where k is a constant: Polynomial.
- 2n: Exponential.
Explain a constant time growth function:
Normally the amount of time that a instruction takes under the RAM model. It does not depend on input.
Explain a Logarithm growth function:
It occurs in algorithms that transform a bigger problem into a smaller version that whose input size is a ration of the original problem. Common in searching and in some tree algorithms.
Explain a Linear growth function:
Algorithms that are forced to pass through all elements of the input (of size n) a number (constant) of times yield linear running time.
Explain a Exponential growth function:
Unfortunately quite a few known solutions to practical problems are in this category. This is as bad as testing all possible answers to a problem. When algorithms fall in this category, algorithm designers go in search of
approximation algorithms.
Explain a Quadratic growth function:
A subset of the polynomial solutions. Quadratic solutions are still acceptable and are relatively efficient to small to medium scale problem sizes. Typical of algorithms that have to analyse all pairs of elements of the input.
Explain a Cubic growth function:
Not very efficient but still polynomial. A classical example of algorithms in this class is the matrix Multiplication.
What algorithm is this?
Algorithm(A[0..n],K)
A[n] = K
i = 0
while (A[i] != K) do
i = i + 1
if (i < n)
return i
else
return -1
Sequential search algorithm.
The worst-case efficiency of an algorithm is its?
efficiency for the worst-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the longest among all possible inputs of that size
[Levitin 2003]
worst case operation is the configuration that will make me do the most number of operations. eg. list size 1 the worst case and best case are the same.
What is Best Case efficiency?
The best-case efficiency of an algorithm is its efficiency for the best-case input of size n, which is an input (or
inputs) of size n for which the algorithm runs the fastest among all the inputs of that size [Levitin 2003]
Best configuration of a problem of size n that will give me the best performance.
Why is Best Case useful ?
One can take advantage of algorithms that run really fast in the best case if the sample inputs to which the algorithm will be applied is approximately the best input:
• For instance, the insertion sort of a list of size n will perform in Cbest(n), when the list is already sorted.
• If the list is close to being completely sorted the best case performance does not degenerate much
• This means that insertion sort may be a good option for lists that are known to be nearly sorted.
What is The Average case ?
Running time of a typical input or a random input.
