Lecture 2 - Fundamentals of algorithms and problem solving

Question 1

Process of designing algorithms:

Answer 1

• Understanding the problem
• Ascertaining the capabilities of the device
• Choosing between exact and approximate problem
• Deciding an appropriate language and data structure
• Implement any solution
• Improve on your initial solution (generate more solutions)

Question 2

Factors that need to be taken into consideration when thinking of a solution:

Answer 2

Very important factors:
• How often are you going to need the algorithm?
• What is the typical problem size you are trying to solve?

Less important factors:
• What language are you going use to implement your algorithm?
• What is the cost-benefit of an efficient algorithm?

Question 3

How do you analyze an algorithm?

Answer 3

By in investigating an algorithm’s efficiency in respect of two resources:
• Running time
• Memory space

Question 4

Why is it not always possible to study algorithm’s efficiency in terms of its input size?

Answer 4

There are algorithms that require more than one parameter (e.g. Graph algorithms)

The input size may not be well-defined as one would wish (eg. Matrix multiplication)

Question 5

What are the problems of using a standard unit for time measurement?

Answer 5

We’d face serious drawbacks:
• Dependence on the speed of a particular computer
• Dependence on the quality of the program implementing the algorithm
• Dependence on the quality of the compiler used to generate the executable
• Difficulty of clocking the time precisely

Question 6

How is time efficiency measured in an algorithm?

Answer 6

The standard approach is to identify the basic operation(s) and count how many times it is executed.

Question 7

What are the basic operations for matrix multiplication ?

Answer 7

multiplications and additions

Question 8

What is the basic operation for sorting ?

Answer 8

comparisons

Question 9

What is the established framework for counting operations?

Answer 9

Count the number of times the algorithm’s basic operation is executed for inputs of size n (Where n is clearly defined)

Question 10

Machine-independent algorithm design depend on what hypothetical computer?

Answer 10

The Random Access Machine or RAM

Question 11

What are the characteristics of the RAM?

Answer 11

• Each simple operation (+, *, -, /, =, memory call) takes 1 time step;
• The count of operations of the algorithm is equal to n 1
• Loops and subroutines are the compositions of many single-step operations.
• Takes no notice of whether an item is in cache or on the disk.

Question 12

Algorithm Design Manual Steve Skiena : RAM advantages

Answer 12

• Under the RAM model, we measure the run time of an algorithm by counting up the number of steps it takes on a given problem instance.
• By assuming that our RAM executes a given number of steps per second, the operation count converts easily to the actual run time.

Question 13

RAM Disadvantages:

Answer 13

• A common complaint is that it is too simple, that these assumptions make the conclusions and analysis too coarse to believe in practice.
• Multiplying two numbers takes more time than adding two numbers on most processors, which violates the first assumption of the model.
• Memory access times differ greatly depending on whether data sits in cache or on the disk, thus violating the third assumption.

Question 14

What type of functions are 2n and n! ?

Answer 14

Exponential. Solutions with these running times are only practical for very small input sizes. Problems having solutions in this class are called “intractable”.

Question 15

Common growth functions:

Answer 15

• ~1: Constant Time.
• ~log n: Logarithmic.
• ~n: Linear.
• ~nk log n: Polylogarithmic.
• n2: Quadratic.
• n3: Cubic.
• nk where k is a constant: Polynomial.
• 2n: Exponential.

Question 16

Explain a constant time growth function:

Answer 16

Normally the amount of time that a instruction takes under the RAM model. It does not depend on input.

Question 17

Explain a Logarithm growth function:

Answer 17

It occurs in algorithms that transform a bigger problem into a smaller version that whose input size is a ration of the original problem. Common in searching and in some tree algorithms.

Question 18

Explain a Linear growth function:

Answer 18

Algorithms that are forced to pass through all elements of the input (of size n) a number (constant) of times yield linear running time.

Question 19

Explain a Exponential growth function:

Answer 19

Unfortunately quite a few known solutions to practical problems are in this category. This is as bad as testing all possible answers to a problem. When algorithms fall in this category, algorithm designers go in search of
approximation algorithms.

Question 20

Explain a Quadratic growth function:

Answer 20

A subset of the polynomial solutions. Quadratic solutions are still acceptable and are relatively efficient to small to medium scale problem sizes. Typical of algorithms that have to analyse all pairs of elements of the input.

Question 21

Explain a Cubic growth function:

Answer 21

Not very efficient but still polynomial. A classical example of algorithms in this class is the matrix Multiplication.

Question 22

What algorithm is this?

Algorithm(A[0..n],K)
A[n] = K
i = 0
while (A[i] != K) do
           i = i + 1
if (i < n)
          return i
else
          return -1

Answer 22

Sequential search algorithm.

Question 23

The worst-case efficiency of an algorithm is its?

Answer 23

efficiency for the worst-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the longest among all possible inputs of that size
[Levitin 2003]

worst case operation is the configuration that will make me do the most number of operations. eg. list size 1 the worst case and best case are the same.

Question 24

What is Best Case efficiency?

Answer 24

The best-case efficiency of an algorithm is its efficiency for the best-case input of size n, which is an input (or
inputs) of size n for which the algorithm runs the fastest among all the inputs of that size [Levitin 2003]

Best configuration of a problem of size n that will give me the best performance.

Question 25

Why is Best Case useful ?

Answer 25

One can take advantage of algorithms that run really fast in the best case if the sample inputs to which the algorithm will be applied is approximately the best input:
• For instance, the insertion sort of a list of size n will perform in Cbest(n), when the list is already sorted.
• If the list is close to being completely sorted the best case performance does not degenerate much
• This means that insertion sort may be a good option for lists that are known to be nearly sorted.

Question 26

What is The Average case ?

Answer 26

Running time of a typical input or a random input.