Lecture 10: Probability Theory

Question 1

What is Probability?

Answer 1

These are statements about outcomes which are possible but may not be guaranteed.

Question 2

How can probabilities be expressed?

Answer 2

Through:

• fractions
• ratios
• percentages

Question 3

Why is probability important?

Answer 3

1. Allows us to reason about a computer sustems reliability
   
   1. What is the likelohood of a critical component failing?
   2. where should we build in redundancy?
2. Helps describe a network server availability

Question 4

List other examples were probability theory is used

Answer 4

• data mining
• pattern recognition
• encryption
• tracking
• robotics
• IS
• data compression
• Bayesian networks
• automatic classifciation
• information fusion

Question 5

what are other factors of probability?

Answer 5

1. Can be considered as “relative frequency”
2. usually expressed as real numbers ranging between zero and one

Question 6

what does the term experiement mean?

Answer 6

It is a process which generates data

1. a single performance of the experiment is call atrial
2. each trial has an outcome
3. e.g. tossing a coin

Question 7

What does the term sample space mean?

Answer 7

The set of all possible outcomes

• e.g single coin tose (h, t)
• e.g2. coin tossed twice (hh, ht etc…)

Question 8

What does the term event mean?

Answer 8

A subset of the sample space

• the outcome with particular characteristics
• e.g. getting the same result in two coin tosses (HH,TT)

Question 9

Describe discrete probability.

Answer 9

considers only experiments for which the sample space contains a finite number of elements or a countably infinite number of elements.

Question 10

What is a probability density function?

Answer 10

assigns probabilities to all of the elements in the sample space

Question 11

Explain Uniform probabilities. (uniform probability density functions)

Answer 11

If a sample space contains N elements, all of which are likely to occur, then probability assigned to each are 1/N.

Question 12

What does it mean by “Counting sample points”?

Answer 12

some problems need to be solved by specifically counting the number of elements in a set or subset, without actually listing each element

Question 13

list the 7 basic counting strategies.

Answer 13

1. addition principle
2. subtraction principle
3. pigeon-hole principle
4. complementation
5. combinations
6. permutations of distinct objects
7. permutations of repeated objects

Question 14

Addition.

Answer 14

suppose one element is to be selected from a collection of disjoint or mutually exclusive sets A1, A2…An

then the total amount of possible choices is given by A1 + A2 + An…

Question 15

What does it mean to be mutually exclusive set?

Answer 15

to have no elements in common

Question 16

Multiplication

Answer 16

a process can be performed in a sequence of m distinct steps.

total outcomes: n = n1 x n2…..x Nm

Question 17

permutations of distinct objects

Answer 17

When the order DOES MATTER in a counting problem

a permutation is a selection (without replacement), of items from a set, where the order of selection is important.

Question 18

What is a circular permutation of distinct objects

Answer 18

• where items are aranged in a circle.
• There are no ends to the arrangements
• 2 arrangements can be similar if one can be transformed into another by a clockwise rotation of elements
• e.g. abc = cba (same circular permutation)

Question 19

Combination

Answer 19

order DOES NOT MATTER

combination is an unordered selection of items in a set

Question 20

Complementation

Answer 20

it may be easier calculation the number of sample points which do not have a particular characteristic

Complementary sets - a set which have a required characteristic and a set which dosent.

Question 21

Pigeon-hole principle

Answer 21

• If m objects are to be placed in n locations and m > n > 0, then at least one location must receive two objects.
• IN counting context: if the total number of elements in a set, is larger then the total number of distinct properties, then at least 2 of the elements have the same property.
• e.g 15000 customers, 1000 PINS, 2 have the same

Question 22

Joint probabilities

Answer 22

two or more events may include the same elements of the sample space

e.g. the characteristics defining the two events may be found in the sample space

Question 23

Conditional Probabilities

Answer 23

Involves calculating the probability of an event when it is known that some other event has occurred.

e.g. A1 occurrence may depend on the occurrence of event A2

Question 24

Independent Events

Answer 24

two events are independent if the probability of one, event is not affected by the occurrence of the other event

Question 25

Binomial Distribution

Answer 25

occurs when

• repeated trials are independent
• probability of success is constant then, experiment is binomal

some experiments consist of repeated trials, where each trial has only two possible outcomes:

• heads or tails
• 0 or 1
• working or defective
• success or failure

Question 26

Summarise. 5 main points to remember.

Answer 26

• Probability can be considered as the relative frequency of occurrence of an event
• The probability of an event is a value between 0 to
• The probabilities for all of the points in the sample space sum to 1
• To calculate the probability of an event you may need to:
  
  • Count sample points
  • Decide if events are conditional or independent
• The binomial distribution is useful for calculating probabilities of events associated with a particular type of experiment