Probability Probability is a numerical description of how likely it is that a particular outcome will occur. It is usually represented as a value on a scale from 0 (impossible) to 1 (certain), although other measures such as percentages or odds are sometimes used.
1
Probability Probability is a numerical description of how likely it is that a particular outcome will occur. It is usually represented as a value on a scale from 0 (impossible) to 1 (certain), although other measures such as percentages or odds are sometimes used. To determine the probability of a particular outcome, we might use
2
Probability Probability is a numerical description of how likely it is that a particular outcome will occur. It is usually represented as a value on a scale from 0 (impossible) to 1 (certain), although other measures such as percentages or odds are sometimes used. To determine the probability of a particular outcome, we might use Relative frequency Pick a student in the room at random; what’s the probability they’re female?
3
Probability Probability is a numerical description of how likely it is that a particular outcome will occur. It is usually represented as a value on a scale from 0 (impossible) to 1 (certain), although other measures such as percentages or odds are sometimes used. To determine the probability of a particular outcome, we might use Relative frequency Pick a student in the room at random; what’s the probability they’re female? Symmetry Toss a coin; what’s the probability of Heads?
4
Probability Probability is a numerical description of how likely it is that a particular outcome will occur. It is usually represented as a value on a scale from 0 (impossible) to 1 (certain), although other measures such as percentages or odds are sometimes used. To determine the probability of a particular outcome, we might use Relative frequency Pick a student in the room at random; what’s the probability they’re female? Symmetry Toss a coin; what’s the probability of Heads? A subjective estimate What are the chances of Andy Murray winning Wimbledon this year?
Probability Mathematically, we can formalise probabilities as follows. Outcome space Let the possible outcomes of an experiment be ω1 , ω2 , . . . , ωn
Probability Mathematically, we can formalise probabilities as follows. Outcome space Let the possible outcomes of an experiment be ω1 , ω2 , . . . , ωn These must satisfy: 1. No two outcomes can be observed simultaneously; 2. Outcomes taken together cover all possibilities.
Probability Mathematically, we can formalise probabilities as follows. Outcome space Let the possible outcomes of an experiment be ω1 , ω2 , . . . , ωn These must satisfy: 1. No two outcomes can be observed simultaneously; 2. Outcomes taken together cover all possibilities. The set of all outcomes Ω = {ω1 , . . . , ωn } is called the outcome space or sample space.
8
Examples
1. The experiment is that of tossing a coin once. â„Ś ={H, T }
9
Examples
1. The experiment is that of tossing a coin once. â„Ś ={H, T } 2. Roll a die. â„Ś ={1, 2, 3, 4, 5, 6}
10
Examples
1. The experiment is that of tossing a coin once. Ω ={H, T } 2. Roll a die. Ω ={1, 2, 3, 4, 5, 6} 3. Toss a coin twice. Ω ={HH, HT, T H, T T }
11
Examples
1. The experiment is that of tossing a coin once. Ω ={H, T } 2. Roll a die. Ω ={1, 2, 3, 4, 5, 6} 3. Toss a coin twice. Ω ={HH, HT, T H, T T } Handout: Probability and Sets
12
Probability Consider an experiment being repeated a large number of times N Suppose f (ωk ) is the number of times outcome ωk occurs, the frequency of ωk
13
Probability Consider an experiment being repeated a large number of times N Suppose f (ωk ) is the number of times outcome ωk occurs, the frequency of ωk Then the probability of ωk is the limit of f (ωk )/ N as N → ∞, P (ωk ) =
14
Probability Consider an experiment being repeated a large number of times N Suppose f (ωk ) is the number of times outcome ωk occurs, the frequency of ωk Then the probability of ωk is the limit of f (ωk )/ N as N → ∞, P (ωk ) =
f (ωk ) N →∞ N lim
15
Probability Consider an experiment being repeated a large number of times N Suppose f (ωk ) is the number of times outcome ωk occurs, the frequency of ωk Then the probability of ωk is the limit of f (ωk )/ N as N → ∞, P (ωk ) =
f (ωk ) N →∞ N lim
e.g. As the number of times a coin is tossed increases we expect f (H)/N to settle around 0.5 Handout: Probability and Sets
16
Probability
Notice that
Probability
Notice that 1. 0 ≤ f (ωk ) ≤ N
Probability
Notice that 1. 0 ≤ f (ωk ) ≤ N 2. f (ω1 ) + f (ω2 ) + · · · + f (ωn ) = N
Probability
Notice that 1. 0 ≤ f (ωk ) ≤ N 2. f (ω1 ) + f (ω2 ) + · · · + f (ωn ) = N Following from this, we have that probabilities must always satisfy
20
Probability
Notice that 1. 0 ≤ f (ωk ) ≤ N 2. f (ω1 ) + f (ω2 ) + · · · + f (ωn ) = N Following from this, we have that probabilities must always satisfy 1. 0 ≤ P (ωk ) ≤ 1
21
Probability
Notice that 1. 0 ≤ f (ωk ) ≤ N 2. f (ω1 ) + f (ω2 ) + · · · + f (ωn ) = N Following from this, we have that probabilities must always satisfy 1. 0 ≤ P (ωk ) ≤ 1 n X P (ωk ) = 1 2. k=1
22
Definitions 1. An event is any subset of the outcome space â„Ś
23
Definitions 1. An event is any subset of the outcome space ℌ 2. A simple event consists of a single outcome ωk
24
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E
25
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes.
26
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
27
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
X
P (ωk )
ωk ∈E
28
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
X
ωk ∈E
2. P (φ) =
P (ωk )
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
X
P (ωk )
ωk ∈E
2. P (φ) = 0
30
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
X
P (ωk )
ωk ∈E
2. P (φ) = 0 3. P Ē =
31
Definitions 1. An event is any subset of the outcome space Ω 2. A simple event consists of a single outcome ωk 3. The complement Ē of an event E is the set of all outcomes not in E, that is Ē = Ω − E 4. The null event φ is the set containing no outcomes. Notes: 1. P (E) =
X
P (ωk )
ωk ∈E
2. P (φ) = 0 3. P Ē = 1 − P (E)
Example
A fair coin is tossed twice. â„Ś = {HH, HT, T H, T T }
Example
A fair coin is tossed twice. â„Ś = {HH, HT, T H, T T } By symmetry, all four outcomes have the same probability, and since the probabilities add up to 1, each outcome must have probability 1/4.
Example
A fair coin is tossed twice. â„Ś = {HH, HT, T H, T T } By symmetry, all four outcomes have the same probability, and since the probabilities add up to 1, each outcome must have probability 1/4. Probability of 2 heads is P (HH) = 1/4
Example
A fair coin is tossed twice. â„Ś = {HH, HT, T H, T T } By symmetry, all four outcomes have the same probability, and since the probabilities add up to 1, each outcome must have probability 1/4. Probability of 2 heads is P (HH) = 1/4 Probability that the results of the 2 tosses are different is
Example
A fair coin is tossed twice. â„Ś = {HH, HT, T H, T T } By symmetry, all four outcomes have the same probability, and since the probabilities add up to 1, each outcome must have probability 1/4. Probability of 2 heads is P (HH) = 1/4 Probability that the results of the 2 tosses are different is P (HT ) + P (T H) = 1/4 + 1/4 = 1/2
Example
A card is drawn at random from an ordinary pack of 52 playing cards. What is the probability that the card is 1. The 4 of spades? 2. Any 7? 3. Not a picture card (J, Q, K)? 4. The 4 of spades or any diamond? 5. A 4 or a diamond?
38
Compound events
Sometimes we are interested in events formed from other events.
39
Compound events
Sometimes we are interested in events formed from other events. e.g. What is the chance of Liverpool winning both the Premiership and the FA Cup this season?
40
Compound events
Sometimes we are interested in events formed from other events. e.g. What is the chance of Liverpool winning both the Premiership and the FA Cup this season? COMPOUND EVENTS are events formed from other events.
41
Compound events
Sometimes we are interested in events formed from other events. e.g. What is the chance of Liverpool winning both the Premiership and the FA Cup this season? COMPOUND EVENTS are events formed from other events. Let E, F ⊂ Ω = {ω1 , ω2 , . . . , ωn } be two events.
Compound events
Sometimes we are interested in events formed from other events. e.g. What is the chance of Liverpool winning both the Premiership and the FA Cup this season? COMPOUND EVENTS are events formed from other events. Let E, F ⊂ Ω = {ω1 , ω2 , . . . , ωn } be two events. The event ‘E or F ’ is written E ∪ F (union)
Compound events
Sometimes we are interested in events formed from other events. e.g. What is the chance of Liverpool winning both the Premiership and the FA Cup this season? COMPOUND EVENTS are events formed from other events. Let E, F ⊂ Ω = {ω1 , ω2 , . . . , ωn } be two events. The event ‘E or F ’ is written E ∪ F (union) The event ‘E and F ’ is written E ∩ F (intersection)
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs } where r ≤ s + 1
F = {ωr , . . . , ωt }
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs } where r ≤ s + 1 Then E ∪ F =
F = {ωr , . . . , ωt }
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt }
47
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt } If r = s + 1, then E ∩ F =
48
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt } If r = s + 1, then E ∩ F = φ
49
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt } If r = s + 1, then E ∩ F = φ If r ≤ s then E ∩ F =
50
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt } If r = s + 1, then E ∩ F = φ If r ≤ s then E ∩ F = {ωr , . . . , ωs }
51
Compound events Suppose (without loss of generality) that E = {ω1 , . . . , ωs }
F = {ωr , . . . , ωt }
where r ≤ s + 1 Then E ∪ F = {ω1 , . . . , ωt } If r = s + 1, then E ∩ F = φ If r ≤ s then E ∩ F = {ωr , . . . , ωs } Addition law: P (E ∪ F ) = P (E) + P (F ) − P (E ∩ F )
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E∪F = E∩F = (E ∩ F ) ∪ G = E∪G= F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
53
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E∩F = (E ∩ F ) ∪ G = E∪G= F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
54
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = E∪G= F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
55
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = E∪G= F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
56
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E∪G= F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
57
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E ∪ G = {1, 3, 5, 6, 7, 8, 9, 10} F ∪G= (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
58
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E ∪ G = {1, 3, 5, 6, 7, 8, 9, 10} F ∪ G = {3, 6, 7, 8, 9, 10} (E ∪ G) ∩ (F ∪ G) = Ē ∩ Ḡ =
59
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E ∪ G = {1, 3, 5, 6, 7, 8, 9, 10} F ∪ G = {3, 6, 7, 8, 9, 10} (E ∪ G) ∩ (F ∪ G) = {3, 6, 7, 8, 9, 10} Ē ∩ Ḡ =
60
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E ∪ G = {1, 3, 5, 6, 7, 8, 9, 10} F ∪ G = {3, 6, 7, 8, 9, 10} (E ∪ G) ∩ (F ∪ G) = {3, 6, 7, 8, 9, 10} Ē ∩ Ḡ = {2, 4, 6, 8, 10} ∩ {1, 2, 3, 4, 5} =
61
Example Let Ω = {1, 2, . . . , 10} and E = ‘odd numbers’ = {1, 3, 5, 7, 9} F = ‘multiples of 3’ = {3, 6, 9} G = ‘greater than 5’ = {6, 7, 8, 9, 10} Then: E ∪ F = {1, 3, 5, 6, 7, 9} E ∩ F = {3, 9} (E ∩ F ) ∪ G = {3, 9} ∪ {6, 7, 8, 9, 10} = {3, 6, 7, 8, 9, 10} E ∪ G = {1, 3, 5, 6, 7, 8, 9, 10} F ∪ G = {3, 6, 7, 8, 9, 10} (E ∪ G) ∩ (F ∪ G) = {3, 6, 7, 8, 9, 10} Ē ∩ Ḡ = {2, 4, 6, 8, 10} ∩ {1, 2, 3, 4, 5} = {2, 4}
62
Independence For independent events A and B
P (A ∊ B) = P (A) Ă— P (B)
63
Independence For independent events A and B
P (A ∩ B) = P (A) × P (B)
eg Toss two fair coins, P (First H, second T) = (1/2) × (1/2) = 1/4
64
Independence For independent events A and B
P (A ∩ B) = P (A) × P (B)
eg Toss two fair coins, P (First H, second T) = (1/2) × (1/2) = 1/4 BUT if we roll a die, then P (“Roll a 2” and “Roll an even number”) 6= P (2) × P (Even) these events are not independent.
65
Independence For independent events A and B
P (A ∩ B) = P (A) × P (B)
eg Toss two fair coins, P (First H, second T) = (1/2) × (1/2) = 1/4 BUT if we roll a die, then P (“Roll a 2” and “Roll an even number”) 6= P (2) × P (Even) these events are not independent. (More on dependence later – conditional probabilities.)
66
Networks
Consider a network of components. A signal enters the network and passes through a component if and only if the component is working.
67
Networks
Consider a network of components. A signal enters the network and passes through a component if and only if the component is working. Let P (X) denote the probability that component X is working. Assuming independence between components, what is the probability that the signal passes successfully through the network?
68
Networks
Consider a network of components. A signal enters the network and passes through a component if and only if the component is working. Let P (X) denote the probability that component X is working. Assuming independence between components, what is the probability that the signal passes successfully through the network? Consider first two simple networks.
69
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) =
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) =
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) = P (A)P (B)
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) = P (A)P (B) 2. OR network T:
C @ R @ @ @
@ R @ @
@D
To get through, either C or D must be working, so using independence, P (T ) =
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) = P (A)P (B) 2. OR network T:
C @ R @ @ @
@ R @ @
@D
To get through, either C or D must be working, so using independence, P (T ) = P (C âˆŞ D)
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) = P (A)P (B) 2. OR network T:
C @ R @ @ @
@ R @ @
@D
To get through, either C or D must be working, so using independence, P (T ) = P (C âˆŞ D) = P (C) + P (D) − P (C ∊ D)
1. AND network S:
-A -B
To get through, both A and B must be working, so because of independence, P (S) = P (A ∊ B) = P (A)P (B) 2. OR network T:
C @ R @ @ @
@ R @ @
@D
To get through, either C or D must be working, so using independence, P (T ) = P (C âˆŞ D) = P (C) + P (D) − P (C ∊ D) = P (C) + P (D) − P (C)P (D)
More complicated networks can be broken down as combinations of these simple networks. Example: S:
C @
R @ B E PP *
PP @
q
P R D @ - A G X : XXX XXX z F X
More complicated networks can be broken down as combinations of these simple networks. Example: S:
C @
R @ B E PP *
PP @
q
P R D @ - A G X : XXX XXX z F X
If P (A) = P (B) = P (D) = 0.8 and P (C) = P (E) = P (F ) = P (G) = 0.9, find the system reliability P (S).
Solution: Consider subsystem U . U:
C @
R @ - B E @ R D @
P (U ) = P (B ∩ (C ∪ D) ∩ E)
79
Solution: Consider subsystem U . U:
C @
R @ - B E @ R D @
P (U ) = P (B ∩ (C ∪ D) ∩ E) = P (B)P (C ∪ D)P (E) (independence)
Solution: Consider subsystem U . U:
C @
R @ - B E @ R D @
P (U ) = P (B ∩ (C ∪ D) ∩ E) = P (B)P (C ∪ D)P (E) (independence) = P (B) (P (C) + P (D) − P (C)P (D)) P (E)
81
Solution: Consider subsystem U . U:
C @
R @ - B E @ R D @
P (U ) = P (B ∩ (C ∪ D) ∩ E) = P (B)P (C ∪ D)P (E) (independence) = P (B) (P (C) + P (D) − P (C)P (D)) P (E) = 0.8 × (0.9 + 0.8 − 0.9 × 0.8) × 0.9
Solution: Consider subsystem U . U:
C @
R @ - B E @ R D @
P (U ) = P (B ∩ (C ∪ D) ∩ E) = P (B)P (C ∪ D)P (E) (independence) = P (B) (P (C) + P (D) − P (C)P (D)) P (E) = 0.8 × (0.9 + 0.8 − 0.9 × 0.8) × 0.9 = 0.7056
83
Next, S:
U @
R @ - A G @ R F @
P (S) = P (A ∩ (U ∪ F ) ∩ G)
84
Next, S:
U @
R @ - A G @ R F @
P (S) = P (A ∩ (U ∪ F ) ∩ G) = P (A) (P (U ) + P (F ) − P (U )P (F )) P (G)
Next, S:
U @
R @ - A G @ R F @
P (S) = P (A ∩ (U ∪ F ) ∩ G) = P (A) (P (U ) + P (F ) − P (U )P (F )) P (G) = 0.8 × (0.7056 + 0.9 − 0.7056 × 0.9) × 0.9
86
Next, S:
U @
R @ - A G @ R F @
P (S) = P (A ∩ (U ∪ F ) ∩ G) = P (A) (P (U ) + P (F ) − P (U )P (F )) P (G) = 0.8 × (0.7056 + 0.9 − 0.7056 × 0.9) × 0.9 = 0.6988 (4 d.p.)
Conditional probability Class of 120 students, 30 smokers, 90 not. Pick a student at random. P (S) =
88
Conditional probability Class of 120 students, 30 smokers, 90 not. Pick a student at random. P (S) =
30 = 0.25 120
89
Conditional probability Class of 120 students, 30 smokers, 90 not. Pick a student at random. P (S) =
M Splitting into male/female, F
30 = 0.25 120 S 15 15 30
NS 65 25 90
80 40
Given that a male is selected, what’s the probability he’s a smoker? P (S|M ) =
90
Conditional probability Class of 120 students, 30 smokers, 90 not. Pick a student at random. P (S) =
M Splitting into male/female, F
30 = 0.25 120 S 15 15 30
NS 65 25 90
80 40
Given that a male is selected, what’s the probability he’s a smoker? P (S|M ) =
15 = 0.1875 80
91
Conditional probability Class of 120 students, 30 smokers, 90 not. Pick a student at random. P (S) =
M Splitting into male/female, F
30 = 0.25 120 S 15 15 30
NS 65 25 90
80 40
Given that a male is selected, what’s the probability he’s a smoker? P (S|M ) =
15 = 0.1875 80
Notice: P (S) 6= P (S|M ) 92
Example A fair die is thrown twice. Given that the score on the first throw is less than 4, what’s the probability that the sum of the two scores is at least 8? The sample space ℌ is (1,1)
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1)
(2,2)
(2,3)
(2,4)
(2,5)
(2,6)
(3,1)
(3,2)
(3,3)
(3,4)
(3,5)
(3,6)
(4,1)
(4,2)
(4,3)
(4,4)
(4,5)
(4,6)
(5,1)
(5,2)
(5,3)
(5,4)
(5,5)
(5,6)
(6,1)
(6,2)
(6,3)
(6,4)
(6,5)
(6,6)
36 outcomes, each equally likely, each having probability 1/36 Define events A = Score on first throw is 1, 2 or 3 B = Sum of two scores is at least 8 Then by counting the number of outcomes in each event, P (A) =
94
36 outcomes, each equally likely, each having probability 1/36 Define events A = Score on first throw is 1, 2 or 3 B = Sum of two scores is at least 8 Then by counting the number of outcomes in each event, P (A) = 18/36 = 1/2
95
36 outcomes, each equally likely, each having probability 1/36 Define events A = Score on first throw is 1, 2 or 3 B = Sum of two scores is at least 8 Then by counting the number of outcomes in each event, P (A) = 18/36 = 1/2 P (B) =
36 outcomes, each equally likely, each having probability 1/36 Define events A = Score on first throw is 1, 2 or 3 B = Sum of two scores is at least 8 Then by counting the number of outcomes in each event, P (A) = 18/36 = 1/2 P (B) = 15/36 = 5/12
Suppose we are told that A has occurred; that is, we know the score on the first throw is less than 4. To find the conditional probability of B, given that A has occurred, written P (B|A), we only need to consider those outcomes in A (the first three rows of the sample space). Within A, there are 18 outcomes in total; for B to occur, there are 3 possible outcomes within A, so P (B|A) =
Suppose we are told that A has occurred; that is, we know the score on the first throw is less than 4. To find the conditional probability of B, given that A has occurred, written P (B|A), we only need to consider those outcomes in A (the first three rows of the sample space). Within A, there are 18 outcomes in total; for B to occur, there are 3 possible outcomes within A, so P (B|A) = 3/18 = 1/6
Note: P (B|A) 6= P (B). That is, knowledge that A has occurred changes the probability that B will occur.
100
Note: P (B|A) 6= P (B). That is, knowledge that A has occurred changes the probability that B will occur. The events A and B are not independent.
Note: P (B|A) 6= P (B). That is, knowledge that A has occurred changes the probability that B will occur. The events A and B are not independent. In general, for any two events E and F , the conditional probability P (E|F ) is defined by P (E|F ) =
Note: P (B|A) 6= P (B). That is, knowledge that A has occurred changes the probability that B will occur. The events A and B are not independent. In general, for any two events E and F , the conditional probability P (E|F ) is defined by P (E|F ) = provided P (F ) 6= 0.
P (E ∊ F ) P (F )
Example Returning to the example of throwing a die twice, define event C by C = Score on second throw is 3 or 4 Counting the number of outcomes in C, find P (C) =
104
Example Returning to the example of throwing a die twice, define event C by C = Score on second throw is 3 or 4 Counting the number of outcomes in C, find P (C) =12/36 = 1/3
105
Example Returning to the example of throwing a die twice, define event C by C = Score on second throw is 3 or 4 Counting the number of outcomes in C, find P (C) =12/36 = 1/3 Now suppose we’re told that A has occurred; then by couting outcomes within A, P (C|A) =
106
Example Returning to the example of throwing a die twice, define event C by C = Score on second throw is 3 or 4 Counting the number of outcomes in C, find P (C) =12/36 = 1/3 Now suppose we’re told that A has occurred; then by couting outcomes within A, P (C|A) = 6/18 = 1/3
107
Example Returning to the example of throwing a die twice, define event C by C = Score on second throw is 3 or 4 Counting the number of outcomes in C, find P (C) =12/36 = 1/3 Now suppose we’re told that A has occurred; then by couting outcomes within A, P (C|A) = 6/18 = 1/3 In this case, P (C|A) = P (C). Knowledge about A doesn’t affect the probability of C. Events A and C are independent.
108
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F )
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F ) For independent events E and F with P (F ) 6= 0, then P (E|F ) =
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F ) For independent events E and F with P (F ) 6= 0, then P (E|F ) =
P (E ∊ F ) = P (F )
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F ) For independent events E and F with P (F ) 6= 0, then P (E|F ) =
P (E ∊ F ) P (E)P (F ) = = P (F ) P (F )
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F ) For independent events E and F with P (F ) 6= 0, then P (E|F ) =
P (E ∊ F ) P (E)P (F ) = = P (E) P (F ) P (F )
Independence
In general, events E and F are called independent if P (E ∊ F ) = P (E)P (F ) For independent events E and F with P (F ) 6= 0, then P (E|F ) =
P (E ∊ F ) P (E)P (F ) = = P (E) P (F ) P (F )
That is, knowledge that F has occurred doesn’t change the probability that E will.
114
Example
Returning to the die throwing example, but now using the definition of conditional probability, P (B|A) =
P (B ∊ A) = P (A)
Example
Returning to the die throwing example, but now using the definition of conditional probability, P (B|A) =
P (B ∊ A) 3/36 = = P (A) 18/36
Example
Returning to the die throwing example, but now using the definition of conditional probability, P (B|A) =
P (B ∊ A) 3/36 3 = = = 1/6 P (A) 18/36 18
P (C|A) =
P (C ∊ A) = P (A)
Example
Returning to the die throwing example, but now using the definition of conditional probability, P (B|A) =
P (B ∊ A) 3/36 3 = = = 1/6 P (A) 18/36 18
P (C|A) =
P (C ∊ A) 6/36 = = P (A) 18/36
Example
Returning to the die throwing example, but now using the definition of conditional probability, P (B|A) =
P (B ∊ A) 3/36 3 = = = 1/6 P (A) 18/36 18
P (C|A) =
P (C ∊ A) 6/36 6 = = = 1/3 P (A) 18/36 18
Probability trees
Example: In a selection of flower seeds, 3/4 have been treated to improve germination, 1/4 left untreated. Treated seeds have probability 0.8 of germination; untreated seeds have probability 0.5 of germination. Find the probability that a seed, selected at random, will germinate.
120
Denote the events of interest by T
= Seed is treated
G = Seed germinates The problem can be represented in a tree diagram:
G
H HH H
T PP P
P P GĚ„
G
HH
H H TĚ„ PP P
PP P
PP P
P P GĚ„
121
Denote the events of interest by T
= Seed is treated
G = Seed germinates The problem can be represented in a tree diagram:
G
T PP 0.75 PP PP
P P GĚ„
H HH G H H
0.25 HH H TĚ„ P PP PP
PP P GĚ„
122
Denote the events of interest by T
= Seed is treated
G = Seed germinates The problem can be represented in a tree diagram:
G 0.8
TP PP 0.75 PP
PP P GĚ„
H HH G H H
0.25 HH H TĚ„ P PP PP
PP P GĚ„
123
Denote the events of interest by T
= Seed is treated
G = Seed germinates The problem can be represented in a tree diagram:
G 0.8
TP PP 0.75 PP
PP 0.2 P GĚ„
H HH G H H
0.25 HH H TĚ„ P PP PP
PP P GĚ„
124
Denote the events of interest by T
= Seed is treated
G = Seed germinates The problem can be represented in a tree diagram:
G 0.8
TP PP 0.75 PP
PP 0.2 P GĚ„
H HH G 0.5 H H
0.25 HH H TĚ„ P PP PP
PP 0.5 P GĚ„
125
From the diagram, P (T ∊ G) =
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6 P T̄ ∩ G =
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6 P T̄ ∩ G = 0.25 × 0.5 = 0.125
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6 P T̄ ∩ G = 0.25 × 0.5 = 0.125 and now P (G) =
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6 P T̄ ∩ G = 0.25 × 0.5 = 0.125 and now P (G) = P (T ∩ G) + P T̄ ∩ G
=
From the diagram, P (T ∩ G) = 0.75 × 0.8 = 0.6 P T̄ ∩ G = 0.25 × 0.5 = 0.125 and now P (G) = P (T ∩ G) + P T̄ ∩ G
= 0.6 + 0.125 = 0.725
132
Law of Total Probability
Suppose we have a collection of events E1 , E2 , . . . , Er
133
Law of Total Probability
Suppose we have a collection of events E1 , E2 , . . . , Er The events {E1 , . . . , Er } are called mutually exclusive if Ei ∊ Ej = φ for every pair of i, j values with i 6= j
134
Law of Total Probability
Suppose we have a collection of events E1 , E2 , . . . , Er The events {E1 , . . . , Er } are called mutually exclusive if Ei ∩ Ej = φ for every pair of i, j values with i 6= j The events {E1 , . . . , Er } are called exhaustive if E1 ∪ E2 ∪ . . . ∪ Er = Ω
Law of Total Probability
Suppose we have a collection of events E1 , E2 , . . . , Er The events {E1 , . . . , Er } are called mutually exclusive if Ei ∩ Ej = φ for every pair of i, j values with i 6= j The events {E1 , . . . , Er } are called exhaustive if E1 ∪ E2 ∪ . . . ∪ Er = Ω Given a set of mutually exclusive and exhaustive events E1 , . . . , Er , then any outcome ωk ∈ Ω falls in one, and only one, of the events Ei
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
137
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
where the events (F ∩ E1 ) , . . . , (F ∩ Er ) are themselves mutually exclusive, and so P (F ) =
138
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
where the events (F ∩ E1 ) , . . . , (F ∩ Er ) are themselves mutually exclusive, and so P (F ) = P (F ∩ E1 ) + · · · + P (F ∩ Er )
139
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
where the events (F ∩ E1 ) , . . . , (F ∩ Er ) are themselves mutually exclusive, and so P (F ) = P (F ∩ E1 ) + · · · + P (F ∩ Er ) and recalling the definition of conditional probability, this becomes P (F ) =
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
where the events (F ∩ E1 ) , . . . , (F ∩ Er ) are themselves mutually exclusive, and so P (F ) = P (F ∩ E1 ) + · · · + P (F ∩ Er ) and recalling the definition of conditional probability, this becomes P (F ) = P (F | E1 ) P (E1 ) +
Law of Total Probability Furthermore, any event F can be written as F
= (F ∩ E1 ) ∪ (F ∩ E2 ) ∪ . . . ∪ (F ∩ Er )
where the events (F ∩ E1 ) , . . . , (F ∩ Er ) are themselves mutually exclusive, and so P (F ) = P (F ∩ E1 ) + · · · + P (F ∩ Er ) and recalling the definition of conditional probability, this becomes P (F ) = P (F | E1 ) P (E1 ) + · · · + P (F | Er ) P (Er )
Law of Total Probability This is the Law of Total Probability: For a mutually exclusive and exhaustive set of events {E1 , . . . , Er } and an event F , P (F ) =
r X i=1
P (F | Ei ) P (Ei )
Law of Total Probability This is the Law of Total Probability: For a mutually exclusive and exhaustive set of events {E1 , . . . , Er } and an event F , P (F ) =
r X
P (F | Ei ) P (Ei )
i=1
Example: A particular person, when hiring a minicab, chooses company A 40% of the time, company B 50% of the time, and company C 10% of the time. Cabs hired from A arrive late with probability 0.09; those from B with probability 0.06; those from C with probability 0.15. Next time this person hires a cab, what is the probability it will arrive late?
144
Denote by L the event that the cab arrives late; then we’re told P (A) = 0.4
P (L|A) = 0.09
P (B) = 0.5
P (L|B) = 0.06
P (C) = 0.1
P (L|C) = 0.15
145
Denote by L the event that the cab arrives late; then we’re told P (A) = 0.4
P (L|A) = 0.09
P (B) = 0.5
P (L|B) = 0.06
P (C) = 0.1
P (L|C) = 0.15
so by the Law of Total Probability, P (L) =
Denote by L the event that the cab arrives late; then we’re told P (A) = 0.4
P (L|A) = 0.09
P (B) = 0.5
P (L|B) = 0.06
P (C) = 0.1
P (L|C) = 0.15
so by the Law of Total Probability, P (L) = P (A)P (L|A) + P (B)P (L|B) + P (C)P (L|C) =
147
Denote by L the event that the cab arrives late; then we’re told P (A) = 0.4
P (L|A) = 0.09
P (B) = 0.5
P (L|B) = 0.06
P (C) = 0.1
P (L|C) = 0.15
so by the Law of Total Probability, P (L) = P (A)P (L|A) + P (B)P (L|B) + P (C)P (L|C) = 0.4 × 0.09 + 0.5 × 0.06 + 0.1 × 0.15 =
Denote by L the event that the cab arrives late; then we’re told P (A) = 0.4
P (L|A) = 0.09
P (B) = 0.5
P (L|B) = 0.06
P (C) = 0.1
P (L|C) = 0.15
so by the Law of Total Probability, P (L) = P (A)P (L|A) + P (B)P (L|B) + P (C)P (L|C) = 0.4 × 0.09 + 0.5 × 0.06 + 0.1 × 0.15 = 0.081
Bayes’ Theorem If E1 and E2 are mutually exclusive and exhaustive events, then for any event F with P (F ) 6= 0, P (E1 |F ) =
150
Bayes’ Theorem If E1 and E2 are mutually exclusive and exhaustive events, then for any event F with P (F ) 6= 0, P (E1 |F ) =
P (F |E1 ) P (E1 ) P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
151
Bayes’ Theorem If E1 and E2 are mutually exclusive and exhaustive events, then for any event F with P (F ) 6= 0, P (E1 |F ) =
P (F |E1 ) P (E1 ) P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
Notes: 1. Events E1 and E2 are complements of one another, so rather than E1 , E2 we could have written E, EĚ„
152
Bayes’ Theorem If E1 and E2 are mutually exclusive and exhaustive events, then for any event F with P (F ) 6= 0, P (E1 |F ) =
P (F |E1 ) P (E1 ) P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
Notes: 1. Events E1 and E2 are complements of one another, so rather than E1 , E2 we could have written E, EĚ„ 2. By symmetry, we also have P (E2 |F ) =
P (F |E2 ) P (E2 ) P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
153
Bayes’ Theorem Proof: P (E1 |F ) =
Bayes’ Theorem Proof: P (E1 |F ) = =
P (E1 ∩ F ) P (F )
Bayes’ Theorem Proof: P (E1 ∩ F ) P (F ) P (F ∩ E1 ) = P (F )
P (E1 |F ) =
=
Bayes’ Theorem Proof: P (E1 ∩ F ) P (F ) P (F ∩ E1 ) = P (F ) P (F |E1 ) P (E1 ) = P (F )
P (E1 |F ) =
=
Bayes’ Theorem Proof: P (E1 ∩ F ) P (F ) P (F ∩ E1 ) = P (F ) P (F |E1 ) P (E1 ) = P (F ) P (F |E1 ) P (E1 ) = P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
P (E1 |F ) =
Bayes’ Theorem Proof: P (E1 ∩ F ) P (F ) P (F ∩ E1 ) = P (F ) P (F |E1 ) P (E1 ) = P (F ) P (F |E1 ) P (E1 ) = P (F |E1 ) P (E1 ) + P (F | E2 ) P (E2 )
P (E1 |F ) =
More generally: If {E1 , E2 , . . . , Er } is a set of mutually exclusive and exhaustive events, and F is any event with P (F ) 6= 0, then for each k, P (Ek |F ) =
P (F |Ek ) P (Ek ) Pr i=1 P (F |Ei ) P (Ei )
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C?
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) =
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) =
P (L|C)P (C) P (L|A)P (A) + P (L|B)P (B) + P (L|C)P (C)
=
162
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) = =
P (L|C)P (C) P (L|A)P (A) + P (L|B)P (B) + P (L|C)P (C) 0.15 × 0.1 0.09 × 0.4 + 0.06 × 0.5 + 0.15 × 0.1
=
163
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) =
P (L|C)P (C) P (L|A)P (A) + P (L|B)P (B) + P (L|C)P (C)
=
0.15 × 0.1 0.09 × 0.4 + 0.06 × 0.5 + 0.15 × 0.1
=
0.015 ≈ 0.1852 0.081
164
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) =
P (L|C)P (C) P (L|A)P (A) + P (L|B)P (B) + P (L|C)P (C)
=
0.15 × 0.1 0.09 × 0.4 + 0.06 × 0.5 + 0.15 × 0.1
=
0.015 ≈ 0.1852 0.081
So the probability the cab is from company C, given that it’s late, is 0.1852;
165
Example Return to the minicab example. Given that the cab arrives late, what’s the probability it was from company C? P (C|L) =
P (L|C)P (C) P (L|A)P (A) + P (L|B)P (B) + P (L|C)P (C)
=
0.15 × 0.1 0.09 × 0.4 + 0.06 × 0.5 + 0.15 × 0.1
=
0.015 ≈ 0.1852 0.081
So the probability the cab is from company C, given that it’s late, is 0.1852; whereas the probability that it’s late, given that it’s from company C, is 0.15
166
Example
A particular disease affects one person in a thousand. A test for the disease exists; people affected by the disease produce a positive test result with probability 0.95; people unaffected produce a positive test result with probability 0.02 Suppose a person chosen at random from the population produces a positive test result. What is the probability that they have the disease?
167
Solution Denote by D the event that a person has the disease, and by + the event that they test positive. Then we’re told P (D) = 0.001 P (+|D) = 0.95 P + | D̄ = 0.02 By Bayes’ theorem, P (D|+) =
Solution Denote by D the event that a person has the disease, and by + the event that they test positive. Then we’re told P (D) = 0.001 P (+|D) = 0.95 P + | D̄ = 0.02 By Bayes’ theorem, P (D|+) = =
P (+|D)P (D) P (+|D)P (D) + P + | D̄ P D̄
Solution Denote by D the event that a person has the disease, and by + the event that they test positive. Then we’re told P (D) = 0.001 P (+|D) = 0.95 P + | D̄ = 0.02 By Bayes’ theorem, P (D|+) = = ≈
P (+|D)P (D) P (+|D)P (D) + P + | D̄ P D̄ 0.95 × 0.001 0.95 × 0.001 + 0.02 × 0.999
Solution Denote by D the event that a person has the disease, and by + the event that they test positive. Then we’re told P (D) = 0.001 P (+|D) = 0.95 P + | D̄ = 0.02 By Bayes’ theorem, P (D|+) = =
P (+|D)P (D) P (+|D)P (D) + P + | D̄ P D̄ 0.95 × 0.001 0.95 × 0.001 + 0.02 × 0.999
≈ 0.045 (3 d.p.)
Solution Denote by D the event that a person has the disease, and by + the event that they test positive. Then we’re told P (D) = 0.001 P (+|D) = 0.95 P + | D̄ = 0.02 By Bayes’ theorem, P (D|+) = =
P (+|D)P (D) P (+|D)P (D) + P + | D̄ P D̄ 0.95 × 0.001 0.95 × 0.001 + 0.02 × 0.999
≈ 0.045 (3 d.p.) So even if a person tests positive, the probability that they actually have the disease is only around 4.5%
172