Example 2a
Let $t$ denote the amount of time the student takes to finish the exam. And, for $0\leq x\leq1$, let \(L_x=\{t:0\leq t<x\}\) denote the event that the student finishes the exam in less than $x$ hours.
If $x_1<x_2$, then $L_{x_1}\subset L_{x_2}$:
\(L_{x_1}=\{t:0\leq t<x_1\}\)
\(L_{x_2}=\{t:0\leq t<x_2\}\)
Let $y\in L_{x_1}$. We want to show that $y\in L_{x_2}$. Since $y\in L_{x_1}$, then $0\leq y<x_1<x_2$ so that $y\in L_{x_2}$ as well.
Suppose $X\subset Y$. Then $Y^c\subset X^c$: Let $u\in Y^c$. We want to show that $u\in X^c$. Since $u\in Y^c$, then $u\notin Y$. And this implies that $u\notin X$: if $u\in X$, then $u\in Y$ which contradicts $u\notin Y$. So we have shown that $u\notin X\iff u\in X^c$.
So $L_{x_2}^c\subset L_{x_1}^c$ if $x_1<x_2$.
Let $F=L_1^c$. Then $F=L_1^c\subset L_{0.75}^c$ since $0.75<1$.
Hence $FL_{0.75}^c=F$ and $P(FL_{0.75}^c)=P(F)$
Example 2b
Let $F=\set{(h,h),(h,t)}$ denote the event that the first flip lands on heads. And let \(E=\{(h,h)\}\) denote the event that both flips land on heads. We wish to compute the conditional probability $\cp{E}{F}$ that \(E\) will occur given that \(F\) has occurred. Well, given that $F$ has occurred, we assume that all outcomes in $F$ become equally likely to occur. So it is equally likely that we will get $(h,h)$ and $(h,t)$:
$$ P(F)=1\quad\quad \cp{\{(h,h)\}}{F}=\cp{\{(h,t)\}}{F}=\frac{1}{2} $$
$$ \cp{E}{F}=\frac{1}{2} $$
Let $G=\set{(h,h),(h,t),(t,h)}$ denote the event where at least one flip lands on heads. What is $\cp{E}{G}$? $\frac{1}{3}$.
Odds Hypothesis
$$ \cp{H}{E}=\frac{P(HE)}{P(E)}=\frac{P}{} $$
Example 3i
Given that $\cp{E}{\bfO}=\frac1{52}$ and $\pr{\bfO}=\frac1{51!}$, we have $\cp{E}{\bfO}\pr{\bfO}=\frac1{52}\frac1{51!}$ and
$$ \pr{E}=\sum_{\bfO}\cp{E}{\bfO}\pr{\bfO}=\sum_{\bfO}\frac1{52}\frac1{51!}=\frac1{52}\frac1{51!}\sum_{\bfO}1=\frac1{52}\frac1{51!}51!=\frac1{52} $$
Mutually Exclusive vs Independent
$$ A,B\text{ Mutually Exclusive }\iff\cases{AB=\empty\\\pr{AB}=0\\\cp{A}{B}=0\\\pr{A\cup B}=\pr{A}+\pr{B}} $$
$$ A,B\text{ Independent }\iff\cases{\pr{AB}=\pr{A}\pr{B}\\\cp{A}{B}=\frac{\pr{A}\pr{B}}{\pr{B}}=\pr{A}\\\pr{A\cup B}=\pr{A}+\pr{B}-\pr{A}\pr{B}} $$
Suppose $A, B$ are mutually exclusive. Then they cannot be independent because $\pr{AB}=0\neq\pr{A}\pr{B}$. The exception is sets of measure zero.
Said another way, mutually exclusive events are dependent. Yet another way to say this: if $A$ and $B$ are mutually exclusive, then the conditional probability that $A$ will occur given that $B$ has occurred is zero. That is, if $B$ occurs, $A$ cannot occur. Hence they are dependent.
Let’s look at an example. Let’s say we roll two dice. Let $A$ denote the event that the sum of the dice is $7$ and let $B$ denote the event that the sum of the dice is $8$. Then
$A=\set{(1,6),(6,1),(2,5),(5,2),(3,4),(4,3)}$
$B=\set{(2,6),(6,2),(3,5),(5,3),(4,4)}$
We see that $A$ and $B$ are mutually exclusive since $AB=\empty$. So we have
$$ \cp{A}{B}=\frac{\pr{AB}}{\pr{B}}=\frac0{\pr{B}}=0\neq\frac16=\pr{A} $$
Hence $A$ and $B$ are dependent since $\cp{A}{B}\neq\pr{A}$.
Conversely, we can show that independent events cannot be mutually exclusive unless one of them is measure zero. Suppose $A, B$ are independent and neither is measure zero. Then $\pr{AB}=\pr{A}\pr{B}\neq0$.
Let’s do another example. Let $A$ denote the event that the sum of the dice is $7$ and let $B$ denote the event that the first die is a $3$. Then
$A=\set{(1,6),(6,1),(2,5),(5,2),(3,4),(4,3)}$
$B=\set{(3,1),(3,2),(3,3),(3,4),(3,5),(3,6)}$
These events are independent since
$$ \cp{A}{B}=\frac{\pr{AB}}{\pr{B}}=\frac{\pr{(3,4)}}{\frac16}=6\frac1{36}=\pr{A} $$
or
$$ \pr{AB}=\pr{(3,4)}=\frac1{36}=\frac16\frac16=\pr{A}\pr{B} $$
We also see that $A$ and $B$ are not mutually exclusive since $AB=\set{(3,4)}\neq\empty$ and $\pr{AB}=\frac1{36}\neq0$.
However, dependent events are not necessarily mutually exclusive. Let’s do a counterexample. Let $A$ denote the event that the sum of the dice is $6$ and let $B$ denote the event that the first die is a $3$. Then
$A=\set{(1,5),(5,1),(2,4),(4,2),(3,3)}$
$B=\set{(3,1),(3,2),(3,3),(3,4),(3,5),(3,6)}$
These events are dependent since
$$ \cp{A}{B}=\frac{\pr{AB}}{\pr{B}}=\frac{\pr{(3,3)}}{\frac16}=6\frac1{36}=\frac16\neq\frac5{36}=\pr{A} $$
But they are not mutually exclusive since $AB=\set{(3,3)}\neq\empty$ and $\pr{AB}=\frac1{36}\neq0$.
A moment ago, we showed that mutually exclusive sets are dependent. And now we just showed that dependent sets are not always mutually exclusive. Hence:
Suppose $B$ is some event in the sample space $\Omega$. Let $X_B$ denote the collection of all events in $\Omega$ that are mutually exclusive with $B$ and let $D_B$ denote the collection of all events in $\Omega$ that are dependent with $B$. Then
$$ X_B\subset D_B\dq\text{and}\dq D_B\not\subset X_B $$
Or is it possible that $D_B=X_B$ in some cases? We’ve shown an example of $D_B\not\subset X_B$. Are there examples where $D_B\subset X_B$?
Example 3o
Assign an index to each ex-con. Let $k$ be the index of AJ Jones, let $M_i$ denote the event that file $i$ matches, and let $G_i$ denote the event that dude $i$ is guilty. We are given that $\cp{M_i}{G_i^c}=10^{-5}$. Hence
$$ \cp{M_i^c}{G_i^c}=1-\cp{M_i}{G_i^c}=1-10^{-5} $$
If we assume that there is exactly $1$ perp, then the $G_i$’s are mutually exclusive and dependent: $G_iG_j=\empty$. Hence the occurrence of $G_k$ implies the occurrence of $G_i^c$, $G_k\subset G_i^c$, and
$$ \cp{M_i^c}{G_k}=1-\cp{M_i}{G_k}=1-\cp{M_i}{G_kG_i^c}=1-10^{-5} $$
The probability that both files don’t match for dudes $i$ and $j$ given $G_k$ is
$$ \cp{M_i^cM_j^c}{G_k}=\cp{M_i^c}{G_k}\cp{M_j^c}{G_k}=\bop1-10^{-5}\bcp\bcp\wt\bop1-10^{-5}\bcp $$
Let $M$ denote event that AJ is the only one of the 10,000 on file to have a match. Then
$$ M=M_{k}\cap\bigcap_{i\neq k}M_i^c $$
$$ \cp{M}{G_k}=\cp{M_k\cap\bigcap_{i\neq k}M_i^c}{G_k\cap\bigcap_{i\neq k}G_i^c} $$
$$ =\cp{M_k}{G_k}\prod_{i\neq k}\cp{M_i^c}{G_k} $$
Example 4f
- sssff
- ssfsf
- sfssf
- fsssf
- ssffs
- sfsfs
- sffss
- ffsss
- fsfss
- fssfs
Example 4i
Let $B_i^x$ be the event that the coupon collected at time $0\leq x\leq k$ was not of type $i$. Since each coupon collected is independently not of type $i$, then $B_i^x$ and $B_i^y$ are independent for any $0\leq x,y\leq k$. Hence
$$ P({\text{no coupon of type i for k collections}})=P(B_i^1B_i^2\dots B_i^k)=P\Bop\bigcap_{x=1}^{k}B_i^x\Bcp $$
$$ =\prod_{x=1}^{k}P(B_i^x)=\prod_{x=1}^{k}(1-p_i)=(1-p_i)^k $$
Let $C_{i,j}^x$ denote the event that the coupon collected at time $0\leq x\leq k$ is of neither type $i$ or $j$. So
$$ C_{i,j}^x=B_i^xB_j^x $$
And
$$ P(C_{i,j}^x)=P(B_i^xB_j^x)=P(B_i^x)+P(B_j^x)-P(B_i^x\cup B_j^x) $$
$$ =(1-p_i)+(1-p_j)-P(B_i^x\cup B_j^x) $$
The probability that $b$ is of type $i$ or $j$ is $p_i+p_j$. So the probability that $b$ is of neither type $i$ nor $j$ is $1-(p_i+p_j)$.
Example 4j Pascal
Let $E_{n,m}$ denote the event that $n$ successes occur before $m$ failures so that $P_{n,m}=P(E_{n,m})$. Let $S_i$ and $F_i$ denote the events of success and failure on the $i^{th}$ trial, respectively.
$$ P_{n,m}=\cp{E_{n,m}}{S_1}\pr{S_1}+\cp{E_{n,m}}{F_1}\pr{F_1} $$
We know that $p$ is the probability of success for any of the independent trials including the first one. So $P(S_1)=p$ and similarly $P(F_1)=1-p$. So we have
$$ P_{n,m}=\cp{E_{n,m}}{S_1}p+\cp{E_{n,m}}{F_1}(1-p) \tag{1} $$
We use a similar argument from example 4h to show that $\cp{E_{n.m}}{S_1}=\pr{E_{n-1,m}}=P_{n-1,m}$. If $S_1$ occurs, then at that point the situation is similar as it was when the problem first started. The only difference is that now we only need $n-1$ successes rather than $n$ successes. Furthermore, the trials are independent; therefore, the outcome of the first trial will have no effect on subsequent trials. That is, the fact that the first trial was a success will have no effect on the success or failure or any subsequent trials. The same argument shows that $\cp{E_{n,m}}{F_1}=\pr{E_{n,m-1}}=P_{n,m-1}$.
So (1) becomes
$$ P_{n,m}=pP_{n-1,m}+(1-p)P_{n,m-1} \tag{2} $$
Example 4j Fermat
In order for $n$ successes to occur before $m$ failures, it is necessary and sufficient that there be at least $n$ successes in the first $m+n−1$ trials.
Proof
First suppose that there are at least $n$ successes in the first $m+n-1$ trials. We want to show that this implies that $n$ success have occurred before $m$ failures. But we’ve only had $m+n-1$ trials, at least $n$ of which are successes. This implies that we’ve had less than or equal to $m+n-1-n=m-1$ failures.
Conversely, suppose that $n$ successes have occurred before $m$ failures. We want to show that this imples that there were at least $n$ successes in the first $m+n-1$ trials. Let $k<m$ be the number of failures that occurred. Then $n+k\leq m+n-1$. If $n+k=m+n-1$, we’re done. If $n+k<m+n-1$, just imagine that the necessary additional trials were performed with any combination of successes and failures.
Example 5a
The solution claims that $\cp{A_2}{AA_1}=0.4$ and $\cp{A_2}{A^cA_1}=0.2$. The author’s actually assuming that $A_1$ and $A_2$ are conditinally independent given $A$, although he doesn’t state this until $3$ pages later. Nice.
Anyway, $A_1$, $A_2$ conditionally independent given $A$ means that
$$ \cp{A_2}{A_1A}=\cp{A_2}{A} $$
We’re given that, during any given year, an accident-prone person will have an accident with probability $0.4$. Hence
$$ \cp{A_2}{A_1A}=\cp{A_2}{A}=0.4 $$
We can similarly show that $\cp{A_2}{A^cA_1}=\cp{A_2}{A^c}=0.2$.
Example 5c Theory of Runs
Let $E$ be the event that a run of $n$ consecutive successes occurs before a run of $m$ consecutive failures. And let $H$ denote the event that the first trial results in a success. Then conditioning $E$ on the outcome of the first trial, we see
$$ P(E)=\cp{E}{H}P(H)+\cp{E}{H^c}P(H^c) $$
$$ =\cp{E}{H}p+\cp{E}{H^c}q \tag{5.2} $$
Now, given that the first trial was successful, one way we can get a run of $n$ successes before a run of $m$ failures would be to have the next $n − 1$ trials all result in successes. So, let us condition on whether or not that occurs. That is, letting $F$ be the event that trials 2 through $n$ all are successes, we obtain
$$ \cp{E}{H}=\cp{E}{FH}\cp{F}{H}+\cp{E}{F^cH}\cp{F^c}{H} \tag{5.3} $$
This just follows from (5.1.b), conditioning on a conditional probability. Now we can see that $\cp{E}{FH}=1$ since $FH$ is the event that the first $n$ trials are successes. But if the event $F^cH$ occurs then the first trial would result in a success but there would be a failure some time during the next $n-1$ trials. However, when this failure occurs, it would wipe out all of the the previous successes, and the situation would be exactly as if we started out with a failure. Hence:
$$ \cp{E}{F^cH}=\cp{E}{H^c} $$
Because the independence of trials implies that $F$ and $H$ are independent, and because $P(F) = p^{n−1}$, it follows from Equation (5.3) that
$$ \cp{E}{H}=p^{n-1}+(1-p^{n-1})\cp{E}{H^c} \tag{5.4} $$
Notice that we have also used the facts that $F^c$ and $H$ are independent and $P(F^c)=1-P(F)=1-p^{n-1}$. In a similar manner, we can compute $\cp{E}{H^c}$. Let $G$ denote the event that trials $2$ through $m$ all result in failure. Then
$$ \cp{E}{H^c}=\cp{E}{GH^c}\cp{G}{H^c}+\cp{E}{G^cH^c}\cp{G^c}{H^c} \tag{5.5} $$
The occurence of $G^cH^c$ implies the occurence of a success in the trials $2$ through $m$. When this success occurs, it would wipe out all of the previous failures and the situation would be exactly as if we started with a success. That is:
$$ \cp{E}{G^cH^c}=\cp{E}{H} $$
Also we see that $\cp{E}{GH^c}=0$ since the event $GH^c$ implies $m$ consecutive failures. The independence of trials implies that $G^c$ and $H^c$ are independent so that $\cp{G^c}{H^c}=P(G^c)=1-P(G)=1-q^{m-1}$. So (5.5) becomes
$$ \cp{E}{H^c}=(1-q^{m-1})\cp{E}{H} \tag{5.6} $$
Solving equations (5.4) and (5.6):
$$ \cp{E}{H}=p^{n-1}+(1-p^{n-1})(1-q^{m-1})\cp{E}{H} $$
$\iff$
$$ \cp{E}{H}-(1-p^{n-1})(1-q^{m-1})\cp{E}{H}=p^{n-1} $$
$\iff$
$$ \cp{E}{H}\bop1-(1-p^{n-1})(1-q^{m-1})\bcp=p^{n-1} $$
$\iff$
$$ \cp{E}{H}=\frac{p^{n-1}}{1-(1-p^{n-1})(1-q^{m-1})} $$
$$ =\frac{p^{n-1}}{1-\bop1-q^{m-1}-p^{n-1}+p^{n-1}q^{m-1}\bcp} $$
$$ =\frac{p^{n-1}}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
$\implies$
$$ \cp{E}{H^c}=\frac{(1-q^{m-1})p^{n-1}}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
So that (5.2) becomes
$$ P(E)=\cp{E}{H}p+\cp{E}{H^c}q $$
$$ =\frac{p^n+q(1-q^{m-1})p^{n-1}}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
$$ =\frac{p^{n-1}\bop p+q(1-q^{m-1})\bcp}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
$$ =\frac{p^{n-1}\bop p+q-q^{m}\bcp}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
$$ =\frac{p^{n-1}\bop p+1-p-q^{m}\bcp}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} $$
$$ P(E)=\frac{p^{n-1}(1-q^{m})}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} \tag{5.7} $$
By the symmetry of the problem, we can similarly show that
$$ P(\{\text{run of m consecutive failures before run of n consecutive successes}\}) $$
$$ =\frac{q^{m-1}(1-p^{n})}{p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}} \tag{5.8} $$
And we can show that (5.7) and (5.8) sum to 1:
$$ p^{n-1}(1-q^m)+q^{m-1}(1-p^n) $$
$$= p^{n-1}-p^{n-1}q^m+q^{m-1}-p^nq^{m-1} $$
$$= p^{n-1}+q^{m-1}-p^nq^{m-1}-p^{n-1}q^m $$
$$= p^{n-1}+q^{m-1}-pp^{n-1}q^{m-1}-p^{n-1}qq^{m-1} $$
$$= p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}(p+q) $$
$$= p^{n-1}+q^{m-1}-p^{n-1}q^{m-1}(p+(1-p)) $$
$$= p^{n-1}+q^{m-1}-p^{n-1}q^{m-1} $$
Since equations (5.7) and (5.8) sum to 1, it follows that, with probability 1, either a run of $n$ successes or a run of $m$ failures will eventually occur.
In [594]: runs=lambda n,m,p: (pw(p,n-1)*(1-pw(1-p,m)))/(pw(p,n-1)+pw(1-p,m-1)-pw(p,n-1)*pw(1-p,m-1))
In [595]: runs(2,3,1/2)
Out[595]: 0.69999999999999996
In [596]: runs(2,4,1/2)
Out[596]: 0.83333333333333337
Conditionally independent given F, p98
We say that the events $E_1$ and $E_2$ are conditionally independent given $F$ if, given that $F$ occurs, the conditional probability that $E_1$ occurs is unchanged by information as to whether or not $E_2$ occurs. More formally, $E_1$ and $E_2$ are said to be conditionally independent given $F$ if
$$ \cp{E_1}{E_2F}=\cp{E_1}{F} $$
The book claims that this is equivalent to
$$ \cp{E_1E_2}{F}=\cp{E_1}{F}\cp{E_2}{F} $$
This follows from
$$ \cp{E_1}{F}=\cp{E_1}{E_2F}=\frac{\pr{E_1E_2F}}{\pr{E_2F}} $$
$$ =\frac{\pr{F}}{\pr{E_2F}}\cp{E_1E_2}{F}=\frac1{\frac{\pr{E_2F}}{\pr{F}}}\cp{E_1E_2}{F} $$
$$ =\frac1{\cp{E_2}{F}}\cp{E_1E_2}{F} $$
Laplace’s rule of succession
This looks confusing:
$$ \cp{H}{F_n}=\sum_{i=0}^k\cp{H}{F_nC_i}\cp{C_i}{F_n} $$
But recall equation (5.1) on p.94 and the subsequent equation
$$ \cp{E_1}{F}=\cp{E_1}{E_2F}\cp{E_2}{F}+\cp{E_1}{E_2^cF}\cp{E_2^c}{F} $$
Mutually Exclusive Sequences
Let’s pick from three letters $a$, $b$, and $c$. We know the count will be $3!=6$.
- abc
- acb
- bac
- bca
- cab
- cba
If the last letter selected is $c$, what is the probability that the first letter selected is $a$?
Let $A$ denote the event that the first letter selected is $a$:
$$ A=\bset{(abc),(acb)} $$
Let $C$ denote the event that the last letter selected is $c$:
$$ C=\bset{(abc),(bac)} $$
The desired probability is
$$ \cp{A}{C}=\frac{P(AC)}{P(C)} $$
$$ =\frac{P(\bset{(abc)})}{P(\bset{(abc),(bac)})} $$
$$ =\frac{1/6}{P(\bset{(abc)})+P(\bset{(bac)})} $$
$$ =\frac{1/6}{2/6}=\frac{1}{2} $$
What is the probability that the first letter selected is $b$?
$$ \prt{b is the first letter selected}=P(\bset{(bac),(bca)}) $$
$$ =P(\bset{(bac)})+P(\bset{(bca)}) $$
$$ =\frac{1}{6}+\frac{1}{6}=\frac{1}{3} $$