(7.1) A player throws a fair die and simultaneously flips a fair coin. If the coin lands heads, then she wins twice, and if tails, then one-half of the value that appears on the die. Determine her expected winnings.
Solution Let $X$ denote the winnings and let $Y$ denote heads or tails. Then we use equation 5.1a on p.333
$$ \evw{X}=\evc{X}{Y=H}\pr{Y=H}+\evc{X}{Y=T}\pr{Y=T} $$
Let $D$ denote the value of the die. Then
$$ \evc{X}{Y=H}=\sum_{i=1}^6\evc{X}{Y=H,D=i}\pr{D=i}=\sum_{i=1}^62i\wts\frac16=\frac13\sum_{i=1}^6i=\frac13\wts21=7 $$
And
$$ \evc{X}{Y=T}=\sum_{i=1}^6\evc{X}{Y=T,D=i}\pr{D=i}=\sum_{i=1}^6\frac{i}2\wts\frac16=\frac1{12}\sum_{i=1}^6i=\frac1{12}\wts21=\frac{21}{12} $$
Hence
$$ \evw{X}=\frac12\cbr{7+\frac{21}{12}}=\frac12\cbr{\frac{84}{12}+\frac{21}{12}}=\frac12\frac{105}{12}=\frac{105}{24}=\frac{3\wts35}{3\wts8}=\frac{35}8 $$
Perhaps a better way to do this: Let $Y=0$ if heads occurs and $Y=1$ if tails occurs. Then
$$ \evw{X}=\sum_{i=0}^{1}\sum_{j=1}^6\evc{X}{Y=i,D=j}\pr{Y=i,D=j} $$
$$ =\sum_{i=0}^{1}\sum_{j=1}^6\evc{X}{Y=i,D=j}\pr{Y=i}\pr{D=j} \tag{Independence} $$
$$ =\sum_{j=1}^6\evc{X}{Y=0,D=j}\pr{Y=0}\pr{D=j}+\sum_{j=1}^6\evc{X}{Y=1,D=j}\pr{Y=1}\pr{D=j} $$
$$ =\sum_{j=1}^62j\wts\frac12\wts\frac16+\sum_{j=1}^6\frac{j}2\wts\frac12\wts\frac16 $$
$$ =\frac16\sum_{j=1}^6j+\frac1{24}\sum_{j=1}^6j=\Prn{\frac16+\frac1{24}}\sum_{j=1}^6j=\Prn{\frac4{24}+\frac1{24}}\frac{6\wts7}2=\frac5{24}\wts21=\frac58\wts7=\frac{35}8 $$
(7.3.a) Gambles are independent, and each one results in the player being equally likely to win or lose $1$ unit. Let $W$ denote the net winnings of a gambler whose strategy is to stop gambling immediately after his first win. Find $\pr{W>0}$.
Solution The winnings are positive IFF we win the first game. After that, the winnings are zero or negative. Hence
$$ \pr{W>0}=\prt{win first gamble}=\frac12 $$
(7.3.b) Find $\pr{W<0}$.
Solution The winnings are zero IFF we win the second game. Hence
$$ \pr{W<0}=1-(\pr{W>0}+\pr{W=0}) $$
$$ =1-\Prn{\frac12+\prt{lose first, win second gamble}} $$
$$ =1-\Prn{\frac12+\prt{lose first}\prt{win second gamble}} \tag{independence} $$
$$ =1-\Prn{\frac12+\frac12\frac12}=1-\frac34=\frac14 $$
(7.3.c) Find $\E{W}$.
Solution Let’s first find a formula for $W$. Let $N$ denote the number of gambles necessary until a win occurs. Then $N$ is a geometric random variable with parameter $p=\frac12$. Of the $N$ gambles, there is $1$ win and $N-1$ losses. So $W=1-(N-1)=2-N$. Hence
$$ \evw{W}=2-\evw{N}=2-\frac1{p}=2-\frac1{\frac12}=2-2=0 $$
Note that $\evw{N}=\frac1p$ follows from ch.4, p.156, example 8b.
Alternative solution: With probability $\frac12$, we win immediately and get a dollar. With probability $\frac12$, we lose the first game, lose a dollar, and the game effectively starts all over. When the game starts all over, we can expect to win $\evw{W}$ dollars. That is, when we lose the first game, we get $-1+\evw{W}$ dollars. Hence
$$ \evw{W}=\frac12\wts1+\frac12\prn{-1+\evw{W}}=\frac12-\frac12+\frac12\evw{W}=\frac12\evw{W} $$
The only values for $\evw{W}$ that satisfy $\evw{W}=\frac12\evw{W}$ are $0$, $\infty$, and $-\infty$.
(7.4.a) If $X$ and $Y$ have joint density function
$$ \pdfa{x,y}{X,Y}=\cases{\frac1y&0<y<1,0<x<y\\0&\text{otherwise}} $$
Find $\E{XY}$.
Solution Let $g(x,y)=xy$. Proposition 2.1 on p.298 gives the second equality:
$$ \evw{XY}=\evw{g(X,Y)}=\int_0^1\int_0^yg(x,y)\pdfa{x,y}{X,Y}dxdy=\int_0^1\int_0^yxy\frac1ydxdy=\int_0^1\int_0^yxdxdy $$
$$ =\int_0^1\frac{y^2}2dy=\frac12\frac13(1^3-0^3)=\frac16 $$
(7.4.b) Find $\E{X}$.
Solution Let $g(x,y)=x$. Proposition 2.1 on p.298 gives the second equality:
$$ \evw{X}=\evw{g(X,Y)}=\int_0^1\int_0^yg(x,y)\pdfa{x,y}{X,Y}dxdy=\int_0^1\int_0^y\frac{x}ydxdy=\int_0^1\frac1y\int_0^yxdxdy $$
$$ =\int_0^1\frac1y\frac{y^2}2dy=\frac12\int_0^1ydy=\frac12\frac12\Sbr{y^2\eval01}=\frac14 $$
(7.4.c) Find $\E{Y}$.
Solution Let $g(x,y)=y$. Proposition 2.1 on p.298 gives the second equality:
$$ \evw{Y}=\evw{g(X,Y)}=\int_0^1\int_0^yg(x,y)\pdfa{x,y}{X,Y}dxdy=\int_0^1\int_0^y\frac{y}ydxdy=\int_0^1\Sbr{x\eval0y}dy $$
$$ =\int_0^1(y-0)dy=\frac12\Sbr{y^2\eval01}=\frac12 $$
Note that $\evw{XY}\neq\evw{X}\evw{Y}$. This is due to the dependent boundary, at least.
(7.5) The county hospital is located at the center of a square whose sides are $3$ miles wide. If an accident occurs within this square, then the hospital sends out an ambulance. The road network is rectangular, so the travel distance from the hospital, whose coordinates are $(0,0)$, to the point $(x,y)$ is $\norm{x}$ + $\norm{y}$. If an accident occurs at a point that is uniformly distributed in the square, find the expected travel distance of the ambulance.
Solution First a useful tool:
Even function integration across origin Suppose that $f$ is an even function. Then
$$ \int_{-a}^af(x)dx=2\int_0^af(x)dx $$
Proof
$$ \int_{-a}^af(x)dx=\int_{-a}^0f(x)dx+\int_0^af(x)dx \tag{7.5.a.1} $$
$$ u=-x\dq x=-u\dq dx=-du\dq u_0=-x_0=-(-a)=a\dq u_1=-x_1=-0=0 $$
$$ \int_{-a}^0f(x)dx=-\int_a^0f(-u)du=\int_0^af(-u)du=\int_0^af(u)du $$
Then 7.5.a.1 becomes
$$ \int_{-a}^af(x)dx=\int_{-a}^0f(x)dx+\int_0^af(x)dx=\int_0^af(x)dx+\int_0^af(x)dx=2\int_0^af(x)dx $$
$\wes$
Now we start the problem. Note that we are given the joint distribution of the coordinates $(X,Y)$. We are not given the marginal distributions of $X$ and $Y$. We can compute the expected value jointly or marginally. Let’s do jointly first.
Recall problem 6.15 in ch.6. Let $R$ denote the square with sides of length $3$ centered at the origin. The area of $R$ is $9$. Since the joint distribution is uniform on $R$, the joint density must be some constant, say $c$, on $R$. Then we can compute the joint density as
$$ 1=\iint_{x,y\in R}\pdfa{x,y}{X,Y}dxdy=c\wts\iint_{x,y\in R}dxdy=c\wts9 $$
The last equality follows because $\iint_{x,y\in R}dxdy$ is simply the area of $R$. Hence $\pdfa{x,y}{X,Y}=\frac19$ for $(x,y)\in R$. For $(x,y)\in R$ to hold, it must be that $-\frac32\leq x\leq\frac32$ and $-\frac32\leq y\leq\frac32$. This is easy to see geometrically. Hence
$$ \pdfa{x,y}{X,Y}=\cases{\frac19&-\frac32\leq x\leq\frac32,-\frac32\leq y\leq\frac32\\0&\text{otherwise}} $$
Let $g(x,y)=\norm{x}+\norm{y}$. Then proposition 2.1 on p.298 gives the second equality:
$$ \evw{\norm{X}+\norm{Y}}=\evw{g(X,Y)}=\iint_{(x,y)\in R}g(x,y)\pdfa{x,y}{X,Y}dxdy $$
$$ =\frac19\int_{-\frac32}^{\frac32}\int_{-\frac32}^{\frac32}(\norm{x}+\norm{y})dxdy $$
$$ =\frac19\int_{-\frac32}^{\frac32}\Sbr{\int_{-\frac32}^{\frac32}\norm{x}dx+\norm{y}\int_{-\frac32}^{\frac32}dx}dy $$
$$ =\frac19\int_{-\frac32}^{\frac32}\Sbr{2\int_0^{\frac32}\norm{x}dx+\norm{y}\int_{-\frac32}^{\frac32}dx}dy \tag{even function} $$
$$ =\frac19\int_{-\frac32}^{\frac32}\Sbr{2\int_0^{\frac32}xdx+\norm{y}\Prn{\frac32--\frac32}}dy $$
$$ =\frac19\int_{-\frac32}^{\frac32}\Sbr{2\wts\frac12\Cbr{x^2\eval0{\frac32}}+3\norm{y}}dy $$
$$ =\frac19\int_{-\frac32}^{\frac32}\Sbr{\frac94+3\norm{y}}dy $$
$$ =\frac19\Cbr{\frac94\int_{-\frac32}^{\frac32}dy+3\int_{-\frac32}^{\frac32}\norm{y}dy} $$
$$ =\frac19\Cbr{\frac94\wts3+3\wts\frac94} \tag{same as above} $$
$$ =\frac14\wts3+3\wts\frac14=\frac32 $$
Check:
In [1]: import scipy.integrate as integrate
In [2]: I2=lambda f_xy,a,b,gfun,hfun: integrate.dblquad(f_xy,a,b,gfun,hfun)[0]
In [3]: import numpy as np
In [4]: I2(lambda x,y:(1/9)*(np.abs(x)+np.abs(y)),-3/2,3/2,lambda y:-3/2,lambda y:3/2)
Out[4]: 1.5
Alternatively, we can compute the expected value with the marginal densities. Note that we can factor the joint density: $\pdfa{x,y}{X,Y}=\frac19=\frac13\frac13=\pdfa{x}{X}\wts\pdfa{y}{Y}$ for $-\frac32\leq x\leq\frac32$ and $-\frac32\leq y\leq\frac32$. Let’s verify the marginals. For $x\in\sbr{-\frac32,\frac32}$, we have:
$$ \pdfa{x}{X}=\int_{-\frac32}^{\frac32}\pdfa{x,y}{X,Y}dy=\frac19\Sbr{y\eval{-\frac32}{\frac32}}=\frac19\wts3=\frac13 $$
And similarly for $\pdfa{y}{Y}$. Hence $X$ and $Y$ are independent and uniformly distributed.
Then equation 2.1 on p.299 gives the first equality and proposition 2.1 on p.191, ch.5 gives the second equality:
$$ \evw{\norm{X}+\norm{Y}}=\evw{\norm{X}}+\evw{\norm{Y}}=\int_{-\frac32}^{\frac32}\norm{x}\pdfa{x}{X}dx+\int_{-\frac32}^{\frac32}\norm{y}\pdfa{y}{Y}dy $$
$$ =\frac13\int_{-\frac32}^{\frac32}\norm{x}dx+\frac13\int_{-\frac32}^{\frac32}\norm{y}dy $$
$$ =\frac13\Cbr{2\int_0^\frac32\norm{x}dx+2\int_0^\frac32\norm{y}dy} \tag{even functions} $$
$$ =\frac23\Cbr{\int_0^\frac32xdx+\int_0^\frac32ydy} $$
$$ =\frac23\Cbr{2\int_0^\frac32xdx}=\frac43\int_0^\frac32xdx=\frac43\frac12\Sbr{x^2\eval0{\frac32}}=\frac23\fracpB32^2=\frac32 $$
(7.6) A fair die is rolled 10 times. Calculate the expected sum of the 10 rolls.
Solution Let $X_i$ denote the value of the $i^{th}$ roll. Then equation 2.2 on p.300 gives the second equality:
$$ \evt{sum of 10 rolls}=\evwB{\sum_{i=1}^{10}X_i}=\sum_{i=1}^{10}\evw{X_i}=\sum_{i=1}^{10}\frac72=35 $$
where we have used the result $\evw{X_i}=\frac72$ from example 3a, p.126, ch.4. Or compute directly:
$$ \E{X_i}=\sum_{j=1}^6j\wts\frac16=\frac16\sum_{j=1}^6j=\frac16\frac{6\wts7}2=\frac72 $$
(7.7.a) Suppose that $A$ and $B$ each randomly and independently choose $3$ of $10$ objects. Find the expected number of objects chosen by both $A$ and $B$.
Solution Let $I_i=1$ if the $i^{th}$ object is chosen by both $A$ and $B$, otherwise $I_i=0$. Then $\sum_{i=1}^{10}I_i$ is the number of objects chosen by both $A$ and $B$. Note that
$$ \evw{I_i}=1\wts\pr{A\text{ chooses }i,B\text{ chooses }i}+0\wts\pr{i\text{ not chosen by both}} $$
$$ =\pr{A\text{ chooses }i}\wts\pr{B\text{ chooses }i} \tag{independence} $$
$$ =\frac3{10}\frac3{10}=\fracpB3{10}^2 $$
Then equation 2.2 on p.300 gives the second equality:
$$ \evt{number of items chosen by both}=\evwB{\sum_{i=1}^{10}I_i}=\sum_{i=1}^{10}\evw{I_i}=10\wts\fracpB3{10}^2=\frac{3^2}{10}=\frac9{10} $$
(7.7.b) Find the expected number of objects not chosen by either $A$ or $B$.
Let $J_i=1$ if the $i^{th}$ object is not chosen by either $A$ or $B$, otherwise $J_i=0$. Then $\sum_{i=1}^{10}J_i$ is the number of objects not chosen by either $A$ or $B$. Note that
$$ \evw{J_i}=1\wts\pr{A\text{ doesn't choose }i,B\text{ doesn't choose }i}+0\wts\pr{i\text{ chosen by } A\text{ or }B\text{ or both}} $$
$$ =\pr{A\text{ doesn't choose }i}\wts\pr{B\text{ doesn't choose }i} \tag{independence} $$
$$ =\fracpB7{10}^2 $$
Then equation 2.2 on p.300 gives the second equality:
$$ \evt{number of items not chosen by either}=\evwB{\sum_{i=1}^{10}J_i}=\sum_{i=1}^{10}\evw{J_i}=10\wts\fracpB7{10}^2=\frac{7^2}{10}=4.9 $$
(7.7.c) Find the expected number of objects chosen by exactly one of $A$ and $B$.
Let $K_i=1$ if the $i^{th}$ object is chosen by exactly one of $A$ and $B$, otherwise $K_i=0$. Then $\sum_{i=1}^{10}K_i$ is the number of objects chosen by exactly one of $A$ and $B$. There are two approaches to solving this problem.
First approach: note that
$$ 10=\text{# of items chosen by both}+\text{# of items chosen by neither}+\text{# of items chosen by exactly 1} $$
$$ =\sum_{i=1}^{10}I_i+\sum_{i=1}^{10}J_i+\sum_{i=1}^{10}K_i $$
Hence
$$ \evt{number of items chosen by exactly 1}=\evwB{\sum_{i=1}^{10}K_i}=\evwB{10-\sum_{i=1}^{10}I_i-\sum_{i=1}^{10}J_i} $$
$$ =10-\evwB{\sum_{i=1}^{10}I_i}-\evwB{\sum_{i=1}^{10}J_i}=10-0.9-4.9=4.2 $$
Alternatively, we can compute this expected value as we did in parts (a) and (b): Note that
$$ \evw{K_i}=1\wts\pr{i\text{ chosen by exactly one}}+0\wts\pr{i\text{ chosen by neither or both}} $$
$$ =\pr{A\text{ chooses }i,B\text{ doesn't choose }i}+\pr{A\text{ doesn't choose }i,B\text{ chooses }i} $$
$$ =\wts\frac3{10}\frac7{10}+\wts\frac7{10}\frac3{10}=2\wts\frac3{10}\frac7{10}=\frac35\frac7{10} $$
Then
$$ \evt{number chosen by exactly 1}=\evwB{\sum_{i=1}^{10}K_i}=\sum_{i=1}^{10}\evw{K_i}=10\wts\frac35\frac7{10}=\frac{21}5=4.2 $$
(7.9.a) A total of $n$ balls, numbered $1$ through $n$, are put into $n$ urns, also numbered $1$ through $n$ in such a way that ball $i$ is equally likely to go into any of the urns $1,2,…,i$. Find the expected number of urns that are empty.
Solution Let $I_j=1$ if urn $j$ is empty, otherwise $I_j=0$. Then $\sum_{j=1}^nI_j$ is the number of urns that are empty. Note that
$$ \pr{\text{ball }i\text{ is not in urn }j}=1-\pr{\text{ball }i\text{ is in urn }j}=\cases{1&i<j\\1-\frac1i&j\leq i} $$
and
$$ \evw{I_j}=\prt{urn j empty}=\pr{\text{for all }i\geq j,\text{ ball }i\text{ is not in urn }j} $$
$$ =\pr{\cap_{i=j}^n\{\text{ball }i\text{ is not in urn }j\}} $$
$$ =\prod_{i=j}^n\pr{\text{ball }i\text{ is not in urn }j} \tag{independence} $$
$$ =\prod_{i=j}^n\Prn{1-\frac1i}=\prod_{i=j}^n\Prn{\frac{i-1}i} $$
$$ =\fracpB{j-1}{j}\fracpB{j}{j+1}\fracpB{j+1}{j+2}\fracpB{j+2}{j+3}\dots\fracpB{n-1}{n}=\frac{j-1}n $$
Hence
$$ \evt{number of empty urns}=\evwB{\sum_{j=1}^nI_j}=\sum_{j=1}^n\evw{I_j}=\sum_{j=1}^n\frac{j-1}n=\frac1n\Prn{\sum_{j=1}^nj-\sum_{j=1}^n1} $$
$$ =\frac1n\Prn{\frac{n(n+1)}2-\frac{2n}2}=\frac{n+1}2-\frac22=\frac{n-1}2 $$
(7.9.b) Find the probability that none of the urns is empty.
Solution Let’s count. Suppose there are $3$ balls and urns. Ball $1$ must go into urn $1$. Suppose ball $2$ goes into urn $1$. Then ball $3$ goes into any of the urns. Then there will be at least $1$ empty urn. But if ball $2$ goes into urn $2$ and ball $3$ goes into urn $3$, then none of the urns are empty.
More generally, we wish to to show that none of the urns are empty IFF each ball $i$ goes into urn $i$. First suppose each ball $i$ goes into urn $i$. Then each urn has a ball in it and none of the urns are empty.
In the other direction, we prove this by induction. Note that ball $1$ always goes into urn $1$. For $n=2$, suppose neither of the two urns are empty. We wish to show that this implies that ball $2$ is in urn $2$. Suppose this is not true. Then both balls are in urn $1$. But then urn $2$ is empty, contradiction. Hence ball $i$ is in urn $i$ for $i\leq2$.
For arbitrary $n$, suppose the claim holds for $n-1$: If none of the first $n-1$ urns are empty, then this implies that every ball $i$ goes into urn $i$ for $i\leq n-1$. Also suppose that none of the $n$ urns are empty. This means that none of the first $n-1$ urns are empty, which in turn means that every ball $i$ goes into urn $i$ for $i\leq n-1$. Since none of the $n$ urns are empty, then this must mean that ball $n$ goes into urn $n$. If that were not the case, then urn $n$ would be empty, contradition. Hence every ball $i$ goes into urn $i$ for $i\leq n$ and we’re done.
Hence
$$ \prt{none empty}=\pr{\text{for all }i\leq n\text{, every ball }i\text{ goes into urn }i} $$
$$ =\pr{\cap_{i=1}^n\set{\text{ball }i\text{ goes into urn }i}} $$
$$ =\prod_{i=1}^n\pr{\text{ball }i\text{ goes into urn }i} \tag{independence} $$
$$ =\prod_{i=1}^n\frac1i=\frac1{n!} $$
(7.10.a) Consider $3$ trials, each having the same probability of success. Let $X$ denote the total number of successes in these trials. If $\E{X}=1.8$, what is the largest possible value of $\pr{X=3}$? Construct a probability scenario that results in $\pr{X=3}$ having the stated value.
Solution Let $X_i=1$ if trial $i$ is a success and $0$ otherwise. Then $\sum_{i=1}^3X_i$ is the number of successes. That is, $X=\sum_{i=1}^3X_i$. Hence
$$ 1.8=\E{X}=\EB{\sum_{j=1}^3X_j}=\sum_{j=1}^3\E{X_j}=\sum_{j=1}^3\prt{trial j success} $$
$$ =3\wts\prt{trial i success}=3\wts\pr{X_i=1}\qd i=1,2,3 $$
Hence $\pr{X_i=1}=\frac{1.8}3=0.6$ for $i=1,2,3,$ and
$$ \pr{X=3}=\pr{X_1=1,X_2=1,X_3=1}\leq\pr{X_i=1}=0.6 $$
where the inequality follows from proposition 4.2, p.29, ch.2. This inequality also follows from the multiplication rule on p.63, ch.3 since
$$ \pr{X_1=1,X_2=1,X_3=1} $$
$$ =\pr{X_1=1}\cp{X_2=1}{X_1=1}\cp{X_3=1}{X_1=1,X_2=1}\leq\pr{X_i=1} $$
Probability scenario: $10$ balls are in an urn. $6$ are red and $4$ are green. We are to pick $3$ balls without replacement. Success is picking a red ball. Whatever color we pick on the first trial, we remove the balls of the other color from the urn for the second and third trials. So if we pick a red ball on the first trial, then we remove all of the green balls for the remaining trials. And vice versa.
First note that
$$ \pr{X_1=1}=0.6\qd\cp{X_2=1}{X_1=1}=1\qd\cp{X_2=1}{X_1=0}=0 $$
Hence
$$ \pr{X_2=1}=\cp{X_2=1}{X_1=0}\pr{X_1=0}+\cp{X_2=1}{X_1=1}\pr{X_1=1} $$
$$ =0\wts0.4+1\wts0.6=0.6 $$
Similarly
$$ \cp{X_3=1}{X_1=1}=1\qd\cp{X_3=1}{X_1=0}=0\qd\pr{X_3=1}=0\wts0.4+1\wts0.6=0.6 $$
So we have shown that $\pr{X_i=1}=0.6$ for $i=1,2,3$. We also want to show that
$$ \cp{X_3=1}{X_1=1,X_2=1}=1 $$
Since $\cp{X_3=1}{X_1=1}=1$, it is sufficient to show that $\set{X_1=1,X_2=1}=\set{X_1=1}$. Clearly $\set{X_1=1,X_2=1}\subset\set{X_1=1}$. In the other direction, suppose $a\in\set{X_1=1}$. Then $a$ is an outcome where the first pick is a red ball. This is implies that the green balls are removed and hence $X_2=1$.
We use the mulitplication rule again:
$$ \pr{X=3}=\pr{X_1=1,X_2=1,X_3=1} $$
$$ =\pr{X_1=1}\cp{X_2=1}{X_1=1}\cp{X_3=1}{X_1=1,X_2=1} $$
$$ =0.6\wts1\wts1=0.6 $$
Hence we have devised a probability scenario where the largest possible value of $\pr{X=3}$ is attained while $\pr{X_i}=0.6$ for $i=1,2,3$.
(7.10.b) If $\E{X}=1.8$, what is the smallest possible value of $\pr{X=3}$? Construct a probability scenario that results in $\pr{X=3}$ having the stated value.
Hint: For part (b), you might start by letting $U$ be a uniform random variable on $(0,1)$ and then defining the trials in terms of the value of $U$.
Solution I think it’s possible for $\pr{X=3}=0$. Define
$$ X_1=\cases{1&0\leq U\leq0.6\\0&\text{otherwise}}\qd X_2=\cases{1&0.4\leq U\leq1\\0&\text{otherwise}}\qd X_3=\cases{1&0\leq U\leq0.3\text{ or }0.7\leq U\leq1\\0&\text{otherwise}} $$
Then $\pr{X=2}=1$ since
$$ 0\leq U\leq0.3\implies X_1=1,X_2=0,X_3=1 $$
$$ 0.3<U<0.7\implies X_1=1,X_2=1,X_3=0 $$
$$ 0.7\leq U\leq1\implies X_1=0,X_2=1,X_3=1 $$
Note that $\set{X=2}\subset\set{X\neq3}$. Hence $1=\pr{X=2}\leq\pr{X\neq3}\leq1$. Hence $\pr{X\neq3}=1$. Hence $\pr{X=3}=0$ and this probability attains it smallest possible value. Also note that $\pr{X_i=1}=0.6$ for $i=1,2,3$.
(7.11) Consider $n$ independent flips of a coin having probability $p$ of landing on heads. Say that a changeover occurs whenever an outcome differs from the one preceding it. For instance, if $n=5$ and the outcome is $HHTHT$, then there are $3$ changeovers. Find the expected number of changeovers.
Hint: Express the number of changeovers as the sum of $n−1$ Bernoulli random variables.
Solution Let $X_i=1$ if a changeover occurs on the $i^{th}$ flip and $0$ otherwise. Notice that $X_1$ is undefined since there is no previous flip. Then $\sum_{i=2}^nX_i$ is the number of changeovers in the $n$ flips. Note that, for $i\geq2$, we have
$$ \evw{X_i}=\pr{\text{changeover occurs on }i^{th}\text{ flip}} $$
$$ =\pr{\text{flip }i-1\text{ is }H,\text{ flip }i\text{ is }T}+\pr{\text{flip }i-1\text{ is }T,\text{ flip }i\text{ is }H} $$
$$ =\pr{\text{flip }i-1\text{ is }H}\pr{\text{flip }i\text{ is }T}+\pr{\text{flip }i-1\text{ is }T}\pr{\text{flip }i\text{ is }H} \tag{independence} $$
$$ =p(1-p)+(1-p)p=2p(1-p)\dq i\geq2 $$
Hence
$$ \evt{number of changeovers in n flips}=\EB{\sum_{i=2}^nX_i}=\sum_{i=2}^n\E{X_i}=(n-1)2p(1-p) $$
(7.14) An urn has $m$ black balls. At each stage, a black ball is removed and a new ball that is black with probability $p$ and white with probability $1−p$ is put in its place. Find the expected number of stages needed until there are no more black balls in the urn.
Solution Let $X$ denote the number of stages needed until there are no more black balls in the run. Then $X$ is a negative binomial random variable with parameters $m$ and $1-p$.
To see this, define a success to be a stage where a white ball replaces a black ball. Then each success means $1$ less black ball in the urn. And there will be no more black balls in the urn exactly when you accumulate $m$ successes. Hence $X$ is the number of independent trials needed to accumulate $m$ successes where a success occurs with probability $1-p$. This matches with the definition of a negative binomial random variable with parameters $m$ and $1-p$.
We know from example 8f, p.159, ch.4, that the expected value of $X$ is $\frac{m}{1-p}$.
(7.15) In Example 2h, say that $i$ and $j$, $i\neq j$, form a matched pair if $i$ chooses the hat belonging to $j$ and $j$ chooses the hat belonging to $i$. Find the expected number of matched pairs.
Solution For $i\neq j$, define $X_{i,j}$ as
$$ X_{i,j}=\cases{1&i,j\text{ form a matched pair}\iff i\text{ picks }j\text{'s hat and }j\text{ picks }i\text{'s hat}\\0&\text{otherwise}} $$
Hence
$$ \E{X_{i,j}}=\pr{i\text{ picks }j\text{'s hat, }j\text{ picks }i\text{'s hat}}=\cp{i\text{ picks }j\text{'s hat}}{j\text{ picks }i\text{'s hat}}\pr{j\text{ picks }i\text{'s hat}} $$
The first probability on the right side is $\frac1{N-1}$ since $i$ has $N-1$ hats to pick from after $j$ has picked $i$’s hat. The second probability is $\frac1N$. Hence
$$ \E{X_{i,j}}=\frac1{N(N-1)} $$
Notice that $\sum_{i\neq j}^NX_{i,j}$ is not the number of matched pairs. The reason is that we want combinations of $i$ and $j$, not permutations. For example, we would double count if we counted both $X_{7,13}$ and $X_{13,7}$ because both random variables are $1$ when persons $7$ and $13$ select each other’s hats and $0$ otherwise. That is, they indicate the same event.
Instead, the sum that represents the number of matched pairs is
$$ \text{number of matched pairs}=\sum_{i=1,i<j}^NX_{i,j}=\sum_{i<j}^NX_{i,j} $$
For instance, if $N=4$, then we have
$$ \sum_{i<j}^4X_{i,j}=X_{1,2}+X_{1,3}+X_{1,4}+X_{2,3}+X_{2,4}+X_{3,4} $$
This is the sum over all $6=\binom42$ combinations of the indices $i,j$. Hence
$$ \evt{number of matched pairs}=\EB{\sum_{i<j}^NX_{i,j}}=\sum_{i<j}^N\E{X_{i,j}} $$
$$ =\binom{N}2\frac1{N(N-1)}=\frac{N(N-1)}2\frac1{N(N-1)}=\frac12 $$
(7.16) Let $Z$ be a standard normal random variable, and, for a fixed $x$, set
$$ X=\cases{Z&Z>x\\0&\text{otherwise}} $$
Show that $\E{X}=\frac1{\sqrt{2\pi}}\e{-\frac{x^2}2}$.
Solution Define $g(z)=z$ if $z>x$ and $g(z)=0$ for $z\leq x$. Then $X=g(Z)$ and proposition 2.1 on p.191, ch.5 gives
$$ \E{X}=\E{g(Z)}=\int_{-\infty}^{\infty}g(z)\wts\pdfa{z}{Z}dz=\int_{-\infty}^x0\wts\pdfa{z}{Z}dz+\int_x^{\infty}z\wts\pdfa{z}{Z}dz $$
$$ =\frac1{\sqrt{2\pi}}\int_x^{\infty}z\wts\e{-\frac{z^2}2}dz $$
$$ u=-\frac{z^2}2\dq du=-\frac12\wts2zdz=-zdz\dq \e{u}du=-\e{-\frac{z^2}2}zdz=-z\e{-\frac{z^2}2}dz $$
$$ u_0=-\frac{z_0^2}2=-\frac{x^2}2\dq u_1=-\frac{z_1^2}2=-\frac{\infty^2}2=-\infty $$
$$ =\frac1{\sqrt{2\pi}}\int_x^{\infty}z\wts\e{-\frac{z^2}2}dz=-\frac1{\sqrt{2\pi}}\int_{-\frac{x^2}2}^{-\infty}\e{u}du $$
$$ =\frac1{\sqrt{2\pi}}\int_{-\infty}^{-\frac{x^2}2}\e{u}du=\frac1{\sqrt{2\pi}}\Sbr{\e{u}\eval{-\infty}{-\frac{x^2}2}}=\frac1{\sqrt{2\pi}}\e{-\frac{x^2}2} $$
(7.17.a) A deck of $n$ cards numbered $1$ through $n$ is thoroughly shuffled so that all possible $n!$ orderings can be assumed to be equally likely. Suppose you are to make $n$ guesses sequentially, where the $i^{th}$ one is a guess of the card in position $i$. Let $N$ denote the number of correct guesses.
If you are not given any information about your earlier guesses show that, for any strategy, $\E{N}=1$.
Hint: For all parts, express $N$ as the sum of indicator (that is, Bernoulli) random variables.
Solution Define
$$ N_i=\cases{1&\text{guess }i\text{ is correct}\\0&\text{otherwise}} $$
Then $\sum_{i=1}^nN_i$ is the number of correct guesses. Hence $N=\sum_{i=1}^nN_i$ and
$$ \E{N}=\EB{\sum_{i=1}^nN_i}=\sum_{i=1}^n\E{N_i}=n\wts\frac1n=1 $$
since $\E{N_i}=\pr{N_i=1}=\pr{\text{guess }i\text{ is correct}}=\frac1n$. Note that we made no assumptions about the guessing strategy. Hence this expected value holds for any strategy.
(7.17.b) Suppose that after each guess you are shown the card that was in the position in question. What do you think is the best strategy? Show that, under this strategy,
$$ \E{N}=\frac1n+\frac1{n-1}+\dots+1\approx\int_1^n\frac{dx}x=\ln{n} $$
Solution Let’s count. The probability of guessing the first card correctly is just $\frac1n$. We are now shown the first card. Hence we don’t want to guess that card for the remaining positions.
Hence, for the second guess, we have $n-1$ cards to guess from. Hence we have a probability of $\frac1{n-1}$ of correctly guessing the second card. We are now shown the second card. Hence, for the remaining guesses, we don’t want to guess the cards in positions $1$ and $2$.
Hence, for the third guess, we have $n-2$ cards to guess from. Hence we have a probability of $\frac1{n-2}$ of correctly guessing the third card. We are now shown the third card. Hence, for the remaining guesses, we don’t want to guess the cards in positions $1$, $2$, and $3$.
Hence, the best strategy is to always guess a card which has not yet appeared. For this strategy, the $i^{th}$ guess will be correct with probability $\frac1{n-i+1}$. Hence
$$ \E{N_i}=\pr{N_i=1}=\pr{\text{guess }i\text{ is correct}}=\frac1{n-i+1} $$
and
$$ \E{N}=\EB{\sum_{i=1}^nN_i}=\sum_{i=1}^n\E{N_i}=\sum_{i=1}^n\frac1{n-i+1}\approx\int_1^n\frac{du}{n-u+1} $$
$$ x=n-u+1\dq u=n-x+1\dq du=-dx $$
$$ x_0=n-u_0+1=n-1+1=n\dq x_1=n-u_1+1=n-n+1=1 $$
$$ \E{N}=\sum_{i=1}^n\frac1{n-i+1}\approx\int_1^n\frac{du}{n-u+1}=\int_n^1\frac{-dx}x=\int_1^n\frac{dx}x=\ln{n}-\ln{1}=\ln{n} $$
(7.17.c) Suppose that you are told after each guess whether you are right or wrong. In this case, it can be shown that the strategy which maximizes $\E{N}$ is one that keeps on guessing the same card until you are told you are correct and then changes to a new card. For this strategy, show that
$$ \E{N}=1+\frac1{2!}+\frac1{3!}+\dots+\frac1{n!}\approx e-1 $$
Solution I don’t understand the stated solution in the solutions manual.
(7.18) Cards from an ordinary deck of $52$ playing cards are turned face up one at a time. If the $1^{st}$ card is an ace, or the $2^{nd}$ a deuce, or the $3^{rd}$ a three, or …, or the $13^{th}$ a king, or the $14^{th}$ an ace, and so on, we say that a match occurs. Note that we do not require that the $(13n+1)^{th}$ card be any particular ace for a match to occur but only that it be an ace. Compute the expected number of matches that occur.
Solution Define
$$ I_i=\cases{1&\text{match on card }i\\0&\text{otherwise}} $$
Note that $\pr{I_1=1}=\frac4{52}=\frac1{13}$ since there are $4$ aces in a deck of $52$. Similarly $\pr{I_{13}=1}=\frac4{52}=\frac1{13}$ since there are $4$ kings in a deck of $52$. Etc. That is, $\pr{I_i=1}=\frac1{13}$ for $1\leq i\leq52$.
Also note that $\sum_{i=1}^{52}I_i$ is the number of matches that occur. Hence
$$ \evt{number of matches}=\EB{\sum_{i=1}^{52}I_i}=\sum_{i=1}^{52}\E{I_i}=52\wts\frac1{13}=4 $$
(7.19.a) A certain region is inhabited by $r$ distinct types of a certain species of insect. Each insect caught will, independently of the types of the previous catches, be of type $i$ with probability
$$ P_i,i=1,...,r\dq \sum_{i=1}^rP_i=1 $$
Compute the mean number of insects that are caught before the first type $1$ catch.
Solution Let $T$ denote the number of catches until the first type $1$ catch. Then $T$ is a geometric random variable with parameter $P_1$.
To see this, define a trial to be the observed type of a catch. Note that the trials are independent since the catches are type independent. Define a success to be the occurence that the observed type is type $1$. Then the probability of success is $P_1$.
Then, by performing catches until the observed type is type $1$, we are performing independent trials, each having a probability $P_1$ of success, until a success occurs. Example 8b, p.156, ch.4 gives the expected value for a geometric random variable with parameter $P_1$: $\E{T}=\frac1{P_1}$.
Note that $T-1$ is number of insects that are caught before the first type $1$ catch. Hence
$$ \evt{number of insects caught before first type 1}=\E{T-1}=\frac1{P_1}-1 $$
(7.19.b) Compute the mean number of types of insects that are caught before the first type $1$ catch.
Solution Let $B_j$ denote the event that type $j$ is caught before the first type $1$ is caught. Let $F_j$ denote the number of the catch that is the first time that either type $j$ or type $1$ is caught.
Suppose, for some $i$, that the $i^{th}$ catch is the first time that either type $j$ or type $1$ is caught. Given $F_j=i$, the first type $j$ will be caught before the first type $1$ IFF type $j$ is caught on this $i^{th}$ catch. The probability that type $j$ will be caught on this $i^{th}$ catch, given that it’s either type $j$ or type $1$, is $\frac{P_j}{P_j+P_1}$. That is
$$ \cp{B_j}{F_j=i}=\cp{\text{catch }i\text{ is type }j}{\text{catch }i\text{ is type }j\text{ or }1}=\frac{P_j}{P_j+P_1} $$
Also note that $F_j$ is a geometric random variable with parameter $p\equiv P_j+P_1$. As shown on p.155:
$$ \sum_{i=1}^\infty\pr{F_j=i}=p\sum_{i=1}^\infty(1-p)^{i-1}=\frac{p}{1-(1-p)}=1 $$
Since the $\set{F_j=i}$ are mutually exclusive, we have
$$ \pr{B_j}=\sum_{i=1}^{\infty}\cp{B_j}{F_j=i}\pr{F_j=i}=\frac{P_j}{P_j+P_1}\sum_{i=1}^{\infty}\pr{F_j=i}=\frac{P_j}{P_j+P_1} $$
For $2\leq j\leq r$, define
$$ I_j=\cases{1&B_j\text{ occurs}\\0&\text{otherwise}} $$
Note that $\sum_{j=2}^rI_j$ is the number of types caught before the first type $1$ catch. Hence
$$ \evt{# types caught before first type 1}=\EB{\sum_{j=2}^rI_j}=\sum_{j=2}^r\E{I_j}=\sum_{j=2}^r\pr{B_j}=\sum_{j=2}^r\frac{P_j}{P_j+P_1} $$
(7.20) In an urn containing $n$ balls, the $i^{th}$ ball has weight $W(i),i=1,…,n$. The balls are removed without replacement, one at a time, according to the following rule: At each selection, the probability that a given ball in the urn is chosen is equal to its weight divided by the sum of the weights remaining in the urn. For instance, if at some time $i_1,…,i_r$ is the set of balls remaining in the urn, then the next selection will be $i_j$ with probability $\frac{W(i_j)}{\sum_{k=1}^rW(i_k)}$, $j=1,…,r$. Compute the expected number of balls that are withdrawn before ball number 1 is removed.
Solution Almost identical to problem 19.b. Let $B_j$ denote the event that ball $j$ is picked before ball $1$. Let $F_j$ denote the number of the first draw where either ball $j$ or ball $1$ is picked. And let $R_i$ denote the sum of the remaining weights after the $i^{th}$ pick:
$$ R_i=\sum_{k=1}^{n-i}W(j_k)\qd\text{balls }j_1,...,j_{n-i}\text{ remaining} $$
Suppose, for some $i$, that the $i^{th}$ draw is the first draw where either ball $j$ or ball $1$ is picked. Given $F_j=i$, ball $j$ will be picked before ball $1$ IFF ball $j$ is picked on this $i^{th}$ draw. The probability that ball $j$ will be picked on this $i^{th}$ draw, given that it’s either ball $j$ or ball $1$, is $\frac{W(j)/R_i}{W(j)/R_i+W(1)/R_i}=\frac{W(j)}{W(j)+W(1)}$. That is
$$ \cp{B_j}{F_j=i}=\cp{\text{draw }i\text{ is ball }j}{\text{draw }i\text{ is ball }j\text{ or }1}=\frac{W(j)}{W(j)+W(1)} $$
Also note that $F_j$ is a geometric random variable with parameter $p\equiv W(j)+W(1)$. As shown on p.155:
$$ \sum_{i=1}^\infty\pr{F_j=i}=p\sum_{i=1}^\infty(1-p)^{i-1}=\frac{p}{1-(1-p)}=1 $$
Since the $\set{F_j=i}$ are mutually exclusive, we have
$$ \pr{B_j}=\sum_{i=1}^{\infty}\cp{B_j}{F_j=i}\pr{F_j=i}=\frac{W(j)}{W(j)+W(1)}\sum_{i=1}^{\infty}\pr{F_j=i}=\frac{W(j)}{W(j)+W(1)} $$
For $2\leq j\leq n$, define
$$ I_j=\cases{1&B_j\text{ occurs}\\0&\text{otherwise}} $$
Note that $\sum_{j=2}^nI_j$ is the number of balls picked before ball $1$. Hence
$$ \evt{# balls picked before ball 1}=\EB{\sum_{j=2}^nI_j}=\sum_{j=2}^n\E{I_j}=\sum_{j=2}^n\pr{B_j}=\sum_{j=2}^n\frac{W(j)}{W(j)+W(1)} $$
(7.21.a) For a group of $100$ people, compute the expected number of days of the year that are birthdays of exactly $3$ people.
Solution Let’s assume the people are chosen randomly from a general population so that birthdays are independent.
Let $X_i=1$ if exactly $3$ people have birthdays on day $i$ of the year, otherwise $0$. Define the observation of each person’s birthday as a trial. Define success as that person’s birthday is day $i$. Then the probability of success is $p\equiv\frac1{365}$.
Hence $X_i$ is a binomial random variable and $\E{X_i}=\binom{100}3p^3(1-p)^{97}$. Note that $\sum_{i=1}^{365}X_i$ is the number of days of the year that are birthdays of exactly $3$ people. Hence
$$ \evt{# bdays of exactly 3}=\EB{\sum_{i=1}^{365}X_i}=\sum_{i=1}^{365}\E{X_i}=365\wts\binom{100}3p^3(1-p)^{97} $$
(7.21.b) For a group of $100$ people, compute the expected number of distinct birthdays.
Solution Again assume the people are chosen randomly from a general population so that birthdays are independent.
Let $X_i=1$ if zero people have b-days on day $i$ of the year, otherwise $0$. Define $Y_i=1$ if at least one person has a b-day on day $i$ of the year, otherwise $0$. Note that $\set{Y_i=1}=\set{X_i=1}^c$. That is, if it’s not the case that zero people have b-days on day $i$, then at least one person does. And vice versa. Hence
$$ \E{Y_i}=\pr{Y_i=1}=\prb{\set{X_i=1}^c}=1-\pr{X_i=1}=1-\binom{100}0p^0(1-p)^{100}=1-(1-p)^{100} $$
$$ \evt{# distinct bdays}=\EB{\sum_{i=1}^{365}Y_i}=\sum_{i=1}^{365}\E{Y_i}=365\wts\prn{1-(1-p)^{100}} $$
In [785]: p=1/365
In [786]: 365*winom(100,3)*pw(p,3)*pw(1-p,97)
Out[786]: 0.930144993183940
In [787]: 365*(1-pw(1-p,100))
Out[787]: 87.575518057196177
(7.22) How many times would you expect to roll a fair die before all $6$ sides appeared at least once?
Solution Notice how similar this is to a negative binomial random variable. If the wording had been “all $6$ sides appeared SEQUENTIALLY and at least once”, then this would have been a nbrv.
As stated, this is a coupon collecting problem. Define a side to be a coupon type and a roll to be a single coupon collection. This particular coupon collecting problem asks “how many coupons must we collect to collect all of the types?”
This problem is identical to example 2i, p.303. Using the formula from that example, we compute
$$ \E{X}=\sum_{i=0}^5\frac6{6-i} $$
In [838]: sum(6/(6-i) for i in range(6))
Out[838]: 14.7
Also note example 1e, ch.4, p.120-121. In that example, $T$ denotes the number of coupons that need to be collected until one obtains a complete set of at least one of each type. So this is the same problem. Two formulas are then derived:
$$ \pr{T>n}=\sum_{i=1}^{N-1}\binom{N}i\fracpB{N-i}N^n(-1)^{i+1} $$
and
$$ \pr{T=n}=\pr{T>n-1}-\pr{T>n} $$
In [850]: w1=lambda N=6,n=1:sum(winom(N,i)*pw((N-i)/N,n)*pw(-1,i+1) for i in range(1,N))
In [851]: w2=lambda N=6,n=1:w1(N=N,n=n-1)-w1(N=N,n=n)
In [852]: sum(i*w2(n=i) for i in range(6,210))
Out[852]: 14.7000000000000
(7.23) Urn $1$ contains $5$ white and $6$ black balls, while urn $2$ contains $8$ white and $10$ black balls. Two balls are randomly selected from urn $1$ and are put into urn $2$. If $3$ balls are then randomly selected from urn $2$, compute the expected number of white balls in the trio.
Hint: Let $X_i=1$ if the $i^{th}$ white ball initially in urn $1$ is one of the three selected, and let $X_i=0$ otherwise. Similarly, let $Y_i=1$ if the $i^{th}$ white ball from urn $2$ is one of the three selected, and let $Y_i=0$ otherwise. The number of white balls in the trio can now be written as $\sum_1^5X_i+\sum_1^8Y_i$.
Solution We have $\pr{Y_i=1}=\frac3{20}$. Let $B_{i,j}$ denote the event that ball $i$ is selected from urn $j$. Then $\set{X_i=1}=B_{i,2}$ and
$$ \pr{X_i=1}=\cp{B_{i,2}}{B_{i,1}}\pr{B_{i,1}}+\cp{B_{i,2}}{B_{i,1}^c}\pr{B_{i,1}^c}=\frac3{20}\frac2{11}+0\wts\frac{17}{20}=\frac3{11\wts10} $$
Hence
$$ \EB{\sum_1^5X_i+\sum_1^8Y_i}=\sum_1^5\E{X_i}+\sum_1^8\E{Y_i}=\frac{5\wts3}{11\wts10}+\frac{24}{20}=\frac{15}{110}+\frac65=\frac{15+132}{110}=\frac{147}{110} $$
(7.24.a) A bottle initially contains m large pills and n small pills. Each day, a patient randomly chooses one of the pills. If a small pill is chosen, then that pill is eaten. If a large pill is chosen, then the pill is broken in two; one part is returned to the bottle (and is now considered a small pill) and the other part is then eaten.
Solution This problem (known as the problem of the Big Pills and Little Pills) was proposed by Donald E. Knuth and John McCarthy in 1991 in the American Mathematical Monthly. There are a few published solutions that can be found online: Brennan-Prodinger, Yale-251, Arxiv
(7.24.b) Let $Y$ denote the day on which the last large pill is chosen. Find $\E{Y}$.
Hint: What is the relationship between $X$ and $Y$?
Solution See the links in part (a).
(7.25) Let $X_1,X_2,…$ be a sequence of independent and identically distributed continuous random variables. Let $N\geq2$ be such that
$$ X_1\geq X_2\geq\dots\geq X_{N-1}<X_N $$
That is, $N$ is the point at which the sequence stops decreasing. Show that $\E{N}=e$.
Hint : First find $\pr{N\geq n}$.
Solution Let’s first show that the sets $\set{N\geq n}$ and $\set{X_1\geq X_2\geq\dots\geq X_{n-1}}$ are equivalent.
Suppose that $\set{X_1\geq X_2\geq\dots\geq X_{n-1}}$ occurs. If $N<n$, then it must be that $X_{i-1}<X_i$ for some $i<n$. But the supposition says that $X_{i-1}\geq X_i$ for all $i\leq n-1<n$. Contradiction.
In the other direction, suppose that $\set{N\geq n}$ occurs. If $X_{i-1}<X_i$ for some $i\leq n-1<n\leq N$, then an increase in the sequence occurred before the $N^{th}$ trial. Contradiction.
Hence $\set{N\geq n}=\set{X_1\geq X_2\geq\dots\geq X_{n-1}}$.
Also, since the $\set{X_i,i=1,…,n-1}$ are IID, then, by symmetry, each of the $(n−1)!$ possible orderings of $X_1,X_2,…,X_{n-1}$ is equally likely to occur. Hence the probability of each possible ordering is $\frac1{(n-1)!}$ and
$$ \pr{N\geq n}=\pr{X_1\geq X_2\geq\dots\geq X_{n-1}}=\frac1{(n-1)!} $$
Alternatively, recall that we explicitly computed $\pr{X_1\geq X_2\geq\dots\geq X_{n-1}}=\frac1{(n-1)!}$ in theoretical exercise 6.11.
Hence
$$ \pr{N=n}=\pr{N\geq n}-\pr{N>n}=\pr{N\geq n}-\pr{N\geq n+1} $$
$$ =\frac1{(n-1)!}-\frac1{(n+1-1)!}=\frac{n}{n!}-\frac1{n!}=\frac{n-1}{n!} $$
And
$$ \E{N}=\sum_{n=2}^{\infty}n\pr{N=n}=\sum_{n=2}^{\infty}n\wts\frac{n-1}{n!}=\sum_{n=2}^{\infty}\frac{n-1}{(n-1)!}=\sum_{n=2}^{\infty}\frac1{(n-2)!}=\sum_{j=0}^{\infty}\frac1{j!}=e $$
In the next-to-last equality, we reindexed $j=n-2$.
Alternatively, recall that, in theoretical exercise 4.4, we proved that $\E{N}=\sum_{i=n}^{\infty}\pr{N\geq n}$ for any nonnegative integer-valued random variable $N$. Hence
$$ \E{N}=\sum_{n=1}^{\infty}\pr{N\geq n}=\sum_{n=1}^{\infty}\frac1{(n-1)!}=\sum_{j=0}^{\infty}\frac1{j!}=e $$
where we reindexed $j=n-1$ and used $\pr{N\geq 1}\equiv1=\frac1{(1-1)!}$.
(7.26.a) If $X_1,X_2,…,X_n$ are independent and identically distributed random variables having uniform distributions over $(0, 1)$, find $\E{\max(X_1,…,X_n)}$.
Solution Define $\bfX=(X_1,…,X_n)$. Let’s first show that, for any $x\in\wR$, the events $\set{\max(\bfX)<x}$ and $\cap_{i=1}^n\set{X_i<x}$ are equivalent. Notice that the latter event can also be written as $\set{X_i<x\text{ for all }1\leq i\leq n}$.
Suppose that $\set{\max(\bfX)<x}$ occurs. Then the largest of the $\set{X_i}$ is smaller than $x$. Hence all of the $\set{X_i}$ are smaller than $x$. Conversely, suppose that all of the $\set{X_i}$ are smaller than $x$. Then the max of the $\set{X_i}$ must be smaller than $x$. Hence these events are equivalent and
$$ \pr{\max(\bfX)<x}=\pr{\cap_{i=1}^n\set{X_i<x}}=\prod_{i=1}^n\pr{X_i<x}=\cases{0&x\leq0\\x^n&x\in(0,1)\\1&x\geq1} $$
where the second equality follows from the independence of the $\set{X_i}$ and last equality follows from the $\set{X_i}$ being uniform on $(0,1)$.
In particular, $\pr{\max(\bfX)<0}=0$ and $\pr{\max(\bfX)>1}=1-\pr{\max(\bfX)<1}=1-1=0$. That is, $\max(\bfX)$ is a nonnegative random variable that takes a value on $(0,1)$. Then by Lemma 2.1, p.191-192, ch.5, we have
$$ \E{\max(\bfX)}=\int_0^\infty\pr{\max(\bfX)>x}dx=\int_0^1(1-x^n)dx $$
$$ =\int_0^1dx-\int_0^1x^ndx=1-\frac1{n+1}\Sbr{x^{n+1}\eval01}=\frac{n+1}{n+1}-\frac1{n+1}=\frac{n}{n+1} $$
(7.26.b) If $X_1,X_2,…,X_n$ are independent and identically distributed random variables having uniform distributions over $(0, 1)$, find $\E{\min(X_1,…,X_n)}$.
Solution Define $\bfX=(X_1,…,X_n)$. Let’s first show that, for any $x\in\wR$, the events $\set{\min(\bfX)>x}$ and $\cap_{i=1}^n\set{X_i>x}$ are equivalent. Notice that the latter event can also be written as $\set{X_i>x\text{ for all }1\leq i\leq n}$.
Suppose that $\set{\min(\bfX)>x}$ occurs. Then the smallest of the $\set{X_i}$ is bigger than $x$. Hence all of the $\set{X_i}$ are bigger than $x$. Conversely, suppose that all of the $\set{X_i}$ are bigger than $x$. Then the min of the $\set{X_i}$ must be bigger than $x$. Hence these events are equivalent and
$$ \pr{\min(\bfX)>x}=\pr{\cap_{i=1}^n\set{X_i>x}}=\prod_{i=1}^n\pr{X_i>x}=\cases{1&x\leq0\\(1-x)^n&x\in(0,1)\\0&x\geq1} $$
where the second equality follows from the independence of the $\set{X_i}$ and last equality follows from the $\set{X_i}$ being uniform on $(0,1)$.
In particular, $\pr{\min(\bfX)<0}=1-\pr{\min(\bfX)>0}=1-1=0$ and $\pr{\min(\bfX)>1}=0$. That is, $\min(\bfX)$ is a nonnegative random variable that takes a value on $(0,1)$. Then by Lemma 2.1, p.191-192, ch.5, we have
$$ \E{\min(\bfX)}=\int_0^\infty\pr{\min(\bfX)>x}dx=\int_0^1(1-x)^ndx $$
$$ u=1-x\dq x=1-u\dq dx=-du\dq (1-x)^ndx=-u^ndu $$
$$ u_0=1-x_0=1-0=1\dq u_1=1-x_1=1-1=0 $$
$$ =-\int_1^0u^ndu=\int_0^1u^ndu=\frac1{n+1}\Sbr{u^{n+1}\eval01}=\frac1{n+1} $$
(7.30) If $X$ and $Y$ are independent and identically distributed with mean $\mu$ and variance $\sigma^2$, find $\E{(X-Y)^2}$.
Solution Note that
$$ \E{X-Y}=\E{X}-\E{Y}=\mu-\mu=0 $$
Hence
$$ \E{(X-Y)^2}=\E{(X-Y)^2}-\prn{\E{X-Y}}^2=\varw{X-Y}=\varw{X}+\varw{-Y}=2\sigma^2 $$
The next-to-last equality follows from equation 4.1, p.324 and the independence of $X$ and $Y$. The last equality follows from $\varw{aY}=a^2\varw{Y}$.
(7.31) In Problem 6, calculate the variance of the sum of the rolls.
(7.6) A fair die is rolled 10 times. Calculate the expected sum of the 10 rolls.
Solution Let $X_i$ denote the value of the $i^{th}$ roll. Then equation 2.2 on p.300 gives the second equality:
$$ \evt{sum of 10 rolls}=\evwB{\sum_{i=1}^{10}X_i}=\sum_{i=1}^{10}\evw{X_i}=\sum_{i=1}^{10}\frac72=35 $$
where we have used the result $\evw{X_i}=\frac72$ from example 3a, p.126, ch.4. Or compute directly:
$$ \E{X_i}=\sum_{j=1}^6j\wts\frac16=\frac16\sum_{j=1}^6j=\frac16\frac{6\wts7}2=\frac72 $$
Since the $X_i$ are pairwise independent, then we have
$$ \vart{sum of 10 rolls}=\varB{\sum_{i=1}^{10}X_i}=\sum_{i=1}^{10}\varw{X_i} $$
Note that
$$ \V{X_i}=\E{X_i^2}-\prn{\E{X_i}}^2=\sum_{j=1}^6j^2\wts\frac16-\fracpB72^2=2.91\overline{6} $$
In [108]: sum(j*j/6 for j in range(1,7))-pw(7/2,2)
Out[108]: 2.9166666666666679
In [109]: 10*(sum(j*j/6 for j in range(1,7))-pw(7/2,2))
Out[109]: 29.166666666666679
$$ \vart{sum of 10 rolls}=10\wts\V{X_1}=29.1\overline{6} $$
(7.32) In Problem 9, compute the variance of the number of empty urns.
(7.9.a) A total of $n$ balls, numbered $1$ through $n$, are put into $n$ urns, also numbered $1$ through $n$ in such a way that ball $i$ is equally likely to go into any of the urns $1,2,…,i$. Find the expected number of urns that are empty.
Solution Let $I_j=1$ if urn $j$ is empty, otherwise $I_j=0$. Then $\sum_{j=1}^nI_j$ is the number of urns that are empty. Note that
$$ \pr{\text{ball }i\text{ is not in urn }j}=1-\pr{\text{ball }i\text{ is in urn }j}=\cases{1&i<j\\1-\frac1i&j\leq i} $$
and
$$ \evw{I_j}=\prt{urn j empty}=\pr{\text{for all }i\geq j,\text{ ball }i\text{ is not in urn }j} $$
$$ =\pr{\cap_{i=j}^n\{\text{ball }i\text{ is not in urn }j\}} $$
$$ =\prod_{i=j}^n\pr{\text{ball }i\text{ is not in urn }j} \tag{independence} $$
$$ =\prod_{i=j}^n\Prn{1-\frac1i}=\prod_{i=j}^n\Prn{\frac{i-1}i} $$
$$ =\fracpB{j-1}{j}\fracpB{j}{j+1}\fracpB{j+1}{j+2}\fracpB{j+2}{j+3}\dots\fracpB{n-1}{n}=\frac{j-1}n $$
Hence
$$ \evt{number of empty urns}=\evwB{\sum_{j=1}^nI_j}=\sum_{j=1}^n\evw{I_j}=\sum_{j=1}^n\frac{j-1}n=\frac1n\Prn{\sum_{j=1}^nj-\sum_{j=1}^n1} $$
$$ =\frac1n\Prn{\frac{n(n+1)}2-\frac{2n}2}=\frac{n+1}2-\frac22=\frac{n-1}2 $$
Note that for $j<k$:
$$ \pr{\text{ball }i\text{ is not in urns }j\text{ or }k}=1-\prb{\set{\text{ball }i\text{ is not in urns }j\text{ or }k}^c} $$
$$ =1-\pr{\text{ball }i\text{ is in urns }j\text{ or }k}=\cases{1&i<j\\1-\frac1i&j\leq i<k\\1-\Prn{\frac1i+\frac1i}&k\leq i} \tag{7.9.a.1} $$
Also note that the variable $I_jI_k$ is an indicator on the event $\settx{urns j and k empty}$. Hence, for $j<k$, we have
$$ \evw{I_jI_k}=\prt{urns j and k empty} $$
$$ =\pr{\text{for all }1\leq i\leq n,\text{ ball }i\text{ is not in urns }j\text{ or }k} $$
$$ =\pr{\cap_{i=1}^n\{\text{ball }i\text{ is not in urns }j\text{ or }k\}} $$
$$ =\prod_{i=1}^n\pr{\text{ball }i\text{ is not in urns }j\text{ or }k} \tag{independence} $$
$$ =\prod_{i=1}^{j-1}\pr{\text{ball }i\text{ not in urns }j\text{ or }k}\prod_{i=j}^{k-1}\pr{\text{ball }i\text{ not in urns }j\text{ or }k}\prod_{i=k}^n\pr{\text{ball }i\text{ not in urns }j\text{ or }k} $$
$$ =\prod_{i=1}^{j-1}(1)\prod_{i=j}^{k-1}\Prn{1-\frac1i}\prod_{i=k}^n\Prn{1-\frac2i} \tag{by 7.9.a.1} $$
$$ =\prod_{i=j}^{k-1}\Prn{1-\frac1i}\prod_{i=k}^n\Prn{1-\frac2i} $$
$$ =\prod_{i=j}^{k-1}\Prn{\frac{i-1}i}\prod_{i=k}^n\Prn{\frac{i-2}i} $$
$$ =\fracpB{j-1}{j}\fracpB{j}{j+1}\fracpB{j+1}{j+2}\dots\fracpB{k-2}{k-1}\wts\fracpB{k-2}{k}\fracpB{k-1}{k+1}\fracpB{k}{k+2}\fracpB{k+1}{k+3}\dots\fracpB{n-2}{n} $$
$$ =\frac{(j-1)(k-2)}{n(n-1)} $$
Hence, for $j<k$, we have
$$ \covw{I_j}{I_k}=\E{I_jI_k}-\E{I_j}\E{I_k}=\frac{(j-1)(k-2)}{n(n-1)}-\frac{j-1}n\frac{k-1}n $$
$$ =\frac{j-1}n\Prn{\frac{k-2}{n-1}-\frac{k-1}n} $$
$$ =\frac{j-1}n\Prn{\frac{nk-n2}{n(n-1)}-\frac{(n-1)(k-1)}{n(n-1)}} $$
$$ =\frac{j-1}n\Prn{\frac{nk-2n}{n(n-1)}-\frac{nk-n-k+1}{n(n-1)}} $$
$$ =\frac{j-1}n\frac{-2n+n+k-1}{n(n-1)} $$
$$ =\frac{j-1}n\frac{k-n-1}{n(n-1)} $$
Also note that $I_j^2=I_j$ since $I_j$ is just an indicator variable. Hence
$$ \V{I_j}=\E{I_j^2}-\prn{\E{I_j}}^2=\E{I_j}-\prn{\E{I_j}}^2=\E{I_j}\prn{1-\E{I_j}} $$
$$ =\frac{j-1}n\Prn{1-\frac{j-1}n}=\frac{j-1}n\Prn{\frac{n}n-\frac{j-1}n}=\frac{j-1}n\frac{n-j+1}n $$
Hence
$$ \vart{number of empty urns}=\varB{\sum_{j=1}^nI_j}=\sum_{j=1}^n\varw{I_j}+2\mathop{\sum\sum}_{j<k}^n\covw{I_j}{I_k} $$
$$ =\sum_{j=1}^n\Sbr{\frac{j-1}n\frac{n-j+1}n}+2\mathop{\sum\sum}_{j<k}^n\Sbr{\frac{j-1}n\frac{k-n-1}{n(n-1)}} $$
In [902]: w732ex=lambda n=7: (n-1)/2
In [903]: w732vr=lambda n=7: sum((j-1)/n*(n-j+1)/n for j in range(1,n+1))+2*sum(sum((j-1)/n*(k-n-1)/(n*(n-1)) for k in
...: range(j+1,n+1)) for j in range(1,n+1))
In [904]: w732ex(),w732vr()
Out[904]: (3.0, 0.6666666666666667)
In [905]: w732ex(14),w732vr(14)
Out[905]: (6.5, 1.2500000000000002)
In [906]: w732ex(23),w732vr(23)
Out[906]: (11.0, 2.0)
We note that for $j<k$:
$$ \covw{I_j}{I_k}=\frac{j-1}n\frac{k-n-1}{n(n-1)}\cases{=0&j=1\\<0&\text{otherwise, since }k\leq n} \tag{7.9.a.2} $$
That is, $I_j$ and $I_k$ are inversely correlated. That is, if urn $j$ is empty, then it is less likely that urn $k$ is empty (relative to the unconditional probability of urn $k$ being empty). That is
$$ \cpt{urn k empty}{urn j empty}<\prt{urn k empty} $$
We can rigorously show this. Recall example 4d on p.329.
$$ 0>\covw{I_j}{I_k}=\E{I_jI_k}-\E{I_j}\E{I_k} $$
$$ =\prt{urn j empty, urn k empty}-\prt{urn j empty}\wts\prt{urn k empty} $$
$\iff$
$$ \prt{urn j empty}\wts\prt{urn k empty}>\prt{urn j empty, urn k empty} $$
$\iff$
$$ \prt{urn k empty}>\frac{\prt{urn j empty, urn k empty}}{\prt{urn j empty}}=\cpt{urn k empty}{urn j empty} $$
Of course this makes sense intuitively: if urn $j$ is empty, then there are more balls to go into urn $k$ for $k>j$. Interestingly, this doesn’t hold for $j=1$:
Note that 7.9.a.2 says $\covw{I_1}{I_k}=0$ for any $k>1$. This says that the events $\settx{urn 1 empty}$ and $\settx{urn k empty}$ are independent since
$$ 0=\covw{I_1}{I_k}=\E{I_1I_k}-\E{I_1}\E{I_k} $$
$$ =\prt{urn 1 empty, urn k empty}-\prt{urn 1 empty}\wts\prt{urn k empty} $$
$\iff$
$$ \prt{urn 1 empty}\wts\prt{urn k empty}=\prt{urn 1 empty, urn k empty} $$
Considering the above discussion where $\settx{urn j empty}$ and $\settx{urn k empty}$ are dependent for $k>j>1$, I was initially surprised by the independence when $j=1$. But then I recalled from part 7.9.b that ball $1$ must go into urn $1$. Then this made perfect sense. I also found this interesting.
(7.33) If $\E{X}=1$ and $\V{X}=5$, find (a) $\E{(2+X)^2}$ and (b) $\V{4+3X}$.
Solution This problem is identical to problem 4.38: Note that
$$ \evw{X^2}=\varw{X}+\bop\evw{X}\bcp^2=5+1=6 $$
so that
$$ \evw{(2+X)^2}=\evw{X^2+4X+4}=\evw{X^2}+4\evw{X}+4=14 $$
Alternatively:
$$ \evw{(2+X)^2}=\varw{2+X}+\bop\evw{2+X}\bcp^2=\varw{X}+\bop2+\evw{X}\bcp^2=5+3^2=14 $$
And part (b):
$$ \varw{4+3X}=\varw{3X}=9\varw{X}=45 $$
(7.34.a) If $10$ married couples are randomly seated at a round table, compute the expected number of wives who are seated next to their husbands.
Solution Define
$$ X_i=\cases{1&\text{if the members of couple }i\text{ are seated next to each other}\\0&\text{otherwise}} $$
Let’s consider the probability that a wife is seated next to her husband. After a wife is seated, her husband can sit in $1$ of the $19$ available seats. Two of these seats are next to his wife. Hence $\E{X_i}=\pr{X_i=1}=\frac2{19}$ and
$$ \evt{number wives seated next to their husbands}=\EB{\sum_{i=1}^{10}X_i}=\sum_{i=1}^{10}\E{X_i}=\frac{20}{19} $$
since $\sum_{i=1}^{10}X_i$ is the number of wives who are seated next to their husbands.
(7.34.b) If $10$ married couples are randomly seated at a round table, compute the variance of the number of wives who are seated next to their husbands.
Solution Note that $X_iX_j$ is the indicator on the event that couple $i$ is seated together and couple $j$ is seated together. Hence, for $i\neq j$, we have
$$ \E{X_iX_j}=\pr{X_i=1,X_j=1}=\cp{X_j=1}{X_i=1}\pr{X_i=1} \tag{7.34.b.1} $$
Let’s consider $\prq{X_j=1}\equiv\cp{X_j=1}{X_i=1}$. Given that couple $i$ is seated together, then there are $18$ seats in a row, not in a circle. That is, the $i^{th}$ couple seated together has cut the circle into a row.
Of the remaining $18$ seats, $2$ of them are located on an end with only a single seat next to it. The probability that the $j^{th}$ wife sits on the end of the row with her husband next to her is $\frac2{18}\frac1{17}$.
Of the remaining $18$ seats, $16$ of them are not located on an end and have two seats next to them. The probability that the $j^{th}$ wife sits not on an end with her husband next to her is $\frac{16}{18}\frac2{17}$.
More formally:
$$ \cp{X_j=1}{X_i=1}=\prq{X_j=1} $$
$$ =\cpq{X_j=1}{\text{wife on end}}\prq{\text{wife on end}}+\cpq{X_j=1}{\text{wife not on end}}\prq{\text{wife not on end}} $$
$$ =\frac1{17}\frac2{18}+\frac2{17}\frac{16}{18}=\frac2{18}\Prn{\frac1{17}+\frac{16}{17}}=\frac2{18} $$
So 7.34.b.1 becomes
$$ \E{X_iX_j}=\pr{X_i=1,X_j=1}=\cp{X_j=1}{X_i=1}\pr{X_i=1}=\frac2{18}\frac2{19} $$
for $i\neq j$. Hence
$$ \covw{X_i}{X_j}=\E{X_iX_j}-\E{X_i}\E{X_j}=\frac2{18}\frac2{19}-\fracpB2{19}^2=\frac2{19}\Prn{\frac2{18}-\frac2{19}} $$
for $i\neq j$. Also note that $X_i^2=X_i$ since $X_i$ is an indicator. Hence
$$ \V{X_i}=\E{X_i^2}-\prn{\E{X_i}}^2=\E{X_i}-\fracpB2{19}^2=\frac2{19}-\fracpB2{19}^2=\frac2{19}\Prn{1-\frac2{19}}=\frac2{19}\frac{17}{19} $$
Hence
$$ \vart{number wives seated next to their husbands}=\VB{\sum_{i=1}^{10}X_i}=\sum_{i=1}^{10}\V{X_i}+2\mathop{\sum\sum}_{i<j}^{10}\covw{X_i}{X_j} $$
$$ =\sum_{i=1}^{10}\frac2{19}\frac{17}{19}+2\mathop{\sum\sum}_{i<j}^{10}\frac2{19}\Prn{\frac2{18}-\frac2{19}} $$
$$ =10\wts\frac2{19}\frac{17}{19}+2\binom{10}2\frac2{19}\Prn{\frac2{18}-\frac2{19}} $$
In [888]: 20/19 #expected
Out[888]: 1.0526315789473684
In [889]: 10*2/19*17/19+2*winom(10,2)*2/19*(2/18-2/19) #variance
Out[889]: 0.997229916897507
(7.35.a) Cards from an ordinary deck are turned face up one at a time. Compute the expected number of cards that need to be turned face up in order to obtain $2$ aces.
Solution Let $X$ denote the number of cards that must be flipped in order to obtain $2$ aces. And Let $Y$ denote the number of cards that must be flipped in order to obtain $1$ ace. Then
$$ \E{X}=\E{\Ec{X}{Y}}=\sum_{y=1}^{49}\Ec{X}{Y=y}\pr{Y=y} $$
$$ =\sum_{y=1}^{49}\sum_{x=y+1}^{50}x\cp{X=x}{Y=y}\pr{Y=y} \tag{7.35.a.1} $$
Explanation of limits $y_0=1$ and $y_1=49$: the first ace can appear on the first card through (and including) the $49^{th}$ card. The limits $x_0=y+1$ and $x_1=50$: the second ace can appear immediately after the first ace and through the $50^{th}$ card.
With no aces yet flipped, on the $i^{th}$ flip, there is a probability of $\frac4{53-i}$ of flipping an ace. With $1$ ace already flipped, on the $i^{th}$ flip, there is a probability of $\frac3{53-i}$ of flipping an ace. Hence
$$ \pr{Y=y}=\frac4{53-y}\prod_{i=1}^{y-1}\Prn{1-\frac4{53-i}} $$
$$ \cp{X=x}{Y=y}=\frac3{53-x}\prod_{i=y+1}^{x-1}\Prn{1-\frac3{53-i}} $$
In [161]: PY=lambda y=2: 4/(53-y)*np.prod([1-4/(53-i) for i in range(1,y)])
In [162]: PXcY=lambda x=4,y=2: 3/(53-x)*np.prod([1-3/(53-i) for i in range(y+1,x)])
In [163]: sum(sum(x*PXcY(x=x,y=y)*PY(y=y) for x in range(y+1,51)) for y in range(1,50))
Out[163]: 21.199999999999996
This solution used conditional expectation. But it can be viewed as probability conditioning. For $2\leq x\leq50$, we have
$$ \pr{X=x}=\sum_{y=1}^{x-1}\cp{X=x}{Y=y}\pr{Y=y} $$
$$ =\sum_{y=1}^{x-1}\Cbr{\frac3{53-x}\Sbr{\prod_{i=y+1}^{x-1}\Prn{1-\frac3{53-i}}}\frac4{53-y}\Sbr{\prod_{i=1}^{y-1}\Prn{1-\frac4{53-i}}}} $$
The limits $y_0=1$ and $y_1=x-1$: the first ace can appear on the first card through the card immediately preceding the second ace. Hence
$$ \E{X}=\sum_{x=2}^{50}x\pr{X=x}=\sum_{x=2}^{50}x\sum_{y=1}^{x-1}\cp{X=x}{Y=y}\pr{Y=y} \tag{7.35.a.2} $$
In [165]: PX=lambda x=4: sum(PXcY(x=x,y=y)*PY(y=y) for y in range(1,x))
In [166]: sum(x*PX(x=x) for x in range(2,51))
Out[166]: 21.20000000000001
For completeness, note that we can change the order of summation in the double sum in 7.35.a.1:
$$ \sum_{y=1}^{49}\sum_{x=y+1}^{50}x\cp{X=x}{Y=y}\pr{Y=y}=\sum_{x=2}^{50}\sum_{y=1}^{x-1}x\cp{X=x}{Y=y}\pr{Y=y} $$
That is, both double sums are over the collection of all integer pairs $(x,y)$ such that $1\leq y<x\leq50$. We performed similar changes of order in theoretical exercise 4.4 and Lemma 2.1 in ch.5. Also note that the right side of this equation is identical to the right side of 7.35.a.2.
In [164]: sum(sum(x*PXcY(x=x,y=y)*PY(y=y) for y in range(1,x)) for x in range(2,51))
Out[164]: 21.200000000000006
Alternatively, we can compute $X$ as a negative hypergeometric random variable. This variable is presented in example 3e, p.319-320. The formula there is
$$ \E{X}=r+m\wts\frac{r}{n+1} $$
where $r=2$ is the number of special cards (aces) to be flipped, $m=48$ is the number of ordinary cards (non-aces) in the deck, and $n=4$ is the number of special cards in the deck.
In [169]: 2+48*2/(4+1)
Out[169]: 21.2
The solutions manual gives a solution that is not clear to me. I need to write it out in more detail. “Let $X_1$ denote the number of non-aces preceding the first ace and $X_2$ the number of non-aces between the first $2$ aces. It is easy to see that”
$$ \pr{X_1=i,X_2=j}=\pr{X_1=j,X_2=i} $$
I don’t easily see this.
Proof: First we do an example:
$$ \pr{X_1=2,X_2=3}=\frac{48}{52}\frac{47}{51}\frac{4}{50}\frac{46}{49}\frac{45}{48}\frac{44}{47}\frac{3}{46} $$
$$ \pr{X_1=3,X_2=2}=\frac{48}{52}\frac{47}{51}\frac{46}{50}\frac{4}{49}\frac{45}{48}\frac{44}{47}\frac{3}{46} $$
We see that $\pr{X_1=2,X_2=3}=\pr{X_1=3,X_2=2}$. Now we can write this for $i<j$:
$$ \pr{X_1=i,X_2=j}=\frac4{52-i}\frac3{52-(i+1)-j}\prod_{k=0}^{i-1}\fracpB{48-k}{52-k}\prod_{k=i}^{i+j-1}\fracpB{48-k}{52-k-1} $$
It won’t be easy to show equivalence with this equation. Instead, we look again at the example and see that
$$ 48\wts47\wts46\wts45\wts44=\frac{48!}{43!}=\frac{48!}{(48-2-3)!} $$
and
$$ 52\wts51\wts50\wts49\wts48\wts47\wts46=\frac{52!}{45!}=\frac{52!}{(52-2-3-2)!} $$
Hence
$$ \pr{X_1=i,X_2=j}=\frac{4\wts3\wts\frac{48!}{(48-i-j)!}}{\frac{52!}{(52-i-j-2)!}}=\pr{X_1=j,X_2=i} $$
where the last equation follows because switching $i$ and $j$ in the middle doesn’t change the value. $\wes$
Next claim: Suppose $\pr{X=i,Y=j}=\pr{X=j,Y=i}$ for all $i,j$. Then $X$ and $Y$ have the same distribution.
Proof: We wish to show that $\pr{X=i}=\pr{Y=i}$ for all $i$.
$$ \pr{X=i}=\sum_{j}\cp{X=i}{Y=j}\pr{Y=j} $$
$$ =\sum_{j}\pr{X=i,Y=j} $$
$$ =\sum_{j}\pr{X=j,Y=i} $$
$$ =\sum_{j}\cp{Y=i}{X=j}\pr{X=j} $$
$$ =\pr{Y=i} $$
Next claim: “$\E{X_1}=\frac{48}5$ by the results of example 3e”. Let’s number the non-aces as $c_1,…,c_{48}$. For $i=1,…,48$, let $A_i$ be the event that card $c_i$ is flipped before $1$ ace has been flipped. Note that $X_1$ is the number of events $A_1,…,A_{48}$ that occur. Since $X_1$ and $X_2$ have the same distribution, we see
$$ \E{X_2}=\E{X_1}=\sum_{i=1}^{48}\pr{A_i} $$
To determine $\pr{A_i}$, consider the $5$ cards consisting of $c_i$ and the $4$ aces. Of these, $c_i$ is equally likely to be the first one flipped, or the second one flipped,…, or the final one flipped. Hence, the probability that it is the first to be flipped (and so is flipped before any aces are flipped) is $\frac15$. That is
$$ \pr{A_i}=\frac15 $$
and
$$ \E{X_2}=\E{X_1}=\sum_{i=1}^{48}\pr{A_i}=48\wts\frac15=\frac{48}5 $$
Hence
$$ \evt{number cards flipped to get 2 aces}=\E{X_1+1+X_2+1} $$
$$ =2+\E{X_1}+\E{X_2}=\frac{10}5+\frac{48}5+\frac{48}5=\frac{106}5=21.2 $$
(7.35.b) Compute the expected number of cards that need to be turned face up in order to obtain $5$ spades.
(Solution)
$$ \E{X}=r+m\wts\frac{r}{n+1}=5+(52-13)\wts\frac{5}{13+1} $$
In [971]: 5+(52-13)*5/(13+1)
Out[971]: 18.92857142857143
(7.35.c) Compute the expected number of cards that need to be turned face up in order to obtain all $13$ hearts.
(Solution)
$$ \E{X}=r+m\wts\frac{r}{n+1}=13+(52-13)\wts\frac{13}{13+1} $$
In [993]: 13+39*13/14
Out[993]: 49.214285714285715
(7.36) Let $X$ be the number of $1$’s and $Y$ the number of $2$’s that occur in $n$ rolls of a fair die. Compute $\covw{X}{Y}$.
Solution Define
$$ X_i=\cases{1&i^{th}\text{ roll is }1\\0&\text{otherwise}}\dq Y_i=\cases{1&i^{th}\text{ roll is }2\\0&\text{otherwise}} $$
Then $X=\sum_{i=1}^nX_i$ is the number of $1$’s rolled, $Y=\sum_{i=1}^nY_i$ is the number of $2$’s rolled, and
$$ \E{X_i}=\prt{roll i is 1}=\frac16\dq\E{Y_i}=\prt{roll i is 2}=\frac16 $$
Note that $X_iY_i=0$ since a roll cannot be both $1$ and $2$. Hence
$$ \covw{X_i}{Y_i}=\E{X_iY_i}-\E{X_i}\E{Y_i}=0-\frac16\frac16=-\frac1{36} $$
Also note that rolls are independent. Hence $X_i$ and $Y_j$ are independent for $i\neq j$. Hence $\covw{X_i}{Y_j}=0$ for $i\neq j$ and
$$ \covw{X}{Y}=\covB{\sum_{i=1}^nX_i}{\sum_{j=1}^nY_j}=\sum_{i=1}^n\sum_{j=1}^n\covw{X_i}{Y_j}=\sum_{i=1}^n\covw{X_i}{Y_i}=-\frac{n}{36} $$
Notice the negative correlation. This makes sense. If you get more $1$’s, you’ll get less $2$’s. Also notice that the correlation goes to zero as the number of rolls increases. This also makes sense. When $n$ is really big, $3$ more $1$’s won’t significantly alter the number of $2$’s you get. But if $n=7$, then $3$ more $1$’s will drastically reduce the number of $2$’s you get.
(7.37) A die is rolled twice. Let $X$ equal the sum of the outcomes, and let $Y$ equal the first outcome minus the second. Compute $\covw{X}{Y}$.
Solution For $i=1,2$, let $W_i$ denote the outcome of the $i^{th}$ roll. Note that $X=W_1+W_2$ and $Y=W_1-W_2$. Since the rolls are independent, $W_1$ and $W_2$ are independent. Hence $\covw{W_1}{W_2}=0$. Hence
$$ \covw{X}{Y}=\covw{W_1+W_2}{W_1-W_2} $$
$$ =\covw{W_1}{W_1}-\covw{W_1}{W_2}+\covw{W_2}{W_1}-\covw{W_2}{W_2} $$
$$ =\covw{W_1}{W_1}-\covw{W_2}{W_2}=\V{W_1}-\V{W_2}=0 $$
The last equality follows because each roll has the same variance.
This result is surprising to me. There is no correlation between the sum and difference of two rolls. Say the sum is $5$. This could be $(1,4),(4,1),(2,3),(3,2)$. The possible differences are $-3,3,-1,1$. If the sum increases to $6$, then this could be $(1,5),(5,1),(2,4),(4,2),(3,3)$. The possible differences are $-4,4,-2,2,0$. Notice that in both cases the average of the possible differences is of course zero.
(7.38) The random variables $X$ and $Y$ have $a$ joint density function given by
$$ \pdf{x,y}=\cases{\frac2x\e{-2x}&0\leq x\leq\infty,0\leq y\leq x\\0&\text{otherwise}} $$
Compute $\covw{X}{Y}$.
Solution Note that $\covw{X}{Y}=\E{XY}-\E{X}\E{Y}$. To compute $\E{XY}$, we use proposition 2.1, p.298:
$$ \E{XY}=\int_0^\infty\int_0^xxy\frac2x\e{-2x}dydx=\int_0^\infty\e{-2x}\int_0^x2ydydx $$
$$ =\int_0^\infty\e{-2x}\Sbr{y^2\eval0x}dx=\int_0^\infty x^2\e{-2x}dx $$
$$ u=x^2\dq du=2xdx\dq dv=\e{-2x}dx\dq v=-\frac12\e{-2x} $$
$$ =\int_0^\infty x^2\e{-2x}dx=\int udv=uv-\int vdu=-\frac12x^2\e{-2x}\eval0\infty+2\wts\frac12\int_0^\infty x\e{-2x}dx $$
$$ =\frac12x^2\e{-2x}\eval\infty0+\int_0^\infty x\e{-2x}dx=0-0+\int_0^\infty x\e{-2x}dx $$
$$ u=x\dq du=dx\dq dv=\e{-2x}dx\dq v=-\frac12\e{-2x} $$
$$ =\int_0^\infty x\e{-2x}dx=-\frac12x\e{-2x}\eval0\infty+\frac12\int_0^\infty\e{-2x}dx $$
$$ =\frac12x\e{-2x}\eval\infty0+\frac12\Sbr{-\frac12\e{-2x}\eval0\infty}=0-0+\frac12\frac12\Sbr{\e{-2x}\eval\infty0}=\frac14(1-0)=\frac14 $$
To compute $\E{X}$, define $g(X,Y)=X$ and again apply proposition 2.1, p.298:
$$ \E{X}=\int_0^\infty\int_0^xx\frac2x\e{-2x}dydx=2\int_0^\infty\e{-2x}\int_0^xdydx $$
$$ =2\int_0^\infty x\e{-2x}dx=2\wts\frac14=\frac12 $$
To compute $\E{Y}$, define $g(X,Y)=Y$ and again apply proposition 2.1, p.298:
$$ \E{Y}=\int_0^\infty\int_0^xy\frac2x\e{-2x}dydx=2\int_0^\infty\frac{\e{-2x}}x\int_0^xydydx $$
$$ =2\int_0^\infty\frac{\e{-2x}}x\frac{x^2}2dx=\int_0^\infty\e{-2x}xdx=\frac14 $$
Hence
$$ \covw{X}{Y}=\E{XY}-\E{X}\E{Y}=\frac14-\frac12\frac14=\frac28-\frac18=\frac18 $$
(7.39) Let $\set{X_i}$ be independent with common mean $\mu$ and common variance $\sigma^2$, and set $Y_n=X_n+X_{n+1}+X_{n+2}$. For $j\geq0$, find $\covw{Y_n}{Y_{n+j}}$.
Solution Since $X_i$ and $X_j$ are independent for $i\neq j$, then $\covw{X_i}{X_j}=0$ for $i\neq j$. Also note that $\covw{X_i}{X_i}=\V{X_i}=\sigma^2$. Hence
$$ \covw{Y_n}{Y_{n}}=\covw{X_{n}+X_{n+1}+X_{n+2}}{X_{n}+X_{n+1}+X_{n+2}} $$
$$ =\covw{X_{n}}{X_{n}}+\covw{X_{n+1}}{X_{n+1}}+\covw{X_{n+2}}{X_{n+2}}=3\sigma^2 $$
and
$$ \covw{Y_n}{Y_{n+1}}=\covw{X_{n}+X_{n+1}+X_{n+2}}{X_{n+1}+X_{n+2}+X_{n+3}} $$
$$ =\covw{X_{n+1}}{X_{n+1}}+\covw{X_{n+2}}{X_{n+2}}=2\sigma^2 $$
and
$$ \covw{Y_n}{Y_{n+2}}=\covw{X_{n}+X_{n+1}+X_{n+2}}{X_{n+2}+X_{n+3}+X_{n+4}} $$
$$ =\covw{X_{n+2}}{X_{n+2}}=\sigma^2 $$
and, for $j\geq3$, we have
$$ \covw{Y_n}{Y_{n+j}}=\covw{X_{n}+X_{n+1}+X_{n+2}}{X_{n+j}+X_{n+j+1}+X_{n+j+2}}=0 $$
(7.40) The joint density function of $X$ and $Y$ is given by
$$ \pdf{x,y}=\frac1y\e{-(y+\frac{x}y)}\dq x>0,y>0 $$
Find $\E{X},\E{Y},$ and show that $\covw{X}{Y}=1$.
Solution For $y\leq0$, $\pdfa{y}{Y}=\int_0^\infty\pdf{x,0}dx=\int_0^\infty0dx=0$. For $y>0$, we have
$$ \pdfa{y}{Y}=\int_0^\infty\frac1y\e{-(y+\frac{x}y)}dx=\e{-y}\int_0^\infty\frac1y\e{-\frac{x}y}dx=\e{-y} $$
The last equality follows because $\frac1y\e{-\frac{x}y}$ for $x\in[0,\infty)$ is the density of the exponential distribution with parameter $\lambda=\frac1y$. Hence the integral must be $1$. In detail:
$$ \int_0^\infty\frac1y\e{-\frac{x}y}dx=-\e{-\frac{x}y}\eval0\infty=\e{-\frac{x}y}\eval\infty0=1-0=1 $$
Hence
$$ \pdfa{y}{Y}=\cases{\e{-y}&y>0\\0&y\leq0} $$
and
$$ \E{Y}=\int_{-\infty}^{\infty}y\pdfa{y}{Y}dy=\int_0^\infty y\e{-y}dy=\GammaF{2}=1 $$
For $x,y>0$, note that
$$ \pdfa{x|y}{X|Y}=\frac{\pdf{x,y}}{\pdfa{y}{Y}}=\frac{\frac1y\e{-(y+\frac{x}y)}}{\e{-y}}=\frac{\frac1y\e{-y}\e{-\frac{x}y}}{\e{-y}}=\frac1y\e{-\frac{x}y} $$
Hence the conditional distribution of $X$ given that $Y=y$ is exponential with parameter $\lambda=\frac1y$. We know from example 5a, p.209-210, ch.5 that $\Ec{X}{Y=y}=\frac1{\frac1y}=y$. Hence
$$ \E{X}=\E{\Ec{X}{Y}}=\int_0^\infty\Ec{X}{Y=y}\wts\pdfa{y}{Y}dy=\int_0^\infty y\e{-y}dy=\E{Y}=1 $$
Then proposition 2.1, p.298 gives
$$ \E{XY}=\int_0^\infty\int_0^\infty xy\pdf{x,y}dxdy=\int_0^\infty\int_0^\infty xy\frac1y\e{-(y+\frac{x}y)}dxdy $$
$$ =\int_0^\infty\e{-y}y\int_0^\infty x\frac1y\e{-\frac{x}y}dxdy \tag{7.40.1} $$
Note that $\int_0^\infty x\frac1y\e{-\frac{x}y}dx=\frac1\lambda=\frac1{\frac1y}=y$ is the mean of the exponential distribution with parameter $\lambda=\frac1y$. We can verify this:
$$ u=x\dq du=dx\dq dv=\frac1y\e{-\frac{x}y}dx\dq v=-\e{-\frac{x}y} $$
$$ =\int_0^\infty x\frac1y\e{-\frac{x}y}dx=-x\e{-\frac{x}y}\eval0\infty+\int_0^\infty\e{-\frac{x}y}dx $$
$$ =x\e{-\frac{x}y}\eval\infty0+\Sbr{-y\e{-\frac{x}y}\eval0\infty}=0-0+y\Sbr{\e{-\frac{x}y}\eval\infty0}=y(1-0)=y $$
Then 7.40.1 becomes
$$ \E{XY}=\int_0^\infty\e{-y}y\int_0^\infty x\frac1y\e{-\frac{x}y}dxdy=\int_0^\infty\e{-y}y^2dy=\GammaF{3}=2 \tag{7.40.2} $$
There is an alternative way to compute $\E{XY}$:
$$ \E{XY}=\E{\Ec{XY}{Y}}=\E{Y\Ec{X}{Y}}=\E{YY}=\E{Y^2} \tag{7.40.3} $$
The third equality follows because $\Ec{X}{Y=y}=y\iff\Ec{X}{Y}=Y$. That is, $\Ec{X}{\wt}$ is the identity map on $Y$. Now let’s look at the second equality: the definition of conditional expectation gives
$$ \Ec{XY}{Y=y}=\int_0^\infty xy\pdfa{x|y}{X|Y}dx=y\int_0^\infty x\pdfa{x|y}{X|Y}dx=y\wts\Ec{X}{Y=y} $$
That is, $\Ec{XY}{Y}=Y\Ec{X}{Y}$. Continuing with 7.40.3, we have
$$ \E{XY}=\E{Y^2}=\int_0^\infty\e{-y}y^2dy=\GammaF{3}=2!=2 $$
This concurs with 7.40.2. Putting it all together, we have
$$ \covw{X}{Y}=\E{XY}-\E{X}\E{Y}=2-1\wts1=1 $$
(7.41) A pond contains $100$ fish, of which $30$ are carp. If $20$ fish are caught, what are the mean and variance of the number of carp among the $20$? What assumptions are you making?
Solution Let $X$ denote the number of carp caught of the $20$ fish caught. We assume that each of the $\binom{100}{20}$ ways to catch the $20$ fish are equally likely. Then $X$ satisfies a hypergeometric distribution with parameters $n=20$, $N=100$, $m=30$.
The formulas for mean and variance of the hypergeometric distribution are computed in two different ways in the text. On p.317, example 3b and on p.162-163, example 8j, we see that
$$ \E{X}=\frac{nm}N=\frac{20\wts30}{100}=6 $$
$$ \V{X}=\frac{nm}N\Sbr{\frac{(n-1)(m-1)}{N-1}+1-\frac{nm}N} $$
In [996]: varhg=lambda n=20,m=30,N=100: n*m/N*(((n-1)*(m-1))/(N-1)+1-n*m/N)
In [997]: varhg()
Out[997]: 3.3939393939393945
(7.42.a) A group of $20$ people consisting of $10$ men and $10$ women is randomly arranged into $10$ pairs of $2$ each. Compute the expectation and variance of the number of pairs that consist of a man and a woman.
Solution Let $X_i$ be the indicator variable for couple $i$ consisting of a man and a woman. Let $X=\sum_{i=1}^{10}X_i$ denote the number of couples consisting of a man and a woman.
Let’s compute the probability that couple $i$ consists of a man and a woman. The probability of first selecting a man and then selecting a woman is $\frac{10}{20}\frac{10}{19}$. The same probability applies to first selecting a woman and then selecting a man. Hence
$$ \E{X_i}=2\wts\frac{10}{20}\frac{10}{19}=\frac{10}{19} $$
and
$$ \E{X}=\sum_{i=1}^{10}\E{X_i}=10\wts\frac{10}{19} $$
Since $X_i$ is an indicator, we have $X_i^2=X_i$. Hence
$$ \V{X_i}=\E{X_i^2}-\prn{\E{X_i}}^2=\E{X_i}-\prn{\E{X_i}}^2=\E{X_i}(1-\E{X_i})=\frac{10}{19}\frac9{19} $$
Note the $X_iX_j$ is the indicator on the intersection event that couple $i$ consists of a man and woman and couple $j$ consists of a man and woman. Hence for $i\neq j$, we have
$$ \E{X_iX_j}=\prt{cpl i m/w, cpl j m/w}=\cpt{cpl j m/w}{cpl i m/w}\prt{cpl i m/w} $$
Note that once couple $i$ is chosen with a man and woman, then couple $j$ will be chosen from $9$ men and $9$ women. Hence $\cpt{cpl j m/w}{cpl i m/w}=2\wts\frac{9}{18}\frac{9}{17}=\frac9{17}$. Hence
$$ \E{X_iX_j}=\cpt{cpl j m/w}{cpl i m/w}\prt{cpl i m/w}=\frac9{17}\frac{10}{19} $$
Hence
$$ \covw{X_i}{X_j}=\E{X_iX_j}-\E{X_i}\E{X_j}=\frac9{17}\frac{10}{19}-\frac{10}{19}\frac{10}{19} $$
$$ =\frac{10}{19}\Prn{\frac9{17}-\frac{10}{19}}=\frac{10}{19}\frac{171-170}{17\wts19}=\frac{10}{19}\frac1{17\wts19} $$
Notice the positive, near-zero correlation. This makes sense. If couple $i$ consists of a man and woman, this means that there are still an equal number of men and women to chose from. This makes it slightly more likely that couple $j$ will consist of a man and woman, which we already showed:
$$ \cpt{cpl j m/w}{cpl i m/w}=\frac9{17}\approx0.529>0.526\approx\frac{10}{19}=\prt{cpl j m/w} $$
And the variance is
$$ \V{X}=\sum_{i=1}^{10}\V{X_i}+2\mathop{\sum\sum}_{i<j}\covw{X_i}{X_j}=10\wts\frac{10\wts9}{19^2}+2\binom{10}2\frac{10}{19}\frac1{17\wts19} $$
In [1015]: 100/19,900/pw(19,2)+2*binom(10,2)*10/19*1/(17*19)
Out[1015]: (5.2631578947368425, 2.63972625061105)
(7.42.b) Now suppose the $20$ people consist of $10$ married couples. Compute the mean and variance of the number of married couples that are paired together.
Solution Let $X_i$ be the indicator variable for couple $i$ consisting of a married couple. Let $X=\sum_{i=1}^{10}X_i$ denote the number of couples consisting of a married couple.
Let’s compute the probability that couple $i$ consists of a married couple. The probability of first selecting a particular person and then selecting the spouse is $\frac{1}{20}\frac{1}{19}$. The same probability applies to first selecting any particular person and then selecting the spouse. Hence
$$ \E{X_i}=20\wts\frac{1}{20}\frac{1}{19}=\frac{1}{19} $$
and
$$ \E{X}=\sum_{i=1}^{10}\E{X_i}=10\wts\frac{1}{19} $$
Since $X_i$ is an indicator, we have $X_i^2=X_i$. Hence
$$ \V{X_i}=\E{X_i^2}-\prn{\E{X_i}}^2=\E{X_i}-\prn{\E{X_i}}^2=\E{X_i}(1-\E{X_i})=\frac{1}{19}\frac{18}{19} $$
Note the $X_iX_j$ is the indicator on the intersection event that couple $i$ consists of a married couple and couple $j$ consists of a married couple. Hence for $i\neq j$, we have
$$ \E{X_iX_j}=\prt{cpl i h/w, cpl j h/w}=\cpt{cpl j h/w}{cpl i h/w}\prt{cpl i h/w} $$
Note that once couple $i$ is chosen with a husband and wife, then couple $j$ will be chosen from $9$ married couples. Hence $\cpt{cpl j h/w}{cpl i h/w}=18\wts\frac{1}{18}\frac{1}{17}=\frac1{17}$. Hence
$$ \E{X_iX_j}=\cpt{cpl j h/w}{cpl i h/w}\prt{cpl i h/w}=\frac1{17}\frac{1}{19} $$
Hence
$$ \covw{X_i}{X_j}=\E{X_iX_j}-\E{X_i}\E{X_j}=\frac1{17}\frac{1}{19}-\frac{1}{19}\frac{1}{19} $$
$$ =\frac{1}{19}\Prn{\frac1{17}-\frac{1}{19}}=\frac{1}{19}\frac{19-17}{17\wts19}=\frac{1}{19}\frac2{17\wts19} $$
Notice the positive, near-zero correlation. This makes sense. If couple $i$ consists of a husband and wife, this means that there are still only married couples to chose from. That is, for every person not yet selected, their spouse also has not been selected. This makes it slightly more likely that couple $j$ will consist of a husband and wife, which we already showed:
$$ \cpt{cpl j h/w}{cpl i h/w}=\frac1{17}\approx0.059>0.053\approx\frac{1}{19}=\prt{cpl j h/w} $$
And the variance is
$$ \V{X}=\sum_{i=1}^{10}\V{X_i}+2\mathop{\sum\sum}_{i<j}\covw{X_i}{X_j}=10\wts\frac{18}{19^2}+2\binom{10}2\frac{1}{19}\frac2{17\wts19} $$
In [1020]: 10/19,180/pw(19,2)+2*binom(10,2)*1/19*2/(17*19)
Out[1020]: (0.5263157894736842, 0.527945250122210)
(7.45) If $X_1,X_2,X_3$, and $X_4$ are (pairwise) uncorrelated random variables, each having mean $0$ and variance $1$, compute the correlations of
- $X_1+X_2$ and $X_2+X_3$
- $X_1+X_2$ and $X_3+X_4$
Solution
$$ \corr{X_1+X_2}{X_2+X_3}=\frac{\covw{X_1+X_2}{X_2+X_3}}{\sigma_{X_1+X_2}\sigma_{X_2+X_3}} $$
$$ =\frac{\covw{X_1}{X_2}+\covw{X_1}{X_3}+\covw{X_2}{X_2}+\covw{X_2}{X_3}}{\sqrt{\V{X_1+X_2}}\sqrt{\V{X_2+X_3}}} $$
$$ =\frac{\covw{X_2}{X_2}}{\sqrt{\V{X_1}+\V{X_2}+2\covw{X_1}{X_2}}\sqrt{\V{X_2}+\V{X_3}+2\covw{X_2}{X_3}}} \tag{7.45.1} $$
$$ =\frac{\V{X_2}}{\sqrt{\V{X_1}+\V{X_2}}\sqrt{\V{X_2}+\V{X_3}}} \tag{7.45.2} $$
$$ =\frac1{\sqrt{1+1}\sqrt{1+1}}=\frac1{\sqrt2\sqrt2}=\frac12 $$
Note that 7.45.1 and 7.45.2 follow because the $\set{X_i}$ are pairwise uncorrelated. Similarly
$$ \corr{X_1+X_2}{X_3+X_4}=\frac{\covw{X_1+X_2}{X_3+X_4}}{\sigma_{X_1+X_2}\sigma_{X_3+X_4}} $$
$$ =\frac{\covw{X_1}{X_3}+\covw{X_1}{X_4}+\covw{X_2}{X_3}+\covw{X_2}{X_4}}{\sqrt{\V{X_1+X_2}}\sqrt{\V{X_3+X_4}}}=0 $$
(7.46) Consider the following dice game, as played at a certain gambling casino: Players $1$ and $2$ roll a pair of dice in turn. The bank then rolls the dice to determine the outcome according to the following rule: Player $i$, $i=1,2$, wins if his roll is strictly greater than the bank’s. For $i=1,2$, let
$$ I_i=\cases{1&i\text{ wins}\\0&\text{otherwise}} $$
and show that $I_1$ and $I_2$ are positively correlated. Explain why this result was to be expected.
Solution Let’s assume conditional independence (p.98) between player $1$ winning and player $2$ winning, given that the bank rolled $b$. That is, given that the bank rolled $b$, whether or not player $2$ wins is irrelevant to the probability that player $1$ wins.
Note that unconditional independence doesn’t hold. If we don’t know what the bank rolled but we know that player $2$ won/lost, then it is more likely player $1$ won/lost too (relative to not knowing the outcome for player $2$). That is, if player $2$ won, this increases the probability that the bank rolled a lower sum, which increases the probability that player $1$ won. This unconditional dependence explains the positive correlation of $I_1$ and $I_2$.
Let $B$ denote the roll sum of the bank. Let $X_i$ denote the roll sum of player $i$. Then
$$ I_i=\cases{1&X_i>B\\0&\text{otherwise}}\dq I_1I_2=\cases{1&X_1>B,X_2>B\\0&\text{otherwise}} $$
and
$$ \Ec{I_1I_2}{B=b}=1\wts\cp{I_1I_2=1}{B=b}=\cp{X_1>B,X_2>B}{B=b} $$
$$ =\cp{X_1>b,X_2>b}{B=b} $$
$$ =\cp{X_1>b}{B=b}\cp{X_2>b}{B=b} \tag{7.46.1} $$
$$ =\pr{X_1>b}\pr{X_2>b}=\prn{\pr{X_1>b}}^2 \tag{7.46.2} $$
7.46.1 holds because of conditional independence. The first equality in 7.46.2 holds because of the independence of $X_i$ and $B$. The second equality in 7.46.2 holds because $X_1$ and $X_2$ are identically distributed. Refer to example 5d, p.335 for similar equations.
Hence
$$ \E{I_1I_2}=\E{\Ec{I_1I_2}{B}}=\sum_{b=2}^{12}\Ec{I_1I_2}{B=b}\pr{B=b} $$
$$ =\sum_{b=2}^{12}\prn{\pr{X_1>b}}^2\pr{B=b} $$
Also note that
$$ \E{I_1}=1\wts\pr{I_1=1}=\pr{X_1>B} $$
$$ =\sum_{b=2}^{12}\cp{X_1>b}{B=b}\pr{B=b}=\sum_{b=2}^{12}\pr{X_1>b}\pr{B=b} $$
Since $X_1$ and $X_2$ are indentically distributed, then so are $I_1$ and $I_2$. Hence $\E{I_2}=\E{I_1}$ and
$$ \E{I_1}\E{I_2}=\Prn{\sum_{b=2}^{12}\pr{X_1>b}\pr{B=b}}^2 $$
To show that $I_1$ and $I_2$ are positively correlated, we want to show that $\E{I_1I_2}>\E{I_1}\E{I_2}$. Or
$$ \sum_{b=2}^{12}\prn{\pr{X_1>b}}^2\pr{B=b}>\Prn{\sum_{b=2}^{12}\pr{X_1>b}\pr{B=b}}^2 $$
This is difficult to prove algegraically (maybe use the multinomial theorem on the right?) but we can compute it:
In [1088]: PB=lambda b: (b-2*max(b-7,0)-1)/36
In [1089]: PXigb=lambda b: sum(PB(x) for x in range(b+1,13)) # B and Xi have same distribution
In [1090]: sum(pw(PXigb(b),2)*PB(b) for b in range(2,13)) # left hand side
Out[1090]: 0.27938528806584367
In [1091]: pw(sum(PXigb(b)*PB(b) for b in range(2,13)),2) # right hand side
Out[1091]: 0.19684558851547024
Below I write out the solution from the solutions manual and indicate where I disagree.
$$ \E{I_1I_2}=\sum_{i=2}^{12}\Ec{I_1I_2}{\text{bank rolls }i}\pr{\text{bank rolls }i} $$
$$ =\sum_{i=2}^{12}\prn{\pr{\text{roll is greater than }i}}^2\pr{\text{bank rolls }i} $$
$$ =\E{I_1^2} \tag{7.46.3} $$
$$ \geq\prn{\E{I_1}}^2 \tag{7.46.4} $$
$$ =\E{I_1}\E{I_2} $$
I disagree with 7.46.3 because $I_1^2=I_1$ hence
$$ \E{I_1^2}=\E{I_1}=\sum_{i=1}^{12}\pr{\text{roll is greater than }i}\pr{\text{bank rolls }i} $$
$$ >\sum_{i=2}^{12}\prn{\pr{\text{roll is greater than }i}}^2\pr{\text{bank rolls }i} $$
On the other hand, I’m not sure that $I_1^2=I_1$ holds. Of course this would hold for a typical indicator. But $I_1$ indicates $X_1>B$. The random variable $B$ on the right side could maybe change this?
(7.47.a) Consider a graph having $n$ vertices labeled $1,2,…,n$, and suppose that, between each of the $\binom{n}2$ pairs of distinct vertices, an edge is independently present with probability $p$. The degree of vertex $i$, designated as $D_i$, is the number of edges that have vertex $i$ as one of their vertices.
What is the distribution of $D_i$?
(Solution) The distribution of $D_i$ is binomial. Consider a success to be that there is an edge present between vertices $i$ and $j\neq i$. Since edges appear independently with probability $p$ and there are $n-1$ nodes besides $i$, then $D_i$ is binomal with parameters $n-1$ and $p$.
(7.47.b) Find $\corr{D_i}{D_j}$.
(Solution) Define
$$ I_{ik}=\cases{1&\text{there is an edge between vertices }i\text{ and }k\\0&\text{otherwise}} $$
Then
$$ D_i=\sum_{k=1,k\neq i}^nI_{ik} $$
Since edges appear independently between different nodes, then the $\set{I_{ik}}$ are independent. That is, $I_{ik}$ is independent of $I_{jl}$ if $i\neq j$ or $k\neq l$. But note that $I_{ik}=I_{ki}$. Hence
$$ I_{ik},I_{jl}\cases{\text{dependent}&(i=j\text{ and }k=l)\text{ or }(i=l\text{ and }k=j)\\\text{independent}&\text{otherwise}} $$
And this implies that
$$ \covw{I_{ik}}{I_{jl}}=\cases{\varw{I_{ik}}&(i=j\text{ and }k=l)\text{ or }(i=l\text{ and }k=j)\\0&\text{otherwise}} \tag{7.47.b.1} $$
Hence, for $i\neq j$, we have
$$ \covw{D_i}{D_j}=\covB{\sum_{k=1,k\neq i}^nI_{ik}}{\sum_{l=1,l\neq j}^nI_{jl}} $$
$$ =\sum_{k=1,k\neq i}^n\sum_{l=1,l\neq j}^n\covw{I_{ik}}{I_{jl}}=\varw{I_{ij}}=p(1-p) $$
The next-to-last equality follows because $i\neq j$. Hence it’s not possible for $i=j\text{ and }k=l$. Hence, by 7.47.b.1, $\covw{I_{ik}}{I_{jl}}=0$ unless $i=l$ and $k=j$. And this only occurs once in the double sum.
The last equality follows because $I_{ij}$ is an indicator. So $\E{I_{ij}}=\pr{I_{ij}=1}=p$, $I_{ij}^2=I_{ij}$, and
$$ \varw{I_{ij}}=\E{I_{ij}^2}-\prn{\E{I_{ij}}}^2=\E{I_{ij}}-p^2=p-p^2=p(1-p) $$
And the correlation is
$$ \corr{D_i}{D_j}=\frac{\covw{D_i}{D_j}}{\sqrt{\varw{D_i}}\sqrt{\varw{D_j}}} $$
$$ =\frac{p(1-p)}{\sqrt{(n-1)p(1-p)}\sqrt{(n-1)p(1-p)}}=\frac{p(1-p)}{(n-1)p(1-p)}=\frac1{n-1} $$
(7.49) There are two misshapen coins in a box; their probabilities for landing on heads when they are flipped are, respectively, $.4$ and $.7$. One of the coins is to be randomly chosen and flipped $10$ times. Given that two of the first three flips landed on heads, what is the conditional expected number of heads in the $10$ flips?
Solution Let $C$ denote the chosen coin. Let $c_i$ denote coin $i$ so that the probability of flipping heads with coin $1$ is $0.4$. Let $F$ denote the number of heads flipped in the first $3$ flips.
Note that $F\bar C=c_1$ is binomial with parameters $3$ and $0.4$ and $F\bar C=c_2$ is binomial with parameters $3$ and $0.7$. Then
$$ \cp{C=c_1}{F=2}=\frac{\cp{F=2}{C=c_1}\pr{C=c_1}}{\cp{F=2}{C=c_1}\pr{C=c_1}+\cp{F=2}{C=c_2}\pr{C=c_2}} $$
$$ =\frac{\binom320.4^2\wts0.6\wts0.5}{\binom320.4^2\wts0.6\wts0.5+\binom320.7^2\wts0.3\wts0.5} $$
$$ =\frac{0.4^2\wts0.6}{0.4^2\wts0.6+0.7^2\wts0.3} $$
In [1093]: (pw(.4,2)*.6)/(pw(.4,2)*.6+pw(.7,2)*.3)
Out[1093]: 0.39506172839506182
Let $N_j$ denote the number of heads in the final $j$ flips. Then
$$ \Ec{N_{10}}{F=2}=\Ec{F+N_7}{F=2}=\Ec{2+N_7}{F=2}=2+\Ec{N_7}{F=2} $$
Recall the remark on p.333 that conditional expectation can be thought of as being an ordinary expectation and satisfies all of the properties of a regular expectation. Let $Q[N_7]=\Ec{N_7}{F=2}$. Since $Q$ is an expectation, we have
$$ Q[N_7]=Q[Q[N_7\bar C]]=Q[N_7\bar C=c_1]\prq{C=c_1}+Q[N_7\bar C=c_2]\prq{C=c_2} $$
Now
$$ Q[N_7\bar C=c_1]=\Ec{N_7}{F=2,C=c_1}=\Ec{N_7}{C=c_1}=2.8 $$
The second equality holds because the occurence of $C=c_1$ makes the occurence of $F=2$ irrelevant. That is, once we know which coin was chosen, the number of heads occurring in the first $3$ flips no longer gives new information.
The last equality holds because $N_7\bar C=c_1$ is binomial with parameters $7$ and $0.4$. Hence its expected value is $7\wts0.4=2.8$. Similarly $N_7\bar C=c_2$ is binomial with parameters $7$ and $0.7$ and
$$ Q[N_7\bar C=c_2]=\Ec{N_7}{F=2,C=c_2}=\Ec{N_7}{C=c_2}=4.9 $$
Also note that
$$ \prq{C=c_1}=\cp{C=c_1}{F=2}\approx0.395 $$
$$ \prq{C=c_2}=\cp{C=c_2}{F=2}=1-\cp{C=c_1}{F=2}\approx0.605 $$
Putting it all together
$$ \evt{number heads in 10 flips given 2 in first 3}=\Ec{N_{10}}{F=2} $$
$$ =2+\Ec{N_7}{F=2}=2+Q[N_7]=2+2.8\wts0.395+4.9\wts0.605 $$
In [1095]: 2+2.8*.395+4.9*.605
Out[1095]: 6.0705
(7.50) The joint density of $X$ and $Y$ is given by
$$ \pdf{x,y}=\frac{\e{-\frac{x}y}\e{-y}}{y}\dq0<x<\infty,\qd0<y<\infty $$
Compute $\Ec{X^2}{Y=y}$.
Solution This problem is very similar to example 5b, p.332. The definition of continous conditional expectation on p.332 and the remark on p.333 give
$$ \Ec{X^2}{Y=y}=\int_0^\infty x^2\pdfa{x|y}{X|Y}dx $$
So we compute the conditional density just as it was done in example 5b:
$$ \pdfa{x|y}{X|Y}=\frac{\pdf{x,y}}{\pdfa{y}{Y}}=\frac{\frac1y\e{-\frac{x}y}\e{-y}}{\int_0^\infty\frac1y\e{-\frac{x}y}\e{-y}dx}=\frac{\frac1y\e{-\frac{x}y}}{\int_0^\infty\frac1y\e{-\frac{x}y}dx}=\frac1y\e{-\frac{x}y} $$
As noted and computed in problem 7.40, $\frac1y\e{-\frac{x}y}$ for $x\in[0,\infty)$ is the density of the exponential distribution with parameter $\lambda=\frac1y$. Hence the integral in the denominator must be $1$ and hence the conditional distribution of $X$, given that $Y=y$, is just the exponential with mean $y$. Hence
$$ \Ec{X^2}{Y=y}=\int_0^\infty\frac{x^2}{y}\e{-\frac{x}y}dx $$
$$ u=\frac{x}y\dq x=uy\dq dx=ydu\dq u_0=\frac{x_0}y=\frac0y=0\dq u_1=\frac{x_1}y=\frac{\infty}y=\infty $$
$$ =\int_0^\infty\frac{x^2}{y}\e{-\frac{x}y}dx=y\int_0^\infty\frac{x^2}{y^2}\e{-\frac{x}y}dx=y\int_0^\infty u^2\e{-u}ydu $$
$$ =y^2\int_0^\infty u^2\e{-u}du=y^2\GammaF{3}=2y^2 $$
(7.51) The joint density of $X$ and $Y$ is given by
$$ \pdf{x,y}=\frac{\e{-y}}{y}\dq0<x<y,\qd0<y<\infty $$
Compute $\Ec{X^3}{Y=y}$.
Solution This is similar to the previous problem:
$$ \pdfa{x|y}{X|Y}=\frac{\pdf{x,y}}{\pdfa{y}{Y}}=\frac{\frac1y\e{-y}}{\int_0^y\frac1y\e{-y}dx}=\frac1{\int_0^y dx}=\frac1y $$
$$ \Ec{X^3}{Y=y}=\int_0^y x^3\pdfa{x|y}{X|Y}dx=\int_0^y x^3\frac1ydx=\frac1{4y}\Sbr{x^4\eval0y}=\frac{y^4}{4y}=\frac{y^3}4 $$
(7.52) A population is made up of $r$ disjoint subgroups. Let $p_i$ denote the proportion of the population that is in subgroup $i,i=1,…,r$. If the average weight of the members of subgroup $i$ is $w_i,i=1,…,r$, what is the average weight of the members of the population?
Solution The average weight of the members of the population is equal to the average weight of a randomly chosen person. Let $W$ denote the weight of the randomly chosen person and let $S$ denote the subgroup to which this randomly chosen person belongs. Also let $s_i,i=1,…,r$ denote each subgroup. Then
$$ \Ec{W}{S=s_i}=w_i\dq\pr{S=s_i}=p_i $$
and
$$ \E{W}=\E{\Ec{W}{S}}=\sum_{i=1}^r\Ec{W}{S=s_i}\pr{S=s_i}=\sum_{i=1}^rw_ip_i $$
(7.53) A prisoner is trapped in a cell containing $3$ doors. The first door leads to a tunnel that returns him to his cell after $2$ days’ travel. The second leads to a tunnel that returns him to his cell after $4$ days’ travel. The third door leads to freedom after $1$ day of travel. If it is assumed that the prisoner will always select doors $1$, $2$, and $3$ with respective probabilities $.5$, $.3$, and $.2$, what is the expected number of days until the prisoner reaches freedom?
Solution Let $N$ denote the number of days until the prisoner reaches freedom. Let $I$ denote the door number initially selected by the prisoner. Then
$$ \E{N}=\E{\Ec{N}{I}}=\sum_{i=1}^3\Ec{N}{I=i}\pr{I=i} $$
$$ =(2+\E{N})\wts0.5+(4+\E{N})\wts0.3+1\wts0.2 \tag{7.53.1} $$
$$ =1+\E{N}\wts0.5+1.2+\E{N}\wts0.3+0.2=2.4+\E{N}\wts0.8 $$
For 7.53.1, similar to example 5c, p.334, when the prisoner selects door $1$, then the time to freedom is $2+\E{N}$ since he must first go through that tunnel and is returned to his original spot. Then it’s as if he is starting all over. So
$$ \E{N}\wts0.2=2.4\iff\E{N}=12 $$
(7.55) Ten hunters are waiting for ducks to fly by. When a flock of ducks flies overhead, the hunters fire at the same time, but each chooses his target at random, independently of the others. If each hunter independently hits his target with probability $.6$, compute the expected number of ducks that are hit. Assume that the number of ducks in a flock is a Poisson random variable with mean $6$.
Solution Define
$$ X_i=\cases{1&i^{th}\text{ duck is hit}\\0&\text{otherwise}} $$
Let $N$ denote the number of ducks in a flock. Then
$$ \EB{\sum_{i=1}^{N}X_i}=\EL{\EBC{\sum_{i=1}^NX_i}{N}}=\sum_{n=1}^\infty\EBC{\sum_{i=1}^NX_i}{N=n}\pr{N=n} $$
Note that
$$ \EBC{\sum_{i=1}^NX_i}{N=n}=\EBC{\sum_{i=1}^nX_i}{N=n}=\sum_{i=1}^n\Ec{X_i}{N=n} $$
$$ =n\Ec{X_1}{N=n} \tag{7.55.1} $$
$$ =n\cp{X_1=1}{N=n} $$
$$ =n\prn{1-\cp{X_1=0}{N=n}} $$
$$ =n\prn{1-\cptone{duck 1 not hit}{N=n}} $$
$$ =n\Prn{1-\Sbr{1-\frac{0.6}n}^{10}} \tag{7.55.2} $$
7.55.1 follows because the $\set{X_i}$ are identically distributed. That is, symmetry.
7.55.2 follows: Given that there are $n$ ducks in the flock, the probability of a given hunter hitting duck $1$ is $\frac{0.6}n$: Define $\prq{\wt}=\cp{\wt}{N=n}$, let $H$ denote the event that a given hunter hits duck $1$, let $T$ denote the event that the given hunter targets duck $1$, and condition on whether or not the given hunter targets duck $1$:
$$ \prq{H}=\cpq{H}{T}\prq{T}+\cpq{H}{T^c}\prq{T^c}=0.6\wts\frac1n+0\wts\frac{n-1}n $$
That is, given $n$ ducks in the flock and given that the hunter targets duck $1$, there is a probability of $0.6$ that he will hit it. And given $n$ ducks in the flock, there is a probability of $\frac1n$ that the given hunter targets duck $1$.
Note that we’re assuming that the given hunter will never hit duck $1$ if he doesn’t target duck $1$. Not a perfect assumption but a reasonable approximation.
Hence, given that there are $n$ ducks in the flock, the probability of the given hunter not hitting duck $1$ is $1-\frac{0.6}n$. Since each hunter independently targets and hits duck $1$, then the probability of all $10$ hunters not hitting duck $1$ is $\Sbr{1-\frac{0.6}n}^{10}$.
Then
$$ \evt{number of ducks hit}=\sum_{n=1}^\infty\EBC{\sum_{i=1}^NX_i}{N=n}\pr{N=n}=\sum_{n=1}^\infty n\Prn{1-\Sbr{1-\frac{0.6}n}^{10}}\e{-6}\frac{6^n}{n!} $$
$$ =\sum_{n=1}^\infty\e{-6}\frac{6^n}{(n-1)!}\Prn{1-\Sbr{1-\frac{0.6}n}^{10}} $$
In [1176]: import scipy
In [1177]: fg=lambda n=4: int(scipy.special.gamma(n+1)) # factorial based on gamma function
In [1178]: w755=lambda l=60,h=10: sum(np.exp(-6)*(pw(6,n)/fg(n-1))*(1-pw(1-(0.6/n),h)) for n in range(1,l))
In [1179]: w755(l=120,h=10) # h is the number of hunters
Out[1179]: 3.6988987144406562
In [1180]: w755(l=120,h=1)
Out[1180]: 0.59851274436804747
In [1181]: w755(l=120,h=5)
Out[1181]: 2.3615273247390691
In [1182]: w755(l=120,h=20)
Out[1182]: 5.0083085111646968
In [1183]: w755(l=120,h=40)
Out[1183]: 5.7790159589275811
In [1184]: w755(l=120,h=80)
Out[1184]: 5.9835415803930712
In [1185]: w755(l=120,h=800)
Out[1185]: 5.9999998180651088
In [1186]: w755(l=120,h=8000)
Out[1186]: 5.9999998180651204
(7.56) The number of people who enter an elevator on the ground floor is a Poisson random variable with mean $10$. If there are $N$ floors above the ground floor, and if each person is equally likely to get off at any one of the $N$ floors, independently of where the others get off, compute the expected number of stops that the elevator will make before discharging all of its passengers.
Solution Let $X$ denote the number of people that enter the elevator on the ground floor. Let $S_i$ equal $1$ if the elevator stops at floor $i$ and $0$ otherwise. Then $\sum_{i=1}^NS_i$ is the number of stops made before discharging all of the passengers and
$$ \EB{\sum_{i=1}^{N}S_i}=\EL{\EBC{\sum_{i=1}^NS_i}{X}}=\sum_{x=0}^\infty\EBC{\sum_{i=1}^NS_i}{X=x}\pr{X=x} $$
Note that the $\set{S_i}$ are symmetric (i.e. identically distributed) and
$$ \EBC{\sum_{i=1}^NS_i}{X=x}=\sum_{i=1}^N\Ec{S_i}{X=x}=N\Ec{S_1}{X=x} $$
$$ =N\cptone{someone gets off on floor 1}{X=x} $$
$$ =N\prn{1-\cptone{no one gets off on floor 1}{X=x}} $$
$$ =N\Prn{1-\Sbr{\frac{N-1}N}^x} $$
The last equation holds because the probability that a given passenger doesn’t get off on the first floor is $\frac{N-1}N$. Hence, because passengers choose a floor independently, the probability that all $x$ of the passengers don’t get off on the first floor is $\sbr{\frac{N-1}N}^x$.
Hence
$$ \evt{number stops before discharging all}=\sum_{x=0}^\infty\EBC{\sum_{i=1}^NS_i}{X=x}\pr{X=x} $$
$$ =\sum_{x=0}^\infty\Cbr{N\Prn{1-\Sbr{\frac{N-1}N}^x}}\frac{\e{-10}10^x}{x!} $$
$$ =\sum_{x=0}^\infty\Prn{N-N\Sbr{\frac{N-1}N}^x}\frac{\e{-10}10^x}{x!} $$
$$ =\sum_{x=0}^\infty\Prn{N\frac{\e{-10}10^x}{x!}-N\Sbr{\frac{N-1}N}^x\frac{\e{-10}10^x}{x!}} $$
$$ =\sum_{x=0}^\infty N\frac{\e{-10}10^x}{x!}-\sum_{x=0}^\infty N\Sbr{\frac{N-1}N}^x\frac{\e{-10}10^x}{x!} $$
$$ =N\e{-10}\sum_{x=0}^\infty\frac{10^x}{x!}-N\e{-10}\sum_{x=0}^\infty\Sbr{\frac{N-1}N}^x\frac{10^x}{x!} $$
$$ =N\e{-10}\e{10}-N\e{-10}\sum_{x=0}^\infty\frac{\prn{\frac{N-1}N10}^x}{x!} $$
$$ =N-N\e{-10}\e{\frac{N-1}N10}\sum_{x=0}^\infty\frac{\e{-\frac{N-1}N10}\prn{\frac{N-1}N10}^x}{x!} $$
The infinite sum on the right is the sum of the mass of the Poisson distribution with parameter $\frac{N-1}N10$. Hence
$$ \evt{number stops before discharging all}=N-N\e{-10}\e{\frac{N-1}N10} $$
$$ =N-N\e{\prn{\frac{N-1}N-1}10}=N-N\e{\frac{N-1-N}N10}=N\prn{1-\e{-\frac{10}N}} $$
(7.57) Suppose that the expected number of accidents per week at an industrial plant is $5$. Suppose also that the numbers of workers injured in each accident are independent random variables with a common mean of $2.5$. If the number of workers injured in each accident is independent of the number of accidents that occur, compute the expected number of workers injured in a week.
Solution Let $A$ denote the number of accidents in a week. Let $W_i$ denote the number of workers injured in accident $i$.
$$ \EB{\sum_{i=1}^AW_i}=\EL{\EBC{\sum_{i=1}^AW_i}{A}}=\sum_{a=0}^\infty\EBC{\sum_{i=1}^AW_i}{A=a}\pr{A=a} \tag{7.57.1} $$
Note that
$$ \EBC{\sum_{i=1}^AW_i}{A=a}=\EBC{\sum_{i=1}^aW_i}{A=a}=\sum_{i=1}^a\Ec{W_i}{A=a}=\sum_{i=1}^a\E{W_i}=2.5a \tag{7.57.2} $$
The third equality holds because the number of workers injured in each accident is independent of the number of accidents that occur. The last equality holds because the $W_i$ have common mean 2.5.
7.57.2 implies that $\Ebc{\sum_{i=1}^AW_i}{A}=2.5A$. Hence 7.57.1 becomes
$$ \EB{\sum_{i=1}^AW_i}=\EL{\EBC{\sum_{i=1}^AW_i}{A}}=\E{2.5A}=2.5\E{A}=2.5\wts5=12.5 $$
If we assume that the numberly of weekly accidents is Poisson, then we can compute 7.57.1 like this:
$$ \EB{\sum_{i=1}^AW_i}=\sum_{a=0}^\infty\EBC{\sum_{i=1}^AW_i}{A=a}\pr{A=a}=\sum_{a=0}^\infty2.5a\e{-5}\frac{5^a}{a!}=2.5\sum_{a=1}^\infty a\e{-5}\frac{5\wts5^{a-1}}{a!} $$
$$ =2.5\wts5\sum_{a=1}^\infty\e{-5}\frac{5^{a-1}}{(a-1)!}=12.5\sum_{n=0}^\infty\e{-5}\frac{5^n}{n!}=12.5 $$
The last equation holds because the infinite series is the sum of the mass of a Poisson distribution.
(7.58.a) A coin having probability $p$ of coming up heads is continually flipped until both heads and tails have appeared. Find the expected number of flips.
Solution Let $F$ denote the outcome of the first flip. Let $N$ denote the number of flips after the first flip until both heads and tails have appeared. Note that $N\bar F=h$ is geometric with parameter $1-p$ and $N\bar F=t$ is geometric with parameter $p$. Then
$$ \E{N}=\E{\Ec{N}{F}}=\Ec{N}{F=h}\pr{F=h}+\Ec{N}{F=t}\pr{F=t}=\frac1{1-p}p+\frac1p(1-p) $$
So
$$ \evt{number flips until both}=\E{1+N}=1+\frac1{1-p}p+\frac1p(1-p) $$
(7.58.b) Find the probability that the last flip lands on heads.
Solution The event that the last flip lands on heads is the same event that the first flip lands on tails (each implies the other). This probability is $1-p$.
(7.59.a) There are $n+1$ participants in a game. Each person independently is a winner with probability $p$. The winners share a total prize of $1$ unit. (For instance, if $4$ people win, then each of them receives $\frac14$, whereas if there are no winners, then none of the participants receive anything.) Let $A$ denote a specified one of the players, and let $X$ denote the amount that is received by $A$.
Compute the expected total prize shared by the players.
Solution Let $Y$ be the total prize shared by the players. If there is no winner, then $Y=0$. If there is at least one winner, then $Y=1$. Then $Y$ is an indicator on the event that there is at least one winner. Since each person independently wins with probability $p$, the probability that there is no winner is $(1-p)^{n+1}$ and
$$ \E{Y}=\pr{Y=1}=1-\pr{Y=0}=1-(1-p)^{n+1} $$
(7.59.b) Argue that
$$ \E{X}=\frac{1-(1-p)^{n+1}}{n+1} $$
Solution Label the players $1$ through $n+1$ and let $Y_i$ denote the prize of the $i^{th}$ player. Note that $Y=\sum_{i=1}^{n+1}Y_i$ is the total prize shared by the players.
Note that each players wins independently of the others, with the same probability, and in equal amounts. Hence the expected prize of each player is the same. That is, symmetry implies equal expected prizes. In particular, $\E{Y_i}=\E{X}$ for all $i=1,…,n+1$ and
$$ 1-(1-p)^{n+1}=\E{Y}=\sum_{i=1}^{n+1}\E{Y_i}=(n+1)\E{X} $$
or
$$ \E{X}=\frac{1-(1-p)^{n+1}}{n+1} $$
(7.59.c) Compute $\E{X}$ by conditioning on whether $A$ is a winner, and conclude that
$$ \E{(1+B)^{-1}}=\frac{1-(1-p)^{n+1}}{(n+1)p} $$
when $B$ is a binomial random variable with parameters $n$ and $p$.
Solution Let $I$ denote the indicator that $A$ wins and let $B$ denote the number of winners excluding $A$. Then
$$ \E{X}=\E{\Ec{X}{I}} $$
$$ =\Ec{X}{I=0}\wts\pr{I=0}+\Ec{X}{I=1}\wts\pr{I=1} $$
$$ =0\wts(1-p)+\EB{\frac1{1+B}}\wts p=p\wts\E{(1+B)^{-1}} $$
Hence
$$ \frac{1-(1-p)^{n+1}}{n+1}=\E{X}=p\wts\E{(1+B)^{-1}} $$
or
$$ \E{(1+B)^{-1}}=\frac{1-(1-p)^{n+1}}{p(n+1)} $$
Also note that $B$ is binomial: define a trial to be the outcome of each non-$A$ player. Then there are $n$ such trials and they’re independent. Also each trial results in success with probability $p$ and loss with probability $1-p$. Hence $B$ is binomial with parameters $n$ and $p$.
(7.61.a) Let $X_1,…$ be independent random variables with the common distribution function $F$, and suppose they are independent of $N$, a geometric random variable with parameter $p$. Let $M=\max(X_1,…,X_N)$.
Find $\pr{M\leq x}$ by conditioning on $N$.
Solution Conditioning on $N$, we get
$$ \pr{M\leq x}=\sum_{n=1}^\infty\cp{M\leq x}{N=n}\pr{N=n} \tag{7.61.a.1} $$
Note that
$$ \cp{M\leq x}{N=n}=\cp{\max(X_1,...,X_N)\leq x}{N=n}=\cp{\max(X_1,...,X_n)\leq x}{N=n} $$
Also note that the sets $\set{\max(X_1,…,X_n)\leq x}$ and $\cap_{i=1}^n\set{X_i\leq x}$ are equivalent. Notice that the latter event can also be written as $\set{X_i\leq x\text{ for all }1\leq i\leq n}$. Then:
Suppose that $\set{\max(X_1,…,X_n)\leq x}$ occurs. Then the largest of the $\set{X_i}$ is less than or equal to $x$. Hence all of the $\set{X_i}$ are less than or equal to $x$. Conversely, suppose that all of the $\set{X_i}$ are less than or equal to $x$. Then the max of the $\set{X_i}$ must be less than or equal to $x$. Hence these events are equivalent.
Hence
$$ \cp{M\leq x}{N=n}=\cp{\max(X_1,...,X_n)}{N=n}=\cp{\cap_{i=1}^n\set{X_i\leq x}}{N=n} $$
$$ =\pr{\cap_{i=1}^n\set{X_i\leq x}}=\prod_{i=1}^n\pr{X_i\leq x}=F^n(x) \tag{7.61.a.2} $$
The third equality holds because the $\set{X_i}$ are independent of $N$. The fourth equality holds because the $\set{X_i}$ are independent. And the last equality holds because the $\set{X_i}$ share the common distribution function $F$.
Since $N$ is geometric with parameter $p$, then $\pr{N=n}=p(1-p)^{n-1}$. Then 7.61.a.1 becomes
$$ \pr{M\leq x}=\sum_{n=1}^\infty\cp{M\leq x}{N=n}\pr{N=n}=\sum_{n=1}^\infty F^n(x)p(1-p)^{n-1} $$
$$ =pF(x)\sum_{n=1}^\infty\prn{F(x)(1-p)}^{n-1}=pF(x)\sum_{m=0}^\infty\prn{F(x)(1-p)}^{m}=\frac{pF(x)}{1-F(x)(1-p)} $$
In the next-to-last equality, we have reindexed $m=n-1\iff 1=n=m+1\iff m=0$. In the last equality, note that $F(x)\leq1$ and $1-p<1$. Hence this is a geometric series.
(7.61.b) Find $\cp{M\leq x}{N=1}$.
Solution In 7.61.a.2, we showed that $\cp{M\leq x}{N=n}=F^n(x)$. In particular $\cp{M\leq x}{N=1}=F(x)$.
(7.61.c) Find $\cp{M\leq x}{N>1}$.
Solution I’m not able to find this. The solution given in the manual is $\cp{M\leq x}{N>1}=F(x)\pr{M\leq x}$.
(7.61.d) Use (b) and (c) to rederive the probability you found in (a).
Solution
$$ \pr{M\leq x}=\cp{M\leq x}{N=1}\pr{N=1}+\cp{M\leq x}{N>1}\pr{N>1} $$
$$ =F(x)p+F(x)\pr{M\leq x}(1-p) $$
$\iff$
$$ F(x)p=\pr{M\leq x}-F(x)\pr{M\leq x}(1-p)=\pr{M\leq x}\prn{1-F(x)(1-p)} $$
$\iff$
$$ \frac{F(x)p}{1-F(x)(1-p)}=\pr{M\leq x} $$
(7.63.a) An urn contains $30$ balls, of which $10$ are red and $8$ are blue. From this urn, $12$ balls are randomly withdrawn. Let $X$ denote the number of red and $Y$ the number of blue balls that are withdrawn. Find $\covw{X}{Y}$ by defining appropriate indicator (that is, Bernoulli) random variables $X_i$ and $Y_j$ such that
$$ X=\sum_{i=1}^{10}X_i\dq Y=\sum_{j=1}^8Y_j $$
Solution Number the red balls and let $X_i$ equal $1$ if the $i^{th}$ red ball is selected and let it be $0$ otherwise. Similarly, number the blue balls and let $Y_j$ equal $1$ if the $j^{th}$ blue ball is selected and let it be $0$ otherwise. Then $X=\sum_{i=1}^{10}X_i$ and $Y=\sum_{j=1}^8Y_j$ and
$$ \covw{X}{Y}=\covB{\sum_{i=1}^{10}X_i}{\sum_{j=1}^8Y_j}=\sum_{i=1}^{10}\sum_{j=1}^{8}\covw{X_i}{Y_j} $$
Since $12$ balls are randomly selected from $30$, the probability that any given ball is selected is $\frac{12}{30}$. In particular, $\E{X_i}=\prt{red ball i selected}=\frac{12}{30}$ and $\E{Y_j}=\prt{blue ball j selected}=\frac{12}{30}$. Also note that
$$ \E{X_iY_j}=\prt{red ball i and blue ball j are both selected}=\frac{\binom22\binom{28}{10}}{\binom{30}{12}} $$
Hence
$$ \covw{X_i}{Y_j}=\E{X_iY_j}-\E{X_i}\E{Y_j}=\frac{\binom{28}{10}}{\binom{30}{12}}-\fracpB{12}{30}^2 $$
and
$$ \covw{X}{Y}=\sum_{i=1}^{10}\sum_{j=1}^{8}\covw{X_i}{Y_j}=\sum_{i=1}^{10}\sum_{j=1}^{8}\SbrL{\frac{\binom{28}{10}}{\binom{30}{12}}-\fracpB{12}{30}^2} $$
$$ =80\SbrL{\frac{\frac{28!}{18!10!}}{\frac{30!}{18!12!}}-\frac{12^2}{30^2}}=80\Sbr{\frac{28!}{18!10!}\frac{18!12!}{30!}-\frac{12^2}{30^2}}=80\Sbr{\frac{12\wts11}{30\wts29}-\frac{12^2}{30^2}} $$
In [1269]: 80*(12*11/(30*29)-12*12/(30*30))
Out[1269]: -0.6620689655172418
(7.63.b) Find $\covw{X}{Y}$ by conditioning (on either $X$ or $Y$) to determine $\E{XY}$.
Solution
$$ \E{XY}=\E{\Ec{XY}{X}} \tag{7.63.b.1} $$
Now, given that $X=x$, then $x$ red balls have been selected. Hence we have $20=30-10$ balls remaining from which we select $12-x$. Hence
$$ \Ec{Y_j}{X=x}=\frac{12-x}{20} $$
and
$$ \Ec{Y}{X=x}=\EBC{\sum_{j=1}^8Y_j}{X=x}=\sum_{j=1}^8\Ec{Y_j}{X=x}=8\wts\frac{12-x}{20} $$
and
$$ \Ec{XY}{X=x}=\Ec{xY}{X=x}=x\wts\Ec{Y}{X=x}=x\wts8\wts\frac{12-x}{20}=\frac25(12x-x^2) $$
This implies that
$$ \Ec{XY}{X}=\frac25(12X-X^2) $$
Since $X$ is hypergeometric, we have
$$ \E{X}=\frac{12\wts10}{30}=4\dq\E{X^2}=4\Sbr{\frac{(12-1)(10-1)}{30-1}+1} $$
Both equations follow from example 8j, p.163, ch.4 where the first and second moments of the hypergeometric variable were computed.
Then 7.63.b.1 becomes
$$ \E{XY}=\E{\Ec{XY}{X}}=\EB{\frac25(12X-X^2)}=\frac25\prn{12\E{X}-\E{X^2}} $$
In [1258]: w763ex2=lambda n=12,m=10,N=30: n*m/N*(((n-1)*(m-1))/(N-1)+1)
In [1259]: ex2=w763ex2()
In [1260]: ex,ex2=4,w763ex2()
In [1261]: 2/5*(12*ex-ex2)
Out[1261]: 12.13793103448276
Alternatively, we can compute $\E{XY}$ by expanding 7.63.b.1:
$$ \E{XY}=\E{\Ec{XY}{X}}=\sum_{x=0}^{10}\Ec{XY}{X=x}\pr{X=x}=\sum_{x=0}^{10}\frac25(12x-x^2)\frac{\binom{10}{x}\binom{20}{12-x}}{\binom{30}{12}} $$
In [1262]: sum(2/5*(12*x-x*x)*winom(10,x)*winom(20,12-x)/winom(30,12) for x in range(11))
Out[1262]: 12.1379310344828
Also note that $Y$ is hypergeometric. So $\E{Y}=\frac{12\wts8}{30}=\frac{6\wts2\wts8}{30}=\frac{16}5$ and
$$ \covw{X}{Y}=\E{XY}-\E{X}\E{Y} $$
In [1266]: exy=2/5*(12*ex-ex2)
In [1267]: ey=12*8/30
In [1268]: exy-ex*ey
Out[1268]: -0.6620689655172409
This matches with the answer in part (a). And of course we expected a negative correlation: the more red balls you select, the less white balls you select, and vice versa.
In [1275]: varx=lambda n=12,m=10,N=30: (n*m/N)*(((n-1)*(m-1))/(N-1)+1-(n*m/N))
In [1276]: vary=lambda n=12,m=8,N=30: (n*m/N)*(((n-1)*(m-1))/(N-1)+1-(n*m/N))
In [1277]: covxy=exy-ex*ey
In [1278]: covxy/(np.sqrt(varx())*np.sqrt(vary()))
Out[1278]: -0.42640143271122061
(7.64) Type $i$ light bulbs function for a random amount of time having mean $\mu_i$ and standard deviation $\sigma_i$, $i=1,2$. A light bulb randomly chosen from a bin of bulbs is a type $1$ bulb with probability $p$ and a type $2$ bulb with probability $1−p$. Let $X$ denote the lifetime of this bulb. Find (a) $\E{X}$ and (b) $\V{X}$.
Solution Let $T$ denote the type of light bulb chosen from the bin. Then
$$ \E{X}=\E{\Ec{X}{T}}=\Ec{X}{T=1}\pr{T=1}+\Ec{X}{T=2}\pr{T=2} $$
$$ =p\mu_1+(1-p)\mu_2 $$
For the variance, define $\mu_T\equiv\Ec{X}{T}$ and $\sigma_T^2\equiv\Vc{X}{T}$. Then $\E{\mu_T}=\E{\Ec{X}{T}}=\E{X}$ and
$$ \Ec{\mu_T^2}{T=i}=\Ec{(\Ec{X}{T})^2}{T=i}=\Ec{(\Ec{X}{T=i})^2}{T=i}=\Ec{\mu_i^2}{T=i}=\mu_i^2 $$
The last equation holds because $\mu_i^2$ is a constant. And we have
$$ \E{\mu_T^2}=\E{\Ec{\mu_T^2}{T}} $$
$$ =\Ec{\mu_T^2}{T=1}\pr{T=1}+\Ec{\mu_T^2}{T=2}\pr{T=2}=p\mu_1^2+(1-p)\mu_2^2 $$
Hence
$$ \V{X}=\E{\Vc{X}{T}}+\V{\Ec{X}{T}}=\E{\sigma_T^2}+\V{\mu_T} $$
$$ =\Ec{\sigma_T^2}{T=1}\pr{T=1}+\Ec{\sigma_T^2}{T=2}\pr{T=2}+\E{\mu_T^2}-\prn{\E{\mu_T}}^2 $$
$$ =\Ec{\sigma_1^2}{T=1}p+\Ec{\sigma_2^2}{T=2}(1-p)+p\mu_1^2+(1-p)\mu_2^2-\prn{p\mu_1+(1-p)\mu_2}^2 $$
$$ =p\sigma_1^2+(1-p)\sigma_2^2+p\mu_1^2+(1-p)\mu_2^2-\prn{p\mu_1+(1-p)\mu_2}^2 \tag{7.64.1} $$
Alternatively, we can compute the variance like this:
$$ \V{X}=\E{X^2}-\prn{\E{X}}^2 $$
$$ =\E{\Ec{X^2}{T}}-\prn{p\mu_1+(1-p)\mu_2}^2 \tag{7.64.2} $$
Recall the equation $\Vc{X}{Y}=\Ec{X^2}{Y}-\prn{\Ec{X}{Y}}^2$ from p.347 (proven in my notes). This gives us
$$ \Ec{X^2}{T=i}=\Vc{X}{T=i}+\prn{\Ec{X}{T=i}}^2=\sigma_i^2+\mu_i^2 $$
Hence
$$ \E{\Ec{X^2}{T}}=\Ec{X^2}{T=1}\pr{T=1}+\Ec{X^2}{T=2}\pr{T=2} $$
$$ =p(\sigma_1^2+\mu_1^2)+(1-p)(\sigma_2^2+\mu_2^2) $$
$$ =p\sigma_1^2+(1-p)\sigma_2^2+p\mu_1^2+(1-p)\mu_2^2 $$
Substituting this into 7.64.2, we arrive at the same result as 7.64.1:
$$ \V{X}=p\sigma_1^2+(1-p)\sigma_2^2+p\mu_1^2+(1-p)\mu_2^2-\prn{p\mu_1+(1-p)\mu_2}^2 $$
(7.65) The number of winter storms in a good year is a Poisson random variable with mean $3$, whereas the number in a bad year is a Poisson random variable with mean $5$. If next year will be a good year with probability $.4$ or a bad year with probability $.6$, find the expected value and variance of the number of storms that will occur.
Solution Let $X$ denote the number of storms. Let $T$ denote the type of year and let $g$ and $b$ denote good and bad years respectively. Since $X\bar T=g$ and $X\bar T=b$ are Poisson with respective parameters $3$ and $5$, then
$$ \E{X}=\E{\Ec{X}{T}}=\Ec{X}{T=g}\pr{T=g}+\Ec{X}{T=b}\pr{T=b} $$
$$ =3\wts.4+5\wts.6=1.2+3=4.2 $$
Recall from p.146 that the second moment of a Poisson variable with parameter $\lambda$ is $\lambda^2+\lambda$. Hence
$$ \E{X^2}=\Ec{X^2}{T=g}\pr{T=g}+\Ec{X^2}{T=b}\pr{T=b} $$
$$ =(3^2+3)\wts.4+(5^2+5)\wts.6=12\wts.4+30\wts.6=4.8+18=22.8 $$
Hence
$$ \V{X}=\E{X^2}-\prn{\E{X}}^2=22.8-(4.2)^2 $$
Notice that we can plug in the formula from problem 7.64 (equation 7.64.1) and get the same answer:
$$ \V{X}=p\sigma_1^2+(1-p)\sigma_2^2+p\mu_1^2+(1-p)\mu_2^2-\prn{p\mu_1+(1-p)\mu_2}^2 $$
In [1293]: p,mu1,sig1,mu2,sig2=.4,3,np.sqrt(3),5,np.sqrt(5)
In [1294]: p*pw(sig1,2)+(1-p)*pw(sig2,2)+p*pw(mu1,2)+(1-p)*pw(mu2,2)-pw(p*mu1+(1-p)*mu2,2)
Out[1294]: 5.1600000000000001
In [1295]: 22.8-pw(4.2,2)
Out[1295]: 5.1600000000000001
(7.66) In Example 5c, compute the variance of the length of time until the miner reaches safety.
Solution Recall that the mean $\E{X}=15$ was computed in the example. Then we have
$$ \E{X^2}=\E{\Ec{X^2}{Y}}=\sum_{y=1}^3\Ec{X^2}{Y=y}\pr{Y=y} $$
$$ =\frac13\prn{9+\E{(5+X)^2}+\E{(7+X)^2}} $$
Note that $\Ec{X^2}{Y=1}=9$ since the random variable $\set{X\bar Y=1}$ is constant. That is, once the miner selects door $1$, then he will, with certainty, reach safety after $3$ hours. That is, the random variable $\set{X\bar Y=1}$ is the constant $3$. Hence the random variable $\set{X^2\bar Y=1}$ is the constant $9$.
Next, we reason that $\Ec{X^2}{Y=2}=\E{(5+X)^2}$: Given that the miner selects door $2$, then he will travel for $5$ hours and start over. That is, the random variable $\set{X\bar Y=2}$ can be written as $5+X$. Hence the random variable $\set{X^2\bar Y=2}$ can be written as $(5+X)^2$.
Similarly $\Ec{X^2}{Y=3}=\E{(7+X)^2}$. Continuing
$$ \E{X^2}=\frac13\prn{9+\E{(5+X)^2}+\E{(7+X)^2}} $$
$$ =\frac13\prn{9+\E{25+10X+X^2}+\E{49+14X+X^2}} $$
$$ =\frac13\prn{83+24\E{X}+2\E{X^2}} $$
$$ =\frac13\prn{83+24\wts15+2\E{X^2}} $$
$$ =\frac13\prn{83+360+2\E{X^2}} $$
$$ =\frac13\prn{443+2\E{X^2}} $$
This is equivalent to
$$ \E{X^2}-\frac23\E{X^2}=\frac13\wts443 $$
Hence $\E{X^2}=443$ and
$$ \V{X}=\E{X^2}-\prn{\E{X}}^2=443-15^2 $$
(7.68.a) The number of accidents that a person has in a given year is a Poisson random variable with mean $\lambda$. However, suppose that the value of $\lambda$ changes from person to person, being equal to $2$ for $60$ percent of the population and $3$ for the other $40$ percent. If a person is chosen at random, what is the probability that he will have $0$ accidents in a certain year?
Solution This problem is similar to 6.43. Let $N$ denote the number of accidents in a certain year for the randomly chosen person. Then $\set{N\bar \lambda=2}$ is Poisson with mean $2$ and $\set{N\bar \lambda=3}$ is Poisson with mean $3$.
$$ \pr{N=0}=\cp{N=0}{\lambda=2}\pr{\lambda=2}+\cp{N=0}{\lambda=3}\pr{\lambda=3} $$
$$ =\e{-2}\frac{2^0}{0!}\wts0.6+\e{-3}\frac{3^0}{0!}\wts0.4=0.6\e{-2}+0.4\e{-3} $$
(7.68.b) If a person is chosen at random, what is the probability that he will have exactly $3$ accidents in a certain year?
Solution
$$ \pr{N=3}=\cp{N=3}{\lambda=2}\pr{\lambda=2}+\cp{N=3}{\lambda=3}\pr{\lambda=3} $$
$$ =\e{-2}\frac{2^3}{3!}\wts0.6+\e{-3}\frac{3^3}{3!}\wts0.4 $$
(7.68.c) What is the conditional probability that he will have $3$ accidents in a given year, given that he had no accidents the preceding year?
Solution Let $N_p$ and $N_c$ denote respectively the number of accidents in the preceding and current years of the randomly chosen person. Then we have
$$ \cp{N_c=3}{N_p=0}=\frac{\pr{N_c=3,N_p=0}}{\pr{N_p=0}} \tag{7.68.c.1} $$
From part (a), we have $\pr{N_p=0}=0.6\e{-2}+0.4\e{-3}$.
Now let’s look at the numerator. Recall problem 3.39. In that problem, we discuss unconditionally dependent and conditionally dependent.
We will assume that $N_p$ and $N_c$ are unconditionally dependent. That is, given no information about the randomly chosen person’s accident parameter $\lambda$, we assume that $N_p$ and $N_c$ are dependent. This is a reasonable assumption: if the person had no accidents the previous year, then they are more likely to have $1$ accident this year than they are to have $7$ accidents this year.
We will also assume that $N_p$ and $N_c$ are conditionally independent. That is, given that the accident parameter is known, say $\lambda=i$ for $i=2$ or $i=3$, then $N_p$ and $N_c$ are independent. That is
$$ \cp{N_c=3,N_p=0}{\lambda=2}=\cp{N_c=3}{\lambda=2}\wts\cp{N_p=0}{\lambda=2} $$
$$ =\e{-2}\frac{2^3}{3!}\e{-2}\frac{2^0}{0!}=\e{-4}\frac{2^3}{3!} $$
and
$$ \cp{N_c=3,N_p=0}{\lambda=3}=\cp{N_c=3}{\lambda=3}\wts\cp{N_p=0}{\lambda=3} $$
$$ =\e{-3}\frac{3^3}{3!}\e{-3}\frac{3^0}{0!}=\e{-6}\frac{3^3}{3!} $$
Then, conditioning the numerator in 7.68.c.1, we get
$$ \pr{N_c=3,N_p=0} $$
$$ =\cp{N_c=3,N_p=0}{\lambda=2}\pr{\lambda=2}+\cp{N_c=3,N_p=0}{\lambda=3}\pr{\lambda=3} $$
$$ =\e{-4}\frac{2^3}{3!}\wts0.6+\e{-6}\frac{3^3}{3!}\wts0.4 $$
Hence
$$ \cp{N_c=3}{N_p=0}=\frac{\pr{N_c=3,N_p=0}}{\pr{N_p=0}}=\frac{0.6\e{-4}\frac{2^3}{3!}+0.4\e{-6}\frac{3^3}{3!}}{0.6\e{-2}+0.4\e{-3}} $$
(7.69.a) Repeat Problem 68 when the proportion of the population having a value of $\lambda$ less than $x$ is equal to $1−\e{−x}$.
Solution The question gives
$$ \pr{\lambda<x}=1-\e{-x} $$
This is the probability that the accident parameter $\lambda$ of the randomly chosen person is less than $x$. This implies that $\lambda$ is an exponential random variable with parameter $1$, p.208.
In problem 68, we noted that $\set{N\bar\lambda}$ is Poisson with parameter $\lambda$. Hence equation 5.8 on p.344 gives
$$ \pr{N=0}=\int_{-\infty}^\infty\cp{N=0}{\lambda=x}\pdfa{x}{\lambda}dx $$
$$ =\int_{0}^\infty\e{-x}\frac{x^0}{0!}1\e{-1x}dx=\int_{0}^\infty\e{-x}\e{-x}dx=\frac12\int_0^\infty2\e{-2x}dx=\frac12 $$
The last equality follows because the integrand is the density of the exponential distribution with parameter $2$.
(7.69.b)
Solution
$$ \pr{N=3}=\int_{-\infty}^\infty\cp{N=3}{\lambda=x}\pdfa{x}{\lambda}dx $$
$$ =\int_{0}^\infty\e{-x}\frac{x^3}{3!}1\e{-1x}dx=\frac1{3!}\int_{0}^\infty\e{-2x}x^3dx $$
$$ y=2x\dq x=\frac{y}2\dq dx=\frac{dy}2\dq y_0=2x_0=0\dq y_1=2x_1=\infty $$
$$ =\frac1{3!}\int_{0}^\infty\e{-2x}x^3dx=\frac1{3!}\int_0^\infty\e{-y}\fracpB{y}2^3\frac{dy}2=\frac1{3!2^4}\int_0^\infty\e{-y}y^3dy $$
$$ =\frac1{3!2^4}\GammaF{4}=\frac{3!}{3!2^4}=\frac1{16} $$
(7.69.c)
Solution As in problem 7.68.c, we will assume that $N_p$ and $N_c$ are conditionally independent. That is, given that the accident parameter $\lambda$ is known, then $N_p$ and $N_c$ are independent. That is
$$ \cp{N_c=3,N_p=0}{\lambda=x}=\cp{N_c=3}{\lambda=x}\wts\cp{N_p=0}{\lambda=x} $$
$$ =\e{-x}\frac{x^3}{3!}\e{-x}\frac{x^0}{0!}=\e{-2x}\frac{x^3}{3!} $$
Equation 5.8 on p.344 gives
$$ \pr{N_c=3,N_p=0}=\int_{-\infty}^\infty\cp{N_c=3,N_p=0}{\lambda=x}\pdfa{x}{\lambda}dx $$
$$ =\int_{0}^\infty\e{-2x}\frac{x^3}{3!}1\e{-1x}dx=\frac1{3!}\int_{0}^\infty\e{-3x}x^3dx $$
$$ y=3x\dq x=\frac{y}3\dq dx=\frac{dy}3\dq y_0=3x_0=0\dq y_1=3x_1=\infty $$
$$ =\frac1{3!}\int_{0}^\infty\e{-3x}x^3dx=\frac1{3!}\int_0^\infty\e{-y}\fracpB{y}3^3\frac{dy}3=\frac1{3!3^4}\int_0^\infty\e{-y}y^3dy $$
$$ =\frac1{3!3^4}\GammaF{4}=\frac{3!}{3!3^4}=\frac1{81} $$
Hence
$$ \cp{N_c=3}{N_p=0}=\frac{\pr{N_c=3,N_p=0}}{\pr{N_p=0}}=\frac{\frac1{81}}{\frac12}=\frac2{81} $$
(7.70.a) Consider an urn containing a large number of coins, and suppose that each of the coins has some probability $p$ of turning up heads when it is flipped. However, this value of $p$ varies from coin to coin. Suppose that the composition of the urn is such that if a coin is selected at random from it, then the $p$-value of the coin can be regarded as being the value of a random variable that is uniformly distributed over $[0, 1]$.
If a coin is selected at random from the urn and flipped twice, compute the probability that the first flip results in a head.
Solution Let $U$ denote the $p$-value of the coin that is selected from the urn. Then $U$ is uniform on $[0,1]$. Let $X$ equal $1$ if the selected coin is flipped to heads, and otherwise $0$. Then $\set{X\bar U=p}$ is Bernoulli with parameter $p$. Hence equation 5.8 on p.344 gives
$$ \pr{X=1}=\int_{-\infty}^\infty\cp{X=1}{U=p}\pdfa{p}{U}dp=\int_0^1p\wts1dp=\frac{1^2}2=\frac12 $$
(7.70.b) If a coin is selected at random from the urn and flipped twice, compute the probability that both flips result in heads.
Solution Let $Y$ denote the number of heads from the two flips. Then $\set{Y\bar U=p}$ is binomial with parameters $n=2$ and $p=p$ and
$$ \pr{Y=2}=\int_{-\infty}^\infty\cp{Y=2}{U=p}\pdfa{p}{U}dp $$
$$ =\int_0^1\binom22p^2(1-p)^{2-2}\wts1dp=\int_0^1p^2dp=\frac{1^3}3=\frac13 $$
(7.71) In Problem 70, suppose that the coin is tossed $n$ times. Let $X$ denote the number of heads that occur. Show that
$$ \pr{X=i}=\frac1{n+1}\qd i=0,1,...,n $$
Hint: Make use of the fact that
$$ \int_0^1x^{a-1}(1-x)^{b-1}dx=\frac{(a-1)!(b-1)!}{(a+b-1)!} $$
when $a$ and $b$ are positive integers.
Solution Note that $\set{X\bar U=p}$ is binomial with parameters $n$ and $p$. Hence
$$ \pr{X=i}=\int_{-\infty}^\infty\cp{X=i}{U=p}\pdfa{p}{U}dp $$
$$ =\int_0^1\binom{n}ip^i(1-p)^{n-i}\wts1dp $$
$$ =\frac{n!}{(n-i)!i!}\int_0^1p^i(1-p)^{n-i}\wts1dp $$
$$ =\frac{n!}{(n-i)!i!}\frac{i!(n-i)!}{(i+n-i+1)!} \tag{7.71.1} $$
$$ =\frac{n!}{(n+1)!}=\frac1{n+1} $$
In 7.71.1, set $a=i+1$ and $b=n-i+1$. This is equivalent to $a-1=i$ and $b-1=n-i$. Hence $a+b-1=a-1+b=i+n-i+1$.
(7.72.a) Suppose that in problem 70 we continue to flip the coin until a head appears. Let $N$ denote the number of flips needed. Find $\pr{N\geq i},i\geq0$.
Solution Note that $\set{N\bar U=p}$ is geometric with parameter $p$. On p.156 and in my ch.4 notes, Example 8a, Part 2, we show that $\cp{N\geq i}{U=p}=(1-p)^{i-1}$. Hence
$$ \pr{N\geq i}=\int_{-\infty}^\infty\cp{N\geq i}{U=p}\pdfa{p}{U}dp=\int_0^1(1-p)^{i-1}\wts1dp $$
$$ u=1-p\dq p=1-u\dq dp=-du\dq u_0=1-p_0=1\dq u_1=1-p_1=0 $$
$$ =\int_0^1(1-p)^{i-1}\wts1dp=-\int_1^0u^{i-1}du=\int_0^1u^{i-1}du $$
$$ =\cases{\frac{1^{i-1+1}}{i-1+1}=\frac1i&i\geq1\\\ln{1}-\ln{0}=0--\infty=\infty=\frac1i&i=0} $$
Succinctly
$$ \pr{N\geq i}=\frac1i\qd i\geq0 $$
Of course it makes no sense that $\pr{N\geq0}=\infty$. I think the author meant to ask this for $i\geq1$.
(7.72.b) Find $\pr{N=i}$.
Solution
$$ \pr{N=i}=\pr{N\geq i}-\pr{N>i}=\pr{N\geq i}-\pr{N\geq i+1} $$
$$ =\frac1i-\frac1{i+1}=\frac{i+1}{i(i+1)}-\frac{i}{i(i+1)}=\frac{i+1-i}{i(i+1)}=\frac1{i(i+1)} $$
(7.72.c) Find $\E{N}$.
Solution Theoretical exercise 4.4 gives the first equality:
$$ \E{N}=\sum_{i=1}^\infty\pr{N\geq i}=\sum_{i=1}^\infty\frac1i\approx\int_1^\infty\frac{dx}x=\ln{\infty}-\ln{1}=\infty $$
Alternatively, we have
$$ \E{N}=\sum_{i=1}^\infty i\pr{N=i}=\sum_{i=1}^\infty i\frac1{i(i+1)}=\sum_{i=1}^\infty\frac1{i+1}\approx\int_1^\infty\frac{dx}{x+1} $$
$$ u=x+1\dq x=u-1\dq dx=du\dq u_0=x_0+1=2\dq u_1=x_1+1=\infty $$
$$ =\int_1^\infty\frac{dx}{x+1}=\int_2^\infty\frac{du}{u}=\ln{\infty}-\ln{2}=\infty $$
(7.73.a) In Example 6b, let $S$ denote the signal sent and $R$ the signal received. Compute $\E{R}$.
Solution Recall from the example that “if a signal value $s$ is sent from location $A$, then the signal value received at location $B$ is normally distributed with parameters $(s,1)$.” That is, given $S=s$, then the mean of $R$ is $s$. That is $\Ec{R}{S=s}=s$. Since this is true for all $s$, this implies that $\Ec{R}{S}=S$ and
$$ \E{R}=\E{\Ec{R}S}=\E{S}=\mu $$
The last equality follows because of the statement “S … is normally distributed with parameters $(\mu,\sigma^2)$”.
(7.73.b) Compute $\V{R}$.
Solution Recall again that “if a signal value $s$ is sent from location $A$, then the signal value received at location $B$ is normally distributed with parameters $(s,1)$.” That is, given $S=s$, then the variance of $R$ is $1$. That is, $\Vc{R}{S=s}=1$. Since this is true for all $s$, this implies that $\Vc{R}{S}=1$ and
$$ \V{R}=\E{\Vc{R}S}+\V{\Ec{R}S}=\E{1}+\V{S}=1+\sigma^2 $$
The first equality follows from proposition 5.2 on p.348. The last equality follows because of the statement “S … is normally distributed with parameters $(\mu,\sigma^2)$”.
(7.73.c) Is $R$ normally distributed?
INCOMPLETE
$$ \int\expCbr{-\frac{\prn{s-\frac{\mu+r\sigma^2}{1+\sigma^2}}^2}{2\frac{\sigma^2}{1+\sigma^2}}}ds $$
$$ \Prn{s-\frac{\mu+r\sigma^2}{1+\sigma^2}}^2=\Prn{\frac{s(1+\sigma^2)}{1+\sigma^2}-\frac{\mu+r\sigma^2}{1+\sigma^2}}^2=\Prn{\frac{s(1+\sigma^2)-(\mu+r\sigma^2)}{1+\sigma^2}}^2 $$
$$ =\frac1{(1+\sigma^2)^2}[s(1+\sigma^2)-(\mu+r\sigma^2)]^2=\frac1{(1+\sigma^2)^2}[s+s\sigma^2-\mu-r\sigma^2]^2 $$
$$ =\frac1{(1+\sigma^2)^2}[(s-\mu)+\sigma^2(s-r)]^2 $$
$$ =\frac1{(1+\sigma^2)^2}[(s-\mu)^2+\sigma^4(s-r)^2+2\sigma^4(s-r)(s-\mu)] $$
(7.73.d) Compute $\covw{R}S$.
Solution Note that
$$ \E{RS}=\E{\Ec{RS}{S}}=\int_{-\infty}^\infty\Ec{RS}{S=s}\pdfa{s}{S}ds $$
$$ =\int_{-\infty}^\infty\Ec{Rs}{S=s}\pdfa{s}{S}ds $$
$$ =\int_{-\infty}^\infty s\Ec{R}{S=s}\pdfa{s}{S}ds $$
$$ =\E{S\Ec{R}{S}}=\E{S^2}=\V{S}+\prn{\E{S}}^2=\sigma^2+\mu^2 $$
If this is confusing, recall that $\Ec{RS}S$ is a function of $S$ and define $g(S)\equiv\Ec{RS}S$. Then
$$ g(s)=\Ec{RS}{S=s}=\Ec{Rs}{S=s}=s\Ec{R}{S=s}=s^2 $$
and
$$ \E{RS}=\E{\Ec{RS}{S}}=\E{g(S)}=\int_{-\infty}^\infty g(s)\pdfa{s}{S}ds=\int_{-\infty}^\infty s^2\pdfa{s}{S}ds=\E{S^2} $$
Succinctly, we have
$$ \E{RS}=\E{\Ec{RS}{S}}=\E{S\Ec{R}S}=\E{S^2}=\sigma^2+\mu^2 $$
Hence
$$ \covw{R}S=\E{RS}-\E{R}\E{S}=\sigma^2+\mu^2-\mu\mu=\sigma^2 $$
7.75.a The moment generating functions of $X$ and $Y$ are given by
$$ \mgf{X}=\e{2\e{t}-2}\dq\mgf{Y}=\prn{\frac34\e{t}+\frac14}^{10} $$
If $X$ and $Y$ are independent, what is $\pr{X+Y=2}$?
Solution From example 7b (or table 7.1, p.358), we see that $X$ is Poisson with mean $2$. From example 7a, we see that $Y$ is binomial with parameters $10, \frac34$. Then
$$ \pr{X+Y=2}=\pr{X=0,Y=2}+\pr{X=1,Y=1}+\pr{X=2,Y=0} $$
$$ =\pr{X=0}\pr{Y=2}+\pr{X=1}\pr{Y=1}+\pr{X=2}\pr{Y=0} $$
$$ =\e{-2}\frac{2^0}{0!}\binom{10}2\fracpB34^2\fracpB14^{10-2}+\e{-2}\frac{2^1}{1!}\binom{10}1\fracpB34^1\fracpB14^{10-1}+\e{-2}\frac{2^2}{2!}\binom{10}0\fracpB34^0\fracpB14^{10-0} $$
7.75.b What is $\pr{XY=0}$
Solution
$$ \pr{XY=0}=\pr{X=0\cup Y=0} $$
$$ =\pr{X=0}+\pr{Y=0}-\pr{X=0=Y} \tag{Inclusion-exclusion} $$
$$ =\e{-2}+\fracpB14^{10}-\e{-2}\fracpB14^{10} $$
7.75.c What is $\E{XY}$
Solution It’s given that $X$ and $Y$ are independent. Proposition 4.1 gives $\E{XY}=\E{X}\E{Y}=2\wts10\wts\frac34$.