Continuous Probability Models

Statistics I

Authors

Affiliation

Disclaimer

These slides are a free translation and adaptation from the slide deck for Estatística I by Prof. Sandra Custódio and Prof. Teresa Ferreira from the Lisbon Accounting and Business School, Polytechnic University of Lisbon.

Uniform Distribution

Continuous Uniform Distribution

A r.v. \(X\) follows a uniform distribution in \([a,b]\subset\mathbb{R}\) with \(-\infty<a<b<\infty\), if its probability density is given by:

\[ f_X(x)=\begin{cases} \frac{1}{b-a} & a\leq x \leq b\\ 0 & otherwise \end{cases} \]

We write \(X\sim U[a,b]\)

Continuous Uniform Distribution

Continuous Uniform Distribution

The cumulative distribution function is given by:

\[ F_X(x)=\begin{cases} 0 & x< a\\ \frac{x-a}{b-a} & a\leq x < b\\ 1 & x\geq b \end{cases} \]

Continuous Uniform Distribution

Note, this distribution is symmetric, and its first two moments are:

\(E[X]=\frac{a+b}{2}\)
\(V[X]=\frac{(b-a)^2}{12}\)

Continuous Uniform Distribution - Example

The length of small spots in a TV network is a r.v. \(X\) distributed \(U[5,12]\).

Find its distribution function.
What is the probability of a small spot last at least 7 seconds?
The probability that a small spot lasts more than 6 seconds, given it never lasts more than 10 seconds is?
Find \(E[X]\) and \(V[X]\)

Solution

Continuous Uniform Distribution - Example

Let the r.v. \(X\) be distributed \(U[2,b]\) with \(b>2\). What value must \(b\) take to make \(P(3\leq X\leq 5)=0.4\)?

Solution

❓ Uniform · Question 1

If \(X\sim U[a,b]\), then \(E[X]\) equals:

A. \(\frac{a+b}{2}\)

B. \(\frac{b-a}{2}\)

C. \(\frac{1}{b-a}\)

D. \(\frac{(b-a)^2}{12}\)

✅ A. The Uniform is symmetric, so its mean sits at the midpoint of the interval. \(\frac{(b-a)^2}{12}\) is the variance.

❓ Uniform · Question 2

For \(X\sim U[0,10]\), the probability \(P(X=5)\) is:

A. \(0.1\)

B. \(1\)

C. \(0.5\)

D. \(0\)

✅ D. \(X\) is continuous, so every single point has probability zero, no matter how flat the density is.

✏️ Uniform · Question 3

Let \(X\sim U[5,15]\).

Compute \(E[X]\), \(V[X]\), \(P(X\leq 8)\) and \(P(7<X\leq 12)\).

✅ Uniform · Solution

\(E[X]=\frac{5+15}{2}=10\) · \(V[X]=\frac{(15-5)^2}{12}=\frac{100}{12}\approx 8.33\)

\(F_X(x)=\frac{x-5}{10}\) on \([5,15]\), so \(P(X\leq 8)=\frac{3}{10}=0.3\)

\(P(7<X\leq 12)=\frac{12-7}{10}=0.5\)

Exponential Distribution

The exponential distribution is rooted in the Poisson distribution, reflecting the waiting time between events originated according to a Poisson process.

Nevertheless, we can apply the exponential distribution to many other phenomena.

Exponential Distribution

A r.v \(X\) is distributed exponentially, with parameter \(\lambda>0\), \(X\sim Exp(\lambda)\), if its probability density function is given by:

\[ f_X(x)=\begin{cases} \lambda e^{-\lambda x} & x\geq 0\\ 0 & x< 0 \end{cases} \]

\(\lambda\) is the rate: the expected number of events per unit of time (or space). The expected waiting time between events is \(\frac{1}{\lambda}\).

Exponential Distribution

Exponential Distribution

The cumulative probability function is:

\[ F_X(x)=\begin{cases} 0 & x< 0\\ 1-e^{-\lambda x} & x\geq 0 \end{cases} \]

And its moments are:

\(E[X]=\frac{1}{\lambda}\)
\(V[X]=\frac{1}{\lambda^2}\)

Exponential Distribution

Property

Lack of memory of the exponential distribution:

Let the r.v. \(X\sim Exp(\lambda)\), then:

\[P(X>x+h|X>x)=P(X>h)\]

with \(x,h>0\)

Considering the survival application of this distribution, this property states that, the time left to live is independent of what it already lived.

Does waiting change anything?

Grey = time you have already waited with nothing happening. Blue = the original distribution from time zero. Orange dashed = the distribution of what is still to come.

already waited x =

extra wait h =

What the chart showed

Drag \(x\) anywhere you like. The orange curve slides along with it, but its shape never changes: it is the blue curve translated, not a flatter or steeper one. And the two probabilities in the panel stay equal to four decimals.

Algebraically: \(P(X>x+h\mid X>x)=\frac{e^{-\lambda(x+h)}}{e^{-\lambda x}}=e^{-\lambda h}=P(X>h)\). The \(e^{-\lambda x}\) cancels, and with it everything the past could have told you.

A component that has run 10 years is exactly as likely to survive another year as a brand new one. That is often the wrong model for wear, which is precisely why you must check whether the exponential fits before using it.

Exponential Distribution - Example

In a factory, the execution time of a piece is random variable distributed exponentially with expected value of 5 minutes.

For this distribution, what is the value of \(\lambda\)?
You know that the piece has been in execution by at least 2 minutes. What is the probability that 4 more minutes are necessary to complete the piece?
Picking randomly 5 pieces. What is the probability that 2 out of the 5 were produced in less than 4 minutes?

Solution

Exponential Distribution - Example

The time it takes until the first consultation, and between consultations, in the clinic of Dr. Shawn are independent and distributed exponentially with \(\lambda=0.1\).

What is the probability that no consultation occurs before the first 10 minutes?

Solution

❓ Exponential · Question 1

Buses arrive following an Exponential model with \(\lambda=2\) per hour. The expected waiting time for the next bus is:

A. 2 hours

B. 0.5 hours

C. 4 hours

D. 1 hour

✅ B. \(\lambda\) is a rate, here 2 buses per hour. The expected waiting time is \(\frac{1}{\lambda}=0.5\) hours, that is 30 minutes.

❓ Exponential · Question 2

Memorylessness says that \(P(X>x+h\,|\,X>x)\) equals:

A. \(P(X>h)\)

B. \(P(X>x)\)

C. \(P(X>x+h)\)

D. \(0\)

✅ A. Having already waited \(x\) tells you nothing. The clock restarts, so only the extra \(h\) matters.

✏️ Exponential · Question 3

Buses arrive with \(\lambda=0.05\) per minute.

Compute the expected waiting time, \(P(X>10)\), and \(P(X>25\,|\,X>15)\).

✅ Exponential · Solution

\(E[X]=\frac{1}{0.05}=20\) minutes.

\(P(X>10)=1-F(10)=e^{-0.05\times 10}=e^{-0.5}\approx 0.6065\)

By memorylessness, \(P(X>25|X>15)=P(X>10)\approx 0.6065\), the same number. ✅

Normal Distribution

The Gaussian or Normal Distributions is one of the most used distributions, playing a key role in statistical inference.

The r.v. \(X\) is normally distributed, \(X\sim N(\mu,\sigma^2)\), if its density and cumulative probability distribution functions:

\[ f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\\ P(X\leq a)=F(a)=\int_{-\infty}^a f(t)dt \]

Where \(E[X]=\mu\) and \(Var[X]=\sigma^2\)

Normal Distribution

Standard Normal

If \(\mu=0\) and \(\sigma^2=1\) we call this the standard normal distribution, where \(Z\sim N(0,1)\).

Theorem

Let \(X\sim N(\mu,\sigma^2)\). Let \(Z=\frac{X-\mu}{\sigma}\), then \[Z\sim N(0,1)\]

Standard Normal

That is, we can standardize a r.v. distributed Normally. Its probability function is denoted \(\Phi\), and its density is \(\phi\).

\[ \phi(z)= \frac{1}{\sqrt{2\pi}}e^{-\frac{z^2}{2}} \\ \Phi(z)=P(Z\leq z) = \int_{-\infty}^z \frac{1}{\sqrt{2\pi}}e^{-\frac{t^2}{2}} dt \]

Standard Normal

How do we know if a r.v. follows a Normal distribution?

There is no rule, you need to look at the data (histogram) and decide.
This distribution is popular because of the Law of Large Numbers and the Central Limit Theorem (covered in Statistics II)
Even though it is by far the most popular, not always it is used correctly.

Standardizing, seen side by side

Left: your \(X\sim N(\mu,\sigma^2)\). Right: the same problem after standardizing. The two shaded areas are always equal, which is why one table serves every Normal.

μ =

σ =

cut-off x =

What the chart showed

Change \(\mu\) and the left curve slides; change \(\sigma\) and it widens or narrows. The right curve never moves: standardizing has already absorbed both.

The orange cut-off tracks the same point in both pictures, at \(x\) on the left and at \(z=\frac{x-\mu}{\sigma}\) on the right, and the shaded areas stay equal to the last decimal.

This is why one page of tabulated \(\Phi(z)\) is enough for every Normal you will ever meet. You never tabulate \(X\), you convert it. Note the readout tells you the exact row and column to look up.

Example 1

Consider the r.v. \(Z\sim N(0,1)\). Find \(P(Z\leq 1.65)\).

\[P(Z\leq 1.65) = F(1.65)=\Phi(1.65)=0.9505\]

Warning

This cannot be solved using the classical integration. Other techniques (approximations) are necessary.
We will rely on Tables to find out the values for the Standard Normal distribution. Remember you can always standardize other Normal random variables.
If \(z<0\), recall that the distribution is symmetric, then \(\Phi(-z)=1-\Phi(z)\)

Example 1

Example 2

Consider the r.v. \(X\sim N(6,25)\). Find \(P(X\leq 12)\).

\[P(X\leq 12)=P\left(\frac{X-6}{5}\leq \frac{12-6}{5}\right)\]

\[P(Z\leq 1.2)=\Phi(1.2)=0.8849\]

Example 2

Example 3

If \(X\sim N(6,25)\), find \(P(6<X\leq 12)\)

\[ P(6<X\leq 12)=F(12)-F(6)= \] \[ P\left(\frac{6-6}{5}<\frac{X-6}{5}\leq \frac{12-6}{5}\right) \]

\[ P(0<Z\leq 1.2)=\Phi(1.2)-\Phi(0)\\0.8849-0.5=0.3849 \]

Example 3

Example 4

Let \(X\sim N(6,25)\). Find \(P(X\leq -4)\) and \(P(X\geq 16)\)

\[ P(X\leq -4)=P\left(\frac{X-6}{5}\leq \frac{-4-6}{5}\right)=\\ P(Z\leq -2)=\Phi(-2)=1-\Phi(2)=0.0228 \]

Example 4

Example 5

Check that \(P(X\leq -4)=P(X\geq 16)\)

Because of symmetry: \(P(X\leq \mu-k)=P(X\geq \mu+k)\) \(\forall k\in\mathbb{R}\)

\(P(X\leq -4)=P(X\leq 6-10)=\\=P(X\geq 6+10)=P(X\geq 16)\)

Corollary

\(\Phi(-k)=P(Z\leq -k)=P(Z\geq k)=1-P(Z\leq k)=1-\Phi(k)\)

Example 6

Consider the distribution \(X\sim N(6,25)\)

Find \(P(0\leq X\leq 8)\)

\(P(0\leq X \leq 8)=P(0<X\leq 8)\) which is equivalent to

\[ P(-1.2<Z\leq 0.4)=\Phi(0.4)-\Phi(-1.2)=\\ \Phi(0.4)-\left[1-\Phi(1.2)\right]=\\ 0.6554-[1-0.8849]=0.5403 \]

Example 7

Consider the r.v. \(X\sim N(6, 25)\). Find \(P(|X-6|>10)\)

\[P(|X-6|>10)=1-P(|X-6|\leq 10)\]

\[1-P(-10\leq X-6\leq 10)=1-P(-2<Z\leq 2)\]

\[1-[\Phi(2)-\Phi(-2)]=1-[\Phi(2)-1+\Phi(2)]\]

\[2-2\Phi(2)=2-2\times 0.9772=0.0456\]

Example 8

Consider the r.v. \(X\sim N(6,25)\), find \(k\) such that \(P(X>k)=0.9\)

\(P(X>k)=0.9\Leftrightarrow 1-P(X\leq k)=0.9\)

\[P(X\leq k)=0.10 \Leftrightarrow P\left(Z\leq \frac{k-6}{5}\right)=0.10\]

\[P\left(Z\leq \frac{k-6}{5}\right)=0.10\Leftrightarrow \Phi\left(\frac{k-6}{5}\right)=0.10\]

Because of symmetry we have \(P(Z\leq -z)=P(Z\geq z)\)

Example 8

Using the table, \(P(Z\geq z)=0.1\) means \(\Phi(z)=0.9\), so \(z=\Phi^{-1}(0.9)\approx 1.282\), and by symmetry \(\Phi^{-1}(0.1)\approx -1.282\). Substituting we get:

\[\Phi(-1.282)=0.1\Leftrightarrow \frac{k-6}{5}=-1.282\] \[k=6-1.282\times 5 = -0.41\]

Example 8

Normal Distribution

Theorem: Normal additivity

If \(X_1\sim N(\mu_1,\sigma_1^2)\) and \(X_2\sim N(\mu_2,\sigma_2^2)\) are independent, then for any \(a,b\in\mathbb{R}\) we have that \(T=aX_1+bX_2\), where \[T\sim N(\mu_T,\sigma_T^2)\]

To find \(\mu_T\) and \(\sigma_T^2\) remember the properties of the mean and variance.

Example 9

Let the r.v.s \(X\sim N(6,4)\) and \(Y\sim N(6,4)\), with \(T=0.5 X-Y\) Find \(\mu_T\) and \(\sigma_T^2\)

\[\mu_T=E[0.5X-Y]=0.5E[X]-E[Y]=\\ 0.5 \times 6 - 6 = -3\]

\[\sigma_T^2=V[0.5 X- Y]=V[0.5 X]+V[-Y]=\\ 0.5^2 V[X]+V[Y]= 0.25 \times 4 + 4 = 5\]

Example 9

Find \(P(T>0)\)

\(T\sim N(-3, 5)\)

\[P(T>0)=P\left(Z>\frac{0+3}{\sqrt{5}}\right)=1-P(Z\leq 1.34)=\\ 1-\Phi(1.34)=0.0901\]

Example 9

Normal Distribution

Corollaries

\(X_i\sim N(\mu,\sigma^2)\) for \(i=1,\dots,n\), i.i.d.
\(T=X_1+\dots+X_n\) with \(\bar{X}=\frac{T}{n}\)

Then:

\(T\sim N(n\times \mu, n\sigma^2)\)
\(\bar{X}\sim N\left(\mu, \frac{\sigma^2}{n}\right)\)

Example 10

Let \(X_i\sim N(120, 64)\) be r.v.s representing the number of bank deposits made in a specific day. Then \(T=X_1+\dots+X_5\) are the weekly deposits.

Find the probability of the weekly deposits exceed 620.

\(T\sim N(600, 320)\) because of the Normal additivity property.

Example 10

\[P(T>620)=P\left(Z>\frac{620-5\times 120}{\sqrt{5}\times 8}\right)=\\1-P(Z\leq 1.12)=1-\Phi(1.12)=0.1314\]

❓ Normal · Question 1

If \(X\sim N(\mu,\sigma^2)\), then \(Z=\frac{X-\mu}{\sigma}\) follows:

A. \(N(\mu,\sigma^2)\)

B. an Exponential

C. a Uniform

D. \(N(0,1)\)

✅ D. Standardizing centres the variable at 0 and rescales it to unit variance. That is why one table is enough for every Normal.

❓ Normal · Question 2

For the standard Normal, \(\Phi(-z)\) equals:

A. \(1-\Phi(z)\)

B. \(\Phi(z)\)

C. \(-\Phi(z)\)

D. \(2\Phi(z)\)

✅ A. The standard Normal is symmetric about 0, so the area to the left of \(-z\) equals the area to the right of \(z\).

✏️ Normal · Question 3

Let \(X\sim N(6,25)\).

a) Compute \(P(X\leq 14)\) and \(P(X>2)\).

b) Find \(k\) such that \(P(X\leq k)=0.90\).

c) Let \(Y\sim N(4,9)\) be independent of \(X\). Compute \(P(X+Y>15)\).

✅ Normal · Solution

Here \(\mu=6\) and \(\sigma=\sqrt{25}=5\).

a) \(P(X\leq 14)=\Phi\left(\frac{14-6}{5}\right)=\Phi(1.6)=0.9452\)

\(P(X>2)=1-\Phi\left(\frac{2-6}{5}\right)=1-\Phi(-0.8)=\Phi(0.8)=0.7881\)

b) \(\Phi(z)=0.90\) gives \(z\approx 1.28\) from the table (\(1.282\) interpolated), so \(k=6+1.282\times 5\approx 12.41\)

c) By additivity, \(X+Y\sim N(6+4,\ 25+9)=N(10,34)\), so \[P(X+Y>15)=1-\Phi\left(\frac{15-10}{\sqrt{34}}\right)=1-\Phi(0.86)=1-0.8051=0.1949\]

References

Murteira, B.; Silva Ribeiro, C.; Andrade e Silva, J. & Pimenta, C., Introdução à Estatística, Escolar Editora, McGraw-Hill, 2010
Paulino, C. D. & Branco, J. A. (2005). Exercícios de Probabilidade e Estatística. Escolar Editora
Pimenta, F., Andrade e Silva, J.; Silva Ribeiro, C. & Murteira, B., Introdução à Estatística, 3ª Edição, Escolar Editora, 2015
Ferreira, T., Custódio, S.G., Modelos Probabilísticos, Síntese Teórica e Exercícios Resolvidos, Edições Sílabo (1ª Edição), 2023

Appendix

Standard Normal Table

z	.00	.01	.02	.03	.04	.05	.06	.07	.08	.09
0.0	0.5000	0.5040	0.5080	0.5120	0.5160	0.5199	0.5239	0.5279	0.5319	0.5359
0.1	0.5398	0.5438	0.5478	0.5517	0.5557	0.5596	0.5636	0.5675	0.5714	0.5753
0.2	0.5793	0.5832	0.5871	0.5910	0.5948	0.5987	0.6026	0.6064	0.6103	0.6141
0.3	0.6179	0.6217	0.6255	0.6293	0.6331	0.6368	0.6406	0.6443	0.6480	0.6517
0.4	0.6554	0.6591	0.6628	0.6664	0.6700	0.6736	0.6772	0.6808	0.6844	0.6879
0.5	0.6915	0.6950	0.6985	0.7019	0.7054	0.7088	0.7123	0.7157	0.7190	0.7224
0.6	0.7257	0.7291	0.7324	0.7357	0.7389	0.7422	0.7454	0.7486	0.7517	0.7549
0.7	0.7580	0.7611	0.7642	0.7673	0.7704	0.7734	0.7764	0.7794	0.7823	0.7852
0.8	0.7881	0.7910	0.7939	0.7967	0.7995	0.8023	0.8051	0.8078	0.8106	0.8133
0.9	0.8159	0.8186	0.8212	0.8238	0.8264	0.8289	0.8315	0.8340	0.8365	0.8389
1.0	0.8413	0.8438	0.8461	0.8485	0.8508	0.8531	0.8554	0.8577	0.8599	0.8621
1.1	0.8643	0.8665	0.8686	0.8708	0.8729	0.8749	0.8770	0.8790	0.8810	0.8830
1.2	0.8849	0.8869	0.8888	0.8907	0.8925	0.8944	0.8962	0.8980	0.8997	0.9015
1.3	0.9032	0.9049	0.9066	0.9082	0.9099	0.9115	0.9131	0.9147	0.9162	0.9177
1.4	0.9192	0.9207	0.9222	0.9236	0.9251	0.9265	0.9279	0.9292	0.9306	0.9319
1.5	0.9332	0.9345	0.9357	0.9370	0.9382	0.9394	0.9406	0.9418	0.9429	0.9441
1.6	0.9452	0.9463	0.9474	0.9484	0.9495	0.9505	0.9515	0.9525	0.9535	0.9545
1.7	0.9554	0.9564	0.9573	0.9582	0.9591	0.9599	0.9608	0.9616	0.9625	0.9633
1.8	0.9641	0.9649	0.9656	0.9664	0.9671	0.9678	0.9686	0.9693	0.9699	0.9706
1.9	0.9713	0.9719	0.9726	0.9732	0.9738	0.9744	0.9750	0.9756	0.9761	0.9767
2.0	0.9772	0.9778	0.9783	0.9788	0.9793	0.9798	0.9803	0.9808	0.9812	0.9817
2.1	0.9821	0.9826	0.9830	0.9834	0.9838	0.9842	0.9846	0.9850	0.9854	0.9857
2.2	0.9861	0.9864	0.9868	0.9871	0.9875	0.9878	0.9881	0.9884	0.9887	0.9890
2.3	0.9893	0.9896	0.9898	0.9901	0.9904	0.9906	0.9909	0.9911	0.9913	0.9916
2.4	0.9918	0.9920	0.9922	0.9925	0.9927	0.9929	0.9931	0.9932	0.9934	0.9936
2.5	0.9938	0.9940	0.9941	0.9943	0.9945	0.9946	0.9948	0.9949	0.9951	0.9952
2.6	0.9953	0.9955	0.9956	0.9957	0.9959	0.9960	0.9961	0.9962	0.9963	0.9964
2.7	0.9965	0.9966	0.9967	0.9968	0.9969	0.9970	0.9971	0.9972	0.9973	0.9974
2.8	0.9974	0.9975	0.9976	0.9977	0.9977	0.9978	0.9979	0.9979	0.9980	0.9981
2.9	0.9981	0.9982	0.9982	0.9983	0.9984	0.9984	0.9985	0.9985	0.9986	0.9986
3.0	0.9987	0.9987	0.9987	0.9988	0.9988	0.9989	0.9989	0.9989	0.9990	0.9990

Uniform Exercise

\[ F_X(x)= \begin{cases} 0 & x<5\\ \frac{x-5}{12-5} & 5\leq x < 12\\ 1 & x\geq 12 \end{cases} \]
\(F_X(12)-F_X(7)=1-\frac{2}{7}=\frac{5}{7}\)
\(\frac{F_X(10)-F_X(6)}{F_X(10)}=\frac{5/7-1/7}{5/7}=4/5\)
\(E[X]=\frac{5+12}{2}=8.5\), \(V[X]=\frac{(12-5)^2}{12}=\frac{49}{12}\approx 4 \Rightarrow \sigma_X\approx\sqrt{4}\approx 2\)

Back to the exercise

Uniform Exercise

\(P(3\leq X\leq 5)=0.4\ \Rightarrow\ F_X(5)-F_X(3)=0.4\)
\(\frac{5-2}{b-2}-\frac{3-2}{b-2}=0.4\ \Rightarrow\ \frac{3}{b-2}-\frac{1}{b-2}=0.4\)
\(\frac{2}{b-2}=0.4\ \Rightarrow\ \frac{1}{b-2}=0.2\), but \(0.2 = \frac{2}{10} = \frac{1}{5}\)
\(\frac{1}{b-2}=\frac{1}{5}\ \Rightarrow\ 5=b-2\ \Rightarrow\ b=7\)

Back to the exercise

Exponential Exercise

\(\lambda=\frac{1}{5}=0.2\). On average 0.2 pieces are produced per minute, that is one piece every 5 minutes. \(X\sim Exp(0.2)\)
\(P(X>6|X>2)=P(X>4)=1-P(T<4)=\) \(1-\left(1-e^{-0.2\times 4}\right)=e^{-0.8}\approx 0.4493\)
This problem calls for the binomial distribution, \(N\sim Bin(n=5, p)\), however, we do not know \(p\). Here \(p\) is the probability that a single piece is produced in 4 minutes or less: \(p=P(X<4)\) with \(X\sim Exp(0.2)\), so \(p=1-e^{-0.2\times 4}\approx 0.55\). Then \(N\sim Bin(5, 0.55)\), and \(P(N=2)\approx 0.2757\).

Back to the exercise

Exponential Exercise

In this case the r.v. \(X\) is the time until the first consultation, and \(X\sim Exp(0.1)\).

\[P(X>10)=1-P(X<10)=1-F(10)\]

\[1-(1-e^{-0.1\times 10})=e^{-1}\approx 0.3679\]

Back to the exercise