7 Further topics on random variables

7.1 Linear functions of random variables

Given random variables \(X_1, X_2, ..., X_n\) and constants \(a_1, a_2,..., a_n \in \mathbb{R}\)

\[ Y=a_1X_1+a_2X_2+...+a_nX_n \]

is a linear combination of \(X_1, X_2, ..., X_n\).

Mean of Linear function

\[ E(Y)= a_1E(X_1)+a_2E(X_2)+...+a_nE(X_n) \]

Variance of Linear function

\[ V(Y)=a_1^2 V(X_1)+a_2^2 V(X_2)+...+a_n^2 V(X_n)+2\sum_{i<j}\sum a_i a_j Cov(X_i,X_j) \]

If \(X_1, X_2, ..., X_n\) are independent then,

\[ V(Y)=a_1^2 V(X_1)+a_2^2 V(X_2)+...+a_n^2 V(X_n) \]

If \(n\) random variables \(X_i\) have common mean \(\mu\) and common variance \(\sigma^2\) then,

\(E(Y) = (a_1 + a_2 + ... + a_n)\mu\)
\(Var(Y) = (a_1^2 + a_2^2 + ... + a_n^2)\sigma^2\)

Average of Independent Random Variables:

\(X_1,X_2,...,X_n\) are \(n\) independent random variables

\(\overline{X} = \frac{X_1+X_2+...+X_n}{n}\)
\(E[\overline{X}] = \frac{1}{n}[E(X_1)+E(X_2)+...+E(X_n)]\)
\(Var[\overline{X}] = \frac{1}{n^2}[Var(X_1)+Var(X_2)+...+Var(X_n)]\)

If \(n\) random variables \(X_i\) have common mean \(\mu\) and common variance \(\sigma^2\) then,

\(E[\overline{X}] = \mu\)
\(Var[\overline{X}] = \frac{\sigma^2}{n}\)

Theorem

Suppose \(X\sim N(\mu_X,\sigma^2_X)\) , \(Y\sim N(\mu_Y,\sigma^2_Y)\) and the Pearson correlation coefficient is \(\rho\) . Consider a new variable \(W=aX \pm bY\).

Then the variable \(W\) also follows normal distribution with

Mean, \(E(W)=a\mu_X+b\mu_Y\)
Variance, \(Var(W)=a^2 Var(X)+b^2 Var(Y) \pm 2abCov(X,Y)\)

Problem 7.1 A random variable X is normally distributed with a mean of 100 and a variance of 100, and a random variable Y is normally distributed with a mean of 200 and a variance of 400. The random variables have a correlation coefficient equal to -0.5. Find the mean and variance of the random variable:

\[ W = 5X + 4Y \]

Problem 7.2 A random variable X is normally distributed with a mean of 500 and a variance of 100, and a random variable Y is normally distributed with a mean of 200 and a variance of 400. The random variables have a correlation coefficient equal to 0.5. Find the mean and variance of the random variable:

\[ W = 5X - 4Y \]

Problem 7.3 The nation of Olecarl, located in the South Pacific, has asked you to analyze international trade patterns. You first discover that each year it exports 10 units and imports 10 units of wonderful stuff. The price of exports is a random variable with a mean of 100 and a variance of 100. The price of imports is a random variable with a mean of 90 and a variance of 400. In addition, you discover that the prices of imports and exports have a correlation of \(\rho = -0.40\). The prices of both exports and imports follow a normal probability density function. Define the balance of trade as the difference between the total revenue from exports and the total cost of imports.

What are the mean and variance of the balance of trade?
What is the probability that the balance of trade is negative?

Problem 7.4 The nation of Waipo has recently created an economic development plan that includes expanded exports and imports. It has completed a series of extensive studies of the world economy and Waipo’s economic capability, following Waipo’s extensive 10-year educational-enhancement program. The resulting model indicates that in the next year exports will be normally distributed with a mean of 100 and a variance of 900 (in billions of Waipo yuan). In addition, imports are expected to be normally distributed with a mean of 105 and a variance of 625 in the same units. The correlation between exports and imports is expected to be +0.70. Define the trade balance as exports minus imports.

Determine the mean and variance of the trade balance (exports minus imports) if the model parameters given above are true.
What is the probability that the trade balance will be positive?

7.2 Functions of random variables: Transformations

7.2.1 Transformation of Discrete random variable

7.2.2 Transformation Continuous random variable

CDF Method:

We know \(P(X\le x)=F_X(x)\).

Let, \(Y=g(X)\). If \(g(X)\) is a one-to-one function we will have the inverse function solving for \(X\) we will have \(X=w(Y)\)

By definition

\[ F_Y (y)=P(Y\le y)=P[g(X)\le y]=P[X\le w(y)]=F_X [w(y)] \]

\[ \therefore f_Y(y)=\frac{d}{dy} F_Y(y)=f_X [w(y)] \cdot \frac{d}{dy} w(y) \]

Example 7.1 Let \(X\) be a continuous r.v with the following PDF

\[ f(x) = \begin{cases} \frac{x}{12}, & 1 < x < 5, \\ 0, & \text{elsewhere.} \end{cases} \]

Find the probability distribution of the random variable \(Y=3X−4\).

Here, \(Y=3X-4\). Solving for \(X\) we have \(X=\frac{Y+4}{3}\)

Now,

\[ f_Y(y)=\frac{d}{dy}F_X \left (\frac{y+4}{3}\right)=f_X\left (\frac{y+4}{3}\right)\cdot \frac{1}{3} \]

\[ f_Y(y)=\frac{\frac{y+4}{3}}{12}\cdot \frac{1}{3}=\frac{y+4}{108} \]

Hence the PDF of \(Y\) is:

\[ f_Y(y) = \begin{cases} \frac{y+4}{108}, & -1<y<11, \\ 0, & \text{elsewhere.} \end{cases} \]

Jacobian Method:

If \(Y=u(X)\) then we will have \(x=w(y)\)

Then the Jacobian \(J=\frac{d}{dy} w(y)\). Finally

\[ f_Y(y)=f_X[w(y)]|J| \]

Example 7.2 From previous example we have

\(x=\frac{y+4}{3}=w(y)\).

So, \(J=\frac{d}{dy}w(y)=\frac{1}{3}\)

Finally, \(f_Y(y)=f_X[w(y)]|J|=\frac{\frac{y+4}{3}}{12}\cdot \frac{1}{3}=\frac{y+4}{108}\)

Example 7.3 If\(X\sim N(\mu, \sigma^2)\) then derive the distribution of \(Z\) where \(Z=\frac{X-\mu}{\sigma}\).

Solution:

We know

\[ f_X(x)=\frac{1}{\sigma \sqrt{2\pi}}e^{-\frac{1}{2} (\frac{x-\mu}{\sigma})^2} \]

Given

\[ z=\frac{x-\mu}{\sigma} \]

\[ \Rightarrow x=\mu+z\sigma \]

Jacobian, \(J=\frac{dx}{dz}=\frac{d}{dz}(\mu+z\sigma)=\sigma\)

For \(-\infty<x<\infty\) ; \(-\infty<z<\infty\)

\[ \therefore f_Z(z)=f_X(z)\cdot |J|=\frac{1}{\sigma \sqrt{2\pi} }e^{-\frac{1}{2}z^2}\cdot\sigma=\frac{1}{\sqrt{2\pi} }e^{-\frac{1}{2}z^2} \]

NOTE: CDF and Jacobian are almost same for uni-variate case; but Jacobian is more robust when we deal with multiple random variables.

7.3 Moments Moment-Generating Functions (MGF)

7.3.1 Moments

The \(r^{th}\) raw moment defined as:

\[ \mu_r^{'} =E(X^r)= \begin{cases} \sum\limits_x x^r f(x) , & \mathrm {if \ \ X \ \ is \ \ dicrete}, \\ \int_{-\infty}^{\infty} x^r f(x), & \mathrm {if \ \ X \ \ is \ \ continuous.} \end{cases} \]

For example \(\mu= \mu_1^{'}=E(X)\) and \(\mu_2^{'}=E(X^2)\). So \(\sigma^2 =\mu_2^{'}-\mu^2\).

7.3.2 MGF

\[ M_X(t)=E(e^{tX}) \]

7.3.3 Finding moments using MGF

\[ \left. \frac{d^r M_X(t)}{dt^r} \right|_{t=0} = \mu'_r. \]

MGF of Binomial distribution

\[ M_X(t)=E(e^{tX})=\sum\limits_{x=0}^n e^{tx} \binom{n}{x} p^x q^{n-x}=\sum\limits_{x=0}^n \binom{n}{x} (pe^t)^x q^{n-x}=(pe^t+q)^n \]

Now

\[ \frac{d M_X(t)}{dt}=n(pe^t+q)^{n-1} pe^t=np(pe^t+q)^{n-1}e^t \]

Setting \(t=0\) we have \(\mu_1^{'}=np(p+q)^{n-1}.1=np.1.1=np\)

MGF of Poisson distribution

Let \(X \sim \text{Poisson}(\lambda)\). The PMF is \(P(X=x) = \frac{e^{-\lambda}\lambda^x}{x!}\) for \(x=0,1,\dots\).

The MGF is derived as follows:

\[ \begin{aligned} M_X(t) &= E[e^{tX}] = \sum_{x=0}^{\infty} e^{tx} \frac{e^{-\lambda} \lambda^x}{x!} \\ &= e^{-\lambda} \sum_{x=0}^{\infty} \frac{(\lambda e^t)^x}{x!} \\ &= e^{-\lambda} e^{\lambda e^t} \quad \text{(using } \sum \frac{z^k}{k!} = e^z \text{)} \\ &= e^{\lambda(e^t - 1)}. \end{aligned} \]

MGF of Normal Distribution

MGF of Standard Normal Distribution

MGF of Chi-square Distribution

Theorem

If \(Z_i\)’s are independent standard normal random variables then \(U=\sum_{i=1}^k Z_i^2\) has a chi-square distribution with \(k\) degrees of freedom.

Proof: