7.6. Expectation

This section contains several results on expectation operator.

Any function $g(x)$ defines a new random variable $g(X)$. If $g(X)$ has a finite expectation, then

$$\mathbb{E}[g(X)]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(x)f_X(x)dx.$$

If several random variables $X_1,\dots ,X_n$ are defined on the same sample space, then their sum $X_1+\cdots +X_n$ is a new random variable. If all of them have finite expectations, then the expectation of their sum exists and is given by

$$\mathbb{E}[X_1+\cdots +X_n]=\mathbb{E}[X_1]+\cdots +\mathbb{E}[X_n].$$

If $X$ and $Y$ are mutually independent random variables with finite expectations, then their product is a random variable with finite expectation and

$$\mathbb{E}(XY)=\mathbb{E}(X)\mathbb{E}(Y).$$

By induction, if $X_1,\dots ,X_n$ are mutually independent random variables with finite expectations, then

$$\mathbb{E}\left[\prod _{i=1}^n{X}_i\right]=\prod _{i=1}^n\mathbb{E}\left[X_i\right].$$

Let $X$ and $Y$ be two random variables with the joint density function $f_{X,Y}(x,y)$. Let the marginal density function of $Y$ given $X$ be $f(y|x)$. Then the conditional expectation is defined as follows:

$$\mathbb{E}[Y|X]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}yf(y|x)dy.$$

$\mathbb{E}[Y|X]$ is a new random variable.

$$\begin{array}{rl}\mathbb{E}[\mathbb{E}[Y|X]]& ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\mathbb{E}[Y|X]f(x)dx\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}yf(y|x)f(x)dydx\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}y({\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x,y)dx)dy\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}yf(y)dy=\mathbb{E}[Y].\end{array}$$

In short, we have

$$\mathbb{E}[\mathbb{E}[Y|X]]=\mathbb{E}[Y].$$

The covariance of $X$ and $Y$ is defined as

$$\mathrm{Cov}(X,Y)=\mathbb{E}[(X-\mathbb{E}[X])(Y-\mathbb{E}[Y])].$$

It is easy to see that

$$\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y].$$

The correlation coefficient is defined as

$$\rho \triangleq \frac{\mathrm{Cov}(X,Y)}{\sqrt{Var(X)Var(Y)}}.$$

7.6.1. Independent Variables

If $X$ and $Y$ are independent, then

$$\mathbb{E}[g_1(x)g_2(y)]=\mathbb{E}[g_1(x)]\mathbb{E}[g_2(y)].$$

If $X$ and $Y$ are independent, then $\mathrm{Cov}(X,Y)=0$.

7.6.2. Uncorrelated Variables

The two variables $X$ and $Y$ are called uncorrelated if $\mathrm{Cov}(X,Y)=0$. Covariance doesn’t imply independence.