7.2. Random Variables

For different random variables, we will characterize their distributions by several parameters. These are listed below

• Probability density function (PDF)

• Cumulative distribution function (CDF)

• Probability mass function (PMF)

• Mean ($\mu$ or $\mathbb{E}(X)$)

• Variance (${\sigma }^2$ or $\mathrm{Var}(X)$)

• Skew

• Kurtosis

• Characteristic function (CF)

• Moment generating function (MGF)

• Second characteristic function

• Cumulant generating function (CGF)

7.2.1. Cumulative Distribution Function

The CDF is defined as

$$F_X(x)=\mathbb{P}(X\le x).$$

Properties of CDF:

$$F_X(x)\ge 0,F_X(-\mathrm{\infty })=0,F_X(\mathrm{\infty })=1.$$

CDF is a monotonically non-decreasing function.

$$x_1<x_2⟹F_X(x_1)\le F_X(x_2).$$

$F_X(-\mathrm{\infty })$ is defined as

$$F_X(-\mathrm{\infty })=\underset{x\to -\mathrm{\infty }}{lim}{F}_X(x).$$

Similarly:

$$F_X(\mathrm{\infty })=\underset{x\to \mathrm{\infty }}{lim}{F}_X(x).$$

$F_X(x)$ is right continuous.

$$\underset{x\to t^{+}}{lim}{F}_X(x)=F_X(t).$$

7.2.2. Probability Density Function

Properties of PDF

$$f_X(x)\ge 0.$$

$${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{f}_X(x)dx=1.$$

The CDF and PDF are related as

$$F_X(x)={\int }_{-\mathrm{\infty }}^x{f}_X(t)dt.$$

7.2.3. Expectation

Expectation of a discrete random variable:

$$\mathbb{E}(X)=\sum _{x}x{p}_X(x).$$

Expectation of a continuous random variable:

$$\mathbb{E}(X)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}t{f}_X(t)dt.$$

Expectation of a function of a random variable:

$$\mathbb{E}[g(X)]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(t)f_X(t)dt.$$

Mean square value:

$$\mathbb{E}[X^2]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{t}^2{f}_X(t)dt.$$

Variance:

$$\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2.$$

$n$-th moment:

$$\mathbb{E}[X^n]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{t}^n{f}_X(t)dt.$$

7.2.4. Characteristic Function

The characteristic function is defined as

$${\mathrm{\Psi }}_X(j\omega )\triangleq \mathbb{E}[\exp (j\omega X)].$$

PDF as Fourier transform of CF.

$${\mathrm{\Psi }}_X(j\omega )={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{j\omega x}{f}_X(x)dx.$$

$$f_X(x)=\frac{1}{2\pi }{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-j\omega x}{\mathrm{\Psi }}_X(j\omega )d\omega$$

$${\mathrm{\Psi }}_X(j0)=\mathbb{E}(1)=1.$$

$${\frac{d}{d\omega }{\mathrm{\Psi }}_X(j\omega )|}_{\omega =0}=j\mathbb{E}[X].$$

$${\frac{d^2}{d{\omega }^2}{\mathrm{\Psi }}_X(j\omega )|}_{\omega =0}=j^2\mathbb{E}[X^2]=-\mathbb{E}[X^2].$$

$$\mathbb{E}[X^k]=\frac{1}{j^k}{\frac{d^k}{d{\omega }^k}{\mathrm{\Psi }}_X(j\omega )|}_{\omega =0}.$$

Let $Y_1,\dots ,Y_k$ be independent. Then

$${\mathrm{\Psi }}_{Y_1+\cdots +Y_k}(j\omega )=\prod _{Y_1,\dots ,Y_K}\mathbb{E}[\exp (j\omega Y_i)].$$

7.2.5. Moment Generating Function

The moment generating function is defined as

$$M_X(t)\triangleq \mathbb{E}[\exp (tX)].$$