Probability· Note 06

Variance

Year 2 · Semester ElulLast updated

Definition

The variance of a random variable XX is a measure of how far XX can (and will) differ from its expected value.
(i.e., V(X)V(X) measures the “spread” of possible values of XX.)

Formula
V(X)=E[X2]−E[X]2V(X)=E[X^2]-E[X]^2
Example

Define X1,X2X_1, X_2 using the following probability functions:

k1k_1 PX1(k1)P_{X_1}(k_1) ∣\mid k2k_2 PX2(k2)P_{X_2}(k_2)
5 13\frac13 ∣\mid 0 13\frac13
10 13\frac13 ∣\mid 10 13\frac13
15 13\frac13 ∣\mid 20 13\frac13

E[X1]=5+10+153=10E[X_1]=\frac{5+10+15}{3} = 10
E[X2]=0+10+203=10E[X_2]=\frac{0 + 10 +20}{3} = 10
They both have the same expected value, but V(X2)V(X_2) should be greater than V(X1)V(X_1) because the values are more spread out from their Expected Values
V(X1)=13(52+02+52)=503=1623V(X_1)=\frac13(5^2+0^2+5^2)=\frac{50}3=16\frac23
V(X2)=13(102+02+102)=2003=6623V(X_2)=\frac13(10^2+0^2+10^2)=\frac{200}3=66\frac23

Properties

  1. Formula: V(X)=E[X2]−E[X]2V(X)=E[X^2]-E[X]^2
  2. For any c∈Rc\in\R: V(cX)=c2⋅V(X)V(cX)=c^2\cdot V(X)
  3. For any c∈Rc\in\R: V(X+3)=V(X)V(X+3)=V(X)
    • Intuition: the spread of a r.v. is unaffected when we merely “shift” all of it’s possible values by a constant cc
    • This is tightly related to property 2 of Expected Values E[X+c]=E[X]+cE[X+c]=E[X]+c
  4. V(X+Y)=V(X)+V(Y)V(X+Y)=V(X)+V(Y) if X,YX,Y are independent.
    • Proof: (This is likely on the #ProbabilityExam
      • V(X+Y)=E[(X+Y)2]−(μ′)2V(X+Y)=\textcolor{orange}{E[(X+Y)^2]}-\textcolor{purple}{(\mu^\prime)^2} Where μ′={=E[X+Y]=E[X]+E[Y]=μ1+μ2\mu^\prime=\begin{cases}&=E[X&+Y]\\&=E[X]&+E[Y]\\&=\mu_1&+\mu_2\end{cases}
      • E[(X+Y)2]=E[X2+2XY+Y2]=E[X2]+E[2XY]+E[Y2]=E[X2]+2E[X]E[Y]+E[Y2](Requires X,Y to be independent)=E[X2]+2μ1μ2+E[Y2]\begin{aligned}\textcolor{orange}{E[(X+Y)^2]}&=E[X^2+2XY+Y^2]\\&=E[X^2]+E[2XY]+E[Y^2]\\&=E[X^2]+2E[X]E[Y]+E[Y^2]\qquad \text{(Requires }X,Y\text{ to be independent)}\\&=\textcolor{orange}{E[X^2]+2\mu_1\mu_2+E[Y^2]}\end{aligned}
      • (μ′)2=(μ1+μ2)2=μ12+2μ1μ2+μ22\textcolor{purple}{(\mu^\prime)^2}=(\mu_1+\mu_2)^2=\textcolor{purple}{\mu_1^2+2\mu_1\mu_2+\mu_2^2}
      • E[(X+Y)2]−(μ′)2=(E[X2]+2μ1μ2+E[Y2])−(μ12+2μ1μ2+μ22)=(E[X2]−μ12)+(E[Y2]−μ22)=V(X)+V(Y)\begin{aligned}\textcolor{orange}{E[(X+Y)^2]}-\textcolor{purple}{(\mu^\prime)^2}&=(\textcolor{orange}{E[X^2]+\cancel{2\mu_1\mu_2}+E[Y^2]})-(\textcolor{purple}{\mu_1^2+\cancel{2\mu_1\mu_2}+\mu_2^2})\\&=(E[X^2]-\mu_1^2)+(E[Y^2]-\mu_2^2)\\&=V(X)+V(Y)\end{aligned}
      • Q.E.D.Q.E.D.

Standard Deviation

Definition
σ(X)=V(X)\sigma(X)=\sqrt{V(X)}
Intuition

Variance “magnifies” the difference E[X2]−E[X]2E[X^2]-E[X]^2 by squaring it, so σ(X)\sigma(X) neutralizes this magnification

Note: As a square root, σ(X)≥0\sigma(X) \ge 0 always.

Properties

  1. σ(cX)=∣c∣⋅σ(X)∣∀c∈R\sigma(cX)=|c|\cdot\sigma(X)\mid\forall c\in\R
  2. σ(X+c)=σ(X)\sigma(X+c)=\sigma(X)