# HG changeset patch
# User Oleksandr Gavenko <gavenkoa@gmail.com>
# Date 1459372520 -10800
# Node ID d4bed13a2c3702d6f3c9a08f3374fc6b41e5cede
# Parent  a80094bd530cf310dca4ab93df3162c361396081
Conditional variance

diff -r a80094bd530c -r d4bed13a2c37 probability-discrete.rst
--- a/probability-discrete.rst	Tue Mar 29 20:47:38 2016 +0300
+++ b/probability-discrete.rst	Thu Mar 31 00:15:20 2016 +0300
@@ -21,11 +21,11 @@
 
    PMF(X = x) = P(X = x) = p_X(x) = P({ω ∈ Ω: X(ω) = x})
 
-   PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b} P(X = x)
+   PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b}\ P(X = x)
 
    p_X(x) ≥ 0
 
-   ∑_x p_X(x) = 1
+   ∑_x\ p_X(x) = 1
 
 where :math:`X` is a random variable on space :math:`Ω` of outcomes which mapped
 to real number via :math:`X(ω)`.
@@ -94,35 +94,35 @@
 
 .. math::
 
-  var(a*X + b) = a²̇ · var[X]
+  var(a*X + b) = a² · var[X]
 
 
 Total probability theorem
 =========================
 
-Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i A_i = Ω`:
+Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i\ A_i = Ω`:
 
 .. math::
 
   p_X(x) = Σ_i P(A_i)·p_{X|A_i}(x)
 
-Conditional PMF
-===============
+Conditional PMF on event
+========================
 
-:def:`Conditional PMF` is:
+:def:`Conditional PMF on event` is:
 
 .. math::
 
   p_{X|A}(x) = P(X=x | A)
 
-  E[X|A] = ∑_x x·p_{X|A}(x)
+  E[X|A] = ∑_x\ x·p_{X|A}(x)
 
 Total expectation theorem
 =========================
 
 .. math::
 
-  E[X] = Σ_i P(A_i)·E[X|A_i]
+  E[X] = Σ_i\ P(A_i)·E[X|A_i]
 
 To prove theorem just multiply total probability theorem by :math:`x`.
 
@@ -141,10 +141,10 @@
 
   E[X+Y] = E[X] + E[Y]
 
-Conditional PMF
-===============
+Conditional joint PMF
+=====================
 
-:def:`Conditional PMF` is:
+:def:`Conditional joint PMF` is:
 
 .. math::
 
@@ -158,7 +158,7 @@
 
   p_{X,Y,Z}(x,y,z) = p_Y(y)·p_{Z|Y}(z|y)·p_{X|Y,Z}(x|y,z)
 
-  ∑_{x,y} p_{X,Y|Z}(x,y|z) = 1
+  ∑_{x,y}\ p_{X,Y|Z}(x,y|z) = 1
 
 Conditional expectation of joint PMF
 ====================================
@@ -167,21 +167,76 @@
 
 .. math::
 
-  E[X|Y=y] = ∑_x x·p_{X|Y}(x|y)
+  E[X|Y=y] = ∑_x\ x·p_{X|Y}(x|y)
 
-  E[g(X)|Y=y] = ∑_x g(x)·p_{X|Y}(x|y)
+  E[g(X)|Y=y] = ∑_x\ g(x)·p_{X|Y}(x|y)
 
 Total probability theorem for joint PMF
 =======================================
 .. math::
 
-  p_X(x) = ∑_y p_Y(y)·p_{X|Y}(x|y)
+  p_X(x) = ∑_y\ p_Y(y)·p_{X|Y}(x|y)
 
 Total expectation theorem for joint PMF
 =======================================
 .. math::
 
-  E[X] = ∑_y p_Y(y)·E[X|Y=y]
+  E[X] = ∑_y\ p_Y(y)·E[X|Y=y]
+
+Proof:
+
+.. math::
+
+   ∑_y\ p_Y(y)·E[X|Y=y] = ∑_y\ p_Y(y)·∑_x\ x·p_{X|Y}(x|y)
+
+   = ∑_y\ ∑_x\ p_Y(y)·x·p_{X|Y}(x|y) = ∑_x\ ∑_y\ x·p_Y(y)·p_{X|Y}(x|y)
+
+   = ∑_x\ x·∑_y\ p_Y(y)·p_{X|Y}(x|y) = ∑_x\ x·p_X(x) = E[X]
+
+Conditional expectation of joint PMF
+====================================
+
+:def:`Conditional expectation of joint PMF` is random variable :math:`E[X|Y]`
+defined as:
+
+.. math:: E[X|Y](y) = E[X|Y=y]
+
+Property:
+
+.. math:: E[E[X|Y]] = E[X]
+
+Proof (using total expectation theorem):
+
+.. math::
+
+   E[E[X|Y]] = ∑_y\ E[X|Y](y) = ∑_y\ E[X|Y=y] = E[X]
+
+Conditional variance
+====================
+
+:def:`Conditional variance` of :math:`X` on :math:`Y` is r.v.:
+
+.. math:: var(X|Y)(y) = var(X|Y=y) = E[(X - E[X|Y=y])²|Y=y]
+
+or in another notation:
+
+.. math:: var(X|Y) = E[X²|Y] - (E[X|Y])²
+
+By applying expected value by :math:`Y` on both sides:
+
+.. math:: E[var(X|Y)] = E[E[X²|Y]] - E[(E[X|Y])²] = E[X²] - E[(E[X|Y])²]
+
+on another hand:
+
+.. math:: var(E[X|Y]) = E[(E[X|Y])²] - (E[E[X|Y]])² = E[(E[X|Y])²] - (E[X])²
+
+By adding last two expression:
+
+.. math:: E[var(X|Y)] + var(E[X|Y]) = E[X²] - (E[X])² = var(X)
+
+So:
+
+.. math:: var(X) = E[var(X|Y)] + var(E[X|Y])
 
 Independence of r.v.
 ====================
@@ -200,6 +255,21 @@
 
   var(X+Y) = var(X) + var(Y)
 
+Convolution formula
+===================
+
+If :math:`Z = X + Y` and X and Y is independent r.v. then:
+
+.. math:: p_Z(z) = ∑_x\ p_X(x)·p_Y(z-x)
+
+Proof:
+
+.. math::
+
+   p_Z(z) = ∑_{x,y:x+y=z}\ p_Z(z) = ∑_{x,y:x+y=z}\ P(X=x,Y=z-x)
+
+   = ∑_{x,y:x+y=z}\ P(X=x)·P(Y=z-x) = ∑_x\ p_X(x)·p_Y(z-x)
+
 Well known discrete r.v.
 ========================