--- a/probability-discrete.rst Tue Mar 29 20:47:38 2016 +0300
+++ b/probability-discrete.rst Thu Mar 31 00:15:20 2016 +0300
@@ -21,11 +21,11 @@
PMF(X = x) = P(X = x) = p_X(x) = P({ω ∈ Ω: X(ω) = x})
- PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b} P(X = x)
+ PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b}\ P(X = x)
p_X(x) ≥ 0
- ∑_x p_X(x) = 1
+ ∑_x\ p_X(x) = 1
where :math:`X` is a random variable on space :math:`Ω` of outcomes which mapped
to real number via :math:`X(ω)`.
@@ -94,35 +94,35 @@
.. math::
- var(a*X + b) = a²̇ · var[X]
+ var(a*X + b) = a² · var[X]
Total probability theorem
=========================
-Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i A_i = Ω`:
+Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i\ A_i = Ω`:
.. math::
p_X(x) = Σ_i P(A_i)·p_{X|A_i}(x)
-Conditional PMF
-===============
+Conditional PMF on event
+========================
-:def:`Conditional PMF` is:
+:def:`Conditional PMF on event` is:
.. math::
p_{X|A}(x) = P(X=x | A)
- E[X|A] = ∑_x x·p_{X|A}(x)
+ E[X|A] = ∑_x\ x·p_{X|A}(x)
Total expectation theorem
=========================
.. math::
- E[X] = Σ_i P(A_i)·E[X|A_i]
+ E[X] = Σ_i\ P(A_i)·E[X|A_i]
To prove theorem just multiply total probability theorem by :math:`x`.
@@ -141,10 +141,10 @@
E[X+Y] = E[X] + E[Y]
-Conditional PMF
-===============
+Conditional joint PMF
+=====================
-:def:`Conditional PMF` is:
+:def:`Conditional joint PMF` is:
.. math::
@@ -158,7 +158,7 @@
p_{X,Y,Z}(x,y,z) = p_Y(y)·p_{Z|Y}(z|y)·p_{X|Y,Z}(x|y,z)
- ∑_{x,y} p_{X,Y|Z}(x,y|z) = 1
+ ∑_{x,y}\ p_{X,Y|Z}(x,y|z) = 1
Conditional expectation of joint PMF
====================================
@@ -167,21 +167,76 @@
.. math::
- E[X|Y=y] = ∑_x x·p_{X|Y}(x|y)
+ E[X|Y=y] = ∑_x\ x·p_{X|Y}(x|y)
- E[g(X)|Y=y] = ∑_x g(x)·p_{X|Y}(x|y)
+ E[g(X)|Y=y] = ∑_x\ g(x)·p_{X|Y}(x|y)
Total probability theorem for joint PMF
=======================================
.. math::
- p_X(x) = ∑_y p_Y(y)·p_{X|Y}(x|y)
+ p_X(x) = ∑_y\ p_Y(y)·p_{X|Y}(x|y)
Total expectation theorem for joint PMF
=======================================
.. math::
- E[X] = ∑_y p_Y(y)·E[X|Y=y]
+ E[X] = ∑_y\ p_Y(y)·E[X|Y=y]
+
+Proof:
+
+.. math::
+
+ ∑_y\ p_Y(y)·E[X|Y=y] = ∑_y\ p_Y(y)·∑_x\ x·p_{X|Y}(x|y)
+
+ = ∑_y\ ∑_x\ p_Y(y)·x·p_{X|Y}(x|y) = ∑_x\ ∑_y\ x·p_Y(y)·p_{X|Y}(x|y)
+
+ = ∑_x\ x·∑_y\ p_Y(y)·p_{X|Y}(x|y) = ∑_x\ x·p_X(x) = E[X]
+
+Conditional expectation of joint PMF
+====================================
+
+:def:`Conditional expectation of joint PMF` is random variable :math:`E[X|Y]`
+defined as:
+
+.. math:: E[X|Y](y) = E[X|Y=y]
+
+Property:
+
+.. math:: E[E[X|Y]] = E[X]
+
+Proof (using total expectation theorem):
+
+.. math::
+
+ E[E[X|Y]] = ∑_y\ E[X|Y](y) = ∑_y\ E[X|Y=y] = E[X]
+
+Conditional variance
+====================
+
+:def:`Conditional variance` of :math:`X` on :math:`Y` is r.v.:
+
+.. math:: var(X|Y)(y) = var(X|Y=y) = E[(X - E[X|Y=y])²|Y=y]
+
+or in another notation:
+
+.. math:: var(X|Y) = E[X²|Y] - (E[X|Y])²
+
+By applying expected value by :math:`Y` on both sides:
+
+.. math:: E[var(X|Y)] = E[E[X²|Y]] - E[(E[X|Y])²] = E[X²] - E[(E[X|Y])²]
+
+on another hand:
+
+.. math:: var(E[X|Y]) = E[(E[X|Y])²] - (E[E[X|Y]])² = E[(E[X|Y])²] - (E[X])²
+
+By adding last two expression:
+
+.. math:: E[var(X|Y)] + var(E[X|Y]) = E[X²] - (E[X])² = var(X)
+
+So:
+
+.. math:: var(X) = E[var(X|Y)] + var(E[X|Y])
Independence of r.v.
====================
@@ -200,6 +255,21 @@
var(X+Y) = var(X) + var(Y)
+Convolution formula
+===================
+
+If :math:`Z = X + Y` and X and Y is independent r.v. then:
+
+.. math:: p_Z(z) = ∑_x\ p_X(x)·p_Y(z-x)
+
+Proof:
+
+.. math::
+
+ p_Z(z) = ∑_{x,y:x+y=z}\ p_Z(z) = ∑_{x,y:x+y=z}\ P(X=x,Y=z-x)
+
+ = ∑_{x,y:x+y=z}\ P(X=x)·P(Y=z-x) = ∑_x\ p_X(x)·p_Y(z-x)
+
Well known discrete r.v.
========================