diff -r a80094bd530c -r d4bed13a2c37 probability-discrete.rst --- a/probability-discrete.rst Tue Mar 29 20:47:38 2016 +0300 +++ b/probability-discrete.rst Thu Mar 31 00:15:20 2016 +0300 @@ -21,11 +21,11 @@ PMF(X = x) = P(X = x) = p_X(x) = P({ω ∈ Ω: X(ω) = x}) - PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b} P(X = x) + PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b}\ P(X = x) p_X(x) ≥ 0 - ∑_x p_X(x) = 1 + ∑_x\ p_X(x) = 1 where :math:`X` is a random variable on space :math:`Ω` of outcomes which mapped to real number via :math:`X(ω)`. @@ -94,35 +94,35 @@ .. math:: - var(a*X + b) = a²̇ · var[X] + var(a*X + b) = a² · var[X] Total probability theorem ========================= -Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i A_i = Ω`: +Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i\ A_i = Ω`: .. math:: p_X(x) = Σ_i P(A_i)·p_{X|A_i}(x) -Conditional PMF -=============== +Conditional PMF on event +======================== -:def:`Conditional PMF` is: +:def:`Conditional PMF on event` is: .. math:: p_{X|A}(x) = P(X=x | A) - E[X|A] = ∑_x x·p_{X|A}(x) + E[X|A] = ∑_x\ x·p_{X|A}(x) Total expectation theorem ========================= .. math:: - E[X] = Σ_i P(A_i)·E[X|A_i] + E[X] = Σ_i\ P(A_i)·E[X|A_i] To prove theorem just multiply total probability theorem by :math:`x`. @@ -141,10 +141,10 @@ E[X+Y] = E[X] + E[Y] -Conditional PMF -=============== +Conditional joint PMF +===================== -:def:`Conditional PMF` is: +:def:`Conditional joint PMF` is: .. math:: @@ -158,7 +158,7 @@ p_{X,Y,Z}(x,y,z) = p_Y(y)·p_{Z|Y}(z|y)·p_{X|Y,Z}(x|y,z) - ∑_{x,y} p_{X,Y|Z}(x,y|z) = 1 + ∑_{x,y}\ p_{X,Y|Z}(x,y|z) = 1 Conditional expectation of joint PMF ==================================== @@ -167,21 +167,76 @@ .. math:: - E[X|Y=y] = ∑_x x·p_{X|Y}(x|y) + E[X|Y=y] = ∑_x\ x·p_{X|Y}(x|y) - E[g(X)|Y=y] = ∑_x g(x)·p_{X|Y}(x|y) + E[g(X)|Y=y] = ∑_x\ g(x)·p_{X|Y}(x|y) Total probability theorem for joint PMF ======================================= .. math:: - p_X(x) = ∑_y p_Y(y)·p_{X|Y}(x|y) + p_X(x) = ∑_y\ p_Y(y)·p_{X|Y}(x|y) Total expectation theorem for joint PMF ======================================= .. math:: - E[X] = ∑_y p_Y(y)·E[X|Y=y] + E[X] = ∑_y\ p_Y(y)·E[X|Y=y] + +Proof: + +.. math:: + + ∑_y\ p_Y(y)·E[X|Y=y] = ∑_y\ p_Y(y)·∑_x\ x·p_{X|Y}(x|y) + + = ∑_y\ ∑_x\ p_Y(y)·x·p_{X|Y}(x|y) = ∑_x\ ∑_y\ x·p_Y(y)·p_{X|Y}(x|y) + + = ∑_x\ x·∑_y\ p_Y(y)·p_{X|Y}(x|y) = ∑_x\ x·p_X(x) = E[X] + +Conditional expectation of joint PMF +==================================== + +:def:`Conditional expectation of joint PMF` is random variable :math:`E[X|Y]` +defined as: + +.. math:: E[X|Y](y) = E[X|Y=y] + +Property: + +.. math:: E[E[X|Y]] = E[X] + +Proof (using total expectation theorem): + +.. math:: + + E[E[X|Y]] = ∑_y\ E[X|Y](y) = ∑_y\ E[X|Y=y] = E[X] + +Conditional variance +==================== + +:def:`Conditional variance` of :math:`X` on :math:`Y` is r.v.: + +.. math:: var(X|Y)(y) = var(X|Y=y) = E[(X - E[X|Y=y])²|Y=y] + +or in another notation: + +.. math:: var(X|Y) = E[X²|Y] - (E[X|Y])² + +By applying expected value by :math:`Y` on both sides: + +.. math:: E[var(X|Y)] = E[E[X²|Y]] - E[(E[X|Y])²] = E[X²] - E[(E[X|Y])²] + +on another hand: + +.. math:: var(E[X|Y]) = E[(E[X|Y])²] - (E[E[X|Y]])² = E[(E[X|Y])²] - (E[X])² + +By adding last two expression: + +.. math:: E[var(X|Y)] + var(E[X|Y]) = E[X²] - (E[X])² = var(X) + +So: + +.. math:: var(X) = E[var(X|Y)] + var(E[X|Y]) Independence of r.v. ==================== @@ -200,6 +255,21 @@ var(X+Y) = var(X) + var(Y) +Convolution formula +=================== + +If :math:`Z = X + Y` and X and Y is independent r.v. then: + +.. math:: p_Z(z) = ∑_x\ p_X(x)·p_Y(z-x) + +Proof: + +.. math:: + + p_Z(z) = ∑_{x,y:x+y=z}\ p_Z(z) = ∑_{x,y:x+y=z}\ P(X=x,Y=z-x) + + = ∑_{x,y:x+y=z}\ P(X=x)·P(Y=z-x) = ∑_x\ p_X(x)·p_Y(z-x) + Well known discrete r.v. ========================