probability-continuous.rst
author Oleksandr Gavenko <gavenkoa@gmail.com>
Fri, 22 Apr 2016 10:25:55 +0300
changeset 18 c18d218b854e
parent 17 db3d7a44583b
permissions -rw-r--r--
Sum of random number of i.i.r.v.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     1
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     2
=============================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     3
 Continuous random variables
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     4
=============================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     5
.. contents::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     6
   :local:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     7
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     8
Probability density function
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     9
============================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    10
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    11
.. role:: def
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    12
   :class: def
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    13
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    14
:def:`Probability density function` (PDF) for continuous random variable
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    15
:math:`x` is function:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    16
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    17
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    18
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
    19
   CDF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∫_{a, b}\ f_X(x) \ dx
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    20
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    21
   f_X(x) ≥ 0
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    22
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    23
   ∫_{-∞, +∞}\ f_X(x) \ dx = 1
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    24
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    25
:math:`f_X(x)` funtion maps values :math:`x` from sample space to real numbers.
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    26
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    27
For continuous random variable:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    28
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    29
.. math:: P(X = a) = 0
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    30
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    31
Expectation
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    32
===========
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    33
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    34
:def:`Expectation` of continuous random variable is:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    35
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    36
.. math:: μ = E[X] = ∫_{-∞, +∞}\ x·f_X(x) \ dx
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    37
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    38
Properties:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    39
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    40
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    41
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    42
   E[X + Y] = E[X] + E[Y]
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    43
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    44
   E[a·X] = a·E[X]
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    45
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    46
   E[a·X + b] = a·E[X] + b
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    47
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    48
Variance
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    49
========
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    50
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    51
:def:`Variance` of continuous random variable is:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    52
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    53
.. math:: var[X] = ∫_{-∞, +∞}\ (x-μ)²·f_X(x) \ dx
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    54
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    55
Properties:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    56
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    57
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    58
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    59
   var[a·X + b] = a²·var[X]
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    60
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    61
   var[X] = E[X²] - E²[X]
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    62
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    63
Standard deviation
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    64
==================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    65
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    66
:def:`Standard deviation` of continuous random variable is:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    67
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    68
.. math:: σ_Χ = sqrt(var[X])
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
    69
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    70
Cumulative distribution functions
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    71
=================================
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    72
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    73
:def:`Cumulative distribution functions` (CDF) of random variable :math:`X` is:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    74
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    75
.. math:: F_X(x) = P(X ≤ x) = ∫_{-∞, x}\ f_X(t) \ dt
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    76
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    77
So:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    78
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    79
.. math::
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    80
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    81
   P(a ≤ X ≤ b) = F_X(b) - F_X(a) + f_X(a) = ∫_{a,b}\ f_X(x) \ dx
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    82
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    83
   F_X(-∞) = 0
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    84
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    85
   F_X(+∞) = 1
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    86
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    87
and :math:`F_X(a) ≤ F_X(b)` for :math:`a ≤ b`.
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    88
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    89
Relation between CDF and PDF:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    90
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    91
.. math:: (d(CDF(t))/dt)(x) = PDF(x)
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    92
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    93
Conditional probability
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    94
=======================
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    95
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    96
:def:`Conditional probability` of continuous random variable is:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    97
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    98
.. math:: P(X ⊆ B | A) = ∫_{B}\ f_{X|A}(x) \ dx = ∫_{A∩B}\ f_X(x) \ dx / P(A)
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
    99
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   100
:def:`Conditional expectation` of continuous random variable is:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   101
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   102
.. math:: E[X|A] = ∫_\ x·f_{X|A}(x) \ dx
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   103
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   104
Properties:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   105
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   106
.. math::
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   107
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   108
   E[g(X)|A] = ∫_\ g(x)·f_{X|A}(x) \ dx
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   109
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   110
Independence
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   111
============
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   112
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   113
Random variable :math:`X`, :math:`Y` are :def:`independent` if:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   114
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   115
.. math:: f_{X,Y}(x, y) = f_X(x)·f_Y(y)
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   116
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   117
Continuous uniform random variable
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   118
==================================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   119
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   120
:def:`Continuous uniform random variable` is :math:`f_X(x)` that is non-zero
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   121
only on :math:`[a, b]` with :math:`f_X(x) = `1/(b-a)`.
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   122
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   123
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   124
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   125
   E[unif(a, b)] = (b+a)/2
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   126
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   127
   var[unif(a, b)] = (b-a)²/12
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   128
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   129
   σ = (b-a)/sqrt(12)
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   130
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   131
Proofs:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   132
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   133
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   134
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   135
   E[unif(a, b)] = ∫_{a, b}\ x·1/(b-a)·dx = x²/2/(b-a) |_{a, b} = (b²-a²)/(b-a)/2 = (b+a)/2
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   136
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   137
   E[unif²(a, b)] = ∫_{a, b} x²·1/(b-a)·dx = x³/3/(b-a) |_{a, b} = (b³-a³)/(b-a)/3 = (b²+b·a+a²)/3
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   138
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   139
   var[unif(a, b)] = E[unif²(a, b)] - E²[unif(a, b)] = (b²+b·a+a²)/3 - (b+a)²/4 = (b-a)²/12
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   140
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   141
.. note::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   142
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   143
   In maxima::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   144
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   145
     (%i4) factor((b^2+b*a+a^2)/3 - (a+b)^2/4);
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   146
                 2
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   147
          (b - a)
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   148
          --------
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   149
             12
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   150
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   151
Exponential random variables
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   152
============================
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   153
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   154
:def:`Exponential random variables` with parameter :math:`λ` is:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   155
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   156
.. math:: f_X(x) = λ·exp(-λ·x)
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   157
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   158
for :math:`x ≥ 0`, and zero otherwise.
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   159
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   160
Properties:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   161
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   162
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   163
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   164
   E[exp(λ)] = 1/λ
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   165
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   166
   var[exp(λ)] = 1/λ²
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   167
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   168
Proof:
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   169
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   170
.. math::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   171
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   172
  ∫_{-∞, +∞}\ f_X(x) \ dx = ∫_{0, +∞}\ λ·exp(-λ·x) \ dx = -exp(-λ·x) |_{0, +∞} = 1
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   173
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   174
  E[exp(λ)] = ∫_{0, +∞}\ x·λ·exp(-λ·x) \ dx = 1/λ
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   175
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   176
  E[exp²(λ)] = ∫_{0, +∞}\ x²·λ·exp(-λ·x) \ dx = 1/λ²
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   177
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   178
.. note::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   179
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   180
   From maxima::
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   181
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   182
    (%i15) assume(lambda>0);
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   183
    (%o15)                           [lambda > 0]
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   184
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   185
    (%i16) integrate(lambda*%e^(-lambda*x),x,0,inf);
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   186
    (%o16)                                 1
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   187
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   188
    (%i17) integrate(x*lambda*%e^(-lambda*x),x,0,inf);
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   189
                                          1
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   190
    (%o17)                              ------
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   191
                                        lambda
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   192
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   193
    (%i18) integrate(x^2*lambda*%e^(-lambda*x),x,0,inf);
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   194
                                           2
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   195
    (%o18)                              -------
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   196
                                              2
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   197
                                        lambda
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   198
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   199
Normal random variables
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   200
=======================
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   201
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   202
:def:`Normal random variables` with parameters :math:`μ, σ` and :math:`σ > 0`
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   203
defined by PDF:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   204
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   205
.. math:: norm(μ, σ²) = 1/sqrt(2·π)/σ·exp(-(x-μ)²/σ²/2)
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   206
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   207
Properties:
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   208
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   209
.. math::
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   210
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   211
   E[norm(μ, σ²)] = μ
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   212
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   213
   var[norm(μ, σ²)] = σ²
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   214
8
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   215
Summa of two normal r.v.
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   216
========================
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   217
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   218
If :math:`Z = X + Y` and X and Y is independent normal r.v. then:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   219
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   220
.. math:: norm(μ_z, σ_z²) = norm(μ_x+μ_y, σ_x²+σ_y²)
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   221
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   222
Proof:
4
5d6cec5fe095 Probability density function. Continuous uniform random variable. Exponential
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   223
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   224
.. math::
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   225
8
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   226
   norm(μ_z, σ_z²) = ∫_x\ f_X(x)·f_Y(z-x)\ dx
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   227
8
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   228
   = ∫_x\ 1/sqrt(2·π)/σ_x·exp(-(x-μ_x)²/σ_x²/2)·1/sqrt(2·π)/σ_y·exp(-(z-x-μ_y)²/σ_y²/2)\ dx
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   229
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   230
   = 1/sqrt(2·π·(σ_x² + σ_y²))·exp(-(x-μ_x-μ_y)²/(σ_x²+σ_y²)/2)
6
9b4e31a03161 Normal random variables.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 4
diff changeset
   231
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   232
Linear function of distribution
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   233
===============================
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   234
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   235
If :math:`Y = a·X + b` then :math:`f_Y(y) = 1/|a|·f_X((y-b)/a)`.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   236
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   237
Proof, for :math:`y > 0`:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   238
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   239
.. math:: F_Y(Y ≤ y) = F_X(a·X + b ≤ y) = F_X(X ≤ (y-b)/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   240
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   241
so:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   242
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   243
.. math:: f_Y(y) = d/dy\ F_Y(Y ≤ y) = d/dy\ F_X(x ≤ (y-b)/a) = 1/a·f_X((y-b)/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   244
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   245
For :math:`y < 0`:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   246
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   247
.. math::
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   248
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   249
   F_Y(Y > y) = F_X(a·X + b > y) = F_X(X < (y-b)/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   250
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   251
   F_Y(Y <= y) = 1 - F_Y(Y > y) = 1 - F_X(X < (y-b)/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   252
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   253
   d/dy\ f_Y(y) = -1/a·f_X((y-b)/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   254
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   255
Combining expression for :math:`a≠0` gives us result.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   256
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   257
If X is uniform distribution with parameters :math:`c, d` then :math:`a·Y + b`
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   258
also is uniform distribution with parameters :math:`a·c+b, a·d+b`.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   259
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   260
If X is exponential distribution with parameters :math:`λ` then :math:`a·Y`
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   261
also is exponential distribution with parameters :math:`λ/a` for :math:`a > 0`.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   262
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   263
If X is normal distribution with parameters :math:`μ, σ²` then
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   264
:math:`a·Y + b` also is normal distribution with parameters :math:`a·μ+b, (a·σ)²`.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   265
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   266
Proofs.
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   267
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   268
When :math:`Χ ~ exp(λ)` and :math:`Y = a·X` then:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   269
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   270
.. math:: f_Y(y) = 1/a·f_X(y/a) = λ/a·e^{-λ·y/a} ~ exp(λ/a)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   271
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   272
When :math:`Χ ~ norm(μ, σ²)` and :math:`Y = a·X + b` then:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   273
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   274
.. math::
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   275
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   276
   f_Y(y) = 1/a·f_X((y-b)/a) = 1/a·1/sqrt(2·π)/σ·e^{-λ·((y-b)/a - μ)²/σ²/2}
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   277
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   278
   = 1/sqrt(2·π)/(a·σ)·e^{-λ·(y - (a·μ+b))²/(a·σ)²/2} = ~ norm(a·μ+b, (a·σ)²)
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   279
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   280
Monotonic function of distribution
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   281
==================================
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   282
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   283
Let's :math:`Y = g(X)` and :math:`g` is monotonic function on range :math:`[a,
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   284
b]`. So there is inverse function :math:`h(Y) = X` on range :math:`[g(a), g(b)]`
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   285
(if :math:`g` is increasing values) or on range :math:`[g(b), g(a)]` (if
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   286
:math:`g` is decreasing values). In that case:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   287
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   288
.. math:: f_Y(y) = f_X(h(y))·(d\ h(t)/dt)(y)
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   289
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   290
Proof. Let :math:`g` is monotonically increasing function. Thus:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   291
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   292
.. math:: F_Y(Y ≤ y) = F_X(g(X) ≤ y) = F_X(X ≤ h(y)) = F_X(h(y))
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   293
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   294
and so:
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   295
10
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   296
.. math:: f_Y(y) = (d\ F_Y(t)/dt)(y) = (d\ F_X(h(t))/dt)(y) = f_X(h(y))·(d\ h(t)/dt)(y)
7
c9c0861c10c2 Linear function of distribution. Monotonic function of distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 6
diff changeset
   297
8
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   298
Convolution formula
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   299
===================
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   300
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   301
If :math:`Z = X + Y` and X and Y is independent r.v. then:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   302
10
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   303
.. math:: f_Z(z) = ∫_x\ f_X(x)·f_Y(z-x)·dx
8
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   304
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   305
Proof:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   306
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   307
Consider :math:`Z` at conditional event :math:`X=x`:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   308
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   309
.. math:: f_{Z|X}(z|X=x) = f_{z|X=x}(z|X=x)
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   310
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   311
Becasue of independence of :math:`X` and :math:`Y`:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   312
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   313
.. math:: f_{Z|X}(z|X=x) = f_{X+Y|X=x}(z|X=x) = f_{x+Y}(z) = f_Y(z-x)
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   314
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   315
Joint PDF of :math:`X` and :math:`Z` is:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   316
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   317
.. math:: f_{X,Z}(x,z) = f_X(x)·f_{Z|X}(z|X=x) = f_X(x)·f_Y(z-x)
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   318
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   319
By integrating by :math:`x` we get:
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   320
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   321
.. math:: f_Z(z) = ∫_x\ f_{X,Z}(x,z)\ dx = ∫_x\ f_X(x)·f_Y(z-x)\ dx
a80094bd530c Convolution formula
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 7
diff changeset
   322
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   323
* https://en.wikipedia.org/wiki/List_of_convolutions_of_probability_distributions
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   324
10
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   325
Covariance
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   326
==========
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   327
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   328
Covariance of two r.v. is:
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   329
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   330
.. math:: cov(X, Y) = E[(X - E[X])·(Y - E[Y])]
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   331
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   332
Properties:
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   333
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   334
.. math:: cov(X, Y) = E[X·Y] - E[X]·E[Y]
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   335
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   336
.. math:: cov(X, X) = var(X)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   337
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   338
.. math:: cov(a·X + b, Y) = a·cov(X, Y)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   339
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   340
.. math:: cov(X, Y + Z) = cov(X, Y) + cov(X, Z)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   341
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   342
.. math:: var(X + Y) = var(X) + var(Y) + 2·cov(X, Y)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   343
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   344
Covariance of two independent r.v. is zero.
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   345
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   346
Proofs:
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   347
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   348
.. math::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   349
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   350
   cov(X, Y) = E[(X - E[X])·(Y - E[Y])] = E[ X·Y - X·E[Y] - E[X]·Y + E[X]·E[Y] ]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   351
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   352
   = E[X·Y] - E[X·E[Y]] - E[E[X]·Y] + E[E[X]·E[Y]]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   353
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   354
   = E[X·Y] - E[X]·E[Y] - E[X]·E[Y] + E[X]·E[Y] = E[X·Y] - E[X]·E[Y]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   355
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   356
.. math::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   357
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   358
   cov(a·X + b, Y) = E[(a·X + b - E[a·X + b])·(Y - E[Y])]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   359
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   360
   = E[(a·X + b - (a·E[X] + b))·(Y - E[Y])] = E[(a·X + a·E[X])·(Y - E[Y])]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   361
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   362
   = a·E[(X + E[X])·(Y - E[Y])] = a·cov(X, Y)
10
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   363
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   364
.. math::
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   365
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   366
   cov(X, Y + Z) = E[(X - E[X])·(Y + Z - E[Y + Z])] = E[(X - E[X])·(Y - E[Y] + Z - E[Z])]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   367
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   368
   = E[(X - E[X])·(Y - E[Y]) + (X - E[X])·(Z - E[Z])]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   369
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   370
   = E[(X - E[X])·(Y - E[Y])] + E[(X - E[X])·(Z - E[Z])] = cov(X, Y) + cov(X, Z)
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   371
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   372
.. math::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   373
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   374
   var(X) + var(Y) + 2·cov(X, Y) = E[X²] - (E[X])² + E[Y²] - (E[Y])² + 2·E[X·Y] - 2·E[X]·E[Y]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   375
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   376
   = E[X² - X·E[X] + Y² - Y·E[Y] + 2·X·Y - X·E[Y] - Y·E[X]]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   377
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   378
   = E[(X+Y)² - (X·E[X] + Y·E[Y] + X·E[Y] + Y·E[X])]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   379
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   380
   = E[(X+Y)²] - E[(X+Y)·(E[X] + E[Y])] = E[(X+Y)²] - E[X+Y]·E[X+Y] = var(X+Y)
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   381
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   382
For independent r.v. :math:`X` and :math:`Y`:
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   383
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   384
.. math:: cov(X, Y) = E[(X - E[X])·(Y - E[Y])] = E[(X - E[X])]·E[(Y - E[Y])] = 0
10
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   385
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   386
Correlation coefficient
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   387
=======================
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   388
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   389
Dimensionless version of covariance:
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   390
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   391
.. math:: ρ(Χ, Υ) = E[(X-E[X])/σ_Χ·(Y-E[Y])/σ_Y] = cov(X, Y)/(σ_X·σ_Y)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   392
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   393
It is defined only for cases when :math:`σ_X ≠ 0` and :math:`σ_Y ≠ 0`.
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   394
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   395
Obviously :math:`-1 ≤ ρ(Χ, Υ) ≤ +1` and :math:`ρ(Χ, X) = 0`.
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   396
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   397
For independent r.v. :math:`ρ(Χ, Y) = 0`.
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   398
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   399
If :math:`|ρ(X, Y)| = 1` then :math:`X` and :math:`Y` is have linear
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   400
dependencies :math:`X = Y` or :math:`X = -Y`.
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   401
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   402
Properties:
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   403
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   404
.. math:: ρ(a·X + b, Y) = sign(a)·ρ(X, Y)
5a09c6837dcb Covariance
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 8
diff changeset
   405
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   406
Conditioned expectation
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   407
=======================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   408
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   409
.. math:: E[X|Y] = ∫_X\ x·f_{X|Y}(x|Y)\ dx
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   410
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   411
Law of total expectation
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   412
========================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   413
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   414
.. math:: E[X] = E[E[X|Y]]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   415
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   416
Proof::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   417
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   418
.. math::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   419
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   420
   E[E[X|Y]] = ∫_Y\ f_Y(y)·∫_X\ x·f_{X|Y}(x|y)\ dx·dy
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   421
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   422
   = ∫_Y\ ∫_X\ x·f_Y(y)·f_{X|Y}(x|y)\ dx·dy = ∫_Y\ ∫_X\ x·f_{X,Y}(x,y)\ dx·dy
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   423
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   424
   = ∫_X\ x·∫_Y\ f_{X,Y}(x,y)\ dy·dx = ∫_X\ x·f_X(x)\ dx = E[X]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   425
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   426
* https://en.wikipedia.org/wiki/Law_of_total_expectation
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   427
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   428
Iterated expectations with nested conditioning sets
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   429
===================================================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   430
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   431
.. math:: E[X|A] = E[E[X|B]|A]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   432
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   433
Conditional variance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   434
====================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   435
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   436
.. math:: var(X|Y=y) = E[(X - E[X|Y=y])² |Y=y]
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   437
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   438
* https://en.wikipedia.org/wiki/Conditional_variance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   439
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   440
Law of total variance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   441
=====================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   442
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   443
.. math:: var(X) = E[var(X|Y)] + var(E[X|Y]) = E_Y[var_X(X|Y)] + var_X(E_Y[X|Y])
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   444
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   445
Proof:
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   446
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   447
.. math::
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   448
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   449
   var(X) = E[X²] - (E[X])² = E[E[X²|Y]] - (E[E[X|Y]])²
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   450
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   451
   = E[var(X|Y) + (E[X|Y])²] - (E[E[X|Y]])²
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   452
17
db3d7a44583b Fix typo.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 16
diff changeset
   453
   = E[var(X|Y)] + E[(E[X|Y])²] - (E[E[X|Y]])² = E[var(X|Y)] + var(E[X|Y])
16
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   454
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   455
* https://en.wikipedia.org/wiki/Law_of_total_variance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   456
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   457
Law of total covariance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   458
=======================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   459
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   460
* https://en.wikipedia.org/wiki/Law_of_total_covariance
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   461
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   462
Sum of normally distributed random variables
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   463
============================================
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   464
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   465
For :math:`X ~ norm(μ_X, σ_X²)` and :math:`Y ~ norm(μ_Y, σ_Y²)` random variable
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   466
:math:`X+Y` is also has normal distribution with parameters:
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   467
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   468
.. math:: norm(μ_X + μ_Y, σ_X² + σ_Y²)
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   469
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   470
https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables
c48b0353e055 Add "Law of total expectation", "Law of total variance", proofs for covariance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 10
diff changeset
   471
18
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   472
Sum of random number of i.i.r.v.
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   473
================================
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   474
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   475
Let :math:`Y = X_1+...+X_N` is a sum of r.v. :math:`N` and all :math:`X_i` are
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   476
i.i.r.v. Thus:
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   477
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   478
.. math:: E[Y] = E[N]·E[X]
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   479
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   480
.. math:: var[Y] = E[N]·var(X) + var(N)·(E[N])²
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   481
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   482
Proofs:
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   483
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   484
.. math::
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   485
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   486
   E[Y] = E[∑_{i=1..N}\ X_i] = E[E[∑_{i=1..n}\ X_i |N=n]] = E[∑_{i=1..n}\ E[X_i|N=n]]
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   487
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   488
   = E[∑_{i=1..n}\ E[X]] = E[N·E[X]] = E[N]·E[X]
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   489
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   490
.. math::
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   491
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   492
   var[Y] = E[var(Y|N)] + var(E[Y|N]) = E[var(∑_{i=1..n}\ X_i |N=n)] + var(E[∑_{i=1..n}\ X_i |N=n])
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   493
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   494
   = E[N·var(X)] + var(N·E[X]) = E[N]·var(X) + var(N)·(E[X])²
c18d218b854e Sum of random number of i.i.r.v.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 17
diff changeset
   495