author | Oleksandr Gavenko <gavenkoa@gmail.com> |
Thu, 21 Apr 2016 16:19:13 +0300 | |
changeset 15 | f606de7b91b0 |
parent 13 | 8a32b268e8bf |
permissions | -rw-r--r-- |
0 | 1 |
|
2 |
============= |
|
3 |
Probability |
|
4 |
============= |
|
5 |
.. contents:: |
|
6 |
:local: |
|
7 |
||
8 |
.. role:: def |
|
9 |
:class: def |
|
10 |
||
11 |
PMF |
|
12 |
=== |
|
13 |
||
14 |
:def:`PMF` or :def:`probability mass function` or :def:`probability law` or |
|
15 |
:def:`probability discribuion` of discrete random variable is a function that |
|
16 |
for given number give probability of that value. |
|
17 |
||
18 |
To denote PMF used notations: |
|
19 |
||
20 |
.. math:: |
|
21 |
||
22 |
PMF(X = x) = P(X = x) = p_X(x) = P({ω ∈ Ω: X(ω) = x}) |
|
23 |
||
9 | 24 |
PMF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∑_{a ≤ x ≤ b}\ P(X = x) |
5 | 25 |
|
26 |
p_X(x) ≥ 0 |
|
27 |
||
9 | 28 |
∑_x\ p_X(x) = 1 |
5 | 29 |
|
0 | 30 |
where :math:`X` is a random variable on space :math:`Ω` of outcomes which mapped |
31 |
to real number via :math:`X(ω)`. |
|
32 |
||
33 |
Expected value |
|
34 |
============== |
|
35 |
||
36 |
:def:`Expected value` of PMF is: |
|
37 |
||
38 |
.. math:: |
|
39 |
||
40 |
E[X] = Σ_{ω∈Ω} Χ(x) * p(ω) = Σ_{x} x * p_X(x) |
|
41 |
||
42 |
We write :math:`a ≤ X ≤ b` for :math:`∀ ω∈Ω a ≤ X(ω) ≤ b`. |
|
43 |
||
44 |
If :math:`X ≥ 0` then :math:`E[X] ≥ 0`. |
|
45 |
||
46 |
if :math:`a ≤ X ≤ b` then :math:`a ≤ E[X] ≤ b`. |
|
47 |
||
48 |
If :math:`Y = g(X)` (:math:`∀ ω∈Ω Y(ω) = g(X(ω))`) then: |
|
49 |
||
50 |
.. math:: |
|
51 |
||
52 |
E[Y] = Σ_{x} g(x) * p_X(x) |
|
53 |
||
54 |
**Proof** TODO: |
|
55 |
||
56 |
.. math:: |
|
57 |
||
58 |
E[Y] = Σ_{y} y * p_Y(y) |
|
59 |
||
60 |
= Σ_{y∈ℝ} y * Σ_{ω∈Ω: Y(ω)=y} p(ω) |
|
61 |
||
62 |
= Σ_{y∈ℝ} y * Σ_{ω∈Ω: g(X(ω))=y} p(ω) |
|
63 |
||
64 |
= Σ_{y∈ℝ} y * Σ_{x∈ℝ: g(x)=y} Σ_{ω∈Ω: X(ω) = x} p(ω) |
|
65 |
||
66 |
= Σ_{y∈ℝ} y * Σ_{x∈ℝ: g(x)=y} p_X(x) |
|
67 |
||
68 |
= Σ_{y∈ℝ} Σ_{x∈ℝ: g(x)=y} y * p_X(x) |
|
69 |
||
70 |
= Σ_{x∈ℝ} Σ_{y∈ℝ: g(x)=y} y * p_X(x) |
|
71 |
||
72 |
= Σ_{x} g(x) * p_X(x) |
|
73 |
||
74 |
.. math:: |
|
75 |
||
76 |
E[a*X + b] = a*E[X] + b |
|
77 |
||
78 |
Variance |
|
79 |
======== |
|
80 |
||
81 |
:def:`Variance` is a: |
|
82 |
||
83 |
.. math:: |
|
84 |
||
15
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
85 |
var[X] = E[(X - E[X])²] = E[X²] - (E[X])² |
0 | 86 |
|
87 |
:def:`Standard deviation` is a: |
|
88 |
||
89 |
.. math:: |
|
90 |
||
91 |
σ_Χ = sqrt(var[X]) |
|
92 |
||
93 |
Property: |
|
94 |
||
95 |
.. math:: |
|
96 |
||
9 | 97 |
var(a*X + b) = a² · var[X] |
0 | 98 |
|
99 |
||
100 |
Total probability theorem |
|
101 |
========================= |
|
102 |
||
9 | 103 |
Let :math:`A_i ∩ A_j = ∅` for :math:`i ≠ j` and :math:`∑_i\ A_i = Ω`: |
0 | 104 |
|
105 |
.. math:: |
|
106 |
||
107 |
p_X(x) = Σ_i P(A_i)·p_{X|A_i}(x) |
|
108 |
||
15
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
109 |
* https://en.wikipedia.org/wiki/Law_of_total_probability |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
110 |
|
9 | 111 |
Conditional PMF on event |
112 |
======================== |
|
0 | 113 |
|
9 | 114 |
:def:`Conditional PMF on event` is: |
0 | 115 |
|
116 |
.. math:: |
|
117 |
||
118 |
p_{X|A}(x) = P(X=x | A) |
|
119 |
||
9 | 120 |
E[X|A] = ∑_x\ x·p_{X|A}(x) |
0 | 121 |
|
122 |
Total expectation theorem |
|
123 |
========================= |
|
124 |
||
125 |
.. math:: |
|
126 |
||
9 | 127 |
E[X] = Σ_i\ P(A_i)·E[X|A_i] |
0 | 128 |
|
129 |
To prove theorem just multiply total probability theorem by :math:`x`. |
|
130 |
||
131 |
Joint PMF |
|
132 |
========= |
|
133 |
||
134 |
:def:`Joint PMF` of random variables :math:`X_1,...,X_n` is: |
|
135 |
||
136 |
.. math:: |
|
137 |
||
138 |
p_{X_1,...,X_n}(x_1,...,x_n) = P(AND_{x_1,...,x_n}: X_i = x_i) |
|
139 |
||
140 |
Properties: |
|
141 |
||
142 |
.. math:: |
|
143 |
||
144 |
E[X+Y] = E[X] + E[Y] |
|
145 |
||
9 | 146 |
Conditional joint PMF |
147 |
===================== |
|
0 | 148 |
|
9 | 149 |
:def:`Conditional joint PMF` is: |
0 | 150 |
|
151 |
.. math:: |
|
152 |
||
153 |
p_{X|Y}(x|y) = P(X=x | Y=y) = P(X=x \& Y=y) / P(Y=y) |
|
154 |
||
155 |
So: |
|
156 |
||
157 |
.. math:: |
|
158 |
||
159 |
p_{X,Y}(x,y) = p_Y(y)·p_{X|Y}(x|y) = p_X(x)·p_{Y|X}(y|x) |
|
160 |
||
161 |
p_{X,Y,Z}(x,y,z) = p_Y(y)·p_{Z|Y}(z|y)·p_{X|Y,Z}(x|y,z) |
|
162 |
||
9 | 163 |
∑_{x,y}\ p_{X,Y|Z}(x,y|z) = 1 |
0 | 164 |
|
165 |
Conditional expectation of joint PMF |
|
166 |
==================================== |
|
167 |
||
168 |
:def:`Conditional expectation of joint PMF` is: |
|
169 |
||
170 |
.. math:: |
|
171 |
||
9 | 172 |
E[X|Y=y] = ∑_x\ x·p_{X|Y}(x|y) |
0 | 173 |
|
9 | 174 |
E[g(X)|Y=y] = ∑_x\ g(x)·p_{X|Y}(x|y) |
0 | 175 |
|
176 |
Total probability theorem for joint PMF |
|
177 |
======================================= |
|
178 |
.. math:: |
|
179 |
||
9 | 180 |
p_X(x) = ∑_y\ p_Y(y)·p_{X|Y}(x|y) |
0 | 181 |
|
182 |
Total expectation theorem for joint PMF |
|
183 |
======================================= |
|
184 |
.. math:: |
|
185 |
||
9 | 186 |
E[X] = ∑_y\ p_Y(y)·E[X|Y=y] |
187 |
||
188 |
Proof: |
|
189 |
||
190 |
.. math:: |
|
191 |
||
192 |
∑_y\ p_Y(y)·E[X|Y=y] = ∑_y\ p_Y(y)·∑_x\ x·p_{X|Y}(x|y) |
|
193 |
||
194 |
= ∑_y\ ∑_x\ p_Y(y)·x·p_{X|Y}(x|y) = ∑_x\ ∑_y\ x·p_Y(y)·p_{X|Y}(x|y) |
|
195 |
||
196 |
= ∑_x\ x·∑_y\ p_Y(y)·p_{X|Y}(x|y) = ∑_x\ x·p_X(x) = E[X] |
|
197 |
||
198 |
Conditional expectation of joint PMF |
|
199 |
==================================== |
|
200 |
||
201 |
:def:`Conditional expectation of joint PMF` is random variable :math:`E[X|Y]` |
|
202 |
defined as: |
|
203 |
||
204 |
.. math:: E[X|Y](y) = E[X|Y=y] |
|
205 |
||
206 |
Property: |
|
207 |
||
11
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
208 |
.. math:: E[g(Y)·X|Y] = g(Y)·E[X|Y] |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
209 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
210 |
For invertible funtion :math:`h`: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
211 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
212 |
.. math:: E[X|h(Y)] = E[X|Y] |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
213 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
214 |
Proof: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
215 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
216 |
.. math:: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
217 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
218 |
E[X|Y=y] = E[X|h(Y)=h(y)] |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
219 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
220 |
Law of Iterated Expectations |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
221 |
============================ |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
222 |
|
9 | 223 |
.. math:: E[E[X|Y]] = E[X] |
224 |
||
225 |
Proof (using total expectation theorem): |
|
226 |
||
227 |
.. math:: |
|
228 |
||
229 |
E[E[X|Y]] = ∑_y\ E[X|Y](y) = ∑_y\ E[X|Y=y] = E[X] |
|
230 |
||
11
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
231 |
Generalisation of Law of Iterated Expectations: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
232 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
233 |
.. math:: E[E[X|Y,Z]|Y] = E[X|Y] |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
234 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
235 |
Proof, for each :math:`y∈Y`: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
236 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
237 |
.. math:: |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
238 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
239 |
E[X|Y=y] = ∑_x\ x·p_{X|Y}(x|Y=y) = ∑_x\ x·p_{X,Y}(x,y)/p_Y(y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
240 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
241 |
= ∑_x\ x·∑_z\ p_{X,Y,Z}(x,y,z)/p_Y(y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
242 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
243 |
= ∑_x\ x·∑_z\ p_{X|Y,Z}(x|Y=y,Z=z)·p_{Y,Z}(y,z)/p_Y(y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
244 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
245 |
= ∑_x\ x·∑_z\ p_{X|Y,Z}(x|Y=y,Z=z)·p_{Z|Y}(z|Y=y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
246 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
247 |
= ∑_x\ ∑_z\ x·p_{X|Y,Z}(x|Y=y,Z=z)·p_{Z|Y}(z|Y=y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
248 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
249 |
= ∑_z\ ∑_x\ x·p_{X|Y,Z}(x|Y=y,Z=z)·p_{Z|Y}(z|Y=y) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
250 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
251 |
= ∑_z\ p_{Z|Y}(z|Y=y)·∑_x\ x·p_{X|Y,Z}(x|Y=y,Z=z) |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
252 |
|
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
253 |
= ∑_z\ p_{Z|Y}(z|Y=y)·E[X|Y,Z] = E[E[X|Y,Z]|Y=y] |
44779fa3053d
Law of Iterated Expectations.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
9
diff
changeset
|
254 |
|
9 | 255 |
Conditional variance |
256 |
==================== |
|
257 |
||
258 |
:def:`Conditional variance` of :math:`X` on :math:`Y` is r.v.: |
|
259 |
||
260 |
.. math:: var(X|Y)(y) = var(X|Y=y) = E[(X - E[X|Y=y])²|Y=y] |
|
261 |
||
262 |
or in another notation: |
|
263 |
||
264 |
.. math:: var(X|Y) = E[X²|Y] - (E[X|Y])² |
|
265 |
||
12
fa0dee44fe1f
Law of total variance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
11
diff
changeset
|
266 |
Law of total variance |
fa0dee44fe1f
Law of total variance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
11
diff
changeset
|
267 |
===================== |
fa0dee44fe1f
Law of total variance.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
11
diff
changeset
|
268 |
|
9 | 269 |
By applying expected value by :math:`Y` on both sides: |
270 |
||
271 |
.. math:: E[var(X|Y)] = E[E[X²|Y]] - E[(E[X|Y])²] = E[X²] - E[(E[X|Y])²] |
|
272 |
||
273 |
on another hand: |
|
274 |
||
275 |
.. math:: var(E[X|Y]) = E[(E[X|Y])²] - (E[E[X|Y]])² = E[(E[X|Y])²] - (E[X])² |
|
276 |
||
277 |
By adding last two expression: |
|
278 |
||
279 |
.. math:: E[var(X|Y)] + var(E[X|Y]) = E[X²] - (E[X])² = var(X) |
|
280 |
||
281 |
So: |
|
282 |
||
283 |
.. math:: var(X) = E[var(X|Y)] + var(E[X|Y]) |
|
0 | 284 |
|
285 |
Independence of r.v. |
|
286 |
==================== |
|
287 |
||
288 |
r.v. :math:`X` and :math:`Y` is :def:`independent` if: |
|
289 |
||
290 |
.. math:: |
|
291 |
||
292 |
∀_{x,y}: p_{X,Y}(x,y) = p_X(x)·p_Y(y) |
|
293 |
||
294 |
So if two r.v. are independent: |
|
295 |
||
296 |
.. math:: |
|
297 |
||
298 |
E[X·Y] = E[X]·E[Y] |
|
299 |
||
300 |
var(X+Y) = var(X) + var(Y) |
|
301 |
||
9 | 302 |
Convolution formula |
303 |
=================== |
|
304 |
||
305 |
If :math:`Z = X + Y` and X and Y is independent r.v. then: |
|
306 |
||
307 |
.. math:: p_Z(z) = ∑_x\ p_X(x)·p_Y(z-x) |
|
308 |
||
309 |
Proof: |
|
310 |
||
311 |
.. math:: |
|
312 |
||
313 |
p_Z(z) = ∑_{x,y:x+y=z}\ p_Z(z) = ∑_{x,y:x+y=z}\ P(X=x,Y=z-x) |
|
314 |
||
315 |
= ∑_{x,y:x+y=z}\ P(X=x)·P(Y=z-x) = ∑_x\ p_X(x)·p_Y(z-x) |
|
316 |
||
13
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
317 |
Sum of a random number of r.v |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
318 |
============================= |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
319 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
320 |
Let :math:`X_i` is independent equally distributed r.v. and let :math:`Y = |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
321 |
∑_{i=1..N}\ X_i`, where :math:`N` is r.v. Then: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
322 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
323 |
.. math:: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
324 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
325 |
E[Y|N=n] = n·E[X] |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
326 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
327 |
E[Y|N] = N·E[X] |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
328 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
329 |
Proof: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
330 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
331 |
.. math:: E[Y|N=n] = E[∑_{i=1..N}\ X_i |N=n] = E[∑_{i=1..n}\ X_i] = ∑_{i=1..n}\ E[X_i] = n·E[X] |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
332 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
333 |
Variance of sum of a random number independent r.v.: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
334 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
335 |
.. math:: var(∑_{i=1..N}\ X_i|N) = E[N]·var(X) + (E[X])²·var(N) |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
336 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
337 |
Proof: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
338 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
339 |
.. math:: |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
340 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
341 |
var(Y|N=n) = var[∑_{i=1..N}\ X_i|N=n] = var[∑_{i=1..n}\ X_i] = ∑_{i=1..n}\ var[X_i] = n·var(X) |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
342 |
|
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
343 |
var(Y) = E[var(Y|N)] + var(E[Y|N]) = E[N]·var(X) + (E[X])²·var(N) |
8a32b268e8bf
Sum of a random number of r.v
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
12
diff
changeset
|
344 |
|
0 | 345 |
Well known discrete r.v. |
346 |
======================== |
|
347 |
||
348 |
Bernoulli random variable |
|
349 |
------------------------- |
|
350 |
||
351 |
:def:`Bernoulli random variable` with parameter :math:`p` is a random variable |
|
352 |
that have 2 outcomes denoted as :math:`0` and :math:`1` with probabilities: |
|
353 |
||
354 |
.. math:: |
|
355 |
||
356 |
p_X(0) = 1 - p |
|
357 |
||
358 |
p_X(1) = p |
|
359 |
||
360 |
This random variable models a trial of experiment that result in success or |
|
361 |
failure. |
|
362 |
||
363 |
:def:`Indicator` of r.v. event :math:`A` is function:: |
|
364 |
||
365 |
I_A = 1 iff A occurs, else 0 |
|
366 |
||
367 |
.. math:: |
|
368 |
||
369 |
P_{I_A} = p(I_A = 1) = p(A) |
|
370 |
||
371 |
I_A*I_B = I_{A∩B} |
|
372 |
||
373 |
.. math:: |
|
374 |
||
375 |
E[bernoulli(p)] = 0*(1-p) + 1*p = p |
|
376 |
||
377 |
var[bernoulli(p)] = E[bernoulli(p) - E[bernoulli(p)]] |
|
378 |
||
379 |
= (0-p)²·(1-p) + (1-p)²·p = p²·(1-p) + (1 - 2p + p²)·p |
|
380 |
||
381 |
= p² - p³ + p - 2·p² + p³ = p·(1-p) |
|
382 |
||
383 |
Discret uniform random variable |
|
384 |
------------------------------- |
|
385 |
||
386 |
:def:`Discret uniform random variable` is a variable with parameters :math:`a` |
|
387 |
and :math:`b` in sample space :math:`{x: a ≤ x ≤ b & x ∈ ℕ}` with equal |
|
388 |
probability of each possible outcome: |
|
389 |
||
390 |
.. math:: |
|
391 |
||
392 |
p_{unif(a,b)}(x) = 1 / (b-a+1) |
|
393 |
||
394 |
.. math:: |
|
395 |
||
396 |
E[unif(a,b)] = Σ_{a ≤ x ≤ b} x * 1/(b-a+1) |
|
397 |
= 1/(b-a+1) * Σ_{a ≤ x ≤ b} x |
|
398 |
||
399 |
= 1/(b-a+1) * (Σ_{a ≤ x ≤ b} a + Σ_{0 ≤ x ≤ b-a} x) |
|
400 |
||
401 |
= 1/(b-a+1) * ((b-a+1)*a + (b-a)*(b-a+1)/2) |
|
402 |
||
403 |
= a + (b-a)/2 |
|
404 |
= (b+a)/2 |
|
405 |
||
406 |
||
407 |
.. math:: |
|
408 |
||
409 |
var[unif(a,b)] = E[unif²(a,b)] - E²[unif(a,b)] |
|
410 |
||
411 |
= ∑_{a≤x≤b} x²/(b-a+1) - (b+a)²/4 |
|
412 |
||
413 |
= 1/(b-a+1)·(∑_{0≤x≤b} x² - ∑_{0≤x≤a-1} x²) - (b+a)²/4 |
|
414 |
||
415 |
= 1/(b-a+1)·(b+3·b²+2·b³ - (a-1)+3·(a-1)²+2·(a-1)³)/6 - (b+a)²/4 |
|
416 |
||
417 |
= (2·b² + 2·a·b + b + 2·a² - a)/6 - (b+a)²/4 |
|
418 |
||
419 |
= (b - a)·(b - a + 2) / 12 |
|
420 |
||
421 |
.. NOTE:: |
|
422 |
||
423 |
From Maxima:: |
|
424 |
||
425 |
sum(i^2,i,0,n), simpsum=true; |
|
426 |
||
427 |
2 3 |
|
428 |
n + 3 n + 2 n |
|
429 |
--------------- |
|
430 |
6 |
|
431 |
||
432 |
factor(b+3*b^2+2*b^3 - (a-1)-3*(a-1)^2-2*(a-1)^3); |
|
433 |
||
434 |
2 2 |
|
435 |
(b - a + 1) (2 b + 2 a b + b + 2 a - a) |
|
436 |
||
437 |
factor((2*b^2 + 2*a*b + b + 2*a^2 - a)/6 - (b+a)^2/4), simp=true; |
|
438 |
||
439 |
(b - a) (2 - a + b) |
|
440 |
------------------- |
|
441 |
12 |
|
442 |
||
443 |
Binomial random variable |
|
444 |
------------------------ |
|
445 |
||
446 |
:math:`Binomial random variable` is a r.v. with parameters :math:`n` (positive |
|
447 |
integer) and p from interval :math:`(0,1)` and sample space of positive integers |
|
448 |
from inclusive region :math:`[0, n]`: |
|
449 |
||
450 |
.. math:: |
|
451 |
||
452 |
p_{binom(n,p)}(x) = n!/(x!*(n-x)!) p^x p^{n-x} |
|
453 |
||
454 |
Binomial random variable models a number of success of :math:`n` independent |
|
455 |
trails of Bernoulli experimants. |
|
456 |
||
457 |
.. math:: |
|
458 |
||
459 |
E[binom(n,p)] = E[∑_{1≤x≤n} bernoulli(p)] = ∑_{1≤x≤n} E[bernoulli(p)] = n·p |
|
460 |
||
461 |
var[binom(n,p)] = var[∑_{1≤x≤n} bernoulli(p)] = ∑_{1≤x≤n} var[bernoulli(p)] = n·p·(1-p) |
|
462 |
||
463 |
Geometric random variable |
|
464 |
------------------------- |
|
465 |
||
466 |
:def:`Geometric random variable` is a r.v. with parameter :math:`p` from |
|
467 |
half open interval :math:`(0,1]`, sample space is all positive numbers: |
|
468 |
||
469 |
.. math:: |
|
470 |
||
471 |
p_{geom(p)}(x) = p (1-p)^(x-1) |
|
472 |
||
473 |
This random variable models number of tosses of biased coin until first success. |
|
474 |
||
475 |
.. math:: |
|
476 |
||
477 |
E[geom(p)] = ∑_{x=1..∞} x·p·(1-p)^(x-1) |
|
478 |
||
479 |
= p·∑_{x=1..∞} x·(1-p)^(x-1) |
|
480 |
||
481 |
= p/(1-p)·∑_{x=0..∞} x·(1-p)^x |
|
482 |
||
483 |
= p/(1-p)·(1-p)/(1-p - 1)² = p/p² = 1/p |
|
484 |
||
485 |
.. NOTE:: |
|
486 |
||
487 |
Maxima calculation:: |
|
488 |
||
489 |
load("simplify_sum"); |
|
490 |
simplify_sum(sum(k * x^k, k, 0, inf)); |
|
491 |
Is abs(x) - 1 positive, negative or zero? |
|
492 |
negative; |
|
493 |
Is x positive, negative or zero? |
|
494 |
positive; |
|
495 |
Is x - 1 positive, negative or zero? |
|
496 |
negative; |
|
497 |
x |
|
498 |
------------ |
|
499 |
2 |
|
500 |
x - 2 x + 1 |
|
15
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
501 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
502 |
.. math:: |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
503 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
504 |
E[(geom(p))²] = ∑_{x=1..∞} x²·p·(1-p)^(x-1) |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
505 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
506 |
= p·∑_{x=1..∞} x²·(1-p)^(x-1) |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
507 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
508 |
= p/(1-p)·∑_{x=0..∞} x²·(1-p)^x |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
509 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
510 |
= p/(1-p)·(1-p)·(1-p+1)/(1 - (1-p))³ = p·(2-p)/p³ = (2-p)/p² |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
511 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
512 |
.. NOTE:: |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
513 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
514 |
Maxima calculation:: |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
515 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
516 |
load("simplify_sum"); |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
517 |
(%i3) assume(x>0); |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
518 |
(%o3) [x > 0] |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
519 |
(%i4) assume(x<1); |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
520 |
(%o4) [x < 1] |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
521 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
522 |
(%i8) simplify_sum(sum(k^2 * x^k, k, 0, inf)); |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
523 |
2 |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
524 |
x + x |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
525 |
(%o8) - ------------------- |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
526 |
3 2 |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
527 |
x - 3 x + 3 x - 1 |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
528 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
529 |
So: |
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
530 |
|
f606de7b91b0
Variance of geometric distribution.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
13
diff
changeset
|
531 |
.. math:: var(geom(p)) = E[(geom(p))²] - E[geom(p)]² = (2-p)/p² - 1/p² = (1-p)/p² |