14 :def:`Probability density function` (PDF) for continuous random variable |
14 :def:`Probability density function` (PDF) for continuous random variable |
15 :math:`x` is function: |
15 :math:`x` is function: |
16 |
16 |
17 .. math:: |
17 .. math:: |
18 |
18 |
19 PDF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∫_{a, b}\ f_X(x) \ dx |
19 CDF(a ≤ X ≤ b) = P(a ≤ X ≤ b) = ∫_{a, b}\ f_X(x) \ dx |
20 |
20 |
21 f_X(x) ≥ 0 |
21 f_X(x) ≥ 0 |
22 |
22 |
23 ∫_{-∞, +∞}\ f_X(x) \ dx = 1 |
23 ∫_{-∞, +∞}\ f_X(x) \ dx = 1 |
24 |
24 |
271 |
271 |
272 When :math:`Χ ~ norm(μ, σ²)` and :math:`Y = a·X + b` then: |
272 When :math:`Χ ~ norm(μ, σ²)` and :math:`Y = a·X + b` then: |
273 |
273 |
274 .. math:: |
274 .. math:: |
275 |
275 |
276 f_Y(y) = 1/a·f_X(y/a) = 1/a·1/sqrt(2·π)/σ·e^{-λ·((y-b)/a - μ)²/σ²/2} |
276 f_Y(y) = 1/a·f_X((y-b)/a) = 1/a·1/sqrt(2·π)/σ·e^{-λ·((y-b)/a - μ)²/σ²/2} |
277 |
277 |
278 = 1/sqrt(2·π)/(a·σ)·e^{-λ·(y - (a·μ+b))²/(a·σ)²/2} = ~ norm(a·μ+b, a·σ) |
278 = 1/sqrt(2·π)/(a·σ)·e^{-λ·(y - (a·μ+b))²/(a·σ)²/2} = ~ norm(a·μ+b, (a·σ)²) |
279 |
279 |
280 Monotonic function of distribution |
280 Monotonic function of distribution |
281 ================================== |
281 ================================== |
282 |
282 |
283 Let's :math:`Y = g(X)` and :math:`g` is monotonic function on range :math:`[a, |
283 Let's :math:`Y = g(X)` and :math:`g` is monotonic function on range :math:`[a, |
339 |
341 |
340 .. math:: var(X + Y) = var(X) + var(Y) + 2·cov(X, Y) |
342 .. math:: var(X + Y) = var(X) + var(Y) + 2·cov(X, Y) |
341 |
343 |
342 Covariance of two independent r.v. is zero. |
344 Covariance of two independent r.v. is zero. |
343 |
345 |
344 Proof: |
346 Proofs: |
345 |
347 |
346 .. math:: |
348 .. math:: |
347 |
349 |
348 cov(X, Y) = E[(X - E[X])·(Y - E[Y])] = E[(X - E[X])]·E[(Y - E[Y])] = 0 |
350 cov(X, Y) = E[(X - E[X])·(Y - E[Y])] = E[ X·Y - X·E[Y] - E[X]·Y + E[X]·E[Y] ] |
|
351 |
|
352 = E[X·Y] - E[X·E[Y]] - E[E[X]·Y] + E[E[X]·E[Y]] |
|
353 |
|
354 = E[X·Y] - E[X]·E[Y] - E[X]·E[Y] + E[X]·E[Y] = E[X·Y] - E[X]·E[Y] |
|
355 |
|
356 .. math:: |
|
357 |
|
358 cov(a·X + b, Y) = E[(a·X + b - E[a·X + b])·(Y - E[Y])] |
|
359 |
|
360 = E[(a·X + b - (a·E[X] + b))·(Y - E[Y])] = E[(a·X + a·E[X])·(Y - E[Y])] |
|
361 |
|
362 = a·E[(X + E[X])·(Y - E[Y])] = a·cov(X, Y) |
|
363 |
|
364 .. math:: |
|
365 |
|
366 cov(X, Y + Z) = E[(X - E[X])·(Y + Z - E[Y + Z])] = E[(X - E[X])·(Y - E[Y] + Z - E[Z])] |
|
367 |
|
368 = E[(X - E[X])·(Y - E[Y]) + (X - E[X])·(Z - E[Z])] |
|
369 |
|
370 = E[(X - E[X])·(Y - E[Y])] + E[(X - E[X])·(Z - E[Z])] = cov(X, Y) + cov(X, Z) |
|
371 |
|
372 .. math:: |
|
373 |
|
374 var(X) + var(Y) + 2·cov(X, Y) = E[X²] - (E[X])² + E[Y²] - (E[Y])² + 2·E[X·Y] - 2·E[X]·E[Y] |
|
375 |
|
376 = E[X² - X·E[X] + Y² - Y·E[Y] + 2·X·Y - X·E[Y] - Y·E[X]] |
|
377 |
|
378 = E[(X+Y)² - (X·E[X] + Y·E[Y] + X·E[Y] + Y·E[X])] |
|
379 |
|
380 = E[(X+Y)²] - E[(X+Y)·(E[X] + E[Y])] = E[(X+Y)²] - E[X+Y]·E[X+Y] = var(X+Y) |
|
381 |
|
382 For independent r.v. :math:`X` and :math:`Y`: |
|
383 |
|
384 .. math:: cov(X, Y) = E[(X - E[X])·(Y - E[Y])] = E[(X - E[X])]·E[(Y - E[Y])] = 0 |
349 |
385 |
350 Correlation coefficient |
386 Correlation coefficient |
351 ======================= |
387 ======================= |
352 |
388 |
353 Dimensionless version of covariance: |
389 Dimensionless version of covariance: |
365 |
401 |
366 Properties: |
402 Properties: |
367 |
403 |
368 .. math:: ρ(a·X + b, Y) = sign(a)·ρ(X, Y) |
404 .. math:: ρ(a·X + b, Y) = sign(a)·ρ(X, Y) |
369 |
405 |
|
406 Conditioned expectation |
|
407 ======================= |
|
408 |
|
409 .. math:: E[X|Y] = ∫_X\ x·f_{X|Y}(x|Y)\ dx |
|
410 |
|
411 Law of total expectation |
|
412 ======================== |
|
413 |
|
414 .. math:: E[X] = E[E[X|Y]] |
|
415 |
|
416 Proof:: |
|
417 |
|
418 .. math:: |
|
419 |
|
420 E[E[X|Y]] = ∫_Y\ f_Y(y)·∫_X\ x·f_{X|Y}(x|y)\ dx·dy |
|
421 |
|
422 = ∫_Y\ ∫_X\ x·f_Y(y)·f_{X|Y}(x|y)\ dx·dy = ∫_Y\ ∫_X\ x·f_{X,Y}(x,y)\ dx·dy |
|
423 |
|
424 = ∫_X\ x·∫_Y\ f_{X,Y}(x,y)\ dy·dx = ∫_X\ x·f_X(x)\ dx = E[X] |
|
425 |
|
426 * https://en.wikipedia.org/wiki/Law_of_total_expectation |
|
427 |
|
428 Iterated expectations with nested conditioning sets |
|
429 =================================================== |
|
430 |
|
431 .. math:: E[X|A] = E[E[X|B]|A] |
|
432 |
|
433 Conditional variance |
|
434 ==================== |
|
435 |
|
436 .. math:: var(X|Y=y) = E[(X - E[X|Y=y])² |Y=y] |
|
437 |
|
438 * https://en.wikipedia.org/wiki/Conditional_variance |
|
439 |
|
440 Law of total variance |
|
441 ===================== |
|
442 |
|
443 .. math:: var(X) = E[var(X|Y)] + var(E[X|Y]) = E_Y[var_X(X|Y)] + var_X(E_Y[X|Y]) |
|
444 |
|
445 Proof: |
|
446 |
|
447 .. math:: |
|
448 |
|
449 var(X) = E[X²] - (E[X])² = E[E[X²|Y]] - (E[E[X|Y]])² |
|
450 |
|
451 = E[var(X|Y) + (E[X|Y])²] - (E[E[X|Y]])² |
|
452 |
|
453 = E[var(X|Y)] + E[(E[X|Y])²} - (E[E[X|Y]])² = E[var(X|Y)] + var(E[X|Y]) |
|
454 |
|
455 * https://en.wikipedia.org/wiki/Law_of_total_variance |
|
456 |
|
457 Law of total covariance |
|
458 ======================= |
|
459 |
|
460 * https://en.wikipedia.org/wiki/Law_of_total_covariance |
|
461 |
|
462 Sum of normally distributed random variables |
|
463 ============================================ |
|
464 |
|
465 For :math:`X ~ norm(μ_X, σ_X²)` and :math:`Y ~ norm(μ_Y, σ_Y²)` random variable |
|
466 :math:`X+Y` is also has normal distribution with parameters: |
|
467 |
|
468 .. math:: norm(μ_X + μ_Y, σ_X² + σ_Y²) |
|
469 |
|
470 https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables |
|
471 |