Class 20: Population and Samples

Computing for Molecular Biology 2

Andrés Aravena, PhD

21 May 2021

Summary of Random Systems

Random integer numbers

  • sample(x,…) can mix any set x of possible outcomes

  • Most times we use sample(x, size, prob, replace=TRUE)

  • A “coin”: sample(0:1, size,prob=c(p,1-p), replace=TRUE)

  • To do many simulations, we use replicate(n, simulation)

Complex systems

When there are many parts, we can use sample to simulate the random parts, and write functions to combine the random parts and get the final result

  • Sum of several coins
  • Number of experiments until success
  • Number of people with epilepsy in our class
  • Probability of two people having the same birthday

Sum of many coins

rbinom(n, p, size) gives the number of successes in size trials, n times

Notice that size has a different meaning in sample()

Application

Answering scientific questions

Where are the cancers?

Highest kidney cancer rates in US (1980–1989) were in rural areas

Highest rates of kidney cancer are in rural areas

Why? Maybe…

  • the health care there is worse than in major cities
  • High stress due to poverty
  • are exposed to more harmful chemicals
  • Drink Alcohol
  • Smoke Tobacco

Lowest rates of kidney cancer are in rural areas

Why? Maybe…

  • No air and water pollution.
  • No stress
  • Eat healthy foods

Something is wrong, of course. The rural lifestyle cannot explain both very high and very low incidence of kidney cancer.

Why smaller genes have extreme GC content

We will answer these question later

Some concepts

Random variable
number that depends on the outcome of a random process
Average of population
a fixed number, that we do not know and want to know
Average of a sample
a random variable, that we get from an experiment

Many things are averages

  • concentration
  • proportion
  • density
  • GC content
  • frequency of events

What is the relationship between sample average and population average?

Can we learn the population average from the sample average?

Example: cell counting

Hemocytometer

Each square is a sample. Volume is fixed. The cell count is an average of cell counts of some squares.

We want population cell density

We have a sample of cell densities

Different samples have different average

N <- 4800
pop_LD <- sample(0:9, size=N, replace=TRUE)
pop_MD <- sample(0:29,size=N, replace=TRUE)
pop_HD <- sample(0:79,size=N, replace=TRUE)
pop_HD
   [1]  4 42 42  0 74 16 28 74 42 63 18 19  6 73  0 54 22 29  8 24 13
  [22] 30  4 20 63  7 41  6 61 10 67 66 33 71 65 48  2 44 59 48 63 62
  [43] 35 54 49 57  3 39 13 67 21 76 54 59 22 57 74 22  6  5 28  5 36
  [64] 31 57  1 10 17  0 79 37 72 54 34 10 69 21 55  4  0 66  0 75 16
  [85] 68 37 60 54 46 59 25  5 19  3 46  9 29 77 77 44 33 10 21 33 41
 [106] 46  7 32 40 64 14 41 57 16 62 59  1 25 39  6 33 64 13  3 45 48
 [127] 67 29 20 27 38 44 16 71 32 49 42 14 42 49  3 21 47 78 79  2 51
 [148] 53 62  4 75 57 32 67  5 42 54 12 71 32  2 21 12 62 51 50  7 53
 [169] 22 20  6 20 33 32 15 40 16 56 24 29 71 37 61 45 62 61 35 58  6
 [190] 53 51 29 79 27 50 33 76  5 48 41 11 16 68 75 66 23 17 21 23  4
 [211] 68 40 47 26 41 10  2 31 12 66 50  3 32 77 22 50  0 41 26 31 30
 [232] 31 39 66  5 74 60 79 54 15 17 49 41 64 61 12 78 69 12 50 39 66
 [253] 35 47  4 29 35 25 30 37 72 23 13 61 76 74 37 26 55 15 21 35 18
 [274] 72 18 32 31 77 54 33 21 43 48 54 58 33 76 11  4 28 13 24 32 23
 [295] 39  7 61 39 13 57 40 38 18 58 27 15 19 36 31 61 13 67 31 38 23
 [316]  0 73 67 31 21 15 39 75 43 65 66 47 75 49 52 60  5 21  5 41  8
 [337] 53 44 57 39  4 51 77 10 71 44  5 30  3 50 57 47 53 30 38 35 62
 [358] 11 59  9 44 25 39 37  3 38 50 22 30 77 29 68 29 17 40 65  1 12
 [379] 36 57 76 30 66 75 26 17 24 17 75  4 51 57 77  1 28  8 57  2 61
 [400] 69 16 45 20 11 48 54 31 38 40  7 17 25 26 52 49 13 37 52 78  4
 [421] 36 64  7  3 47 70 32 55 30 64 47 26 64 12 55 18 38 34  9  9 62
 [442] 43 61  8 67 49 15 13 64  6  9 19 42 23 19 37 78 63 50 45  5 40
 [463] 26  9 39 65 30 64 61 56 24 65 10 18 23 50 41  2 44 30 72 32  2
 [484] 45 47  5  1 67 21 32 77 74 22  8  0 67  2  5 77 71 26 73 53 22
 [505] 33 13 50 47 44 71 69 33 52 70 14 26 32 28 62 57 73 60 68 44 35
 [526] 55 78 77 12 78 66  9 54 11 70 38  0 30 59 32 59 15 35 50 27 60
 [547] 59 41 18 64 62 29  8 15 30 67 34 67 20 64 19 33  5 19  4 65 56
 [568] 73 77  0 29 60 16 42 77  1 66 79 73 37 69 76 64 64 29 50 49 32
 [589] 16 22 21  4 21  7 41 59 64 51 30  2 11 19 53 48  2 44 14  4 76
 [610] 38 46 49 36 31 26 69 22 17 28 50 31 48 36 64 79 72 64 37 33 14
 [631] 46 32 20 73 65 71 62 47 36 52 34 40  8 77 28 71 63 28 60  3 66
 [652]  5 68 53 75 50 13 71 77 27 53 42 65 73 31 29 58 50 69 64 51  6
 [673] 20 75 36 32 61 50 39 75 12 34 54 78 62 20 21 79 69 28 43 73 18
 [694]  3  5  9 61 49 12 46 57 25 18 34 57 43 25 43 62 45 33 12  1 64
 [715] 62 62 76 53 21 30  8 18 72 50 21 61 63 45 22 49 39  0 55 51 40
 [736] 32 25  6 47 59 28 13 16 37 61  0 37 58 37 35 46 71 53  8 32 64
 [757] 51 65 76 53 11 32  8  7  5 17 16 60 16 15 73 63 21 45 71 32 56
 [778] 53 71 46 69 58 41 66 69 63  2 51 35 27 62 28 58 34 79 79 74 23
 [799] 54 77  9 57 65 70 69 34 78 77  7  7 67  2 13 48  1 74 42 30 70
 [820] 76 61  8  5 30 64 34 68 38 22 74 34 70 10 55 51 36 27 66  6 59
 [841] 70  0 55 10 19 28 49 28 57 79 15  2 57  9 64 46 63 10 21 14 13
 [862] 24 38 62 62 69 79 20 23 77 75 33 19 62 38 72 27 50  6 71 65 76
 [883] 62  9 55 52 30 76 37 33 61 36 66 36 37 26 58 24 52 14  5 55 49
 [904] 35 46 50 41 67 36 42  2  8 66 32 62 47 14 42 60  1 23 39  7 73
 [925] 41 11 52  4 13 16 30 23 41 19  3 31 53 60 42 45 63 45 19 66 73
 [946] 35 58 62 61 51 25 46 12 20 28 67 67 53 18 62 70  2 62 18 28 72
 [967]  1 76 27 40  3 23 75 34 63 42 57  8 39 22 10 65 56 32 36 71 47
 [988] 78 16 40 47 16 23 40 23 49 56 60 68 17 49 70 42  1 44 21  5 77
[1009] 41 27 58 45 55 10 74 47 67 15 45 70 43 75 76 56 49  9  8 37 42
[1030] 10 27 13 39 53  1 15 18 53 46  7 43 29 63 55 31 12 30 62 18  1
[1051] 20 40 43 77 54 35 35  7 10 12 15 51 61 63  1 69 61 53 47 54 18
[1072] 17 34 10  5 10 60  0  1  5 31  9 49 16  2 12 30  1 50 34 15 52
[1093] 71 18  1 39 41 48 37 27 55 37 35 70 54 78 36 71 24 32 75 19 58
[1114] 58 41 67 73 70  4 73 36 50 41 39 46 40 14  8 17 52 14 57 48 68
[1135]  6 70 56 56 58 76 36 42 68 23 46 23 46 60 32 57 29 50 16 53  8
[1156] 47  9 61 23 10 45 69 71 36 72 38 38 36 69 36 56 75 19  4 20 45
[1177] 49 51 33 75 18 26 46 18 20 28 24 68 31 57 45 15 62 60 46 12 34
[1198] 37 14 71 70 23  3 27 42  3 62 16  1 21 43  5 65 25 23  7 42 30
[1219] 67 41 57 22 10 33 59 29 27 30 13 19 24 33 79 45 51 67 72 58 67
[1240] 50  6 25 12  4 79 44 65 16 63 69  2 32 39 35  2 16 36 64 25 45
[1261] 59 22 31 35 70 73 66 60  2 57 69 17 36 10  8 65  7 55 39 35 60
[1282] 45 25 52 64 62 26 72 25 19 74 51 53 71  0 56  6 20 12 58 47 56
[1303] 25 20 20 59 75  6  7 51 62 34 51  8 41  1 21 58 15 62 24 26 36
[1324]  5 44 36 59 15  0 43 29 79  7 72 14 56 75  9 19 69 68  3 52 25
[1345] 59 66 16 54 71 67 64 40 57 39 41 14 20 22  7 22 61 33 68 74 29
[1366] 18 54 17  9 78 67 69 44 39 34 35 59 73 66 53 50 63 17 63 12 17
[1387] 39 28  3 74 20 71 19 22 78 43 78 23 21 13 12 15 29 39 58 15 25
[1408] 18 42 40 26 24  9 51 56 37 52 76 38 72 35 29 49 79 77 36 13 38
[1429] 41 21 63 22 48 55 67 20  3 62 72  4 69 30 78 61 65 42 51 20  1
[1450]  1 31 53 68 78  4 30 39 48  8  8 47 74 37 77 60 25 59 13  2 48
[1471] 36 37 28 22 76 20 75 27 65 71 46 66 11 41  3 52 33 45 11 42 17
[1492] 38 37 56 44 78 30 15 74 35 33 77 27 34  2 44  0 46  8 61 76 57
[1513] 78 64 61 40 24 27 25  3 53 38 33 78 44 17 20 34 29 40 64 70 42
[1534]  8 11 48 59 27  1 55 79 39 32 25 65 72 72 29 33 60 50 23 38 48
[1555] 29 26 46 40 52 52 36 40 18 28 37 11 20 53 18 56 57 21 49 64 38
[1576] 58 57 44 31 70 22 63 44 45 78 76 29  0 48  8 17 63 42 28 50 78
[1597] 17 50 52 66 38 77 19 78 16 49 11 66  0 22 62 42 57 16 31 43 40
[1618]  7 10 39 29 66 48 17 43 59 77 78 69 33 60 38 52 51 40 65 55 65
[1639] 36 12 54 30 17 18 31 68 35 24 11 46 24 20 63 21  5 78 44 13  2
[1660] 57 40 15 56 34 31 37 40 30 48 12 27  5 26 13 11 47 63 67 77 40
[1681] 62 78 68  8 25 35 60 49 24 77 61 79 29  4 55 62 14  9 42 17 38
[1702] 10 10  3 77 55 29 48 62  6 12 42 46 36 50 11 79 24 11 77 30 50
[1723] 28 76 21  1 71 76 53 42  7  2 20 64 44 17 57  1 51 73 18 19 22
[1744] 14 10 48 52 50 27 60 36 22 36 25 28 28 37 14 79 43 59  4 10  7
[1765] 30 28 53 66 41  5 64 16 77 75 31 26 76 39 21 42 29 60 15 27 61
[1786] 72  4 66 59 47 67 77 53 75 45 59 31 35  0 31 17 59 59  6 20 36
[1807] 18 65 16 16  0 72 32 41 44 65 49 48 30 21  8 25 35 27 44 22 51
[1828] 12 38 41 72  5 75 61 78 24  3 63 28 61  8 24 36 55 39 63 39  0
[1849] 51 76 71 52 78  0 49 78 56  5  0 58  5 42 22 14 28 40 78 17 37
[1870]  4 16 67 66 53 44 57 17  3 37 16 21 77 55 63 33 27 55  8 11 79
[1891]  1 29 27 13  0 10 28  5 62 78 73 34 34 67  0 63  6  9 46 43 66
[1912] 43 76 40 75  4 48 70 55 49 63 78 48 71  1 17 27 56 55 40 46 69
[1933] 21 48 43 57 42 29 67 59 12 41 44  0 35 32 76 62 48 33 49  4  6
[1954] 71 71 16 25  0 51 35 49 44 69 12 18 17 69  8 43 17 55 28 28 23
[1975] 42 11 13 36 16 43 14 60  3 17 68 50 64 25 33 57  0  0 50  8 29
[1996] 38 21 63 28 55 70 37 50 40 53 69 79 16 17 61 56 44 37 13 43 28
[2017] 50 43 50  2 13 52 13 35 34 37 42 12 20 26 40  6 26 60 44 65 22
[2038] 62 56  6 50 59  7 52  1 75 73 60 62 77 23  8 55 58 13 26 58 41
[2059] 23 15 15 67  6 49 50 29  8 28 78 58 46 77 17 47 65 22 35 30 25
[2080] 19 57 31 34  4  0 24 18 58 58 41 61  0 36 63 61 36 26 72 28  7
[2101] 58 17  0 48 40 40 61 77 64 39 76 18  1 37 31 26 49 77 52 50 64
[2122] 10  6 38 74 47 36 63 17 78 33 15 25 60 22  9  3 28 24 41 38 47
[2143] 75 16 67 55 78  7 18  7 50 51  3 76 34 24 67 24 37 57 62  2  1
[2164]  1 77 31 43 75 57 41 17 47 63  6 17 34 70 46 22 18 43 65 72 61
[2185] 72 22 77 71 55 34 15 70  0 57 50 77 65 36  1 48  1 74 44 24 40
[2206] 59 30 50 47 76 49 35  3 57 49 50 29 15 57 17 27 36 39 26 11 13
[2227] 57 42 60 61 40 37 13 11 63 25 20 51 77  4 64 32 38  6 48 17 64
[2248] 68 14 44  2  4 69  4 12 68 65 12 28 21 44 73 11 76 66 15 17 16
[2269] 16 29 55 43 21 37 33 15 68 41 76 24 46 68 31 49  2 75 52 37 29
[2290] 12  0 53 32 48 48 68 44 60  1  6 19 78  7 61 55 30 78 74 63 39
[2311]  9  5 16 64 66 48 23 13 72 77 63 72 35 60 67 10 55 64 56  0 21
[2332] 64 29 61 65 51 70 50  6  2 36 48 50 70 18  0  1 55 24 77 37 54
[2353] 50 21 21 68  1 34  2 35 34 35 25 30 13  6 21 77 17 43 42 53  8
[2374] 54 57 63 45 66 61 55 27 51 46 32 11 52 28 20 42 37  8  2 15 19
[2395] 67 11  7  1 79 59 51 61 18 18 21 16 55 38 72 48 28 75  4 43 67
[2416] 57 27 42 28 72 59 72  3 69 14 39 10 34 69  3 19 25  1  2 69 52
[2437] 49 15 60 70  2  5 11 40  7 32  5 25 39  2 48 32 73  6 79 18 20
[2458]  8 29 62  2 70 23 67 12 64 60 15 10 29 73 53 42 14 58 56  8 77
[2479] 37 54  8 40 12 15 35  5 62 30 21 50 32 10 39 33 52 27 26 77 73
[2500]  2 51 23  6  4 46 33 33 29 65 43 38 24 33 60 44 14  2 54 78 67
[2521] 15 71 48  8 33 69 38 57 69 34 47 43 12  5 70 77  0  5 17 15 72
[2542] 44 72 32 55 42 24 51 56 75  2  2 65 31  8 29 79  1 69  1 31 31
[2563] 10 40 38 78 19 56 72 58 65 35 35 61 78 44 18 28 66 62 69 23 61
[2584] 38 62 72  4 70 77 54 39 68 34 12 44 71 21 69 51 18 13 48 20 66
[2605] 46 60 67 47 70 35 15 10 55  1 59  0 60 27 41 19 16 38 73 73 78
[2626] 44 21 40 55 68  9  5 17 14 38 22 50 52 15 40  3 67 10 60 28 12
[2647] 11 77 71 20 37 28  8 79 46 75 70 17 64 33 53  6 65 36 22 22 32
[2668] 66  7 45 38 36 37 42 30 19 49 61  0 73 43 29  0 21 77 55 78 58
[2689] 36 17 64 35 21 75 10 39 46 36  3 78  0  6 42 65 74  6 53  0 17
[2710]  6 62 59  3 14 74 34 35 42 15 22 48 12 13 73 31  5  8 21  9 16
[2731] 13 75 44 30  0 19 35 44 70 35 12 20 17 67 75 30 37 46 45 60 42
[2752] 72  7 63 11 71 46 36 68 50  8  9 31 45 44  2 61 43 50 60 23 72
[2773]  1 43  2 22 32 43  4 40  1  6 24 58  7  1 61 54 65 58 76 55 49
[2794] 72 25 26  3 62 63 73 24 28 17  1  9 13 31 21 12 24 47 25 54 68
[2815] 71 34 34 55 57 57 20 15 66 14 60 50 48  2 18 49 26 55 19 55 17
[2836] 62 29 78 62  8  9 14 55 12 17 51 62 52 37 25 63 51 68 59  5 49
[2857] 64 29 72 20 37 56 17 38 47 76 26 40 31 35 36 31 60 60  5 26 15
[2878]  9  6  8 47 61 69  9 21 34 49  9 42 56 10 46 22 44 62 54 39 24
[2899] 13 70 53 73 36 62 71 42 23 79 20 59 68 25 52 26 39 54 55 25 11
[2920]  6 50 68 54 36  6  4 54 49 38 46 62 12 59 49 52 17 12 29 64  5
[2941] 52 57 53 60  4  5 53 32 41 18  9 78 31 36 10 76 43 47 18 33 15
[2962] 44 68 66 49 12 18 51  0 76 17 66 69 65 74 28 19 49 74  2 26 63
[2983] 15 29 44 63 48 11 44 55 24 61 31 69  7  7 50 64 65 26 13 45 10
[3004] 73 35 45 67 11 44 22 23  4 26 10 17 71 15 60 59 45 74  4 73 71
[3025] 47 47 43 12 17 72 37 40 13 36 77 72 53 70 64 60 21 31 34 34 55
[3046]  6 51 16 50 16 14  3 25 65 66 38  0 37 45  8 35 14  3 77 66 35
[3067] 57 57 37 31 10 69 22 27 37 78 75 74 75 40 10  7 25 74 78  2 10
[3088] 71 55 28 77 64 46  7 58 47 12 14 13 35 25 58 29 30 75 64 17 12
[3109] 36 46 33 40  6 70 45 18 69 31 17 55 47 42 41  6 50  2 56 30 37
[3130] 57 62 61 50 32 48 44 70 34 55 52 51 64 37 39 42 31 68  4 77 71
[3151] 24 53 23 19 15 46 48 68 54 17 55 79 34 72 41 41 35 19 71  6 21
[3172] 19 47  0 46 20 57 77 74 10 54 20 51 57 24 27 19 57 55 69 54 23
[3193] 23 33 67 18  2 63 23 26  5 51 73 21 47 20 48 78 24  6 46 47 71
[3214] 28  0 39 38 45  4 66 74 36 55 42 42 44  3 17 35 27 55 48 67 32
[3235] 41 76 10 51 28 52 20 10 51 11 14  0 41 36 48 20 47 20 27 19 56
[3256] 66 63 44  3 64 60 66 63 46 74 27 21  0 28 40 17 36 22 17 38 30
[3277] 46 55 74 41  1 79 41 14 61 49 57 40 46 64 78 18 28 66 20 18 50
[3298] 46  9 39 75 75 51 56 27  3  7  7 72  9 16 26 33 78 53 48 56 71
[3319] 22 79 68  8 61 47 17  7 65 50 62 32  8 46 38 60 19 42 47 25 14
[3340] 65 45 77 13 40 69 53 45 39 55 76 63 55 38 41 69 73 32 68 45 42
[3361] 11 65 19  5 65 46 64 18 16 78 15 22 19 62 19 72 47 15 42 60  1
[3382] 73 34 52  4 22 64 25 76 13 79 43 51 43 11 65 34 36 41 44 60  7
[3403] 14 59 31 62 74 38 29 27 23  1 51 72 22 72 56 66 38 49 13 11 38
[3424] 25 53 18 76 47 52 11 72 32 24  2 16 25 50 52 21 54  2 43 18  3
[3445] 54 56  5  0 69 19 57 55 49  0  6 28 78 69 41 76 40 33 58  9 49
[3466] 63 18 60 31 59 55 71 56 12 55 77 77 12 10 17 37  6 30 27 71 23
[3487] 49 22 28 63 39 42  5  4 21 35 17 44  0  9 73 37  0  0 39 71 33
[3508] 14 28 37 46 22 69 14 12 47 26 66 58 10 75 58  4 60 38 60 29 58
[3529] 33  0  6 19 59 25 72 30 37  2 79  8 36 76 27 21 58  6 29  9 56
[3550]  6 12 40 21  2 70 57 13 48 48  2 59 41 70  1 51 10 65 73  9 58
[3571]  1 24 70 63 18 18 11 46 37 10 59 30 39 72 48 51 23 26 11 17 37
[3592] 33  6 69 59 63 68 29 13 78  3 79 24 19 76 67 70 34 48  9 27 53
[3613] 17 16 73  7 49 64 50 20  5  5 44 13 18 72 24 73 58 28  7 19 57
[3634] 25 51  3 44 43 55 21 52 49 73 65 49 19  9 26 42  4 41  4 43 20
[3655]  2 19 74  6 50 21 61 20 53 22  7 24  5 60  5 68 78 13 24 79 63
[3676] 50 28  1  9 60 20 53 53 13 30 55 21 11 16 12 38 42  9  5 12 11
[3697] 67 49 23 31 60 14 30 67 61 75 64 26 39  1 68 10 13 63 66 38 45
[3718] 15 45 40 11  7 66 62 72 25 16 47 64 27 64 48 13 76 47 47 21 73
[3739] 52 78 67  7 18 12 50 46 11 74 65 23 44 56 12 69 53 34 39 21 61
[3760] 63 30 18 54 54  7  7 59 60 52  6 24 25 10 31  9 61  3 66 47 71
[3781]  8 43 25 12 76 71  6 65 24 45 74 57 74 31 61 52 62 78 43 10 58
[3802] 35 47 62 51 21 22 39 75 24 61 50 16 46 49 41 23 75 44 60 61  5
[3823]  3 55 51 64 20 15 74 27 50 39 32 50 44 78 46 47 74 57 26 63 25
[3844] 13 64 31 72 33 66 56 38 21 64 47 47 23 57 45  3 57  5 62 14 17
[3865] 67 43 36 41 33 74 55 65 77 30 23 66 54 61 14  4 22 34 35 36 31
[3886]  8 35  4  7 66 17 43 11 71 31 63 19 37 57 64 58 72 42 21 52 18
[3907] 36  0  6 50 67 44 56 13 56 51 38 58 52 26 71 21 33 38 66 28 39
[3928] 10 28 25  5 48 35 21 55 14 68 48 52  3  9 51 33 31 77 51 18 21
[3949] 30 41 11 61  7 77 72 43 32 52  8 59 45 66 19 74 55 15 36 47  4
[3970] 12 62 25 42 64 58 60 13 14 56 34 77 50 22 46 14 71  2 46 72 48
[3991] 53 79 12 32 71 57 17 50 29 28 31  2 26 53 17 45 48 50  3 47 57
[4012] 53 43 35  5 35 39 56 36 31 67 24  2 75 69 73  1 55 71 69  3 23
[4033] 59 46 27 74 10 71 17 76 24  0 52 25 45 66  6 25 15 48 25 75 79
[4054] 19 33 23  8 14 52 30 65 21 72  4 36 27 54 38 27  9 79 42 66 75
[4075]  3 28 31 47 24 38 61 11 63 38 30 57 10 56 24 27  1 25 45  3 75
[4096] 15 11 50 56  1  8 37 57 79 52 64 58 34 47 44 14 45 31 24 12 56
[4117] 39 39 35  7 52 30 10 39 16 25  3 64 71 45 19 25 72 76 20 19 18
[4138] 33  4 23 61 18 11 59  6 31 71 76 54  6 56 71 48  5 18 44 25 23
[4159] 17 53 15  1  9 64 25  4  9 13 17 56 56 17 57 71 19 51 68 36 69
[4180] 22 16 41 65 32 50 66 42 65 76  2 50 68 50 54 13 14 75 14 63 47
[4201] 55 26  2 71 43  1 37 35  1 68  5 33 41 63 11 26  2 67 23 77 32
[4222] 28 13 42 51 30 21 42  0  4 22  5 25 48 57 55 55 65 46 21  1 79
[4243] 19 49 13 38  1 22 57 17 67 52 61 73 34 58 31 36 13 38 14 63 65
[4264] 23 69 50 56  2  6 64 48  8 58  2 22 72 38 32 57 50 41 20 72 78
[4285] 43 68 12  3 42 55 39  8  4 43 74  2 77 35 56 60 12 74 69 52 11
[4306] 25 76 26 45 33 43 67 13 13 38 61 33 66 52 18 59 28 52 30 69  2
[4327] 73 40 22 51 47 74 17 67 67  5 21 44  5  7 62 23 31 21  7 58 39
[4348] 33 65 27 20 21 31 59 19 15  4 70 57 38  6  7 53 64 66 58 66  3
[4369]  5 74 21 49  0  0  4 52 65 49 12 28 41 27 21  9  1 14 35 51  4
[4390] 77 26 26 63 45 40  3 71  6 58 38 53 34 45 43 59  5 28  2 40 17
[4411] 78 27 58 47 19  2 68 76 23 70 47 51 50 16 27 65 31 49 62 76 45
[4432] 14 23  0  9  8 11 49 34 15 55 21 75 71 29 16 77 68 17 29 39 45
[4453]  3 31 77 68 21 61 36 34 13 36 35 26 66  7 77 40 28 50 17 77 43
[4474] 79  9 59 10 10 46 75  7 27 49 34 28 10 37 49 33 51 49 63 55  2
[4495] 25 54 78 34 43 34 46 22 24 79 25 40  9 38 26 58 10 23 49 69 75
[4516] 55  5 59 17 32 23  2 50 51 46 43 71 32 72 15 30  6 33 32 30 20
[4537] 19 48  8 41 78 62 40 70 62 27  4 50 64 18  3 48 59 29 74 49 28
[4558] 18 22 71 58 55 70 72 77 51 30 78 49 30 59 40 44 31  1 44 67 56
[4579] 22  3 41 16 26 66 21  7 62 79 22 45 24 10 53 42 53  3 21 78 37
[4600] 24 53 15 71 47 18 10 64 30 15 66 65 37 75 47 78 30 72 16 22 40
[4621] 38 66 78 57 36  3 37  2 61 23 15 60 76 15 29 38 38 68 42 71 62
[4642] 54 37 66 27 59 11 44 76 52 21 47  2 56 65  6 27 78 44 46  6 48
[4663] 79 62 53 45 78 57 78 54 20 72 10 65 50 50 62 19 42 67 70 47 52
[4684] 34 78 46 61  8 34 65 50  2  1 36 39 15 66 23 13  5  6 75 54 37
[4705] 74 37 24 14 16 71 44 51  1 53 19  1 35 49 73 40 47 13 44 17 34
[4726] 20 18 61  3 32  2 49  0 40 67 53 43 10 45  6 35 64 56 60  1  4
[4747] 44 68 37 25 34 17 65 32 59 60 32 37 28 57 74 75 60 14 57 33 19
[4768] 50 48 63 27 44 59 12 72 18 38 12 46 79 63 18 48 77 17 11 60 72
[4789] 54  3 56 40 36 63 24  3 20 77 77  9

Visualization

Bar plot

Representing the sample with a single value

\[\begin{aligned} \text{mean}(\mathbf x)&=\frac{1}{N}\sum_i x_i\\ &=\bar{\mathbf x}\end{aligned}\]

mean(x) is the average of x

\(\bar{\mathbf x}\) is the value that results in the smallest squared error

 
 
 
 

mean(pop_LD)
[1] 4.509583
mean(pop_MD)
[1] 14.45521
mean(pop_HD)
[1] 39.08687

Variance: Quality of representation

\[\text{var}(\mathbf x)=\frac{1}{N}\sum_i (x_i-\bar{\mathbf x})^2\]

Variance is the mean squared error of the average

We explained this on the last semester

 
 
 
 

var(pop_LD)
[1] 8.44125
var(pop_MD)
[1] 74.48059
var(pop_HD)
[1] 522.7903

Standard Deviation

\[\begin{aligned} \text{sd}(\mathbf x)&=\sqrt{\text{var}(\mathbf x)}\\ &=\sqrt{\frac{1}{N}\sum_i (x_i-\bar{\mathbf x})^2}\end{aligned}\]

 
 
 
 
 
 

sd(pop_LD)
[1] 2.905383
sd(pop_MD)
[1] 8.630214
sd(pop_HD)
[1] 22.86461

Standard Deviation is the “width” of population

We care about sd(x) because it tells us how close is the mean to most of the population

It can be proved that always \[\Pr(\vert x_i-\bar{\mathbf x}\vert\geq k\cdot\text{sd}(\mathbf x))\leq 1/k^2\]

In other words, the probability that “the distance between the mean \(\bar{\mathbf x}\) and any element \(x_i\) is bigger than \(k\cdot\text{sd}(\mathbf x)\)” is less than \((1/k^2)\)

This is Chebyshev’s inequality

It is always valid, for any probability distribution

(Later we will see better rules valid only sometimes)

It can also be written as \[\Pr(\vert x_i-\bar{\mathbf x}\vert\leq k\cdot\text{sd}(\mathbf x))\geq 1-1/k^2\]

The probability that “the distance between the mean \(\bar{\mathbf x}\) and any element \(x_i\) is less than \(k\cdot\text{sd}(\mathbf x)\)” is greater than \(1-1/k^2\)

Some examples of Chebyshev’s inequality

Another way to understand the meaning of this theorem is \[\Pr(\bar{\mathbf x} -k\cdot\text{sd}(\mathbf x)\leq x_i \leq \bar{\mathbf x} +k\cdot\text{sd}(\mathbf x))\geq 1-1/k^2\] Replacing \(k\) for some values, we get

\[\begin{aligned} \Pr(\bar{\mathbf x} -1\cdot\text{sd}(\mathbf x)\leq x_i \leq \bar{\mathbf x} +1\cdot\text{sd}(\mathbf x))&\geq 1-1/1^2=0\\ \Pr(\bar{\mathbf x} -2\cdot\text{sd}(\mathbf x)\leq x_i \leq \bar{\mathbf x} +2\cdot\text{sd}(\mathbf x))&\geq 1-1/2^2=0.75\\ \Pr(\bar{\mathbf x} -3\cdot\text{sd}(\mathbf x)\leq x_i \leq \bar{\mathbf x} +3\cdot\text{sd}(\mathbf x))&\geq 1-1/3^2=0.889 \end{aligned}\]

From stats.libretexts.org

For any numerical data set

  • at least 3/4 of the data lie within two standard deviations of the mean, that is, in the interval with endpoints \(\bar{\mathbf x}±2\cdot\text{sd}(\mathbf x)\)
  • at least 8/9 of the data lie within three standard deviations of the mean, that is, in the interval with endpoints \(\bar{\mathbf x}±3\cdot\text{sd}(\mathbf x)\)
  • at least \(1-1/k^2\) of the data lie within \(k\) standard deviations of the mean, that is, in the interval with endpoints \(\bar{\mathbf x}±k\cdot\text{sd}(\mathbf x),\) where \(k\) is any positive whole number greater than 1

The Empirical Rule and Chebyshev’s Theorem. (2021, January 11). Retrieved May 25, 2021, from https://stats.libretexts.org/@go/page/559

Example: the case of pop_HD

These values should be more than 0, 0.75 and 0.889

mean(abs(pop_HD-mean(pop_HD))<= 1*sd(pop_HD))
[1] 0.576875
mean(abs(pop_HD-mean(pop_HD))<= 2*sd(pop_HD))
[1] 1
mean(abs(pop_HD-mean(pop_HD))<= 3*sd(pop_HD))
[1] 1

Sampling

Simulating cell counting

one_sample <- function(m, population) {
    return(sample(population, size=m))
}
one_sample(30, pop_HD)
 [1] 35 50 42  5 35 20 25 73 43 64 53 33 40 27 12 22 23 17 25 41  5  0
[23] 72 22 14 64 44 61 12 43

Sample mean is a random variable

Bad news!
The sample average changes every time

Moreover, it is often different from the population average

Sample average v/s size

Sample average v/s size

Average of smaller samples are more extreme

Average of smaller samples are more extreme

This explains why rural areas have the highest and lowest cancer rates

It is because the groups are smaller, so averages are taken from smaller groups

We saw the same on the GC content

Good news

When the sample size is big,

the sample average is closer to

the population average

Variation depends on sample size

Log-log plot (for high density population)

plot(log(sd_sample_mean)~log(size))
model_HD <- lm(log(sd_sample_mean)~log(size))
lines(predict(model_HD)~log(size))

Linear model

coef(model_HD)
(Intercept)   log(size) 
  3.2796685  -0.5362221

\[\log(\text{sd_sample_mean}) = 3.2796685 + -0.5362221\cdot\log(\text{size})\] \[\begin{aligned}\text{sd_sample_mean} &= \exp(3.2796685) \cdot\text{size}^{-0.5362221}\\ & =26.5669633\cdot\text{size}^{-0.5362221} \end{aligned}\]

Models for all populations

\[\text{sd_sample_mean} = A\cdot \text{size}^B\]

  A B std dev population
pop_LD 3.038 -0.5165 2.905
pop_MD 9.85 -0.5337 8.63
pop_HD 26.57 -0.5362 22.86

Coefficient \(A\) is the standard deviation of the population
Coefficient \(B\) is -0.5

What does this mean

If we know the population standard deviation, we can predict the sample standard deviation

\[\text{sd(sample mean)}=\frac{\text{sd(population)}}{\sqrt{\text{sample size}}}\]

Law of Large Numbers

Using Chebyshev formula, we know that, with high probability \[\vert \text{mean(sample)} -\text{mean(population)}\vert < k\cdot\frac{\text{sd(population)}}{\sqrt{\text{sample size}}}\]

Therefore the population average is inside the interval \[\text{mean(sample)} \pm k\cdot\frac{\text{sd(population)}}{\sqrt{\text{sample size}}}\] (probably)

How to know population standard deviation?

Remember that we do not know neither the population mean nor the population variance

So we do not know the population standard deviation 😕

In most cases we can use the sample standard deviation

Visually