Heavy Tails added

based on Nassim Taleb's work about fat tails

bootstrap/buckets.Rmd

@@ -496,8 +496,7 @@ legend("topright", c("from population", "from bootstrap", "true mean"), col=c(1,
 
 # Comparing statistics from different distributions
 
-Permutation tests are applicable whenever we can freely exchange labels under
-the null hypothesis; for eg, when testing the hypothesis that both samples are drawn from the same population.
+Permutation tests are applicable whenever we can freely exchange labels under the null hypothesis; for eg, when testing the hypothesis that both samples are drawn from the same population.
 
 To test, say, if two samples have the same mean but _assuming_ that they came from distributions with different variance, it is no longer possible to permute them between groups. One way to deal with it, is to resample twice, one per group, and check if the confidence interval of the differences includes the observed difference.
 

bootstrap/buckets.html

@@ -712,7 +712,7 @@ <h1>Replacing <span class="math inline">\(\chi^2\)</span>-Test</h1>
 </blockquote>
 <center>
 <!-- html table generated in R 3.6.1 by xtable 1.8-4 package -->
-<!-- Tue Nov 05 14:55:13 2019 -->
+<!-- Wed Dec 11 17:39:57 2019 -->
 <table border="1">
 <tr>
 <th>

bootstrap/stat_resampling.Rmd

@@ -371,14 +371,63 @@ abline(v=mean(group.F)-mean(group.M), col="red", lwd=2)
 quantile(results, c(.025,.975))
 ```
 
+# Replacing Friedman Rank Sum Test
 
+The Friedman Test is a frequencist test used to detect differences in several different treatments. Example: _n wine judges each rate k different wines. Are any of the k wines ranked consistently higher or lower than the others?_. To do this, this procedure ranks the same values across treatments, and has connections with ANOVA.
 
+The used statistic for the permutation test will be the [F value](https://en.wikipedia.org/wiki/F-test).
 
+```{r}
+# Computing the F statistic
+#  = variance of group means / 
+#    mean of within group variances
+# ref: https://statisticsbyjim.com/anova/f-tests-anova/
+f.stat <- function(df) {
+  means <- apply(df, 2, mean)
+  vars  <- apply(df, 2, var)
+  var(means) / mean(vars)
+}
+```
 
+So, what consists of the resampling procedure? We will compute the ranks for the observed data and find its F value. Then, we will resample these ranks by shuffling, for each row (each datum), the ranks of the different treatments and compute the resampling F values.
 
+```{r}
+rank.test <- function(observed.data, size=2500) {
+  # function for column-wise shuffling of data frame (row by row)
+  permute.rows <- function(df) {
+    apply(df, 1, function(row) sample(row))
+  }
 
+  observed.ranks   <- apply(observed.data, 1, rank) %>% t
+  permuted.data    <- replicate(size, observed.ranks %>% permute.rows %>% t)
+  permuted.effects <- apply(permuted.data, 3, f.stat)
 
+  list(permuted.effects = permuted.effects,
+       observed.effect  = observed.ranks %>% f.stat)
+}
+```
 
+Let's see it in action:
 
+```{r}
+# create fake data
+set.seed(101)
+n <- 100
+observed.data <- data.frame( gp1 = rnorm(n, 1.00, 0.5)
+                           , gp2 = rnorm(n, 1.00, 0.5)
+                           , gp3 = rnorm(n, 1.15, 0.5)
+                           , gp4 = rnorm(n, 1.00, 0.5)
+                           )
+
+results <- rank.test(observed.data) 
+
+present_results(results$permuted.effects, results$observed.effect, "Difference of Ranks")
+```
 
+Here's the  classic Friedman Rank Sum Test to compare its p-value:
+
+```{r}
+f.test <- observed.data %>% as.matrix %>% friedman.test
+f.test$p.value
+```
 

circular/index.Rmd

@@ -1,6 +1,6 @@
 ---
 title: "Circular Statistics with BUGS"
-author: "João Neto"
+author: "João Neto"
 date: September 2015
 output: 
   html_document:
@@ -56,7 +56,8 @@ The package includes a rose diagram plot:
 
 ```{r}
 par(mfrow=c(1,2))
-rose.diag(df$eventhour, bins = 24, col = "lightblue", main = "Events by Hour (sqrt scale)",
+rose.diag(df$eventhour, bins = 24, col = "lightblue", 
+          main = "Events by Hour (sqrt scale)",
           radii.scale = "sqrt", prop=3)
 
 rose.diag(df$eventhour, bin = 24, col = "lightblue", main = "Events by Hour (linear scale)", 

fat_tails/_fat tails refs/NCEI Hazard Earthquake Results.URL

@@ -0,0 +1,6 @@
+[InternetShortcut]
+URL=https://www.ngdc.noaa.gov/hazel/view/hazards/earthquake/event-data?maxYear=2020&minYear=1800
+IDList=
+HotKey=0
+IconFile=C:\Users\jpn3t\AppData\Local\Mozilla\Firefox\Profiles\s0d0utr7.default-release-1576271083429\shortcutCache\zHiLqr4fY6PFDAFLNK7rTA==.ico
+IconIndex=0

fat_tails/_fat tails refs/Quantitative Risk Management.URL

@@ -0,0 +1,6 @@
+[InternetShortcut]
+URL=https://www.youtube.com/playlist?list=PLgCR5H4IzggHyHw8dalrVHqHAqZfmTeWa
+IDList=
+HotKey=0
+IconFile=C:\Users\jpn3t\AppData\Local\Mozilla\Firefox\Profiles\s0d0utr7.default-release-1576271083429\shortcutCache\FSFJ+ftajWsjjq8tb+9uiA==.ico
+IconIndex=0

fat_tails/_fat tails refs/Wierman - Chapter Download.URL

@@ -0,0 +1,6 @@
+[InternetShortcut]
+URL=https://docs.google.com/document/d/18AisjULqnRvs4i_ccdxvNVnvlEr9JIaeGdjIoH_cKEA/edit#
+IDList=
+HotKey=0
+IconFile=C:\Users\jpn3t\AppData\Local\Mozilla\Firefox\Profiles\s0d0utr7.default-release-1576271083429\shortcutCache\Ia0Tl6+3foohAjj2o88kRQ==.ico
+IconIndex=0