A sorcerer’s tale around campfire, magic potions, and grouped distributions

[rempsyc]

In psychology, we frequently have to produce descriptive statistics by group. The psych package allows this.

library(psych)
describeBy(iris, "Species")
#> 
#>  Descriptive statistics by group 
#> Species: setosa
#>              vars  n mean   sd median trimmed  mad min max range skew kurtosis
#> Sepal.Length    1 50 5.01 0.35    5.0    5.00 0.30 4.3 5.8   1.5 0.11    -0.45
#> Sepal.Width     2 50 3.43 0.38    3.4    3.42 0.37 2.3 4.4   2.1 0.04     0.60
#> Petal.Length    3 50 1.46 0.17    1.5    1.46 0.15 1.0 1.9   0.9 0.10     0.65
#> Petal.Width     4 50 0.25 0.11    0.2    0.24 0.00 0.1 0.6   0.5 1.18     1.26
#> Species*        5 50 1.00 0.00    1.0    1.00 0.00 1.0 1.0   0.0  NaN      NaN
#>                se
#> Sepal.Length 0.05
#> Sepal.Width  0.05
#> Petal.Length 0.02
#> Petal.Width  0.01
#> Species*     0.00
#> ------------------------------------------------------------ 
#> Species: versicolor
#>              vars  n mean   sd median trimmed  mad min max range  skew kurtosis
#> Sepal.Length    1 50 5.94 0.52   5.90    5.94 0.52 4.9 7.0   2.1  0.10    -0.69
#> Sepal.Width     2 50 2.77 0.31   2.80    2.78 0.30 2.0 3.4   1.4 -0.34    -0.55
#> Petal.Length    3 50 4.26 0.47   4.35    4.29 0.52 3.0 5.1   2.1 -0.57    -0.19
#> Petal.Width     4 50 1.33 0.20   1.30    1.32 0.22 1.0 1.8   0.8 -0.03    -0.59
#> Species*        5 50 2.00 0.00   2.00    2.00 0.00 2.0 2.0   0.0   NaN      NaN
#>                se
#> Sepal.Length 0.07
#> Sepal.Width  0.04
#> Petal.Length 0.07
#> Petal.Width  0.03
#> Species*     0.00
#> ------------------------------------------------------------ 
#> Species: virginica
#>              vars  n mean   sd median trimmed  mad min max range  skew kurtosis
#> Sepal.Length    1 50 6.59 0.64   6.50    6.57 0.59 4.9 7.9   3.0  0.11    -0.20
#> Sepal.Width     2 50 2.97 0.32   3.00    2.96 0.30 2.2 3.8   1.6  0.34     0.38
#> Petal.Length    3 50 5.55 0.55   5.55    5.51 0.67 4.5 6.9   2.4  0.52    -0.37
#> Petal.Width     4 50 2.03 0.27   2.00    2.03 0.30 1.4 2.5   1.1 -0.12    -0.75
#> Species*        5 50 3.00 0.00   3.00    3.00 0.00 3.0 3.0   0.0   NaN      NaN
#>                se
#> Sepal.Length 0.09
#> Sepal.Width  0.05
#> Petal.Length 0.08
#> Petal.Width  0.04
#> Species*     0.00

datawizard also has a nice function to produce descriptive statistics:

library(datawizard)
#> 
#> Attaching package: 'datawizard'
#> The following object is masked from 'package:psych':
#> 
#>     rescale
describe_distribution(iris)
#> Variable     | Mean |   SD |  IQR |        Range | Skewness | Kurtosis |   n | n_Missing
#> ----------------------------------------------------------------------------------------
#> Sepal.Length | 5.84 | 0.83 | 1.30 | [4.30, 7.90] |     0.31 |    -0.55 | 150 |         0
#> Sepal.Width  | 3.06 | 0.44 | 0.52 | [2.00, 4.40] |     0.32 |     0.23 | 150 |         0
#> Petal.Length | 3.76 | 1.77 | 3.52 | [1.00, 6.90] |    -0.27 |    -1.40 | 150 |         0
#> Petal.Width  | 1.20 | 0.76 | 1.50 | [0.10, 2.50] |    -0.10 |    -1.34 | 150 |         0

However it does not support groups directly (there are no group arguments). Fortunately, datawizard, like any respectable magician, has a trick or two hiding in a hat (or behind someone’s ear): datawizard::data_group. Legends say that This function is comparable to dplyr::group_by(). This is great news, because it would then seem just like the right tool to satisfy our appetite for adventure and mysticism. Because we don’t really want to rely on the muggle dplyr, do we? Given this is a scientific enterprise, after all, let’s empirically verify the claims of tales passed down through generations.

iris |>
  data_group(Species) |>
  describe_distribution(centrality = "all")
#> # Species=setosa
#> 
#> Variable     | Median |  MAD | Mean |   SD |  MAP |  IQR |        Range | Skewness | Kurtosis |  n | n_Missing
#> --------------------------------------------------------------------------------------------------------------
#> Sepal.Length |   5.00 | 0.30 | 5.01 | 0.35 | 5.02 | 0.40 | [4.30, 5.80] |     0.12 |    -0.25 | 50 |         0
#> Sepal.Width  |   3.40 | 0.37 | 3.43 | 0.38 | 3.40 | 0.52 | [2.30, 4.40] |     0.04 |     0.95 | 50 |         0
#> Petal.Length |   1.50 | 0.15 | 1.46 | 0.17 | 1.45 | 0.20 | [1.00, 1.90] |     0.11 |     1.02 | 50 |         0
#> Petal.Width  |   0.20 | 0.00 | 0.25 | 0.11 | 0.20 | 0.10 | [0.10, 0.60] |     1.25 |     1.72 | 50 |         0
#> 
#> # Species=versicolor
#> 
#> Variable     | Median |  MAD | Mean |   SD |  MAP |  IQR |        Range | Skewness | Kurtosis |  n | n_Missing
#> --------------------------------------------------------------------------------------------------------------
#> Sepal.Length |   5.90 | 0.52 | 5.94 | 0.52 | 5.75 | 0.70 | [4.90, 7.00] |     0.11 |    -0.53 | 50 |         0
#> Sepal.Width  |   2.80 | 0.30 | 2.77 | 0.31 | 2.90 | 0.50 | [2.00, 3.40] |    -0.36 |    -0.37 | 50 |         0
#> Petal.Length |   4.35 | 0.52 | 4.26 | 0.47 | 4.46 | 0.60 | [3.00, 5.10] |    -0.61 |     0.05 | 50 |         0
#> Petal.Width  |   1.30 | 0.22 | 1.33 | 0.20 | 1.31 | 0.30 | [1.00, 1.80] |    -0.03 |    -0.41 | 50 |         0
#> 
#> # Species=virginica
#> 
#> Variable     | Median |  MAD | Mean |   SD |  MAP |  IQR |        Range | Skewness | Kurtosis |  n | n_Missing
#> --------------------------------------------------------------------------------------------------------------
#> Sepal.Length |   6.50 | 0.59 | 6.59 | 0.64 | 6.40 | 0.75 | [4.90, 7.90] |     0.12 |     0.03 | 50 |         0
#> Sepal.Width  |   3.00 | 0.30 | 2.97 | 0.32 | 2.98 | 0.40 | [2.20, 3.80] |     0.37 |     0.71 | 50 |         0
#> Petal.Length |   5.55 | 0.67 | 5.55 | 0.55 | 5.55 | 0.80 | [4.50, 6.90] |     0.55 |    -0.15 | 50 |         0
#> Petal.Width  |   2.00 | 0.30 | 2.03 | 0.27 | 1.86 | 0.50 | [1.40, 2.50] |    -0.13 |    -0.60 | 50 |         0

WOW! Wonderful! A breathtaking trick worthy even of Max Maven’s most powerful illusions. Of course, even the most learned warlock can still improve some aspects of their craft. Possible additional improvements, for example, would be to add info from psych:describeBy that is missing from datawizard::describe_distribution:

• se (only one left after discovering the centrality = "all" argument 😱🙊🤯)
• mode (from including mode for the distribution in describe_distribution #160)
• 95% CI? (maybe not, but adding just in case)

One might also wonder whether these features should stay within the psych package proper, which is a reasonable possibility. After all, it is a package made for psychologists. But given datawizard has a function so similar available already, it is worth considering, if only so researchers and sorcerers alike may rely on one fewer dependency within our magical scripts and grimoires.

PS: Note that the skewness and kurtosis values differ slightly between the two packages, and though I know it not, there is probably a reason for this.

PS2: I have gone through many iterations of this post before realizing that datawizard, in fact, does address all of my needs. Realizing that my feature requests were now unnecessary, I have turned this issue into a discussion instead. Perhaps it would be nice to add the group_by to the readme and vignettes for more visibility (I mean, combined with describe_distribution, specifically).

^{Created on 2022-09-03 by the reprex package (v2.0.1)}

[strengejacke]

At PS1: there are different types to compute the kurtosis out skewness (https://easystats.github.io/datawizard/reference/skewness.html)

At PS2: I just found the function data_group(), just to realize we already have it for some time, and that it was even me who implemented that function...