Skip to content

colRowVars

Jump to bottom Edit New page
Henrik Bengtsson edited this page Sep 11, 2019 · 6 revisions

matrixStats: Benchmark report

colVars() and rowVars() benchmarks

This report benchmark the performance of colVars() and rowVars() against alternative methods.

Alternative methods

  • apply() + var()
  • colVarColMeans() and rowVarColMeans()
  • genefilter::rowVars(t(.)) and genefilter::rowVars()

where

> colVarColMeans <- function(x, na.rm = TRUE) {
+     if (na.rm) {
+         n <- colSums(!is.na(x))
+     }     else {
+         n <- nrow(x)
+     }
+     var <- colMeans(x * x, na.rm = na.rm) - (colMeans(x, na.rm = na.rm))^2
+     var * n/(n - 1)
+ }

and

> rowVarRowMeans <- function(x, na.rm = TRUE) {
+     if (na.rm) {
+         n <- rowSums(!is.na(x))
+     }     else {
+         n <- ncol(x)
+     }
+     mu <- rowMeans(x, na.rm = na.rm)
+     var <- rowMeans(x * x, na.rm = na.rm) - mu^2
+     var * (n/(n - 1))
+ }

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3217965 171.9    5709258 305.0  5709258 305.0
Vcells 6641007  50.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215471 171.8    5709258 305.0  5709258 305.0
Vcells 6633758  50.7   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.001460 0.0020040 0.0025969 0.0026205 0.0028940 0.010744
2 colVarColMeans 0.008972 0.0099950 0.0115287 0.0115365 0.0124145 0.028553
4 genefilter::rowVars(t(.)) 0.035187 0.0366470 0.0456826 0.0390445 0.0407525 0.708376
3 apply+var 0.074956 0.0776335 0.0795909 0.0790310 0.0803820 0.121227
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 6.145206 4.987525 4.439352 4.402404 4.289737 2.657576
4 genefilter::rowVars(t(.)) 24.100685 18.286926 17.590926 14.899638 14.081721 65.932241
3 apply+var 51.339726 38.739272 30.647943 30.158748 27.775397 11.283228

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.001435 0.0018395 0.0024751 0.0024095 0.0027640 0.011424
2 rowVarRowMeans 0.010436 0.0112590 0.0130313 0.0121740 0.0129220 0.043333
4 genefilter::rowVars 0.029881 0.0313825 0.0342078 0.0336050 0.0354945 0.077415
3 apply+var 0.074680 0.0765430 0.0804742 0.0781900 0.0795675 0.179592
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.00000 1.000000 1.000000
2 rowVarRowMeans 7.272474 6.120685 5.265056 5.05250 4.675109 3.793155
4 genefilter::rowVars 20.822996 17.060343 13.821006 13.94688 12.841715 6.776523
3 apply+var 52.041812 41.610764 32.514036 32.45072 28.787084 15.720588

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+10x10 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars 1.435 1.8395 2.47506 2.4095 2.764 11.424
1 colVars 1.460 2.0040 2.59694 2.6205 2.894 10.744
expr min lq mean median uq max
2 rowVars 1.000000 1.000000 1.000000 1.00000 1.000000 1.0000000
1 colVars 1.017422 1.089427 1.049243 1.08757 1.047033 0.9404762

Figure: Benchmarking of colVars() and rowVars() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3214048 171.7    5709258 305.0  5709258 305.0
Vcells 6250325  47.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3214039 171.7    5709258 305.0  5709258 305.0
Vcells 6255363  47.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.028924 0.0308180 0.0320355 0.0318570 0.0330285 0.040462
2 colVarColMeans 0.047736 0.0519650 0.0552907 0.0539490 0.0563815 0.120782
4 genefilter::rowVars(t(.)) 0.176005 0.1842785 0.1918426 0.1882460 0.1949230 0.274957
3 apply+var 0.732728 0.7632955 0.7924034 0.7863065 0.8000800 1.460130
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.650394 1.686190 1.725920 1.693474 1.707056 2.985072
4 genefilter::rowVars(t(.)) 6.085085 5.979574 5.988442 5.909094 5.901661 6.795438
3 apply+var 25.332872 24.767847 24.735182 24.682378 24.223928 36.086452

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.029560 0.0316060 0.0323945 0.0323015 0.033135 0.040801
2 rowVarRowMeans 0.107017 0.1105695 0.1146396 0.1143530 0.116681 0.166548
4 genefilter::rowVars 0.158899 0.1668470 0.1717770 0.1716305 0.175746 0.220166
3 apply+var 0.725154 0.7627570 0.7811029 0.7811415 0.798055 0.855579
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 3.620331 3.498371 3.538864 3.540176 3.521382 4.081959
4 genefilter::rowVars 5.375474 5.278966 5.302663 5.313391 5.303938 5.396093
3 apply+var 24.531597 24.133298 24.112230 24.182824 24.084955 20.969560

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+100x100 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 28.924 30.818 32.03548 31.8570 33.0285 40.462
2 rowVars 29.560 31.606 32.39447 32.3015 33.1350 40.801
expr min lq mean median uq max
1 colVars 1.000000 1.00000 1.000000 1.000000 1.000000 1.000000
2 rowVars 1.021989 1.02557 1.011206 1.013953 1.003224 1.008378

Figure: Benchmarking of colVars() and rowVars() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3214816 171.7    5709258 305.0  5709258 305.0
Vcells 6254405  47.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3214810 171.7    5709258 305.0  5709258 305.0
Vcells 6259448  47.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.029384 0.0301155 0.0311118 0.0310525 0.0314355 0.051402
2 colVarColMeans 0.048120 0.0496390 0.0515907 0.0510435 0.0519615 0.115216
3 apply+var 0.165527 0.1698840 0.1742890 0.1728510 0.1750000 0.241151
4 genefilter::rowVars(t(.)) 0.368303 0.3710325 0.3751062 0.3734090 0.3768670 0.433726
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.637626 1.648287 1.658232 1.643781 1.652956 2.241469
3 apply+var 5.633236 5.641082 5.602014 5.566412 5.566955 4.691471
4 genefilter::rowVars(t(.)) 12.534134 12.320317 12.056700 12.025087 11.988580 8.437921

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.030134 0.0310535 0.0318158 0.0317710 0.0322115 0.040320
3 apply+var 0.165937 0.1695655 0.1731759 0.1719755 0.1741725 0.237098
2 rowVarRowMeans 0.300213 0.3011130 0.3026639 0.3023300 0.3035415 0.315096
4 genefilter::rowVars 0.351820 0.3551775 0.3587482 0.3578755 0.3595995 0.429584
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
3 apply+var 5.506637 5.460431 5.443083 5.412971 5.407153 5.880407
2 rowVarRowMeans 9.962600 9.696588 9.513014 9.515911 9.423389 7.814881
4 genefilter::rowVars 11.675184 11.437600 11.275797 11.264219 11.163699 10.654365

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+1000x10 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 29.384 30.1155 31.11185 31.0525 31.4355 51.402
2 rowVars 30.134 31.0535 31.81577 31.7710 32.2115 40.320
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars 1.025524 1.031147 1.022625 1.023138 1.024685 0.7844053

Figure: Benchmarking of colVars() and rowVars() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215040 171.8    5709258 305.0  5709258 305.0
Vcells 6255360  47.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215031 171.8    5709258 305.0  5709258 305.0
Vcells 6260398  47.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.025268 0.0279980 0.0303644 0.0294665 0.0314475 0.042629
2 colVarColMeans 0.052979 0.0591515 0.0652482 0.0624990 0.0680285 0.140843
4 genefilter::rowVars(t(.)) 0.162822 0.1796470 0.1945063 0.1870080 0.1974035 0.343357
3 apply+var 6.109926 6.6011225 7.2480532 7.0200620 7.2867205 13.687952
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.00000 1.000000
2 colVarColMeans 2.096684 2.112704 2.148837 2.121019 2.16324 3.303925
4 genefilter::rowVars(t(.)) 6.443802 6.416423 6.405733 6.346461 6.27724 8.054540
3 apply+var 241.804892 235.771216 238.702258 238.238746 231.71064 321.094841

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.024447 0.0276580 0.0304014 0.0293110 0.0322280 0.044348
2 rowVarRowMeans 0.094529 0.1030295 0.1100027 0.1080685 0.1118065 0.181899
4 genefilter::rowVars 0.146207 0.1590590 0.1733496 0.1686255 0.1809965 0.255233
3 apply+var 6.113934 6.6625510 7.3587865 7.0683160 7.3025360 20.319613
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 3.866691 3.725125 3.618347 3.686961 3.469235 4.101628
4 genefilter::rowVars 5.980570 5.750922 5.702032 5.752977 5.616126 5.755231
3 apply+var 250.089336 240.890556 242.054358 241.148920 226.589798 458.185555

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+10x1000 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars 24.447 27.658 30.40138 29.3110 32.2280 44.348
1 colVars 25.268 27.998 30.36441 29.4665 31.4475 42.629
expr min lq mean median uq max
2 rowVars 1.000000 1.000000 1.0000000 1.000000 1.0000000 1.0000000
1 colVars 1.033583 1.012293 0.9987839 1.005305 0.9757819 0.9612384

Figure: Benchmarking of colVars() and rowVars() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215266 171.8    5709258 305.0  5709258 305.0
Vcells 6256030  47.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215254 171.8    5709258 305.0  5709258 305.0
Vcells 6306063  48.2   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.275615 0.300162 0.3109073 0.3078700 0.3151855 0.395928
2 colVarColMeans 0.388405 0.425314 0.4463002 0.4345805 0.4442360 0.589709
4 genefilter::rowVars(t(.)) 1.301353 1.379997 1.5185748 1.4121535 1.4680055 2.273539
3 apply+var 7.176277 7.820049 8.6066052 8.0836630 8.3058275 19.486092
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.409230 1.416948 1.435477 1.411571 1.409443 1.489435
4 genefilter::rowVars(t(.)) 4.721633 4.597507 4.884333 4.586850 4.657592 5.742304
3 apply+var 26.037324 26.052763 27.682223 26.256742 26.352188 49.216251

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.289459 0.3146985 0.3247815 0.3188465 0.327698 0.419879
2 rowVarRowMeans 0.801433 0.8602290 0.8776261 0.8724590 0.888086 1.071445
4 genefilter::rowVars 1.167510 1.2446495 1.3809156 1.2760420 1.305341 11.102619
3 apply+var 7.217139 7.8124385 8.5300494 8.1436900 8.316660 19.425627
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 2.768727 2.733502 2.702204 2.736298 2.710075 2.551795
4 genefilter::rowVars 4.033421 3.955054 4.251829 4.002057 3.983364 26.442425
3 apply+var 24.933200 24.825153 26.263960 25.541099 25.379037 46.264821

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+100x1000 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 275.615 300.1620 310.9073 307.8700 315.1855 395.928
2 rowVars 289.459 314.6985 324.7815 318.8465 327.6980 419.879
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars 1.050229 1.048429 1.044625 1.035653 1.039699 1.060493

Figure: Benchmarking of colVars() and rowVars() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215492 171.8    5709258 305.0  5709258 305.0
Vcells 6256798  47.8   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215486 171.8    5709258 305.0  5709258 305.0
Vcells 6306841  48.2   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.269730 0.2879225 0.2998887 0.2966835 0.3088765 0.367661
2 colVarColMeans 0.392556 0.4262895 0.5085648 0.4389355 0.4582800 6.883671
4 genefilter::rowVars(t(.)) 1.460079 1.5259420 1.6693504 1.5723330 1.6429030 8.278764
3 apply+var 1.528480 1.6058140 1.8092533 1.6467375 1.7313355 8.541611
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 colVarColMeans 1.455367 1.480570 1.695845 1.479474 1.483700 18.72288
4 genefilter::rowVars(t(.)) 5.413113 5.299836 5.566567 5.299698 5.318964 22.51738
3 apply+var 5.666704 5.577244 6.033083 5.550486 5.605268 23.23230

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.294086 0.3060105 0.3263007 0.318160 0.330092 0.520573
2 rowVarRowMeans 0.976035 1.0134155 1.0933112 1.059063 1.096492 1.661798
4 genefilter::rowVars 1.339642 1.4060590 1.5466028 1.440475 1.495157 8.205882
3 apply+var 1.509747 1.5914370 4.1817890 1.628740 1.691283 247.044896
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 3.318876 3.311702 3.350625 3.328710 3.321777 3.192248
4 genefilter::rowVars 4.555273 4.594806 4.739808 4.527516 4.529516 15.763172
3 apply+var 5.133692 5.200596 12.815753 5.119248 5.123672 474.563406

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on integer+1000x100 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 269.730 287.9225 299.8887 296.6835 308.8765 367.661
2 rowVars 294.086 306.0105 326.3007 318.1600 330.0920 520.573
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars 1.090298 1.062823 1.088073 1.072389 1.068686 1.415905

Figure: Benchmarking of colVars() and rowVars() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215733 171.8    5709258 305.0  5709258 305.0
Vcells 6372765  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215715 171.8    5709258 305.0  5709258 305.0
Vcells 6372888  48.7   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.001556 0.0020510 0.0027148 0.0027855 0.0030660 0.011572
2 colVarColMeans 0.009149 0.0100160 0.0117929 0.0116530 0.0125135 0.027150
4 genefilter::rowVars(t(.)) 0.034330 0.0358905 0.0390460 0.0386250 0.0397710 0.128296
3 apply+var 0.075539 0.0776635 0.0800162 0.0792815 0.0810990 0.126887
expr min lq mean median uq max
1 colVars 1.00000 1.000000 1.000000 1.00000 1.000000 1.00000
2 colVarColMeans 5.87982 4.883471 4.343861 4.18345 4.081376 2.34618
4 genefilter::rowVars(t(.)) 22.06298 17.499025 14.382360 13.86645 12.971624 11.08676
3 apply+var 48.54692 37.866163 29.473507 28.46222 26.451076 10.96500

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.001387 0.0018590 0.0024424 0.0024515 0.0027560 0.011180
2 rowVarRowMeans 0.007707 0.0088080 0.0097236 0.0094245 0.0104575 0.028320
4 genefilter::rowVars 0.028887 0.0305515 0.0330583 0.0330115 0.0343530 0.073501
3 apply+var 0.075817 0.0771945 0.0799806 0.0788150 0.0797735 0.178445
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 5.556597 4.738031 3.981142 3.844381 3.794449 2.533095
4 genefilter::rowVars 20.826965 16.434373 13.535131 13.465837 12.464804 6.574329
3 apply+var 54.662581 41.524744 32.746607 32.149704 28.945392 15.961091

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+10x10 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars 1.387 1.859 2.44241 2.4515 2.756 11.180
1 colVars 1.556 2.051 2.71485 2.7855 3.066 11.572
expr min lq mean median uq max
2 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colVars 1.121846 1.103281 1.111546 1.136243 1.112482 1.035063

Figure: Benchmarking of colVars() and rowVars() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215951 171.8    5709258 305.0  5709258 305.0
Vcells 6373811  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3215942 171.8    5709258 305.0  5709258 305.0
Vcells 6383849  48.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.028936 0.031528 0.0326631 0.0327440 0.0335710 0.045097
2 colVarColMeans 0.033583 0.037066 0.0391861 0.0389680 0.0408225 0.052679
4 genefilter::rowVars(t(.)) 0.195892 0.207596 0.2148097 0.2118340 0.2189415 0.302210
3 apply+var 0.776041 0.813462 0.8292739 0.8308275 0.8420170 0.879850
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.160596 1.175653 1.199704 1.190081 1.216005 1.168127
4 genefilter::rowVars(t(.)) 6.769837 6.584496 6.576520 6.469399 6.521745 6.701333
3 apply+var 26.819222 25.801256 25.388693 25.373427 25.081678 19.510167

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.029752 0.0312025 0.0321571 0.0320755 0.0330060 0.039333
2 rowVarRowMeans 0.056404 0.0595410 0.0616717 0.0610280 0.0627370 0.110391
4 genefilter::rowVars 0.138317 0.1463465 0.1509628 0.1502880 0.1539290 0.195537
3 apply+var 0.724393 0.7502555 0.7712210 0.7770370 0.7875495 0.831908
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 1.895805 1.908213 1.917824 1.902636 1.900776 2.806575
4 genefilter::rowVars 4.648998 4.690217 4.694534 4.685445 4.663667 4.971322
3 apply+var 24.347708 24.044724 23.982892 24.225250 23.860798 21.150383

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+100x100 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars 29.752 31.2025 32.15713 32.0755 33.006 39.333
1 colVars 28.936 31.5280 32.66312 32.7440 33.571 45.097
expr min lq mean median uq max
2 rowVars 1.0000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colVars 0.9725733 1.010432 1.015735 1.020841 1.017118 1.146544

Figure: Benchmarking of colVars() and rowVars() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216174 171.8    5709258 305.0  5709258 305.0
Vcells 6375033  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216168 171.8    5709258 305.0  5709258 305.0
Vcells 6385076  48.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.029681 0.0306250 0.0314739 0.0313115 0.0319810 0.043013
2 colVarColMeans 0.035095 0.0375480 0.0395102 0.0392315 0.0401390 0.084315
3 apply+var 0.200188 0.2049125 0.2087912 0.2074580 0.2101885 0.268354
4 genefilter::rowVars(t(.)) 0.291321 0.2946895 0.3000324 0.2983215 0.3031665 0.355143
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.182406 1.226057 1.255334 1.252942 1.255089 1.960221
3 apply+var 6.744651 6.691020 6.633793 6.625617 6.572293 6.238905
4 genefilter::rowVars(t(.)) 9.815067 9.622514 9.532743 9.527538 9.479582 8.256643

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.031358 0.0327635 0.0336445 0.0334860 0.0339690 0.042074
2 rowVarRowMeans 0.056548 0.0581170 0.0591101 0.0587185 0.0596780 0.068328
3 apply+var 0.157372 0.1602390 0.1634272 0.1621590 0.1642375 0.227331
4 genefilter::rowVars 0.235624 0.2379485 0.2409243 0.2399785 0.2421960 0.318149
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 1.803304 1.773834 1.756901 1.753524 1.756837 1.623996
3 apply+var 5.018560 4.890778 4.857469 4.842591 4.834923 5.403123
4 genefilter::rowVars 7.514000 7.262609 7.160877 7.166532 7.129913 7.561653

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+1000x10 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 29.681 30.6250 31.47388 31.3115 31.981 43.013
2 rowVars 31.358 32.7635 33.64453 33.4860 33.969 42.074
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowVars 1.056501 1.069829 1.068967 1.069447 1.062162 0.9781694

Figure: Benchmarking of colVars() and rowVars() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216398 171.8    5709258 305.0  5709258 305.0
Vcells 6375184  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216389 171.8    5709258 305.0  5709258 305.0
Vcells 6385222  48.8   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 0.022920 0.0259430 0.0291069 0.0273310 0.0312695 0.073139
2 colVarColMeans 0.038597 0.0434255 0.0493707 0.0468000 0.0526110 0.080217
4 genefilter::rowVars(t(.)) 0.152831 0.1637990 0.2377787 0.1725215 0.1887395 5.921388
3 apply+var 6.188453 6.5538515 7.1599027 6.9743715 7.2102750 13.837310
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colVarColMeans 1.683988 1.673881 1.696183 1.712341 1.682502 1.096775
4 genefilter::rowVars(t(.)) 6.668019 6.313803 8.169144 6.312301 6.035898 80.960746
3 apply+var 270.002312 252.625043 245.986188 255.181717 230.584915 189.191950

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.023676 0.0254230 0.0280148 0.0266790 0.0292275 0.043177
2 rowVarRowMeans 0.058777 0.0634995 0.0692870 0.0669680 0.0712410 0.136778
4 genefilter::rowVars 0.131994 0.1434805 0.1563137 0.1497585 0.1597975 0.256463
3 apply+var 6.166386 6.5458450 7.1699319 6.9323980 7.1196035 13.766868
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 2.482556 2.497719 2.473225 2.510139 2.437465 3.167844
4 genefilter::rowVars 5.575013 5.643728 5.579676 5.613348 5.467368 5.939806
3 apply+var 260.448809 257.477284 255.933444 259.844747 243.592627 318.847257

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+10x1000 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowVars 23.676 25.423 28.01483 26.679 29.2275 43.177
1 colVars 22.920 25.943 29.10693 27.331 31.2695 73.139
expr min lq mean median uq max
2 rowVars 1.0000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colVars 0.9680689 1.020454 1.038983 1.024439 1.069866 1.693934

Figure: Benchmarking of colVars() and rowVars() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216624 171.8    5709258 305.0  5709258 305.0
Vcells 6376639  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216612 171.8    5709258 305.0  5709258 305.0
Vcells 6476672  49.5   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 colVarColMeans 0.211638 0.253694 0.2783711 0.2667265 0.2981645 0.495933
1 colVars 0.274078 0.290533 0.3031686 0.3000850 0.3102310 0.373429
4 genefilter::rowVars(t(.)) 1.489826 1.654986 1.8198173 1.7153285 1.7810035 10.620372
3 apply+var 7.511483 8.009573 8.8056520 8.2848490 8.5225565 26.018903
expr min lq mean median uq max
2 colVarColMeans 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colVars 1.295032 1.145210 1.089081 1.125066 1.040469 0.7529828
4 genefilter::rowVars(t(.)) 7.039501 6.523552 6.537378 6.431039 5.973224 21.4149331
3 apply+var 35.492128 31.571787 31.632776 31.061214 28.583405 52.4645527

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.289432 0.3031925 0.3394843 0.3198170 0.3534725 0.500898
2 rowVarRowMeans 0.440619 0.4814455 0.5062973 0.4985045 0.5123345 0.755062
4 genefilter::rowVars 0.968830 1.0926175 1.2465513 1.1379740 1.1809715 9.777972
3 apply+var 7.117417 7.5680465 8.6340100 7.9284915 8.8480955 19.268878
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 1.522358 1.587920 1.491372 1.558718 1.449432 1.507417
4 genefilter::rowVars 3.347349 3.603709 3.671897 3.558204 3.341056 19.520885
3 apply+var 24.590982 24.961193 25.432723 24.790713 25.031920 38.468666

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+100x1000 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 274.078 290.5330 303.1686 300.085 310.2310 373.429
2 rowVars 289.432 303.1925 339.4843 319.817 353.4725 500.898
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars 1.056021 1.043573 1.119787 1.065755 1.139385 1.341347

Figure: Benchmarking of colVars() and rowVars() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216850 171.8    5709258 305.0  5709258 305.0
Vcells 6376786  48.7   22345847 170.5 56666022 432.4
> colStats <- microbenchmark(colVars = colVars(X, na.rm = FALSE), colVarColMeans = colVarColMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 2L, FUN = var, na.rm = FALSE), `genefilter::rowVars(t(.))` = genefilter_colVars(X, 
+     na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3216844 171.8    5709258 305.0  5709258 305.0
Vcells 6476829  49.5   22345847 170.5 56666022 432.4
> rowStats <- microbenchmark(rowVars = rowVars(X, na.rm = FALSE), rowVarRowMeans = rowVarRowMeans(X, 
+     na.rm = FALSE), `apply+var` = apply(X, MARGIN = 1L, FUN = var, na.rm = FALSE), `genefilter::rowVars` = genefilter_rowVars(X, 
+     na.rm = FALSE), unit = "ms")

Table: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 colVarColMeans 0.230082 0.2452775 0.4582199 0.2825605 0.4763775 10.352064
1 colVars 0.268699 0.2763685 0.3032927 0.2976315 0.3291365 0.385134
4 genefilter::rowVars(t(.)) 1.193049 1.2922330 1.7129067 1.4956065 2.0847410 7.467183
3 apply+var 1.395430 1.4664860 1.8258231 1.6921730 1.9289650 8.239629
expr min lq mean median uq max
2 colVarColMeans 1.000000 1.000000 1.0000000 1.000000 1.0000000 1.0000000
1 colVars 1.167840 1.126758 0.6618933 1.053337 0.6909153 0.0372036
4 genefilter::rowVars(t(.)) 5.185321 5.268453 3.7381758 5.293049 4.3762373 0.7213231
3 apply+var 6.064925 5.978885 3.9845999 5.988710 4.0492362 0.7959407

Table: Benchmarking of rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowVars 0.295673 0.3097960 0.3525580 0.327192 0.3823360 0.602889
2 rowVarRowMeans 0.447263 0.4719035 0.5568056 0.520182 0.5852035 1.420983
4 genefilter::rowVars 1.059860 1.0947750 1.4357536 1.161610 1.3250435 7.669609
3 apply+var 1.427955 1.4776015 1.8454647 1.565719 1.8629885 8.765901
expr min lq mean median uq max
1 rowVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVarRowMeans 1.512695 1.523272 1.579331 1.589837 1.530600 2.356956
4 genefilter::rowVars 3.584568 3.533858 4.072390 3.550240 3.465652 12.721428
3 apply+var 4.829508 4.769595 5.234500 4.785323 4.872647 14.539826

Figure: Benchmarking of colVars(), colVarColMeans(), apply+var() and genefilter::rowVars(t(.))() on double+1000x100 data as well as rowVars(), rowVarRowMeans(), apply+var() and genefilter::rowVars() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colVars() and rowVars() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colVars 268.699 276.3685 303.2927 297.6315 329.1365 385.134
2 rowVars 295.673 309.7960 352.5580 327.1920 382.3360 602.889
expr min lq mean median uq max
1 colVars 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowVars 1.100387 1.120953 1.162435 1.099319 1.161633 1.565401

Figure: Benchmarking of colVars() and rowVars() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 35.3 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colVars')

Copyright Henrik Bengtsson. Last updated on 2019-09-10 20:55:08 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer

Benchmark reports

Clone this wiki locally