Skip to content

colRowOrderStats_subset

Jump to bottom Edit New page
Henrik Bengtsson edited this page Sep 11, 2019 · 3 revisions

matrixStats: Benchmark report

colOrderStats() and rowOrderStats() benchmarks on subsetted computation

This report benchmark the performance of colOrderStats() and rowOrderStats() on subsetted computation.

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3160715 168.9    5709258 305.0  5709258 305.0
Vcells 6170406  47.1   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159093 168.8    5709258 305.0  5709258 305.0
Vcells 6165814  47.1   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.001679 0.0017115 0.0028944 0.001741 0.001896 0.110035
2 colOrderStats(X, rows, cols) 0.001831 0.0019415 0.0020506 0.001986 0.002089 0.003936
3 colOrderStats(X[rows, cols]) 0.002197 0.0024350 0.0026608 0.002531 0.002690 0.009370
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 colOrderStats(X, rows, cols) 1.090530 1.134385 0.708472 1.140724 1.101793 0.0357704
3 colOrderStats(X[rows, cols]) 1.308517 1.422729 0.919295 1.453762 1.418776 0.0851547

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.001644 0.001686 0.0018210 0.0017225 0.0019030 0.004463
2 rowOrderStats(X, cols, rows) 0.001829 0.001910 0.0031059 0.0019580 0.0021190 0.108610
3 rowOrderStats(X[cols, rows]) 0.002221 0.002407 0.0026516 0.0025335 0.0027135 0.007356
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.112530 1.132859 1.705618 1.136720 1.113505 24.335649
3 rowOrderStats(X[cols, rows]) 1.350973 1.427639 1.456139 1.470827 1.425906 1.648219

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+10x10 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowOrderStats_X_S 1.644 1.6860 1.82100 1.7225 1.903 4.463
1 colOrderStats_X_S 1.679 1.7115 2.89437 1.7410 1.896 110.035
expr min lq mean median uq max
2 rowOrderStats_X_S 1.000000 1.000000 1.00000 1.00000 1.0000000 1.00000
1 colOrderStats_X_S 1.021289 1.015125 1.58944 1.01074 0.9963216 24.65494

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3157722 168.7    5709258 305.0  5709258 305.0
Vcells 5834137  44.6   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3157715 168.7    5709258 305.0  5709258 305.0
Vcells 5839219  44.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.045465 0.0464495 0.0468889 0.0468450 0.0473160 0.051183
2 colOrderStats(X, rows, cols) 0.050398 0.0515100 0.0520990 0.0518795 0.0523495 0.060443
3 colOrderStats(X[rows, cols]) 0.053829 0.0549355 0.0558574 0.0553235 0.0558250 0.093930
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.108501 1.108946 1.111117 1.107471 1.106380 1.180919
3 colOrderStats(X[rows, cols]) 1.183966 1.182693 1.191273 1.180991 1.179833 1.835180

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.047157 0.0479810 0.0487817 0.0484515 0.0489790 0.057191
2 rowOrderStats(X, cols, rows) 0.050720 0.0515625 0.0525264 0.0519995 0.0523430 0.091926
3 rowOrderStats(X[cols, rows]) 0.055143 0.0566260 0.0581534 0.0571055 0.0578135 0.097305
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.075556 1.074644 1.076763 1.073228 1.068682 1.607351
3 rowOrderStats(X[cols, rows]) 1.169349 1.180176 1.192115 1.178612 1.180373 1.701404

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+100x100 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 45.465 46.4495 46.88887 46.8450 47.316 51.183
2 rowOrderStats_X_S 47.157 47.9810 48.78173 48.4515 48.979 57.191
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.037215 1.032971 1.040369 1.034294 1.035147 1.117383

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158479 168.7    5709258 305.0  5709258 305.0
Vcells 5838506  44.6   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158469 168.7    5709258 305.0  5709258 305.0
Vcells 5843583  44.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.040612 0.0416015 0.0422597 0.042175 0.0425080 0.053002
2 colOrderStats(X, rows, cols) 0.046201 0.0468570 0.0477479 0.047254 0.0476280 0.091428
3 colOrderStats(X[rows, cols]) 0.048938 0.0500370 0.0508741 0.050405 0.0508885 0.064984
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.137619 1.126330 1.129867 1.120427 1.120448 1.724992
3 colOrderStats(X[rows, cols]) 1.205013 1.202769 1.203843 1.195139 1.197151 1.226067

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.043081 0.0440415 0.0451259 0.044399 0.0449625 0.088380
2 rowOrderStats(X, cols, rows) 0.047081 0.0481515 0.0491258 0.048543 0.0489885 0.072370
3 rowOrderStats(X[cols, rows]) 0.052782 0.0536900 0.0553989 0.054027 0.0548870 0.092495
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowOrderStats(X, cols, rows) 1.092848 1.093321 1.088639 1.093335 1.089541 0.8188504
3 rowOrderStats(X[cols, rows]) 1.225181 1.219078 1.227652 1.216852 1.220728 1.0465603

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+1000x10 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 40.612 41.6015 42.25973 42.175 42.5080 53.002
2 rowOrderStats_X_S 43.081 44.0415 45.12588 44.399 44.9625 88.380
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.060795 1.058652 1.067822 1.052733 1.057742 1.667484

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158685 168.7    5709258 305.0  5709258 305.0
Vcells 5839445  44.6   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158675 168.7    5709258 305.0  5709258 305.0
Vcells 5844522  44.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.035543 0.0379300 0.0395150 0.0393480 0.0406585 0.047658
2 colOrderStats(X, rows, cols) 0.043625 0.0459880 0.0472274 0.0468690 0.0477795 0.089594
3 colOrderStats(X[rows, cols]) 0.045421 0.0473875 0.0491550 0.0491935 0.0505995 0.060045
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.227386 1.212444 1.195178 1.191141 1.175142 1.879936
3 colOrderStats(X[rows, cols]) 1.277917 1.249341 1.243958 1.250216 1.244500 1.259914

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.036867 0.038759 0.0406265 0.040116 0.0413585 0.088381
2 rowOrderStats(X, cols, rows) 0.042657 0.045875 0.0466163 0.046849 0.0476395 0.050586
3 rowOrderStats(X[cols, rows]) 0.044834 0.047037 0.0485303 0.048292 0.0496065 0.067829
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowOrderStats(X, cols, rows) 1.157051 1.183596 1.147435 1.167838 1.151867 0.5723628
3 rowOrderStats(X[cols, rows]) 1.216101 1.213576 1.194548 1.203809 1.199427 0.7674613

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+10x1000 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 35.543 37.930 39.51499 39.348 40.6585 47.658
2 rowOrderStats_X_S 36.867 38.759 40.62648 40.116 41.3585 88.381
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.037251 1.021856 1.028128 1.019518 1.017217 1.854484

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158884 168.8    5709258 305.0  5709258 305.0
Vcells 5862181  44.8   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3158874 168.8    5709258 305.0  5709258 305.0
Vcells 5912258  45.2   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.511953 0.5136960 0.5164118 0.5144935 0.5156305 0.582297
2 colOrderStats(X, rows, cols) 0.553590 0.5558820 0.5663287 0.5565210 0.5574335 0.980120
3 colOrderStats(X[rows, cols]) 0.584424 0.5857075 0.5918786 0.5873245 0.5894085 0.730817
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.081330 1.082123 1.096661 1.081687 1.081072 1.683196
3 colOrderStats(X[rows, cols]) 1.141558 1.140183 1.146137 1.141559 1.143083 1.255059

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.524200 0.5264185 0.5313409 0.5273445 0.5287520 0.685987
2 rowOrderStats(X, cols, rows) 0.562353 0.5651015 0.5770728 0.5664360 0.5684105 1.222077
3 rowOrderStats(X[cols, rows]) 0.588049 0.5904985 0.6012168 0.5920775 0.5935995 0.986854
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.072783 1.073483 1.086069 1.074129 1.075004 1.781487
3 rowOrderStats(X[cols, rows]) 1.121803 1.121728 1.131508 1.122753 1.122643 1.438590

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+100x1000 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 511.953 513.6960 516.4118 514.4935 515.6305 582.297
2 rowOrderStats_X_S 524.200 526.4185 531.3409 527.3445 528.7520 685.987
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.023922 1.024767 1.028909 1.024978 1.025448 1.178071

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159100 168.8    5709258 305.0  5709258 305.0
Vcells 5862952  44.8   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159090 168.8    5709258 305.0  5709258 305.0
Vcells 5913029  45.2   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.468832 0.470952 0.4723511 0.4715670 0.4726180 0.509464
2 colOrderStats(X, rows, cols) 0.510461 0.511813 0.5131895 0.5126730 0.5137085 0.531828
3 colOrderStats(X[rows, cols]) 0.533324 0.534661 0.5369369 0.5352865 0.5364225 0.646107
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.088793 1.086763 1.086458 1.087169 1.086942 1.043897
3 colOrderStats(X[rows, cols]) 1.137559 1.135277 1.136733 1.135123 1.135002 1.268209

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.491882 0.4935045 0.4950448 0.4943935 0.4958175 0.520219
2 rowOrderStats(X, cols, rows) 0.526540 0.5287630 0.5295081 0.5293495 0.5302955 0.533636
3 rowOrderStats(X[cols, rows]) 0.562691 0.5647765 0.5680307 0.5658485 0.5683605 0.672219
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.070460 1.071445 1.069616 1.070705 1.069538 1.025791
3 rowOrderStats(X[cols, rows]) 1.143955 1.144420 1.147433 1.144531 1.146310 1.292185

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on integer+1000x100 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 468.832 470.9520 472.3511 471.5670 472.6180 509.464
2 rowOrderStats_X_S 491.882 493.5045 495.0448 494.3935 495.8175 520.219
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 rowOrderStats_X_S 1.049165 1.047887 1.048044 1.048406 1.049087 1.02111

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159318 168.8    5709258 305.0  5709258 305.0
Vcells 5954077  45.5   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159299 168.8    5709258 305.0  5709258 305.0
Vcells 5954239  45.5   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.001858 0.0019260 0.0021853 0.0019720 0.0020450 0.018716
2 colOrderStats(X, rows, cols) 0.002082 0.0021705 0.0023088 0.0022235 0.0023195 0.005371
3 colOrderStats(X[rows, cols]) 0.002534 0.0028185 0.0029899 0.0029035 0.0030220 0.008190
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 colOrderStats(X, rows, cols) 1.120560 1.126947 1.056518 1.127536 1.134230 0.2869737
3 colOrderStats(X[rows, cols]) 1.363832 1.463396 1.368193 1.472363 1.477751 0.4375935

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.001846 0.0018910 0.0020326 0.0019495 0.0020870 0.004615
2 rowOrderStats(X, cols, rows) 0.002075 0.0021185 0.0026252 0.0021980 0.0023625 0.038320
3 rowOrderStats(X[cols, rows]) 0.002401 0.0026410 0.0028597 0.0027630 0.0028915 0.008576
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.124052 1.120307 1.291514 1.127469 1.132008 8.303359
3 rowOrderStats(X[cols, rows]) 1.300650 1.396616 1.406877 1.417286 1.385482 1.858288

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+10x10 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowOrderStats_X_S 1.846 1.891 2.03263 1.9495 2.087 4.615
1 colOrderStats_X_S 1.858 1.926 2.18532 1.9720 2.045 18.716
expr min lq mean median uq max
2 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.0000000 1.000000
1 colOrderStats_X_S 1.006501 1.018509 1.075119 1.011541 0.9798754 4.055471

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159520 168.8    5709258 305.0  5709258 305.0
Vcells 5960022  45.5   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159510 168.8    5709258 305.0  5709258 305.0
Vcells 5970099  45.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.077925 0.0785155 0.0787999 0.0787215 0.078998 0.082369
2 colOrderStats(X, rows, cols) 0.082524 0.0831375 0.0836289 0.0833135 0.083621 0.098135
3 colOrderStats(X[rows, cols]) 0.094363 0.0947905 0.0957399 0.0949730 0.095258 0.152458
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.059018 1.058867 1.061282 1.058332 1.058520 1.191407
3 colOrderStats(X[rows, cols]) 1.210946 1.207284 1.214975 1.206443 1.205828 1.850915

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.078604 0.0792060 0.0794728 0.0793895 0.0795815 0.084939
2 rowOrderStats(X, cols, rows) 0.082436 0.0829105 0.0834196 0.0832010 0.0834545 0.091404
3 rowOrderStats(X[cols, rows]) 0.086782 0.0872640 0.0881903 0.0875140 0.0877785 0.126035
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.048751 1.046770 1.049662 1.048010 1.048667 1.076113
3 rowOrderStats(X[cols, rows]) 1.104040 1.101735 1.109690 1.102337 1.103001 1.483830

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+100x100 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 77.925 78.5155 78.79991 78.7215 78.9980 82.369
2 rowOrderStats_X_S 78.604 79.2060 79.47285 79.3895 79.5815 84.939
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.00000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.008714 1.008794 1.00854 1.008486 1.007386 1.031201

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159729 168.8    5709258 305.0  5709258 305.0
Vcells 5961532  45.5   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159719 168.8    5709258 305.0  5709258 305.0
Vcells 5971609  45.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.072105 0.0727140 0.0733995 0.0729070 0.0732475 0.089554
2 colOrderStats(X, rows, cols) 0.077293 0.0777605 0.0786936 0.0779940 0.0782940 0.138389
3 colOrderStats(X[rows, cols]) 0.088435 0.0889675 0.0895629 0.0891695 0.0894280 0.103972
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.071951 1.069402 1.072127 1.069774 1.068896 1.545313
3 colOrderStats(X[rows, cols]) 1.226475 1.223526 1.220212 1.223058 1.220902 1.160998

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.074353 0.0748150 0.0756492 0.0751230 0.0754320 0.114917
2 rowOrderStats(X, cols, rows) 0.078350 0.0789755 0.0794441 0.0792230 0.0795345 0.088957
3 rowOrderStats(X[cols, rows]) 0.083684 0.0844845 0.0851351 0.0848625 0.0852920 0.092688
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowOrderStats(X, cols, rows) 1.053757 1.055611 1.050165 1.054577 1.054387 0.7740978
3 rowOrderStats(X[cols, rows]) 1.125496 1.129245 1.125394 1.129647 1.130714 0.8065647

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+1000x10 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 72.105 72.714 73.39946 72.907 73.2475 89.554
2 rowOrderStats_X_S 74.353 74.815 75.64915 75.123 75.4320 114.917
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.00000 1.000000 1.000000 1.000000
2 rowOrderStats_X_S 1.031177 1.028894 1.03065 1.030395 1.029824 1.283215

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159935 168.8    5709258 305.0  5709258 305.0
Vcells 5961669  45.5   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3159925 168.8    5709258 305.0  5709258 305.0
Vcells 5971746  45.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.066852 0.067833 0.0685552 0.0683925 0.0691210 0.073122
2 colOrderStats(X, rows, cols) 0.072981 0.074263 0.0750012 0.0747020 0.0751530 0.100147
3 colOrderStats(X[rows, cols]) 0.077055 0.077892 0.0785175 0.0782685 0.0788555 0.084143
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.091680 1.094792 1.094025 1.092254 1.087267 1.369588
3 colOrderStats(X[rows, cols]) 1.152621 1.148291 1.145317 1.144402 1.140833 1.150721

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.067635 0.0685505 0.0697352 0.0692105 0.0698205 0.109806
2 rowOrderStats(X, cols, rows) 0.072822 0.0740610 0.0746043 0.0746665 0.0749840 0.077564
3 rowOrderStats(X[cols, rows]) 0.075961 0.0768790 0.0780130 0.0774315 0.0779910 0.104113
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowOrderStats(X, cols, rows) 1.076691 1.080386 1.069824 1.078832 1.073954 0.7063731
3 rowOrderStats(X[cols, rows]) 1.123102 1.121494 1.118705 1.118783 1.117021 0.9481540

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+10x1000 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 66.852 67.8330 68.55525 68.3925 69.1210 73.122
2 rowOrderStats_X_S 67.635 68.5505 69.73516 69.2105 69.8205 109.806
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.00000 1.00000 1.000000
2 rowOrderStats_X_S 1.011712 1.010577 1.017211 1.01196 1.01012 1.501682

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3160134 168.8    5709258 305.0  5709258 305.0
Vcells 6007240  45.9   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3160124 168.8    5709258 305.0  5709258 305.0
Vcells 6107317  46.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.770434 0.7737435 0.7862686 0.7747015 0.7772540 1.049748
2 colOrderStats(X, rows, cols) 0.813697 0.8165185 0.8245568 0.8181240 0.8207455 1.011181
3 colOrderStats(X[rows, cols]) 0.920819 0.9243660 0.9314293 0.9267720 0.9303025 1.184210
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 colOrderStats(X, rows, cols) 1.056154 1.055283 1.048696 1.056051 1.055955 0.9632607
3 colOrderStats(X[rows, cols]) 1.195195 1.194667 1.184620 1.196296 1.196909 1.1280898

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.778297 0.7809520 0.7886945 0.7828105 0.7874945 0.930093
2 rowOrderStats(X, cols, rows) 0.830279 0.8324975 0.8447693 0.8343380 0.8397160 1.433900
3 rowOrderStats(X[cols, rows]) 0.848244 0.8535030 0.8672251 0.8564900 0.8610175 1.026660
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.066789 1.066003 1.071098 1.065824 1.066313 1.541674
3 rowOrderStats(X[cols, rows]) 1.089872 1.092901 1.099570 1.094122 1.093363 1.103825

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+100x1000 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 770.434 773.7435 786.2686 774.7015 777.2540 1049.748
2 rowOrderStats_X_S 778.297 780.9520 788.6945 782.8105 787.4945 930.093
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
2 rowOrderStats_X_S 1.010206 1.009316 1.003085 1.010467 1.013175 0.8860155

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> rows <- sample.int(nrow(X), size = nrow(X) * 0.7)
> cols <- sample.int(ncol(X), size = ncol(X) * 0.7)
> X_S <- X[rows, cols]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3160350 168.8    5709258 305.0  5709258 305.0
Vcells 6007387  45.9   22267496 169.9 56666022 432.4
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats_X_S = colOrderStats(X_S, which = which, na.rm = FALSE), 
+     `colOrderStats(X, rows, cols)` = colOrderStats(X, rows = rows, cols = cols, which = which, na.rm = FALSE), 
+     `colOrderStats(X[rows, cols])` = colOrderStats(X[rows, cols], which = which, na.rm = FALSE), 
+     unit = "ms")
> X <- t(X)
> X_S <- t(X_S)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3160340 168.8    5709258 305.0  5709258 305.0
Vcells 6107464  46.6   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowOrderStats_X_S = rowOrderStats(X_S, which = which, na.rm = FALSE), 
+     `rowOrderStats(X, cols, rows)` = rowOrderStats(X, rows = cols, cols = rows, which = which, na.rm = FALSE), 
+     `rowOrderStats(X[cols, rows])` = rowOrderStats(X[cols, rows], which = which, na.rm = FALSE), 
+     unit = "ms")

Table: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 0.758792 0.7602540 0.7652907 0.7610550 0.7629300 0.877707
2 colOrderStats(X, rows, cols) 0.799374 0.8007675 0.8056904 0.8018910 0.8032150 0.921177
3 colOrderStats(X[rows, cols]) 0.828195 0.8329110 0.8397628 0.8339365 0.8365295 1.022333
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 colOrderStats(X, rows, cols) 1.053482 1.053289 1.052790 1.053657 1.052803 1.049527
3 colOrderStats(X[rows, cols]) 1.091465 1.095569 1.097312 1.095764 1.096469 1.164777

Table: Benchmarking of rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowOrderStats_X_S 0.781644 0.7833675 0.7925041 0.7850245 0.7877940 0.915818
2 rowOrderStats(X, cols, rows) 0.826801 0.8288840 0.8411113 0.8307185 0.8351465 1.166112
3 rowOrderStats(X[cols, rows]) 0.859268 0.8636510 0.8827061 0.8655500 0.8711405 1.611106
expr min lq mean median uq max
1 rowOrderStats_X_S 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 rowOrderStats(X, cols, rows) 1.057772 1.058104 1.061334 1.058207 1.060108 1.273301
3 rowOrderStats(X[cols, rows]) 1.099309 1.102485 1.113819 1.102577 1.105797 1.759199

Figure: Benchmarking of colOrderStats_X_S(), colOrderStats(X, rows, cols)() and colOrderStats(X[rows, cols])() on double+1000x100 data as well as rowOrderStats_X_S(), rowOrderStats(X, cols, rows)() and rowOrderStats(X[cols, rows])() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats_X_S 758.792 760.2540 765.2907 761.0550 762.930 877.707
2 rowOrderStats_X_S 781.644 783.3675 792.5041 785.0245 787.794 915.818
expr min lq mean median uq max
1 colOrderStats_X_S 1.000000 1.000000 1.00000 1.000000 1.00000 1.000000
2 rowOrderStats_X_S 1.030116 1.030402 1.03556 1.031495 1.03259 1.043421

Figure: Benchmarking of colOrderStats_X_S() and rowOrderStats_X_S() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 24.72 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colRowOrderStats_subset')

Copyright Dongcan Jiang. Last updated on 2019-09-10 20:46:42 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer

Benchmark reports

Clone this wiki locally