Skip to content

colRowRanks

Jump to bottom Edit New page
Henrik Bengtsson edited this page Sep 11, 2019 · 6 revisions

matrixStats: Benchmark report

colRanks() and rowRanks() benchmarks

This report benchmark the performance of colRanks() and rowRanks() against alternative methods.

Alternative methods

  • apply() + rank()

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3193853 170.6    5709258 305.0  5709258 305.0
Vcells 6468219  49.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192173 170.5    5709258 305.0  5709258 305.0
Vcells 6463307  49.4   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.009151 0.0102300 0.0131554 0.0127835 0.014636 0.045409
2 apply+rank 0.156443 0.1608635 0.1730241 0.1640870 0.169303 0.321996
expr min lq mean median uq max
1 colRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.000000
2 apply+rank 17.09573 15.72468 13.15227 12.83584 11.56757 7.091017

Table: Benchmarking of rowRanks() and apply+rank() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.003241 0.0039220 0.0051267 0.0053805 0.0057575 0.017792
2 apply+rank 0.120576 0.1234595 0.1295284 0.1247940 0.1274035 0.233164
expr min lq mean median uq max
1 rowRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+rank 37.20333 31.47871 25.26555 23.19376 22.12827 13.10499

Figure: Benchmarking of colRanks() and apply+rank() on integer+10x10 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 3.241 3.922 5.12668 5.3805 5.7575 17.792
1 colRanks 9.151 10.230 13.15545 12.7835 14.6360 45.409
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 2.823511 2.608363 2.566076 2.375894 2.542076 2.552214

Figure: Benchmarking of colRanks() and rowRanks() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3190729 170.5    5709258 305.0  5709258 305.0
Vcells 6079813  46.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3190720 170.5    5709258 305.0  5709258 305.0
Vcells 6084851  46.5   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.312976 0.3156005 0.3214385 0.3220275 0.3247015 0.384026
2 apply+rank 1.651695 1.6822950 1.6998098 1.6904005 1.7038450 1.947765
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.00000 1.000000
2 apply+rank 5.277386 5.330457 5.288133 5.249243 5.24742 5.071961

Table: Benchmarking of rowRanks() and apply+rank() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.297921 0.299919 0.3046728 0.3034425 0.3086655 0.325643
2 apply+rank 1.655421 1.676748 1.6957418 1.6888720 1.7049320 1.990512
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 5.556577 5.590669 5.565779 5.565707 5.523559 6.112559

Figure: Benchmarking of colRanks() and apply+rank() on integer+100x100 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 297.921 299.9190 304.6728 303.4425 308.6655 325.643
1 colRanks 312.976 315.6005 321.4385 322.0275 324.7015 384.026
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.050534 1.052286 1.055028 1.061247 1.051953 1.179285

Figure: Benchmarking of colRanks() and rowRanks() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191456 170.5    5709258 305.0  5709258 305.0
Vcells 6083319  46.5   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191450 170.5    5709258 305.0  5709258 305.0
Vcells 6088362  46.5   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.385546 0.390501 0.3954847 0.3957785 0.3983605 0.451537
2 apply+rank 1.028308 1.037556 1.0488994 1.0498740 1.0549680 1.132388
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.667147 2.656988 2.652187 2.652681 2.648275 2.507852

Table: Benchmarking of rowRanks() and apply+rank() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.375090 0.3784755 0.3875547 0.383650 0.385605 0.531404
2 apply+rank 1.028457 1.0436115 1.0878877 1.052121 1.065198 1.618831
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.741894 2.757408 2.807056 2.742399 2.762406 3.046328

Figure: Benchmarking of colRanks() and apply+rank() on integer+1000x10 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 375.090 378.4755 387.5547 383.6500 385.6050 531.404
1 colRanks 385.546 390.5010 395.4847 395.7785 398.3605 451.537
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colRanks 1.027876 1.031773 1.020462 1.031613 1.033079 0.8497057

Figure: Benchmarking of colRanks() and rowRanks() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191647 170.5    5709258 305.0  5709258 305.0
Vcells 6084008  46.5   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191638 170.5    5709258 305.0  5709258 305.0
Vcells 6089046  46.5   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.203743 0.2094645 0.2294692 0.2264965 0.2401895 0.340512
2 apply+rank 10.201566 10.4951780 11.3892725 10.6452085 11.2344635 26.840616
expr min lq mean median uq max
1 colRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+rank 50.07076 50.10481 49.63312 46.99944 46.77333 78.82429

Table: Benchmarking of rowRanks() and apply+rank() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.185777 0.190884 0.202331 0.200224 0.210046 0.257102
2 apply+rank 10.248381 10.555267 11.289273 10.768982 11.236743 18.301513
expr min lq mean median uq max
1 rowRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+rank 55.16496 55.29676 55.79606 53.78467 53.49658 71.18386

Figure: Benchmarking of colRanks() and apply+rank() on integer+10x1000 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 185.777 190.8840 202.3310 200.2240 210.0460 257.102
1 colRanks 203.743 209.4645 229.4692 226.4965 240.1895 340.512
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.096707 1.097339 1.134128 1.131215 1.143509 1.324424

Figure: Benchmarking of colRanks() and rowRanks() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191828 170.5    5709258 305.0  5709258 305.0
Vcells 6084494  46.5   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3191822 170.5    5709258 305.0  5709258 305.0
Vcells 6134537  46.9   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 3.077105 3.118673 3.160381 3.147074 3.209983 3.336213
2 apply+rank 16.456213 16.686357 17.977450 16.921699 17.188270 33.735491
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.00000 1.00000
2 apply+rank 5.347953 5.350468 5.688381 5.376962 5.35463 10.11191

Table: Benchmarking of rowRanks() and apply+rank() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 2.974055 3.029072 3.106812 3.067885 3.121558 3.751149
2 apply+rank 16.474844 16.808226 18.051511 17.006084 17.240157 33.977104
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.00000 1.000000 1.000000
2 apply+rank 5.539522 5.548969 5.810301 5.54326 5.522932 9.057786

Figure: Benchmarking of colRanks() and apply+rank() on integer+100x1000 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 2.974055 3.029072 3.106812 3.067885 3.121558 3.751149
1 colRanks 3.077105 3.118673 3.160381 3.147074 3.209983 3.336213
expr min lq mean median uq max
2 rowRanks 1.00000 1.00000 1.000000 1.000000 1.000000 1.0000000
1 colRanks 1.03465 1.02958 1.017243 1.025812 1.028327 0.8893843

Figure: Benchmarking of colRanks() and rowRanks() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192020 170.5    5709258 305.0  5709258 305.0
Vcells 6085062  46.5   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192014 170.5    5709258 305.0  5709258 305.0
Vcells 6135105  46.9   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 3.823106 3.841194 4.094267 3.853652 3.931769 12.78651
2 apply+rank 9.965379 10.026457 10.396685 10.086535 10.236059 18.43050
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.606619 2.610245 2.539327 2.617396 2.603423 1.441402

Table: Benchmarking of rowRanks() and apply+rank() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 3.732511 3.744989 3.850598 3.753887 3.818224 5.680193
2 apply+rank 9.978090 10.038864 10.717244 10.103415 10.296952 19.265572
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.673291 2.680612 2.783268 2.691455 2.696791 3.391711

Figure: Benchmarking of colRanks() and apply+rank() on integer+1000x100 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 3.732511 3.744989 3.850598 3.753887 3.818224 5.680193
1 colRanks 3.823106 3.841194 4.094267 3.853652 3.931769 12.786509
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.024272 1.025689 1.063281 1.026577 1.029738 2.251069

Figure: Benchmarking of colRanks() and rowRanks() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192231 170.5    5709258 305.0  5709258 305.0
Vcells 6201433  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192213 170.5    5709258 305.0  5709258 305.0
Vcells 6201556  47.4   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.006912 0.0075880 0.0093864 0.0084405 0.010515 0.036475
2 apply+rank 0.122983 0.1254835 0.1282255 0.1265915 0.128378 0.239726
expr min lq mean median uq max
1 colRanks 1.00000 1.0000 1.00000 1.0000 1.00000 1.000000
2 apply+rank 17.79268 16.5371 13.66078 14.9981 12.20903 6.572337

Table: Benchmarking of rowRanks() and apply+rank() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.002952 0.003937 0.0048741 0.0050595 0.0053925 0.019921
2 apply+rank 0.119666 0.122002 0.1256963 0.1242670 0.1259480 0.256685
expr min lq mean median uq max
1 rowRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+rank 40.53726 30.98857 25.78873 24.56112 23.35614 12.88515

Figure: Benchmarking of colRanks() and apply+rank() on double+10x10 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 2.952 3.937 4.87408 5.0595 5.3925 19.921
1 colRanks 6.912 7.588 9.38640 8.4405 10.5150 36.475
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 2.341463 1.927356 1.925779 1.668248 1.949931 1.830982

Figure: Benchmarking of colRanks() and rowRanks() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192411 170.5    5709258 305.0  5709258 305.0
Vcells 6201545  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192402 170.5    5709258 305.0  5709258 305.0
Vcells 6211583  47.4   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.327309 0.3451305 0.3493351 0.3483775 0.354750 0.399367
2 apply+rank 1.703618 1.7740350 1.7936778 1.7950170 1.809705 1.966386
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 5.204923 5.140186 5.134548 5.152506 5.101355 4.923757

Table: Benchmarking of rowRanks() and apply+rank() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.318951 0.336027 0.3383823 0.339328 0.3449145 0.349451
2 apply+rank 1.670506 1.715305 1.7441626 1.744661 1.7595735 2.233143
expr min lq mean median uq max
1 rowRanks 1.0000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 5.2375 5.104666 5.154414 5.141518 5.101477 6.390432

Figure: Benchmarking of colRanks() and apply+rank() on double+100x100 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 318.951 336.0270 338.3823 339.3280 344.9145 349.451
1 colRanks 327.309 345.1305 349.3351 348.3775 354.7500 399.367
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.026205 1.027092 1.032368 1.026669 1.028516 1.142841

Figure: Benchmarking of colRanks() and rowRanks() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192598 170.6    5709258 305.0  5709258 305.0
Vcells 6202429  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192592 170.6    5709258 305.0  5709258 305.0
Vcells 6212472  47.4   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.449337 0.4574915 0.4679399 0.467756 0.4776545 0.507614
2 apply+rank 1.039772 1.0547325 1.0687176 1.066174 1.0822980 1.123068
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.314014 2.305469 2.283878 2.279338 2.265859 2.212445

Table: Benchmarking of rowRanks() and apply+rank() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.445564 0.453902 0.4600217 0.4593425 0.4660655 0.489765
2 apply+rank 0.988491 1.008866 1.0186755 1.0154935 1.0262895 1.101602
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.218516 2.222652 2.214407 2.210755 2.202028 2.249246

Figure: Benchmarking of colRanks() and apply+rank() on double+1000x10 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 445.564 453.9020 460.0217 459.3425 466.0655 489.765
1 colRanks 449.337 457.4915 467.9399 467.7560 477.6545 507.614
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.008468 1.007908 1.017213 1.018316 1.024866 1.036444

Figure: Benchmarking of colRanks() and rowRanks() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192789 170.6    5709258 305.0  5709258 305.0
Vcells 6203467  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192780 170.6    5709258 305.0  5709258 305.0
Vcells 6213505  47.5   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 0.2075 0.218528 0.2355714 0.2361095 0.2485725 0.296547
2 apply+rank 10.3790 10.911312 11.7866420 11.3245100 11.6416920 28.030995
expr min lq mean median uq max
1 colRanks 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+rank 50.01928 49.93096 50.03427 47.96296 46.83419 94.52463

Table: Benchmarking of rowRanks() and apply+rank() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 0.194874 0.2075985 0.2192731 0.2144825 0.2250525 0.326257
2 apply+rank 10.172275 10.9990000 12.0164536 11.4397480 11.9099350 19.626885
expr min lq mean median uq max
1 rowRanks 1.00000 1.00000 1.0000 1.00000 1.0000 1.00000
2 apply+rank 52.19924 52.98208 54.8013 53.33651 52.9207 60.15774

Figure: Benchmarking of colRanks() and apply+rank() on double+10x1000 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 194.874 207.5985 219.2731 214.4825 225.0525 326.257
1 colRanks 207.500 218.5280 235.5714 236.1095 248.5725 296.547
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colRanks 1.064791 1.052647 1.074329 1.100833 1.104509 0.9089368

Figure: Benchmarking of colRanks() and rowRanks() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192970 170.6    5709258 305.0  5709258 305.0
Vcells 6203583  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3192964 170.6    5709258 305.0  5709258 305.0
Vcells 6303626  48.1   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 3.214433 3.420603 3.463363 3.465956 3.523589 3.664443
2 apply+rank 17.136738 17.997701 19.274386 18.223185 18.519274 36.069326
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 5.331185 5.261558 5.565223 5.257765 5.255798 9.843058

Table: Benchmarking of rowRanks() and apply+rank() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 3.169005 3.355858 3.478543 3.442334 3.512765 5.077613
2 apply+rank 16.795541 17.470779 21.270506 17.734878 18.130592 277.089230
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 apply+rank 5.299942 5.206054 6.114774 5.151993 5.161345 54.57077

Figure: Benchmarking of colRanks() and apply+rank() on double+100x1000 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 3.169005 3.355858 3.478543 3.442334 3.512765 5.077613
1 colRanks 3.214433 3.420603 3.463363 3.465956 3.523589 3.664443
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colRanks 1.014335 1.019293 0.995636 1.006862 1.003081 0.7216862

Figure: Benchmarking of colRanks() and rowRanks() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3193162 170.6    5709258 305.0  5709258 305.0
Vcells 6204802  47.4   22343563 170.5 56666022 432.4
> colStats <- microbenchmark(colRanks = colRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 2L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3193156 170.6    5709258 305.0  5709258 305.0
Vcells 6304845  48.2   22343563 170.5 56666022 432.4
> rowStats <- microbenchmark(rowRanks = rowRanks(X, na.rm = FALSE), `apply+rank` = apply(X, MARGIN = 1L, 
+     FUN = rank, na.last = "keep", ties.method = "max"), unit = "ms")

Table: Benchmarking of colRanks() and apply+rank() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colRanks 4.456471 4.632466 4.815742 4.727482 4.752999 13.35861
2 apply+rank 9.657767 10.056055 10.502975 10.114044 10.181708 19.44552
expr min lq mean median uq max
1 colRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.167133 2.170778 2.180967 2.139414 2.142165 1.455655

Table: Benchmarking of rowRanks() and apply+rank() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowRanks 4.417688 4.538609 4.658729 4.651197 4.68947 5.660156
2 apply+rank 9.713633 10.009710 10.574110 10.146669 10.19068 19.566589
expr min lq mean median uq max
1 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+rank 2.198805 2.205457 2.269742 2.181518 2.173098 3.456899

Figure: Benchmarking of colRanks() and apply+rank() on double+1000x100 data as well as rowRanks() and apply+rank() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colRanks() and rowRanks() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowRanks 4.417688 4.538609 4.658729 4.651197 4.689470 5.660156
1 colRanks 4.456471 4.632466 4.815742 4.727482 4.752999 13.358608
expr min lq mean median uq max
2 rowRanks 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colRanks 1.008779 1.020679 1.033703 1.016401 1.013547 2.360113

Figure: Benchmarking of colRanks() and rowRanks() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 43.37 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colRanks')

Copyright Henrik Bengtsson. Last updated on 2019-09-10 20:52:42 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer

Benchmark reports

Clone this wiki locally