Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run rcmdcheck as cran in CI #116

Closed
wants to merge 9 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,31 @@ permissions:

jobs:
pre_deploy:
runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
matrix:
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
env:
cache-version: 5
steps:
- uses: actions/checkout@v2
- name: Set up libraries for Ubuntu
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libsodium-dev libharfbuzz-dev libfribidi-dev libcurl4-openssl-dev texlive-latex-base texlive-fonts-extra pandoc libmagick++-dev libhdf5-dev
- name: Set up libraries for MacOS
if: runner.os == 'macOS'
run: |
brew install libsodium harfbuzz fribidi curl texlive pandoc imagemagick hdf5 basictex
- name: Set up libraries for Windows
if: runner.os == 'Windows'
run: |
choco install rtools
choco install pandoc
choco install tinytex
choco install imagemagick
- name: Set up R 4.0
uses: r-lib/actions/setup-r@v2
with:
Expand Down Expand Up @@ -52,17 +68,26 @@ jobs:
- name: rcmdcheck
run: |
rcmdcheck::rcmdcheck(
error_on = "error", # TODO: switch back to "warning"
error_on = "error",
check_dir = "check"
)
shell: Rscript {0}
env:
_R_CHECK_FORCE_SUGGESTS_: false
- name: rcmdcheck as CRAN
run: |
rcmdcheck::rcmdcheck(
args = "--as-cran",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-manual to avoid the pdftatex errors. Not sure we need to build the full manual.

error_on = "warning",
check_dir = "cran_check"
)
shell: Rscript {0}
- name: Run coverage report
run: |
covr::package_coverage()
shell: Rscript {0}
- name: Downgrade pkgdown
# Reference: https://github.com/r-lib/pkgdown/issues/2205#issuecomment-1377793532
run: |
remotes::install_version("pkgdown", "2.0.3")
shell: Rscript {0}
Expand Down
3 changes: 1 addition & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: pizzarr
Type: Package
Title: Slice into Zarr arrays in R
Title: Slice into Zarr Arrays in R
Version: 0.1.0
Authors@R: c(
person(
Expand Down Expand Up @@ -39,7 +39,6 @@ RoxygenNote: 7.3.2
Suggests:
testthat,
knitr,
covr,
bslib,
pkgdown,
rmarkdown,
Expand Down
4 changes: 2 additions & 2 deletions R/numcodecs.R
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ ZstdCodec <- R6::R6Class("ZstdCodec",
#' @param zarr_arr The ZarrArray instance.
#' @return Compressed data.
encode = function(buf, zarr_arr) {
# Reference: https://github.com/traversc/qs/blob/84e30f4/R/RcppExports.R#L16
# Reference: https://github.com/qsbase/qs/blob/84e30f4/R/RcppExports.R#L16
result <- zstd_compress_raw(buf, self$level)
return(result)
},
Expand Down Expand Up @@ -114,7 +114,7 @@ Lz4Codec <- R6::R6Class("Lz4Codec",
#' @param zarr_arr The ZarrArray instance.
#' @return Compressed data.
encode = function(buf, zarr_arr) {
# Reference: https://github.com/traversc/qs/blob/84e30f4/R/RcppExports.R#L24
# Reference: https://github.com/qsbase/qs/blob/84e30f4/R/RcppExports.R#L24
body <- lz4_compress_raw(buf, self$acceleration)

# The compressed output includes a 4-byte header storing the original size
Expand Down
58 changes: 29 additions & 29 deletions R/zarr-array.R
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
#' oindex TODO
#' @keywords internal
oindex = NULL,
#' method_description
#' @description
#' (Re)load metadata from store without synchronization (file locking).
load_metadata_nosync = function() {

Expand Down Expand Up @@ -135,27 +135,27 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
private$dtype <- normalize_dtype(meta$dtype, object_codec = object_codec)
},
#' method_description
#' @description
#' Load or reload metadata from store.
load_metadata = function() {
private$load_metadata_nosync()
# TODO: support for synchronization
},
#' method_description
#' @description
#' Referesh metadata if not cached without synchronization (file locking).
refresh_metadata_nosync = function() {
if(!private$cache_metadata && !private$is_view) {
private$load_metadata_nosync()
}
},
#' method_description
#' @description
#' Refresh metadata from store if not cached.
refresh_metadata = function() {
if(!private$cache_metadata) {
private$load_metadata()
}
},
#' method_description
#' @description
#' Write metadata to store without synchronization (file locking).
flush_metadata_nosync = function() {
if(private$is_view) {
Expand Down Expand Up @@ -188,13 +188,13 @@ ZarrArray <- R6::R6Class("ZarrArray",
encoded_meta <- private$store$metadata_class$encode_array_metadata(zarray_meta)
private$store$set_item(mkey, encoded_meta)
},
#' method_description
#' @description
#' TODO
chunk_key = function(chunk_coords) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0/zarr/core.py#L2063
return(paste0(private$key_prefix, do.call(paste, c(as.list(chunk_coords), sep = private$dimension_separator))))
},
#' method_description
#' @description
#' TODO
compute_cdata_shape = function() {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0/zarr/core.py#L428
Expand All @@ -212,7 +212,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
cdata_shape <- as.numeric(cdata_shape)
return(cdata_shape)
},
#' method_description
#' @description
#' Resize an array without synchronization (file locking)
resize_nosync = function(...) {
# Note: When resizing an array, the data are not rearranged in any way.
Expand Down Expand Up @@ -252,7 +252,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
}
},
#' method_description
#' @description
#' TODO
get_basic_selection_zd = function(selection = NA, out = NA, fields = NA) {
# Special case basic selection for zero-dimensional array
Expand Down Expand Up @@ -296,13 +296,13 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
return(out)
},
#' method_description
#' @description
#' TODO
get_basic_selection_nd = function(selection = NA, out = NA, fields = NA) {
indexer <- BasicIndexer$new(selection, self)
return(private$get_selection(indexer, out = out, fields = fields))
},
#' method_description
#' @description
#' TODO
get_selection = function(indexer, out = NA, fields = NA) {
# Reference: https://github.com/gzuidhof/zarr.js/blob/292804/src/core/index.ts#L304
Expand Down Expand Up @@ -369,7 +369,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
return(out)

},
#' method_description
#' @description
#' TODO
set_basic_selection_zd = function(selection, value, fields = NA) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0e6cdc04c6413e14f57f61d389972ea937c/zarr/core.py#L1625
Expand Down Expand Up @@ -432,13 +432,13 @@ ZarrArray <- R6::R6Class("ZarrArray",
c_data <- private$encode_chunk(chunk_raw)
self$get_chunk_store()$set_item(c_key, c_data)
},
#' method_description
#' @description
#' TODO
set_basic_selection_nd = function(selection, value, fields = NA) {
indexer <- BasicIndexer$new(selection, self)
return(private$set_selection(indexer, value = value, fields = fields))
},
#' method_description
#' @description
#' TODO
set_selection = function(indexer, value, fields = NA) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0/zarr/core.py#L1682
Expand Down Expand Up @@ -515,13 +515,13 @@ ZarrArray <- R6::R6Class("ZarrArray",
return()
}
},
#' method_description
#' @description
#' TODO
process_chunk = function(out, cdata, chunk_selection, drop_axes, out_is_ndarray, fields, out_selection, partial_read_decode = FALSE) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0/zarr/core.py#L1755
# TODO
},
#' method_description
#' @description
#' TODO
get_chunk_value = function(proj, indexer, value, selection_shape) {
# Reference: https://github.com/gzuidhof/zarr.js/blob/15e3a3f00eb19f0133018fb65f002311ea53bb7c/src/core/index.ts#L550
Expand All @@ -541,12 +541,12 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
return(chunk_value)
},
#' method_description
#' @description
#' TODO
chunk_buffer_to_raw_array = function(decoded_chunk) {
# TODO
},
#' method_description
#' @description
#' For parallel usage
chunk_getitem_part1 = function(chunk_coords, chunk_selection, out, out_selection, drop_axes = NA, fields = NA) {
if(length(chunk_coords) != length(private$chunks)) {
Expand All @@ -567,7 +567,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
})
return(result)
},
#' method_description
#' @description
#' For parallel usage
chunk_getitem_part2 = function(part1_result, chunk_coords, chunk_selection, out, out_selection, drop_axes = NA, fields = NA) {
c_key <- private$chunk_key(chunk_coords)
Expand Down Expand Up @@ -610,7 +610,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
}
},
#' method_description
#' @description
#' For non-parallel usage
chunk_getitem = function(chunk_coords, chunk_selection, out, out_selection, drop_axes = NA, fields = NA) {
# TODO
Expand Down Expand Up @@ -658,12 +658,12 @@ ZarrArray <- R6::R6Class("ZarrArray",
}
})
},
#' method_description
#' @description
#' TODO
chunk_getitems = function(lchunk_coords, lchunk_selection, out, lout_selection, drop_axes = NA, fields = NA) {
# TODO
},
#' method_description
#' @description
#' TODO
chunk_setitem = function(chunk_coords, chunk_selection, value, fields = NA) {
# Reference: https://github.com/gzuidhof/zarr.js/blob/15e3a3f00eb19f0133018fb65f002311ea53bb7c/src/core/index.ts#L625
Expand Down Expand Up @@ -741,30 +741,30 @@ ZarrArray <- R6::R6Class("ZarrArray",
chunk_data <- private$encode_chunk(chunk_raw)
self$get_chunk_store()$set_item(chunk_key, chunk_data)
},
#' method_description
#' @description
#' TODO
chunk_setitem_nosync = function(chunk_coords, chunk_selection, value, fields = NA) {
# TODO
},
#' method_description
#' @description
#' TODO
chunk_setitems = function(lchunk_coords, lchunk_selection, values, fields = NA) {
# TODO
},
#' method_description
#' @description
#' TODO
process_for_setitem = function(ckey, chunk_selection, value, fields = NA) {
# TODO
},
chunk_delitem = function(ckey) {
# TODO
},
#' method_description
#' @description
#' TODO
chunk_delitems = function(ckeys) {
# TODO
},
#' method_description
#' @description
#' TODO
decode_chunk = function(cdata, start = NA, nitems = NA, expected_shape = NA) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0e6cdc04c6413e14f57f61d389972ea937c/zarr/core.py#L2066
Expand Down Expand Up @@ -806,7 +806,7 @@ ZarrArray <- R6::R6Class("ZarrArray",
# ensure correct chunk shape
return(chunk)
},
#' method_description
#' @description
#' TODO
encode_chunk = function(chunk_as_raw) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0e6cdc04c6413e14f57f61d389972ea937c/zarr/core.py#L2105
Expand Down Expand Up @@ -837,7 +837,7 @@ ZarrArray <- R6::R6Class("ZarrArray",

return(cdata)
},
#' method_description
#' @description
#' TODO
append_nosync = function(data, axis = 0) {
# Reference: https://github.com/zarr-developers/zarr-python/blob/5dd4a0/zarr/core.py#L2141
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ print(selection$data)
| `Unicode` | ✔ / ✔ | Converted to `character` in R. |
| `void *` | ❌ / ❌ | |
| Structured data types | ❌ / ❌ | |
| Object data type - [VLenUTF8](https://numcodecs.readthedocs.io/en/stable/vlen.html#vlenutf8) | ✔ / ✔ | Converted to `character` in R. |
| Object data type - [VLenUTF8](https://numcodecs.readthedocs.io/en/stable/other/vlen.html) | ✔ / ✔ | Converted to `character` in R. |


Note: no effort is made to assess loss of precision due to conversion.
Expand Down Expand Up @@ -120,14 +120,14 @@ pkgdown::build_site()
- Note: `pizzarr` has an optional dependency on Rarr for Blosc (de)compression.
- R package development
- [R packages](https://r-pkgs.org/)
- [roxygen2 syntax](https://cran.r-project.org/web/packages/roxygen2/vignettes/rd-formatting.html)
- [roxygen2 syntax](https://CRAN.R-project.org/package=roxygen2/vignettes/rd-formatting.html)
- [R6](https://r6.r-lib.org/index.html)
- [R6 roxygen2 syntax](https://www.tidyverse.org/blog/2019/11/roxygen2-7-0-0/#r6-documentation)
- [pkgdown](https://pkgdown.r-lib.org/)
- Zarr implementation
- [zarr_implementations](https://github.com/zarr-developers/zarr_implementations)
- [zarr-python](https://github.com/zarr-developers/zarr-python)
- [LZ4 and ZSTD compression in R](https://github.com/traversc/qs)
- [LZ4 and ZSTD compression in R](https://github.com/qsbase/qs)
- [zarr.js](https://github.com/gzuidhof/zarr.js)
- [zarrita.js](https://github.com/manzt/zarrita.js)
- [v2 spec](https://zarr.readthedocs.io/en/stable/spec/v2.html)
Loading