Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

checkobjmem checks against the memory size of the entire RasterStack when I only want to read one band #733

Open
asinghvi17 opened this issue Sep 10, 2024 · 3 comments
Labels

Comments

@asinghvi17
Copy link
Collaborator


julia> sstr = read(rs.SST)
ERROR: required memory 235358208 is greater than system memory 38846464.
Use `lazy=true` if you are loading dataset, and only call `read` on a subset after `view`.
If you beleive this is not correct, pass the keyword `checkmem=false` or set `Rasters.checkmem!(false)`
and try again. These options may crash your system if the file is actually larger than memory.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] _no_memory_error(f::Rasters.var"#217#218", bytes::Int64)
   @ Rasters ~/.julia/dev/Rasters/src/utils.jl:352
 [3] _checkmem
   @ ~/.julia/dev/Rasters/src/utils.jl:340 [inlined]
 [4] _checkobjmem
   @ ~/.julia/dev/Rasters/src/utils.jl:338 [inlined]
 [5] _checkobjmem
   @ ~/.julia/dev/Rasters/src/utils.jl:336 [inlined]
 [6] #read#276
   @ ~/.julia/dev/Rasters/src/read.jl:15 [inlined]
 [7] read(x::Raster{…})
   @ Rasters ~/.julia/dev/Rasters/src/read.jl:12
 [8] top-level scope
   @ REPL[138]:1
Some type information was truncated. Use `show(err)` to see complete types.

but rs.SST is just a 5000x5000 float64 grid.

@rafaqz
Copy link
Owner

rafaqz commented Sep 11, 2024

Hmm maybe there is mistake in dispatch.

For now there is Rastets.checkmem!(false)

@rafaqz
Copy link
Owner

rafaqz commented Sep 18, 2024

Do you have an MWE? I can't reproduce this

@asinghvi17
Copy link
Collaborator Author

asinghvi17 commented Sep 18, 2024

Ah unfortunately my only example involved quite a few hacks. Let me see if I can replicate it now though.

@rafaqz rafaqz added the memory label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants