Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GNU OpenMP forbids xr.align #35

Open
j-haacker opened this issue Nov 23, 2024 · 1 comment
Open

GNU OpenMP forbids xr.align #35

j-haacker opened this issue Nov 23, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@j-haacker
Copy link
Owner

Describe the bug
Processing L1b data in parallel fails. The reasons seems to be that GNU OpenMP does not allow xarray to fork processes within a forked worker.

To Reproduce
Steps to reproduce the behavior:
Run l2.from_id(list, cores=2) with list containing multiple track IDs per month.

Error message
Terminating: fork() called from a process already using GNU OpenMP, this is unsafe.

Environment
Linux kernel 5.15

@j-haacker j-haacker added the bug Something isn't working label Nov 23, 2024
@j-haacker
Copy link
Owner Author

Work-around

Two work-arounds worked for me:

  1. If you have access to a compute node, try do the processing there. The compute node may use a different message passing interface (MPI).
  2. Pass cores=1 to disable parallel processing. You can still manually start multiple process, e.g., to compute different periods each. In that case and if you will continue gridding the data, make use of kwarg cache_fullname to specify different cache names for each job. You can later merge the cache files using misc.merge_l2_cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant