Thread decompose()

[mrzv]

Issue by Dmitriy Morozov
Friday Nov 14, 2014 at 18:20 GMT

---

Currently, decompose() calls the create callback serially. This is useful because it allows one to call MPI functions from create(), where the most common use-case is calling MPI-IO functions to load the data into the block. However, since it's desirable for IO-efficiency to combine the loading of the data with the first computation, it is common to perform initial local computation inside create(). When DIY is allowed to use multiple threads, this creates a problem since it forces serial execution of the local computation.

Proposed solution. Pass two callbacks to decompose(): serial and parallel. The first one would be called in serial and is allowed to use MPI routines directly. The second one will be spun into a new thread, with multiple threads executing at the same time. It's allowed to use only DIY routines for communication.

Note: this would require much tighter coupling between Master and decompose() than exists now.