Feature Request: split hostfile for tasks in a job

[walkloud]

MVAPICH requires the passing of a hostfile at the mpirun launch. If you are bundling tasks, currently, you give them all the complete hostlist for the job. Would it be possible to have METAQ split the hostfile into parts and assign different tasks different sub-hostfiles, and then clean them up when a task completes, so METAQ knows these specific resources are free?

[kenmcelvain]

I have python code in startjm.py that collects the hosts on Sierra/summit and splits the hosts into groups. For startjm.py the groups are “lumps” of nodes that are started independently.

…

On Jan 18, 2019, at 10:23 AM, walkloud ***@***.***> wrote: MVAPICH requires the passing of a hostfile at the mpirun launch. If you are bundling tasks, currently, you give them all the complete hostlist for the job. Would it be possible to have METAQ split the hostfile into parts and assign different tasks different sub-hostfiles, and then clean them up when a task completes, so METAQ knows these specific resources are free? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#16>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFE8qszTpTkivmB7UWtvMUvB2PJKTmaqks5vEhENgaJpZM4aIaJN>.

[walkloud]

ChrisK and I may have figure out how to do the entire thing within the task.sh file - so no need to modify METAQ - we'll let you know