Compute resource descriptors and callbacks

[tclose]

What would you like changed/added and why?

To be able to specify/recommend compute resources to be requested for a job by the Worker object

What would be the benefit? Does the change make something easier to use?

Resource allocation is important for distributed workers (e.g. SLURM)

If it is a new feature, what is the benefit?

Required resources are allocated to jobs

[tclose]

@satra's notes from BrainHack discussion Pydra.pdf

[tclose]

@attrs.define
class ComputeResources:
    cpus:  int | Range[int]
    num_threads: int | Range[int]
    memory: int | Range[int]
    acceleration: Enum[...]
    ...

Should check CWL and other workflow engines for other options

Each attribute should take either a fixed value (i.e. int or float) or a Range object defining min/max values (max would be the default, min the minimum below which the task won't run)

Add resources argument to compose decorators, which can either take a fixed ComputeResources or a callback function, which takes the task object as an input and returns a ComputeResources object.

Similarly resources objects or callbacks could be provided to workflow.add() and Submitter.__call__() (as a dict mapping node name to resources objects)

ComputeResources objects would then be resolved and stored within Job objects, and it would be on the Worker class to choose to use them how want to

[tclose]

What do you think @effigies @djarecka ?

[effigies]

Seems reasonable. I don't really know what acceleration is here. Possibly to indicate that the job needs GPU? In nipype1, someone added a GPU resource to MultiProc, but I think the idea there was that it might be able to consume multiple GPUs, so you'd want to make sure that your design doesn't imply GPU usage is purely boolean. Likewise, I wouldn't necessarily assume that every thread a GPU-enabled process has is consuming a GPU.

[anibalsolon]

Accellaration is regarding hardware accell, such as GPU and TPU.

From our discussion yesterday, there are some nuances to it--

• Accelleration sometimes may be required, sometimes may be recommended.
• Different acceleration types have their own specificities

Checking into CWL specs, there is this proposal regarding GPUs, but they do not seem to have gone detailed as we were discussing.

common-workflow-language/common-workflow-language#587
https://github.com/common-workflow-language/cwltool/pull/1629/files

They work with CUDA devices, and only specify the # of devices and CUDA minimal version.

requirements:
  cwltool:CUDARequirement:
    cudaVersionMin: "1.0"
    cudaComputeCapability: "1.0"
    cudaDeviceCountMax: 4

Most likely the safest way to go as a starting point, as controlling for memory/cores/etc may be too fine-grained.

[djarecka]

this class would be very useful, but I don't fully understand the idea of ranges, i.e., min/max value. How would the worker use min? would this be just the information for the user?
Perhaps, if we want to have it, I would add it as an additional optional value.