Optimize for requesting 1 device #18
Conversation
Signed-off-by: kerthcet <[email protected]>
Signed-off-by: kerthcet <[email protected]>
1. The required device number is equal to the size 2. Require 1 GPU, in this case, we will pick the device with the minimum sum of scores with available devices. Signed-off-by: kerthcet <[email protected]>
| } | ||
|
|
||
| // Optimize for the case when size == 1. | ||
| // We'll pick the device with the minimum sum of scores with available devices. |
There was a problem hiding this comment.
We can discuss here, from my POV, since 1 device no longer needs the communication with other GPUs, so the lower score the better.
There was a problem hiding this comment.
For requests with 1 device is the choice not irrellevant from the perspective of the application requesting the device? It may be that assigning the device with the lowest connectivity in this case provides some advantage since other more connected devices are available for future requests that require multple devices.
Do you have benchmarks to show whether this change improves performance at all?
There was a problem hiding this comment.
It may be that assigning the device with the lowest connectivity in this case provides some advantage since other more connected devices are available for future requests that require multple devices.
Yes, exactly the case, benefit to the future request for multi-devices.
I have no benchmark yet, but I can provide one if needed, generally, this is not quite aimed for the performance but for the application friendly and rational utilization as you described above.
kerthcet
changed the title
|
cc @elezar |
| } | ||
|
|
||
| // Optimize for the case when size == 1. | ||
| // We'll pick the device with the minimum sum of scores with available devices. |
There was a problem hiding this comment.
For requests with 1 device is the choice not irrellevant from the perspective of the application requesting the device? It may be that assigning the device with the lowest connectivity in this case provides some advantage since other more connected devices are available for future requests that require multple devices.
Do you have benchmarks to show whether this change improves performance at all?
Signed-off-by: kerthcet <[email protected]>
What I mainly do in the PR:
size.