Support for hierarchical resource constraints of process containers

[jterry75]

Is your feature request related to a problem? Please describe.
All major orchestrators and runtimes now consider multiple containers as a unit. Pods, Tasks, Groups, etc. For resource scheduling and constraints to be accurate, Windows needs containers (process isolation) to be considered hierarchical in terms of resource constrains.

Imagine the following group of containers:

Pod: 2 CPU, 2 GiB Mem
C1 (Workload): CPU, Memory
C2 (Sidecar): .5 CPU, 256 MiB Mem

Many people "want" to do this for Windows and are already doing this for Linux containers. The Workload wants to run beyond an upper limit if the sidecar is not using resources. So in this way we only limit the sidecar to a max, but we limit the Workload to "all resources the pod has available".

However, on Windows, because there is no hierarchy this creates a problem. C1 is now running in the view of the Host resources not the Pod. And so the scheduler now has no idea how much cpu or memory C1 is actually constrained to.

To work around this we ask that all containers on Windows have limits in the configs. Thus C1 becomes 1.5 CPU and 1.75 GiB Mem. But again, this artificially limits the workload even if the sidecar is using nothing.

Note: What I am not asking for is Container within a Container (I understand that is very hard for Windows). And I am not asking for Hypervisor isolated containers where the UVM is considered the group level constraints as this does not work in the vast majority of environments that don't allow nested hypervisors.

Describe the solution you'd like
I would like to see Windows add support for process isolated containers to consider hierarchical resource constraints as described above.

I would be ok with this as a container primitive like:

1. Create a "Container Group" of size X/Y
2. Create a "Container" in a CG of size A/B
3. Create another "Container" in a CG with no limit on size thus inherits its group size.

Or it would be ok by using Windows OS primitives like:

1. Create a CPU group of Host CPU's 1-2 called C1
2. Create a Memory group of Host Memory of size X called M1
3. Create a container with resource limits of size A/B in CPU Group C1 and in Memory Group M1
4. Create another container with no resource limits in C1 and M1.

Thus it's the OS primitive that acts as the parent resource limits and not any container concept itself.

Or it could be a parent NT_JobObject.

1. Create a JobObject with limits called J1
2. Create container with resource limits A/B in J1
3. Create another container with no resource limits in J1

Again, letting the current Job infra in Windows to handle this case for us considering all of the container silo within a given Job Object.

Describe alternatives you've considered
The above describes alternative approaches Windows could consider.

Additional context
N/A

[ntrappe-msft]

Thanks for bringing this to our attention. We're going to discuss it and get back to you soon. Internal ticket (#45034676).

[jterry75]

How did the discussion go?

[ntrappe-msft]

I'll update you as soon as I can. We're still waiting to hear back from some of our technical experts. Thank you for your patience!

[microsoft-github-policy-service]

This issue has been open for 90 days with no updates.
no assignees, please provide an update or close this issue.