Run kpack-image-builder, job-task-runner and statefulset-runner as standalone Deployments

[pbusko]

Is there a related GitHub Issue?

Fixes #3519
Fixes #3495

What is this change about?

Implementation for #3519 (and a fix for #3495)

Does this PR introduce a breaking change?

no

Acceptance Steps

Tag your pair, your PM, and/or team

[danail-branekov]

Overall, looks good, minor comments.

I played a bit with the change:

• All tests pass
• Manual testing looks good
• Upgrade from 0.13 seems seamless, helm takes care of the new deployments nicely
• Kind installer works

I was trying to reason about resource requirements and how they are affected by splitting controllers into multiple deployments:

Monolit controllers deployments

• Requests
  cpu: 50m
  memory: 100Mi
• Limits
  cpu: 1000m
  memory: 1Gi

Multiple controllers deployments

Controllers:

• Requests
  cpu: 50m
  memory: 100Mi
• Limits
  cpu: 1000m
  memory: 1Gi

Statefulset runner:

• Requests
  cpu: 50m
  memory: 100Mi
• Limits
  cpu: 1000m
  memory: 1Gi

Kpack builder:

• Requests
  cpu: 50m
  memory: 100Mi
• Limits
  cpu: 1000m
  memory: 1Gi

Task runner:

• Requests
  cpu: 50m
  memory: 100Mi
• Limits
  cpu: 1000m
  memory: 1Gi

Total:

• Requests
  cpu: 200m
  memory: 400Mi
• Limits
  cpu: 4000m
  memory: 4Gi

What I see is that with the default request/limit values, the multiple deployment model requires 4 times more resources in comparisson with the monolit deployment model. I wonder whether we could come up with reasonable defaults that total up to similar numbers

[georgethebeatle]

@pbusko any thoughts on the comment by @danail-branekov above with regards to limits and requests?

[pbusko]

What I see is that with the default request/limit values, the multiple deployment model requires 4 times more resources in comparisson with the monolit deployment model. I wonder whether we could come up with reasonable defaults that total up to similar numbers

On the other hand we're talking about 200m CPU and 400mb requests in total, which should be doable on pretty much every workstation/VM (unless it's running on a toaster). Limits themselves should not harm, since they do not allocate anything.
We also should not forget that it's just the default values that can always be adjusted by the end user if needed.