Separate template processing step from reconciliation to make for better batching capabilities

[etsauer]

Been thinking about this for a while, and would like to propose a solution.

Currently, all templates we process are reconciled at the time of processing.. essentially creating a loop like:

for template, params in my_inventory:
  oc process -f $template --param-file=$params | kubectl apply -f -
done

This creates a 1:1 ratio of API calls to templates, which can take some time if I have a lot of templates to process, or multiple sets of params to the template.

I would like to propose a different strategy where we cache the output of the template processing, and then reconcile the resulting objects en masse. This would look something like:

for template, params in my_inventory:
  oc process -f $template --param-file=$params > tmp/manifests/$action/$counter.yml
done

kubectl apply -f tmp/manifests/apply/
kubectl create -f tmp/manifests/create/
kubectl create -f tmp/manifests/delete/

[pcarney8]

I'm curious to know what failures would look like in this case

[etsauer]

@pcarney8 can you elaborate?