Use market prices to determine regions to run in.

[jschwartzentruber]

When we migrated to Taskcluster we lost the ability to automatically run fuzzing in the cheapest regions at a given time (based on spot-market fluctuations).

Since fuzzing-decision generates a list of machine/region configurations in which tasks could run, we could periodically calculate which regions are currently the cheapest and use that to filter the list of configurations so we run in cheaper regions.

This would have a few moving pieces:

1. Periodically scrape provider prices for machine/regions we use (both spot and demand pricing).
   • Machine specs are generated by fuzzing-decision based on a list of potential machines in fuzzing-tc-config.
   • Regions are pulled from community-tc-config which has to allocate static resources in each region it supports (subnets, security groups, etc.).
2. Update fuzzing pools to utilize acceptable machines in the cheapest regions at the current time.
   • This affects future machine generation only. It doesn't kill currently running machines.
   • "Cheapest" needs a definition. (lower than mean?)
3. Set a maximum price to avoid spot price fluctuations.
   • Spot providers support setting a bid price which will not be exceeded. We currently don't utilize this.
   • Setting it to x% above the current market price would prevent us from being affected by large price variations by preempting our machine if the price goes up. This would allow a new machine to be allocated using more recent price data.

We could get in a situation where we are preempted because of price fluctuations and don't re-run until the next fuzzing cycle. This can be mitigated by raising the threshold for "cheapest," by raising the threshold for max price, or by asking for additional regions in community-tc-config.