Integrated Boot into Proto-X

[wangpatrick57]

Summary: Integrated Boot to accelerate the HPO and tuning loops of Proto-X

Demo:
On TPC-H SF10, using Boot (image 1), Proto-X finds a configuration with a "Best Seen Metric" of 69.67s after ~1hr of tuning. Without Boot (image 2), Proto-X only finds a configuration with a "Best Seen Metric" of 103.10s after ~1hr of tuning. Note that "Best Seen Metric" refers to the total runtime of the last run of all 22 queries TPC-H. "Best Metric" is set to "Best Seen Metric" whenever a run has no timed-out queries. Both runs used the same hyperparameters and the same per-query timeout.

Details:

• Boot is automatically installed when running postgres build.
• Redis (needed by Boot) is automatically started when Boot is enabled. This Redis does not conflict with the Redis instance used by Ray.
• Boot can be toggled on and off with the CLI arg --enable-boot-during-[hpo|tune]. Boot is enabled separately for HPO and for tuning.
• Boot can be configured with a .yaml file passed in through the CLI.
• Currently, the results from Boot being enabled show the estimated runtime of the TPC-H workload, not the actual runtime. Thus, the results should be taken with a grain of salt. A future PR will resolve this.
• Boot could potentially be even more effective if the per-query timeout was also decreased to reflect how Boot accelerates queries. This may be investigated in the future.
• Currently, Boot's entire intelligent cache is cleared every time the configuration changes. A future PR will implement logic to selectively clear parts of the cache when the configuration changes.

[lmwnshn]

LGTM, only comments are minor thoughts as I read it. Nice work independently integrating everything.

[lmwnshn]

I'm guessing this is temporary? Not sure if you wanted to remove this.

[wangpatrick57]

This is to run specific commands within main.sh. It's inconvenient to copy them directly because they rely on ENV_VARS, so this is my way of "copying" them. I'll add a comment better explaining this, but I imagine it'll be permanent.

[lmwnshn]

Similar comment, but doesn't matter much.

[lmwnshn]

This leaks abstractions between modules somewhat, (proto-x knowing about Boot), but from briefly thinking about it, it doesn't seem like fixing the abstraction leak is worth it.

I think in an ideal world, the connection handling logic would be independent of the tuner -- then stuff like Boot doesn't need to get pushed down into the tuner.

However, given that the tuner currently manages its own connections, this is fine. As long as there aren't more than 3-5 things that leak, it probably won't be too bad.

[wangpatrick57]

Good point, that's something I didn't realize. I think there is an argument for why Proto-X should know about Boot though, because when you use Proto-X to tune, you need to indicate whether Boot was used during tuning or not to know whether the numbers are real runtimes or estimated. We can revisit this later though.