[batch] Consider retention periods for batches

[daniel-goldstein]

What happened?

Hail Batch never forgets a batch. All batches, jobs, and attempts are forever persisted in the Batch database. This is rarely a performance problem, as the indexes ensure that old rows are rarely ever looked at, but the fact that the database storage is monotonically increasing is something that we have to reckon with, and it makes migrations very time intensive. There are certainly many improvements that can be made to waste less space in the database (like #14623), but ultimately we will need to make a decision about how long we should persist batches.

We should quantify the utility of historic batches, what might be a good cutoff or alternative process for expiring batches, and whether we should provide some sort of export that users can use to own information about their batches. I imagine the most relevant information would be cost and logs.

Version

0.2.132

Relevant log output

No response