[Issue #3527] Modify the logic around the opportunity change tracking table to never delete records

[mikehgrantsgov]

Summary

Fixes #3527

Time to review: 30 mins

Changes proposed

Add a new table to track task progress
Rename queue table and add all opportunities to it. Add any new opportunities that come in (should basically always be 1:1)
For index job, load data based on last_loaded_at date rather than using has_update field or deleting records as we go.

Context for reviewers

Cascade deleting orphan relationship remains unchanged
JobTable is meant to be general purpose and can be used, possibly ended, by other tasks.

Additional information

See unit tests

[chouinar]

Let's use our lookup tables for this, just requires setting up a boilerplatey table and a mapping in the lookup models file.

[chouinar]

Just to have slightly fewer places we put commits, what if we had this:

def update_job_status(self, job_status: JobStatus) -> None:
      self.job.job_status = job_status
     self.db_session.commit()

[chouinar]

Just brainstorming.

Should job_type be an enum?
Should we store the metrics object as well as JSON?

[mikehgrantsgov]

I was thinking about the metrics object and will add it for flexibility down the road (although I think NR is place to store and query these). When trying to make this an enum the migration would try creating these enum types that I wasn't sure was right. Will make this something like LkJobStatus

[chouinar]

Let's have a slightly longer name, what about task_log or something like that?