DM-47507: Add workflow for automatically generating migration scripts

[JeremyMcCormick]

This will generate migration scripts automatically.

It accepts a branch name and a commit SHA representing the PR merge in sdm_schemas which made the schema changes.

[Vebop]

We will need to include the glob again after I add the startracker tables

[JeremyMcCormick]

We will need to include the glob again after I add the startracker tables

Or you can just add the new schemas to the instrument list. It's a small enough set of files that it could just be maintained by hand in this script for now.

[Vebop]

I was at first agreeing with you, then thinking through what it would mean to add new instruments. (will we continue to add any more new instruments ever??)

When one adds a new table, that instrument would have to be manually added here;
If they try to add a new table to the schema, but don't know to edit this list, they won't get an error.
(wouldn't receive the 'no section in alembic.ini file' error)

I think I would prefer it to give an error when one attempts to add a cdb_*.yml rather than a success on an empty or no migration created. maybe accidentally skipping the 'need to test in TTS, etc' step.

opinions welcome.

[Vebop]

Conversation welcome on the hard coded list of instruments, I'm convince-able either way just want us to be knowingly intentional with the direction we choose.

[Vebop]

I think we should make sure people understand the ticket name will be the name of the version migration, I think we generally try to keep the migration names more concise, and they are directly put into the migration file name via underscores.

[JeremyMcCormick]

For automatically generated migrations, I can't think of a much better way to do it. Agree users should be made aware that the scripts will reference tickets in this new way.

[Vebop]

I was at first agreeing with you, then thinking through what it would mean to add new instruments. (will we continue to add any more new instruments ever??)

When one adds a new table, that instrument would have to be manually added here;
If they try to add a new table to the schema, but don't know to edit this list, they won't get an error.
(wouldn't receive the 'no section in alembic.ini file' error)

I think I would prefer it to give an error when one attempts to add a cdb_*.yml rather than a success on an empty or no migration created. maybe accidentally skipping the 'need to test in TTS, etc' step.

opinions welcome.

[JeremyMcCormick]

Conversation welcome on the hard coded list of instruments, I'm convince-able either way just want us to be knowingly intentional with the direction we choose.

I'm planning to merge this soon (probably within a day) so let's change to the hardcoded list, as it won't work otherwise.

I am fine with changing it back to the generic file glob once all the schema migrations are handled without errors. You could do this in your PR which adds the startracker schema migrations.

[JeremyMcCormick]

I would like to point out that merging this PR won't make the migrations trigger automatically until this PR is merged on sdm_schemas:

lsst/sdm_schemas#278

I still have to verify that the trigger works - the sdm_schemas workflow needs to use a GitHub Personal Access Token (PAT) on my account for authorization of the call. I'll have to play with it a bit to get this working.