insert valueset function

[fzhao99]

PULL REQUEST

Summary

Adds a insertValueSet function that takes our internal ValueSet struct and attempts to insert the ValueSet, associated concepts, and associated concept <> valueset joins into the db.

I pushed a branch https://github.com/CDCgov/dibbs-query-connector/tree/bob/38-insert-valueset-test that 1) deletes migrations 2-6 so that on spinup, the db gets created without any data that also 2) has a button on the homepage that will insert the single example valueset that we added in a previous PR into the database. You can test that the code is working locally by running that branch, connecting to your db using your favorite Postgres viewer, and then hitting the below button. Hopefully, the example data should insert the valueset / concept / joins accordingly, and after, subsequent attempts at joining won't do anything .

Related Issue

Fixes #57

Acceptance Criteria

Given the a valueset’s name, codes, system, external_id, author, relevant reportable conditions, and the clinical concept type the codes represent add the value set to the QC DB
- Valuesets are uniquely identified by OID and version number
- The behavior of this function should be idempotent

Idempotency considerations:

• ValueSet ID's are set to be OID + ValueSet version. This uniqueness constraint means duplicate valuesets of the same OID/version won't get created
• Concept ID's are set to be system + code version. This uniqueness constraint means that assuming codes are unique within the system in question (ie SNOMED codes are all unique within SNOMED), duplicate concepts won't get created
• ValueSet <> Concept join ID's are the combination of the two individual table ID's. Thus, duplicate joins won't get created.
  • Since this table's columns are just a deconstruction of this ID, let me know if we want to enforce the uniqueness constraint using the columns instead. Doing it this way because I couldn't think of an ID that was preferable, so thought just to smash the two together.

Additional consideration

Did a little more thinking around the conversation around concept-level fault tolerance at our previous meeting and made a ticket to expand this work rather than address it here. I think the core scope of the ticket is available as is, and I think that line of work can be parallelized with the next default query building step to flesh out the rest of the db creation refactoring.

[robertandremitchell]

@fzhao99 , is it okay if I push some changes to this branch: https://github.com/CDCgov/dibbs-query-connector/tree/bob/38-insert-valueset-test ? I had to touch up some things to get linting to pass

[robertandremitchell]

looks good to me. had some minor issues getting the test branch set up to pass linting, but insertion worked.

probably beyond the scope of the ticket, but I wonder if we want

tefca-viewer-1  | ValueSet <> concept join insert failed
tefca-viewer-1  | error: duplicate key value violates unique constraint "valueset_to_concept_pkey"

this to be an overwrite instead? I guess it will depend on how often we expect STLTs to update a valueset versus it maybe being a one-time thing they do per condition/topic area. I guess if we eventually add a remove option it's not the biggest deal, and it's probably better to have the remove/re-insert as two steps to ensure things aren't happening automatically/accidentally without people being aware of what's happening.

[fzhao99]

I wonder if we want

tefca-viewer-1  | ValueSet <> concept join insert failed
tefca-viewer-1  | error: duplicate key value violates unique constraint "valueset_to_concept_pkey"

...I guess it will depend on how often we expect STLTs to update a valueset versus it maybe being a one-time thing they do per condition/topic area. I guess if we eventually add a remove option it's not the biggest deal, and it's probably better to have the remove/re-insert as two steps to ensure things aren't happening automatically/accidentally without people being aware of what's happening.

Thanks for raising! I think the handling of the valueset update flow deserves some larger consideration since I'm not sure when valuesets will be fully deleted versus just being updated with a new version. Depending on where we land on version addition vs update-in-place, think this join interaction would need different considerations.