Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenMRS sync creates duplicate resources. #141

Open
witash opened this issue Oct 10, 2024 · 2 comments
Open

OpenMRS sync creates duplicate resources. #141

witash opened this issue Oct 10, 2024 · 2 comments
Assignees
Labels
Type: Technical issue Improve something that users won't notice

Comments

@witash
Copy link
Contributor

witash commented Oct 10, 2024

OpenMRS sync currently subtracts the 2 * SYNC_INTERVAL from the SYNC_PERIOD to decide how long ago to sync. So if it's syncing 1 hour ago every minute, it only syncs resources which were updated in the last 58 minutes.

This works around an issue where it was creating duplicate resources based on resources it had already created, but it does not really fix the core issues and is very confusing.

This workaround should be removed, and duplicate resources should be prevented by fixing the bug in the code that is supposed to be checking unique keys.

@witash witash added the Type: Technical issue Improve something that users won't notice label Oct 10, 2024
@witash witash self-assigned this Oct 10, 2024
@witash
Copy link
Contributor Author

witash commented Oct 10, 2024

The 2 * SYNC_INTERVAL from the SYNC_PERIOD is because there is a gap between when a resource is created in one system and when it is created in another system. So resources created near the boundary of SYNC_PERIOD may appear in the query for one system, but be absent from the other, causing the sync to try and create them again.

So the idea was to add a window where a resource in one system that is not in the other is not created. It's not a good solution because creation times for corresponding resources in both systems can be different by much more than 2 * SYNC_INTERVAL if the sync wasn't running or there was an error during the initial sync and it was resolved later.

@njuguna-n njuguna-n self-assigned this Oct 14, 2024
@witash
Copy link
Contributor Author

witash commented Oct 17, 2024

One fix for this could be to make sure that the FHIR server doesn't allow duplicate resources, based on id or any of the other identifiers: OpenMRS ID, CHT Document ID

So if a resource appeared in the comparison list for OpenMRS but not FHIR, but was actually in the FHIR server, the sync would still send a creation request. But it wouldn't matter, it would get a 40X error and not do anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Technical issue Improve something that users won't notice
Projects
Status: In Progress
Development

No branches or pull requests

2 participants