Merge two records into a single record

[george-neha]

As a user/Social Worker (SW), I want to merge two similar records in an entity into one to avoid duplicates and ensure consistency and accuracy.

Objective
In the system, two users/SWs may record notes/tasks/schools/children twice, which can lead to duplication of work and waste of their time. To avoid this, there should be an option to merge two records of a single record. After merging the two records in the system, there should be only one record with the details captured from both records to ensure no loss of unique data.

Proposed Solutions & Alternatives
Acceptance Criteria:

1. I should be able to select or search two records (even archived/inactive ones) from the list I want to merge. If the toggle for archive functionality is ON then user should be able to search an archived record for merge as well.
2. I should be able to select data from either or both records. (see exceptions below)
3. I should be able to preview the data fields of both records simultaneously and select the data I want to merge.
4. Merging of records should be a one-time activity. Once the records are merged into one, the deleted data should not be available.
5. After merging two records, the system should notify the SW/user that “ assigned to you has been merged.”

Exceptions:

1. If there is a date field, I should be able to select only one date (field-level logic)
2. For fields that have only single-select dropdown values (e.g., gender, status, category, address), I should be able to select only one field.

Out of scope

1. Merging more than two records. After getting user feedback, we can scale it up to 5 records.
2. Automatic detection of duplicate records and merging them
3. Audit trail of the records that are merged.

Low fidelity skteches for merging two records

[Abhinegi2]

To implement the merge functionality this is my approach:

• Add a "Merge" button in the bulk actions item.

• Manage Selection Validation:
  If more than two rows are selected and the user clicks on "Merge," show a warning: "You can only select 2 rows for merging right now."

• Create a new MergePopupComponent:

• Display a popup showing:

  - Child A and Child B are side by side for comparison.
  - Merge Preview to show the resulting merged data.
  

• Allow users to:

  • Select data from either or both records.
  • Choose only one option for date fields and single-select dropdowns.
  • Include action buttons: Cancel and Merge.
    - Show a confirmation dialog before merging.

• After merging, show a success message: "Records merged successfully."

Let me know if it looks good to you and if you have any suggestions

[Abhinegi2]

@george-neha, I've created the first draft of the merge screen. You can check it out (https://pr-2879.aam-digital.net/) by navigating to Bulk Action -> Merge.

I'm also attaching a screenshot of the current UI. Let me know if you have any feedback or suggestions. In the meantime, I'll continue working on the functionality.

[Abhinegi2]

@sleidig
I have a question about the merge feature. Currently, I am merging all the updated details into the first record, updating that entity, and then deleting the second record. Is this the best approach, or should I create a new entity record(with updated details) and delete both selected records instead?

[sleidig]

That's a good question, @Abhinegi2 ...

I think updating one of the existing records is good. It means we have to make less changes to related records (other entities that reference this only have to be changed if they point to the second, deleted ID - although this is only part of a second step #2858 ). At the moment I can't think of a disadvantage of this compared to creating a completely new record. Do you see any potential issues?

[Abhinegi2]

Currently, the only disadvantage I see with creating a completely new record is that the related records from any of the selected entities do not merge. However, when merging into an existing record, we at least retain the references from the selected record(which we are going to cover in 2858).

[george-neha]

1. For fields other than the exceptions (mentioned above), as a user, I should be allowed to select values from records A & B for merging.

2. Bug- sometimes, on selecting a field, both fields get selected

3. Address is not displayed

4. After merging, only one record should exist. currently, after merging till the page is loaded, both records are visible in the list.

[george-neha]

@Abhinegi2 In notes, when merging two records, the system displays an internal student ID instead of the child's name.

[sleidig]

In notes, when merging two records, the system displays an internal student ID instead of the child's name.

@Abhinegi2 for this as well as some similar remarks: I would suggest to use EntityFieldViewComponents and/or EntityFieldEditComponents here, so that we always use the dataType specific UI

[Abhinegi2]

I have integrated the EnfityFieldViewComponent for now, and it seems to be working fine. @george-neha please check now also currently after merging the Notes record related entities not working as expected which I will covering in #2858

[george-neha]

1. During merge, can the fields listed in an order such that fields of at least one tab are together? Currently, during the merge of 2 children's records, fields from the dropout and health tabs come in between the fields of general information.
2. Before the records are merged, display a warning message to alert the user that this is one time activity and cannot be undone.

[sleidig]

1. During merge, can the fields listed in an order such that fields of at least one tab are together? Currently, during the merge of 2 children's records, fields from the dropout and health tabs come in between the fields of general information.

This is not trivial (e.g. the same field can be displayed under multiple tabs; some fields may not be shown in any tab but still contain old data that has to be merged to avoid data loss; etc.). Let's discuss this separately as a sub-issue to the overall feature

[Abhinegi2]

@george-neha I have added a confirmation dialog similar to other bulk actions, Please check now

[sleidig]

Looks good so far!

I have a few extra points on my UX wishlist still:

• I would like to manually edit the values in the "Merge Preview" column, using the normal form fields (EditComponent). (e.g. instead of using the exact name from record A or B, I want to type a new name that may be a custom combination of the two previous names)
• I would like to have "smart" pre-selection of merged values:
  • if the values are identical -> select the value
  • if only one record has a value and the other's value is empty -> select the one with a value
  • ...?
  • for other fields, by default select the one from record A, so that I don't have to click through all the fields

@george-neha , are these part of any of the other sub-issues already?
Maybe at least the second point should become its own sub-issue, to keep this manageable here.