Linked issue is lost during the import/export process

[roerjo]

Describe the problem
I'm testing the export and import of two tickets that are using issue links to connect them. I'm noticing that if the two tickets are linked multiple times, only one of the links is showing up in ADO.

Within the import logs, I see the following which seems applicable to the problem

[I][14:17:52] Processing 5/6 - wi '36', jira 'AMP-29679, rev 2'.
[W][14:17:52] Duplicate work item link detected to related workitem id: https://dev.azure.com/roerjowork/dd77a350-c9ce-44dd-bab4-76a24b443c56/_apis/wit/workItems/37, Skipping link

Is there a reason why a duplicate work item link cannot be imported? After importing, I can manually add duplicate work item links within ADO (see video below)

To Reproduce

1. Create two Jira tickets
2. Add a "relates" link between them
3. Add a "blocks" link between them
4. Export
5. Import
6. See that only one of the links came through

Tool version

• 3.0.420

Attachments

Please attach the following files:

config.json

{
    "source-project": "AMP",
    "target-project": "test-migration",
    "query": "key = AMP-29680 OR key = AMP-29679 ORDER BY created ASC",
    "using-jira-cloud": true,
    "workspace": "C:\\Users\\rejohnso\\Documents\\jira-ado\\export",
    "epic-link-field": "Epic Link",
    "sprint-field": "Sprint",
    "batch-size": 20,
    "download-options": 7,
    "log-level": "Debug",
    "attachment-folder": "C:\\Users\\rejohnso\\Documents\\jira-ado\\export\\attachments",
    "base-area-path": "",
    "base-iteration-path": "",
    "ignore-failed-links": true,
    "include-link-comments": false,
    "include-jira-css-styles": true,
    "ignore-empty-revisions": false,
    "process-template": "Agile",
    "user-mapping-file": "C:\\Users\\rejohnso\\Documents\\jira-ado\\users-roerjo-test.txt",
    "link-map": {
      "link": [
        {
          "source": "Epic",
          "target": "System.LinkTypes.Hierarchy-Reverse"
        },
        {
          "source": "Parent",
          "target": "System.LinkTypes.Hierarchy-Reverse"
        },
        {
          "source": "Child",
          "target": "System.LinkTypes.Hierarchy-Forward"
        },
        {
          "source": "Duplicate",
          "target": "System.LinkTypes.Duplicate-Forward"
        },
        {
          "source": "Relates",
          "target": "System.LinkTypes.Related"
        },
        {
          "source": "Cloners",
          "target": "System.LinkTypes.Duplicate-Forward"
        },
        {
          "source": "Blocks",
          "target": "System.LinkTypes.Dependency-Forward"
        }
      ]
    },
    "type-map": {
      "type": [
        {
          "source": "Bug",
          "target": "Bug"
        },
        {
          "source": "New Feature",
          "target": "New Feature"
        },
        {
          "source": "Task",
          "target": "Task"
        },
        {
          "source": "Improvement",
          "target": "Improvement"
        },
        {
          "source": "Epic",
          "target": "Epic"
        },
        {
          "source": "Story",
          "target": "Story"
        },
        {
          "source": "Initiative",
          "target": "Initiative"
        },
        {
          "source": "Sub-task",
          "target": "Task"
        }
      ]
    },
    "field-map": {
      "field": [
        {
          "source": "customfield_11626",
          "target": "Microsoft.VSTS.Common.AcceptanceCriteria",
          "comment": "customfield_11626 is `Acceptance Criteria` field in Jira"
        },
        {
          "source": "assignee",
          "target": "System.AssignedTo",
          "mapper": "MapUser"
        },
        {
          "source": "customfield_11100",
          "target": "Microsoft.VSTS.Common.BusinessValue",
          "comment": "customfield_11100 is the `Business Impact` field in Jira",
          "mapping": {
            "values": [
              {
                "source": "Low",
                "target": "4"
              },
              {
                "source": "Medium",
                "target": "3"
              },
              {
                "source": "High",
                "target": "2"
              },
              {
                "source": "Critical",
                "target": "1"
              }
            ]
          }
        },
        {
          "source": "customfield_11625",
          "target": "Custom.ClientID",
          "comment": "customfield_11625 is the `Client ID` field in Jira"
        },
        {
          "source": "customfield_11670",
          "type": "datetime",
          "target": "Microsoft.VSTS.Common.ClosedDate",
          "comment": "customfield_11670 is the `Closed` field in Jira"
        },
        {
          "source": "comment",
          "target": "System.History",
          "mapper": "MapRendered"
        },
        {
          "source": "description",
          "target": "System.Description",
          "mapper": "MapRendered"
        },
        {
          "source": "labels",
          "target": "System.Tags",
          "mapper": "MapTags"
        },
        {
          "source": "customfield_11650",
          "target": "Custom.Product",
          "comment": "customfield_11650 is the `Primary Product` field in Jira"
        },
        {
          "source": "priority",
          "target": "Microsoft.VSTS.Common.Priority",
          "mapping": {
            "values": [
              {
                "source": "Trivial",
                "target": "4"
              },
              {
                "source": "Minor",
                "target": "3"
              },
              {
                "source": "Major",
                "target": "2"
              },
              {
                "source": "Blocker",
                "target": "1"
              },
              {
                "source": "Critical",
                "target": "1"
              }
            ]
          }
        },
        {
          "source": "customfield_11655",
          "target": "Custom.Tester",
          "comment": "customfield_11655 is the `QA Tester` field in Jira",
          "mapper": "MapUser"
        },
        {
          "source": "resolution",
          "target": "Microsoft.VSTS.Common.ResolvedReason"
        },
        {
          "source": "status",
          "target": "System.State",
          "mapping": {
            "values": [
              {
                "source": "New",
                "target": "New"
              },
              {
                "source": "Ticketed",
                "target": "Ticketed"
              },
              {
                "source": "To Do",
                "target": "To Do"
              },
              {
                "source": "Assigned",
                "target": "Assigned"
              },
              {
                "source": "In Progress",
                "target": "In Progress"
              },
              {
                "source": "QA Ready",
                "target": "QA Ready"
              },
              {
                "source": "QA Assigned",
                "target": "QA Assigned"
              },
              {
                "source": "QA Testing",
                "target": "QA Testing"
              },
              {
                "source": "Reopened",
                "target": "Reopened"
              },
              {
                "source": "Resolved",
                "target": "Resolved"
              },
              {
                "source": "Closed",
                "target": "Closed"
              }
            ]
          }
        },
        {
          "source": "summary",
          "target": "System.Title",
          "mapper": "MapTitle"
        }
      ]
    }
  }

jira-export-log.txt

====================================================================
Jira Export Log
====================================================================
Tool version : 3.0.420
Start time   : 05/31/2024 2:16:49 PM
Telemetry    : Enabled
Session id   : d1b4863c-8899-40f5-b3a3-6029dab8b6fd
Tool user    : US\rejohnso
Config       : ..\config-roerjo.json
Force        : no
Log level    : Debug
Machine      : RMT-545281L
System       : Microsoft Windows 10.0.19045
Jira url     : https://aireal.atlassian.net
Jira user    : [email protected]
Jira version : 1001.0.0-SNAPSHOT
Jira type    : Cloud
====================================================================
[I][14:16:48] Connecting to Jira...
[I][14:16:48] Retrieving Jira fields...
[I][14:16:49] Retrieving Jira link types...
[D][14:16:49]    Get item count using query: 'key = AMP-29680 OR key = AMP-29679 ORDER BY created ASC'
[I][14:16:49] Export started. Selecting 2 items.
[I][14:16:49] Initializing Jira field mapping...
[D][14:16:49]    Enumerate remote issues
[I][14:16:50] Processing 1/2 - 'AMP-29679'.
[D][14:16:50]    Downloaded item.
[D][14:16:50]    Rendered field customfield_10500 contains unparsable type Null, using text
[D][14:16:50]    Rendered field customfield_11679 contains unparsable type Null, using text
[D][14:16:50]    Rendered field priority contains unparsable type Null, using text
[D][14:16:50]    Rendered field labels contains unparsable type Null, using text
[D][14:16:50]    Rendered field issuelinks contains unparsable type Null, using text
[D][14:16:50]    Rendered field status contains unparsable type Null, using text
[D][14:16:50]    Rendered field components contains unparsable type Null, using text
[D][14:16:50]    Rendered field customfield_11300 contains unparsable type Null, using text
[D][14:16:50]    Rendered field creator contains unparsable type Null, using text
[D][14:16:50]    Rendered field reporter contains unparsable type Null, using text
[D][14:16:50]    Rendered field issuetype contains unparsable type Null, using text
[D][14:16:50]    Rendered field project contains unparsable type Null, using text
[D][14:16:50]    Rendered field workratio contains unparsable type Null, using text
[D][14:16:50]    Rendered field customfield_11631 contains unparsable type Null, using text
[D][14:16:50]    Rendered field customfield_10007 contains unparsable type Null, using text
[D][14:16:50]    Rendered field summary contains unparsable type Null, using text
[D][14:16:50]    Downloaded issue: AMP-29679 changelog.
[D][14:16:50]    Undone link 'Added [Blocks] AMP-29679->AMP-29680'.
[D][14:16:50]    Link is non-directional (Relates) and sourceItem (AMP-29679) is older then target item (AMP-29680). Link change will be part of target item.
[D][14:16:51]    Created 3 history revisions.
[D][14:16:51]    Mapping revision 0.
[D][14:16:51]    Mapped value 'Test 1' to field 'Microsoft.VSTS.Common.AcceptanceCriteria'.
[D][14:16:51]    Mapped value '<style>div {
    display: block;
}

table.confluenceTable {
    border-collapse: collapse;
    margin: 5px 0 5px 2px;
    width: auto;
}

table {
    display: table;
    border-collapse: separate;
    border-spacing: 2px;
    border-color: grey;
}

tbody {
    display: table-row-group;
    vertical-align: middle;
    border-color: inherit;
}

tr {
    display: table-row;
    vertical-align: inherit;
    border-color: inherit;
}

th.confluenceTh {
    border: 1px solid #ccc;
    background: #f5f5f5;
    padding: 3px 4px;
    text-align: center;
}

th {
    font-weight: bold;
    text-align: -internal-center;
}

td, th {
    display: table-cell;
    vertical-align: inherit;
}

    td.confluenceTd {
        border: 1px solid #ccc;
        padding: 3px 4px;
    }

dfn, cite {
    font-style: italic;
}

    cite:before {
        content: "\2014 \2009";
    }
</style><p>Test 1</p>' to field 'System.Description'.
[D][14:16:51]    Mapped value 'Testing;a3Detected' to field 'System.Tags'.
[D][14:16:51]    Mapped value '2' to field 'Microsoft.VSTS.Common.Priority'.
[D][14:16:51]    Mapped value 'New' to field 'System.State'.
[D][14:16:51]    Mapped value '[AMP-29679] Jira|ADO Testing 1' to field 'System.Title'.
[I][14:16:51] Email is not public for user '620d0754f4d8af0070fdcd45' in Jira, using usernameOrAccountId '620d0754f4d8af0070fdcd45' for mapping. You may safely ignore this warning, unless there is a subsequent warning about the username/accountId being missing in the usermapping file.
[I][14:16:51] Specified user '620d0754f4d8af0070fdcd45' does not exist or you do not have required permissions, using accountId '620d0754f4d8af0070fdcd45'
[W][14:16:51] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[D][14:16:51]    Mapping revision 1.
[W][14:16:51] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[D][14:16:51]    Mapping revision 2.
[W][14:16:51] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[I][14:16:51] Processing 2/2 - 'AMP-29680'.
[D][14:16:51]    Downloaded item.
[D][14:16:51]    Rendered field customfield_10500 contains unparsable type Null, using text
[D][14:16:51]    Rendered field customfield_11679 contains unparsable type Null, using text
[D][14:16:51]    Rendered field priority contains unparsable type Null, using text
[D][14:16:51]    Rendered field labels contains unparsable type Null, using text
[D][14:16:51]    Rendered field issuelinks contains unparsable type Null, using text
[D][14:16:51]    Rendered field status contains unparsable type Null, using text
[D][14:16:51]    Rendered field components contains unparsable type Null, using text
[D][14:16:51]    Rendered field customfield_11300 contains unparsable type Null, using text
[D][14:16:51]    Rendered field creator contains unparsable type Null, using text
[D][14:16:51]    Rendered field reporter contains unparsable type Null, using text
[D][14:16:51]    Rendered field issuetype contains unparsable type Null, using text
[D][14:16:51]    Rendered field project contains unparsable type Null, using text
[D][14:16:51]    Rendered field workratio contains unparsable type Null, using text
[D][14:16:51]    Rendered field customfield_11631 contains unparsable type Null, using text
[D][14:16:51]    Rendered field customfield_10007 contains unparsable type Null, using text
[D][14:16:51]    Rendered field summary contains unparsable type Null, using text
[D][14:16:52]    Downloaded issue: AMP-29680 changelog.
[D][14:16:52]    Link with description 'This issue is blocked by AMP-29679' is either not found or this issue (AMP-29680) is not inward issue.
[D][14:16:52]    No link to undo for 'Added [Relates] AMP-29680->AMP-29679'
[D][14:16:52]    Created 3 history revisions.
[D][14:16:52]    Mapping revision 0.
[D][14:16:52]    Mapped value 'Test 2' to field 'Microsoft.VSTS.Common.AcceptanceCriteria'.
[D][14:16:52]    Mapped value '<style>div {
    display: block;
}

table.confluenceTable {
    border-collapse: collapse;
    margin: 5px 0 5px 2px;
    width: auto;
}

table {
    display: table;
    border-collapse: separate;
    border-spacing: 2px;
    border-color: grey;
}

tbody {
    display: table-row-group;
    vertical-align: middle;
    border-color: inherit;
}

tr {
    display: table-row;
    vertical-align: inherit;
    border-color: inherit;
}

th.confluenceTh {
    border: 1px solid #ccc;
    background: #f5f5f5;
    padding: 3px 4px;
    text-align: center;
}

th {
    font-weight: bold;
    text-align: -internal-center;
}

td, th {
    display: table-cell;
    vertical-align: inherit;
}

    td.confluenceTd {
        border: 1px solid #ccc;
        padding: 3px 4px;
    }

dfn, cite {
    font-style: italic;
}

    cite:before {
        content: "\2014 \2009";
    }
</style><p>Test 2</p>' to field 'System.Description'.
[D][14:16:52]    Mapped value 'Testing;a3Detected' to field 'System.Tags'.
[D][14:16:52]    Mapped value '2' to field 'Microsoft.VSTS.Common.Priority'.
[D][14:16:52]    Mapped value 'New' to field 'System.State'.
[D][14:16:52]    Mapped value '[AMP-29680] Jira|ADO Testing 2' to field 'System.Title'.
[W][14:16:52] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[D][14:16:52]    Mapping revision 1.
[W][14:16:52] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[D][14:16:52]    Mapping revision 2.
[W][14:16:52] Could not find user '620d0754f4d8af0070fdcd45' identity in user map. Using default identity '[email protected]'.
[D][14:16:52]    Exported as type 'Improvement'.
[D][14:16:52]    Exported as type 'Improvement'.
[I][14:16:52] Export complete. Exported 2 items (0 errors, 1 warnings) in 00:00:03.

wi-import-log.txt

====================================================================
Azure DevOps Work Item Import Log
====================================================================
Tool version         : 3.0.420
Start time           : 05/31/2024 2:17:50 PM
Telemetry            : Enabled
Session id           : 8803303b-6238-45e3-9377-180bf3c6404c
Tool user            : US\rejohnso
Config               : ..\config-roerjo.json
User                 : US\rejohnso
Force                : no
Log level            : Debug
Machine              : RMT-545281L
System               : Microsoft Windows 10.0.19045
Azure DevOps url     : https://dev.azure.com/roerjowork
Azure DevOps version : n/a
Azure DevOps type    : Cloud
====================================================================
[I][14:17:48] Connecting to Azure DevOps/TFS...
[I][14:17:49] Retreiving project info from Azure DevOps/TFS...
[I][14:17:49] Building iteration cache...
[D][14:17:50]    Iteration 'Sprint 1' added to cache
[I][14:17:50] Building area cache...
[I][14:17:50] Building execution plan...
[D][14:17:50]    Analyzing item 'AMP-29679'.
[D][14:17:50]    Analyzing item 'AMP-29680'.
[I][14:17:50] Import started. Importing 2 items with 6 revisions.
[I][14:17:50] Processing 1/6 - wi '36', jira 'AMP-29679, rev 0'.
[D][14:17:50]    Mapped AreaPath 'test-migration'.
[D][14:17:50]    Mapped IterationPath 'test-migration'.
[W][14:17:50] '[Added] AMP-29679/-1->AMP-29680/-1 [System.LinkTypes.Related]' - target work item for Jira 'AMP-29680' is not yet created in Azure DevOps/TFS. You can safely ignore this warning if this work item is scheduled for import later in your migration.
[W][14:17:50] ''AMP-29679', rev 0' - not all changes were saved.
[D][14:17:50]    Correcting acceptance criteria on separate revision on ''AMP-29679', rev 0'.
[D][14:17:51]    Imported revision.
[I][14:17:51] Processing 2/6 - wi '37', jira 'AMP-29680, rev 0'.
[D][14:17:51]    Mapped AreaPath 'test-migration'.
[D][14:17:51]    Mapped IterationPath 'test-migration'.
[D][14:17:51]    Correcting acceptance criteria on separate revision on ''AMP-29680', rev 0'.
[D][14:17:51]    Imported revision.
[I][14:17:51] Processing 3/6 - wi '36', jira 'AMP-29679, rev 1'.
[D][14:17:51]    Correcting acceptance criteria on separate revision on ''AMP-29679', rev 1'.
[D][14:17:51]    Imported revision.
[I][14:17:51] Processing 4/6 - wi '37', jira 'AMP-29680, rev 1'.
[I][14:17:52] Updated new work item Id:37 with link to work item ID:36
[D][14:17:52]    Correcting acceptance criteria on separate revision on ''AMP-29680', rev 1'.
[D][14:17:52]    Imported revision.
[I][14:17:52] Processing 5/6 - wi '36', jira 'AMP-29679, rev 2'.
[W][14:17:52] Duplicate work item link detected to related workitem id: https://dev.azure.com/roerjowork/dd77a350-c9ce-44dd-bab4-76a24b443c56/_apis/wit/workItems/37, Skipping link
[W][14:17:52] ''AMP-29679', rev 2' - not all changes were saved.
[D][14:17:52]    Correcting acceptance criteria on separate revision on ''AMP-29679', rev 2'.
[D][14:17:52]    Imported revision.
[I][14:17:52] Processing 6/6 - wi '37', jira 'AMP-29680, rev 2'.
[D][14:17:52]    Correcting acceptance criteria on separate revision on ''AMP-29680', rev 2'.
[D][14:17:53]    Imported revision.
[I][14:17:53] Import complete. Imported 2 items, 6 revisions (0 errors, 4 warnings) in 00:00:04.

Screenshots
Ticket 1:

Ticket 2:

Here is a short video showing the result of the import in ADO. Only the "relates" link has come over:

2024-05-31.14-19-28.mp4

BTW thank you so much for this tooling!

[Alexander-Hjelm]

Hello @roerjo ! Thank you for the detailed repro steps.

Upon reviewing the WitClientUtils, I found that there does not seem to be an check for uniqueness with regards to the link type:

jira-azuredevops-migrator/src/WorkItemMigrator/WorkItemImport/WitClient/WitClientUtils.cs

Lines 42 to 58 in 304e6e2

	public bool IsDuplicateWorkItemLink(IEnumerable<WorkItemRelation> links, WorkItemRelation relatedLink)
	{
	if (links == null || relatedLink == null)
	{
	return false;
	}
	var containsRelatedLink = links.Contains(relatedLink);
	var hasSameRelatedWorkItemId = links.OfType<WorkItemRelation>()
	.Any(l => l.Url == relatedLink.Url);
	if (!containsRelatedLink && !hasSameRelatedWorkItemId)
	return false;
	Logger.Log(LogLevel.Warning, $"Duplicate work item link detected to related workitem id: {relatedLink.Url}, Skipping link");
	return true;
	}

I believe that a potential fix could be as easy as to add a similarity check on the WorkItemRelation.Rel property when filtering the links with the same WorkItem ID.