Skip to content

Commit

Permalink
Drop Tackle 1.2 export from CLI
Browse files Browse the repository at this point in the history 
Removing export Tackle 1.2 function from CLI.

Use older CLI versions if the Tackle 1.2 export is still needed.

Signed-off-by: Marek Aufart <[email protected]>
  • Loading branch information
@aufi
aufi committed Nov 27, 2023
1 parent ce5a30e commit 2c1420a
Show file tree
Hide file tree
Showing 3 changed files with 2 additions and 248 deletions.
32 changes: 2 additions & 30 deletions hack/tool/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ For more details about the Tackle project, see [Tackle2-Hub README](https://gith

## Scenarios

### Export and Import data between Tackle 2 instances
### Export and Import data between Tackle/Konveyor instances

Tackle 2 can be exported and imported to another Tackle 2 instance, expecting it is cleaned-up before import. This focuses on application inventory, but exports and imports most of Tackle 2 objects except TaskGroups&Tasks and Keycloak identitty provider data.

Expand All @@ -24,21 +24,6 @@ Steps for import (tackle-config.yml file points to destination Tackle 2, existin

**Note: This export/import functionality doesn't aim to be a backup/restore tool. E.g. it doesn't dump Keycloak data and it requires have the Tackle installation to be running on the destination cluster before importing the data.**

### Migrate data from running Tackle 1.2 to running Tackle 2 instance

Migrate data updating refs to the seed objects matching to the destination Tackle 2 (tags, tag-types and job functions seeded).

- ```tackle export-tackle1```
- ```tackle import```

### Export data from Tackle 1.2 to be later imported to some Tackle 2 instance

Exports full data dump including seeds to be later imported to currently not available Tackle 2 instance, which needs a cleanup before running the import.

- ```tackle --skip-destination-check export-tackle1```
- ```tackle clean-all```
- ```tackle import```

### If the import failed

The ```tackle import``` command could fail in a pre-import check phase which ensures there are no resources of given type with the same ID in the destination Tackle 2 (error after ```Checking tagtypes in destination Tackle2..``` etc.). In this case, run ```tackle clean``` command, which will remove such objects from the destination Tackle 2 API or remove it manually either from the destination Tackle 2 or from the JSON data files.
Expand Down Expand Up @@ -72,7 +57,6 @@ Run the tackle tool:

### Supported actions
- ```export``` exports Tackle 2 objects into local JSON files and bucket data directory
- ```export-tackle1``` exports Tackle 1.2 API objects into local JSON files
- ```import``` creates objects in Tackle 2 from local JSON files and buckets data
- ```clean``` deletes objects uploaded to Tackle 2 from local JSON files
- ```clean-all``` deletes ALL data returned by Tackle 2 (including seeds, additional to ```clean```), skips some pathfinder stuff without index action
Expand All @@ -85,12 +69,6 @@ The export dommand connects to Tackle2 Hub API and dumps relevant resources to l

**Note: the encryption_passphase needs to be the same when running the import.**

### Export Tackle 1.2

Run ```tackle export-tackle1``` to get dump of Tackle 1.2 objects into JSON files in local directory ```./tackle-data```.

The ```export-tackle1``` command looks into Tackle2 to grab seed resources first, then downloads all resources from Tackle 1.2 API, transforms it to format expected by the Tackle 2 Hub and re-map resources to seeds already existing in destination Tackle2 Hub API. The result is serialized and stored into local JSON files.

### Import to Tackle 2

Check local JSON dump files in ```./tackle-data``` directory (if needed) and create objects in Tackle 2 Hub running ```tackle import```.
Expand Down Expand Up @@ -133,7 +111,7 @@ usage: tackle [-h] [-c [CONFIG]] [-d [DATA_DIR]] [-v] [-s] [-w] [-i] [-n] [-b] [
Konveyor Tackle maintenance tool.
positional arguments:
action One or more Tackle commands that should be executed, options: export export-tackle1 import clean clean-all
action One or more Tackle commands that should be executed, options: export import clean clean-all
options:
-h, --help show this help message and exit
Expand Down Expand Up @@ -164,12 +142,6 @@ password:
# Export of Identitiy (credentials) password and key fields should be encrypted, set the passphase
encryption_passphase:
# Tackle 1.2 endpoint and credentials, e.g. to dump data and migrate to Tackle2
tackle1:
url: https://tackle-tackle.apps.mta01.cluster.local
username: tackle
password:
```

Unverified HTTPS warnings from Python could be hidden by ```export PYTHONWARNINGS="ignore:Unverified HTTPS request"``` or with ```-w``` command option.
212 changes: 0 additions & 212 deletions hack/tool/tackle
View file
Original file line number Diff line number Diff line change
Expand Up @@ -301,196 +301,6 @@ class TackleTool:
print("ERROR: %s record ID %d not found." % (objType, id))
exit(1)

# Gather Tackle 1.2 API objects and map seeded Tackle2 API objects
def dumpTackle1(self):
### TAG TYPES & TAGS ###
collection = apiJSON(self.tackle1Url + "/api/controls/tag-type", self.tackle1Token)
for tt1 in collection:
# Temp holder for tags
tags = []
# Prepare TagCategories's Tags
for tag1 in tt1['tags']:
tag = Tackle2Object(tag1)
tag.name = tag1['name']
# TagCategory is injected from tagType processing few lines below
# Store Tag only if doesn't exist in Tackle2 destination already
self.add('origin-tags', tag) # tmp tags for merge with seed tags lookup
if tag.name.lower() not in self.destData['tags']:
self.add('tags', tag)
tags.append(tag)
# Prepare TagCategory
tt = Tackle2Object(tt1)
tt.name = tt1['name']
tt.colour = tt1['colour']
tt.rank = tt1['rank']
tt.username = tt1['createUser']
for tag in tags:
tag.tagCategory = copy.deepcopy(tt)
tt.tags = tags
# Store only if doesn't exist in Tackle2 destination already
if tt.name.lower() not in self.destData['tagcategories']:
self.add('tagcategories', tt)

### APPLICATION ###
collection = apiJSON(self.tackle1Url + "/api/application-inventory/application?page=0&size=10000", self.tackle1Token)
for app1 in collection:
# Temp holder for tags
tags = []
# Prepare Tags
debugPrint(app1)
if app1['tags']:
for tagId in app1['tags']:
appTag = self.findById('origin-tags', int(tagId))
# Check if Tag exists in Tackle2 destination
if appTag.name.lower() in self.destData['tags']:
# Re-map to existing Tackle2 Tag
tags.append(self.destData['tags'][appTag.name.lower()])
else:
# Use imported Tag, creating a new one to cut association to Tag type
tag = Tackle2Object()
tag.id = appTag.id
tag.name = appTag.name
tags.append(tag)
# Prepare Application
app = Tackle2Object(app1)
app.name = app1['name']
app.description = app1['description']
app.tags = tags
if app1['businessService']:
app.businessService = {'id':int(app1['businessService'])}
else:
print("Warning: Application %d %s has not businessService, which is required by Tackle2. Set it manually in %s/applications.json" % (app.id, app.name, self.dataDir))
### APPLICATION REVIEW ###
if app1['review']:
rev = Tackle2Object(app1['review'])
rev.proposedAction = app1['review']['proposedAction']
rev.effortEstimate = app1['review']['effortEstimate']
rev.businessCriticality = app1['review']['businessCriticality']
rev.workPriority = app1['review']['workPriority']
rev.comments = app1['review']['comments']
rev.application = {'id': app.id, 'name': app.name}
if app1['review']['copiedFromReviewId']:
rev.copiedFromReviewId = app1['review']['copiedFromReviewId']
self.add('reviews', rev)
#app.repository = app1['repository'] # Not part of 1.2 API
#app.binary = app1['binary']
#app.facts = app1['facts']
self.add('applications', app)

### DEPENDENCIES ###
collection = apiJSON(self.tackle1Url + "/api/application-inventory/applications-dependency", self.tackle1Token)
for dep1 in collection:
# Prepare Dependency
dep = Tackle2Object(dep1)
dep.to = {'id': dep1['to']['id'], 'name': dep1['to']['name']}
setattr(dep, 'from', {'id': dep1['from']['id'], 'name': dep1['from']['name']}) # Cannot use "from" as an attribute name directly
self.add('dependencies', dep)

### ASSESSMENTS & RISKS (per Application) ###
for app in self.data['applications']:
collection = apiJSON(self.tackle1Url + "/api/pathfinder/assessments?applicationId=%d" % app.id, self.tackle1Token)
for assm1 in collection:
# Prepare Assessment
assm = Tackle2Object()
assm.id = assm1['id']
assm.applicationId = assm1['applicationId']
assm.status = assm1['status']
# Prepare Assessment questions and answers
asqa1 = apiJSON(self.tackle1Url + "/api/pathfinder/assessments/%d" % assm.id, self.tackle1Token)
asqa = Tackle2Object()
asqa.id = asqa1['id']
asqa.applicationId = asqa1['applicationId']
asqa.status = asqa1['status']
asqa.stakeholders = asqa1['stakeholders']
asqa.stakeholderGroups = asqa1['stakeholderGroups']
asqa.questionnaire = asqa1['questionnaire']
self.add('assessments', asqa)

collection = apiJSON(self.tackle1Url + "/api/pathfinder/assessments/assessment-risk", self.tackle1Token, data=[{"applicationId": app.id}], method='POST')
for assmr1 in collection:
# Prepare Assessment Risk
assmr = Tackle2Object()
assmr.assessmentId = assmr1['assessmentId']
assmr.applicationId = assmr1['applicationId']
assmr.risk = assmr1['risk']
self.add('assessments--assessment-risk', assmr)

collection = apiJSON(self.tackle1Url + "/api/pathfinder/assessments/confidence", self.tackle1Token, data=[{"applicationId": app.id}], method='POST')
for conf1 in collection:
# Prepare Confidence
conf = Tackle2Object()
conf.assessmentId = conf1['assessmentId']
conf.applicationId = conf1['applicationId']
conf.confidence = conf1['confidence']
self.add('assessments--confidence', conf)

### STAKEHOLDER ###
collection = apiJSON(self.tackle1Url + "/api/controls/stakeholder", self.tackle1Token)
for sh1 in collection:
# Temp holder for stakeholder's groups
shgs = []
# Prepare StakeholderGroups
for shg1 in sh1['stakeholderGroups']:
shg = Tackle2Object(shg1)
shg.name = shg1['name']
shg.description = shg1['description']
self.add('stakeholdergroups', shg)
shgs.append(shg)
# Prepare StakeHolder
sh = Tackle2Object(sh1)
sh.name = sh1['displayName']
sh.email = sh1['email']
sh.stakeholderGroups = shgs
if sh1['jobFunction']:
if sh1['jobFunction']['role'] in self.destData['jobfunctions']:
# Re-map to JobFunction existing in Tackle2 destination
sh.jobFunction = self.destData['jobfunctions'][sh1['jobFunction']['role']]
else:
# Prepare new JobFunction
jf = Tackle2Object(sh1['jobFunction'])
jf.name = sh1['jobFunction']['role']
self.add('jobfunctions', jf)
sh.jobFunction = jf
self.add('stakeholders', sh)

### STAKEHOLDER GROUPS ###
collection = apiJSON(self.tackle1Url + "/api/controls/stakeholder-group", self.tackle1Token)
for shg1 in collection:
# Prepare StakeholderGroup
shg = Tackle2Object(shg1)
shg.name = shg1['name']
shg.description = shg1['description']
self.add('stakeholdergroups', shg)

### JOB FUNCTION ###
collection = apiJSON(self.tackle1Url + "/api/controls/job-function", self.tackle1Token)
for jf1 in collection:
# Temp holder for stakeholders
shs = []
# Prepare JobFunction's Stakeholders
for sh1 in jf1['stakeholders']:
sh = Tackle2Object(sh1)
sh.name = sh1['displayName']
sh.email = sh1['email']
shs.append(sh)
# Prepare JobFunction
jf = Tackle2Object(jf1)
jf.name = jf1['role']
jf.stakeholders = shs
# Store only if doesn't exist in Tackle2 destination already
if jf.name not in self.destData['jobfunctions']:
self.add('jobfunctions', jf)

### BUSINESS SERVICE ###
collection = apiJSON(self.tackle1Url + "/api/controls/business-service", self.tackle1Token)
for bs1 in collection:
# Prepare JobFunction
bs = Tackle2Object(bs1)
bs.name = bs1['name']
bs.description = bs1['description']
bs.owner = bs1['owner'] # Stakeholder
self.add('businessservices', bs)

# Gather Tackle 2 API objects
def dumpTackle2(self):
### TAG TYPES & TAGS ###
Expand Down Expand Up @@ -920,28 +730,6 @@ disableSSlWarnings(args.disableSslWarnings)
c = loadConfig(args.config)
cmdExecuted = False

# Tackle 1 export steps
if cmdWanted(args, "export-tackle1"):
cmdExecuted = True
# Gather Keycloak access tokens for Tackle1&2
token1 = getKeycloakToken(c['tackle1']['url'], c['tackle1']['username'], c['tackle1']['password'])
token2 = ""
if not args.skipDestCheck:
token2 = getKeycloakToken(c['url'], c['username'], c['password'])

# Setup Tackle 1.2->2.0 data migration object
tool = TackleTool(args.data_dir, c['tackle1']['url'], token1, c['url'], token2)

# Run the export
if not args.skipDestCheck:
print("Loading seed objects from Tackle 2")
tool.loadTackle2Seeds()
print("Exporting Tackle 1.2 objects (this might take a while..)")
tool.dumpTackle1()
print("Writing JSON data files into %s" % args.data_dir)
tool.store()
print("Done. The data could be imported to Tackle 2 using command \"tackle import\"")

# Tackle 2 export steps
if cmdWanted(args, "export"):
cmdExecuted = True
Expand Down
6 changes: 0 additions & 6 deletions hack/tool/tackle-config.yml.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,3 @@ password:

# Export of Identitiy (credentials) password and key fields should be encrypted, set the passphase
encryption_passphase:

# Tackle 1.2 endpoint and credentials, e.g. to dump data and migrate to Tackle2
tackle1:
url: https://tackle-tackle.apps.mta01.cluster.local
username: tackle
password:

0 comments on commit 2c1420a

Please sign in to comment.