Skip to content

Latest commit

 

History

History
100 lines (88 loc) · 3.38 KB

README.md

File metadata and controls

100 lines (88 loc) · 3.38 KB
Raw

A set of scripts for easier parsing and batch processing of the Jenkins test reports.

Usage Examples:

Parse all tiers for the last completed builds of the job

In [1]: import claims

In [2]: claims.config
Out[2]: 
{'bld': 'lastCompletedBuild',
 'job': 'automation-6.2-tier{0}-rhel{1}',
 'pwd': 'nbusr123',
 'url': 'https://jenkins.server.com',
 'usr': 'uradnik1'}

In [5]: reports
Out[5]: 
['t1':
	[
		'el6': [
			{
			u'className': u'tests.foreman.cli.test_syncplan.SyncPlanTestCase',
			 u'errorDetails': None,
			 u'errorStackTrace': None,
			 u'name': u'test_negative_synchronize_custom_product_past_sync_date',
			 u'status': u'PASSED',
			 u'testActions': [],
			 'url': u'https://jenkins.server.com/job/automation-6.2-tier4-rhel7/lastCompletedBuild/testReport/junit/tests.foreman.cli.test_syncplan/SyncPlanTestCase/test_negative_synchronize_custom_product_past_sync_date'
			}
		],
		'el7': [
			{...}
		]
	],
't2': [...],
...
]

Get a flat list of all failed tests:

In [6]: failures = []

In [7]: for i in reports.keys():
   ...:     for j in reports[i].keys():
   ...:         failures += claims.parse_fails(reports[i][j])
   ...:         

In [8]: len(failures)
Out[8]: 324
In [9]: failures
Out[9]: 
[
	{u'className': u'tests.foreman.cli.test_syncplan.SyncPlanTestCase',
	 u'errorDetails': u'AssertionError: Repository contains invalid number of content entities',
	 u'errorStackTrace': u'self = <tests.foreman.cli.test_syncplan.SyncPlanTestCase ...',
	 u'name': u'test_positive_synchronize_custom_product_past_sync_date',
	 u'status': u'FAILED',
	 u'testActions': [{u'reason': None}],
	 'url': u'https://jenkins.server.com/job/automation-6.2-tier4-rhel6/lastCompletedBuild/testReport/junit/tests.foreman.cli.test_syncplan/SyncPlanTestCase/test_positive_synchronize_custom_product_past_sync_date'
	},
	{...},
	...
]

Claim unclaimed tests, containing 'foo' in the errorDetails with a reason: 'bar':

for f in [i for i in failures if u'foo' in i['errorDetails'] and not i['testActions'][0]['reason']]:
	claims.claim(f, 'bar')

Iterate over a list of failures and claim them based on the provided ruleset:

#load the ruleset (from kb.json)
In [10]: rules = claims.load_rules()

In [11]: rules
Out[11]: 
[{u'pattern': u'StaleElementReferenceException',
  u'reason': u'https://github.com/SatelliteQE/robottelo/issues/4162'},
 {u'pattern': u'HTTPError: 500 Server Error: Internal Server Error for url: https://jenkins.server.com/docker/api/v2/containers',
  u'reason': u'https://bugzilla.redhat.com/show_bug.cgi?id=1414821'},
 {u'pattern': u'Unable to log in to this Docker Registry - 404 Resource Not Found: 404 page not found',
  u'reason': u'docker_infra_error'},
 {u'pattern': u'VirtualMachineError: Failed to fetch virtual machine IP address information',
  u'reason': u'https://github.com/SatelliteQE/robottelo/issues/4578'},
 {u'pattern': u'Forbidden - server refused to process the request[^$]+Could not enable repository',
  u'reason': u'outdated_subscription'},
 {u'pattern': u'HTTPError: 403 Client Error: Forbidden for url[^$]+repository_sets[^$]+enable',
  u'reason': u'outdated_subscription'},
 {u'pattern': u'TaskTimedOutError[^$]+action[^$]+Manifest',
  u'reason': u'https://bugzilla.redhat.com/show_bug.cgi?id=1339696'},
 {u'pattern': u'SSHCommandTimeoutError:[^$]+hammer[^$+subscription]',
  u'reason': u'https://bugzilla.redhat.com/show_bug.cgi?id=1339696'}]

In [12]: claims.claim_by_rules(failures, rules)