[IS-11] Make test cases independent #836
Conversation
Libraries like python-zeroconf are starting to depend on more recent Python features
python-zeroconf 0.75.0 strict=False for _nmos-registration._tcp
So that same one is used by Python tests and testssl.sh tests Co-authored-by: Simon Lo <[email protected]>
Co-authored-by: Simon Lo <[email protected]>
Co-authored-by: jonathan-r-thorpe <[email protected]>
Pass IP address of API under test to testssl.sh
Mocks warm up
|
Tests of Outputs:
|
| data=VALID_EDID) | ||
| if not valid or response.status_code != 204: | ||
| return test.FAIL("Unexpected response from " | ||
| "the Stream Compatibility Management API: {}".format(response)) | ||
|
|
There was a problem hiding this comment.
Add reasonable delay after put command.
start_time = time.time()
while time.time() < start_time + CONFIG.API_PROCESSING_TIMEOUT:
time.sleep(0.2)
There was a problem hiding this comment.
This code PUTs and then immediately GETs /edid/base. It's clear that handling the new EDID by an external baseband source takes some time but the API is the matter of the Node, and the spec doesn't allow the following situation:
- value is ABC
- user sets it to CBA
- user requests it and gets ABC
- user waits some time
- user requests it again and gets CBA
| valid, response = self.do_request("DELETE", self.compat_url + "inputs/" + inputId + "/edid/base") | ||
| if not valid or response.status_code != 204: | ||
| return test.FAIL("Unexpected response from " | ||
| "the Stream Compatibility Management API: {}".format(response)) | ||
|
|
There was a problem hiding this comment.
Add reasonable delay after Delete command.
start_time = time.time()
while time.time() < start_time + CONFIG.API_PROCESSING_TIMEOUT:
time.sleep(0.2)
There was a problem hiding this comment.
The same case as #836 (comment)
nmostesting/suites/IS1101Test.py
Outdated
| valid, response = self.do_request("PUT", | ||
| self.compat_url + "inputs/" + inputId + "/edid/base", | ||
| headers={"Content-Type": "application/octet-stream"}, | ||
| data=VALID_EDID) |
There was a problem hiding this comment.
Add reasonable delay after Put command.
start_time = time.time()
while time.time() < start_time + CONFIG.API_PROCESSING_TIMEOUT:
time.sleep(0.2)
There was a problem hiding this comment.
The version of an Input should increment immediately after a PUT, isn't it?
| if not valid or response.status_code != 405: | ||
| return test.FAIL("Unexpected response " | ||
| "for DELETE /edid/base: {}".format(response)) | ||
|
|
There was a problem hiding this comment.
Add reasonable delay after Delete command.
start_time = time.time()
while time.time() < start_time + CONFIG.API_PROCESSING_TIMEOUT:
time.sleep(0.2)
There was a problem hiding this comment.
We expect that a Node rejects the request, why should we wait here?
| valid, response = self.do_request("GET", self.compat_url + "inputs/" + inputId + "/edid/base") | ||
| if not valid or response.status_code != 204: | ||
| return test.FAIL("Unexpected response from " | ||
| "the Stream Compatibility Management API: {}".format(response)) | ||
|
|
There was a problem hiding this comment.
Add reasonable delay after Delete command before getting the effective EDID
start_time = time.time()
while time.time() < start_time + CONFIG.API_PROCESSING_TIMEOUT:
time.sleep(0.2)
# Conflicts: # nmostesting/IS11Utils.py
|
Let's merge this PR in is-11 |
This PR unifies IS-11 test suite with the rest in terms of test cases independence.
The first reworked part is tests of Inputs:
test_1deactivate_connection_resourcescalled inset_up_testsdelete_active_constraintscalled inset_up_testsset_up_testsset_up_teststest_2test_3test_4test_5test_6