requests_common.py should not raise an error when HTTP response code is greater than 299

[Heyji2]

Description

I am using the Utilities: Call Rest API function in a resilient worklow. The workflow crashes with the following error message:

File "/usr/local/lib/python3.6/site-packages/resilient_lib/components/requests_common.py", line 138, in execute_call_v2 raise IntegrationError(msg) resilient_lib.components.integration_errors.IntegrationError: '404 Client Error: Processed for url: https://url_modified_in_public_forum:8843/ressource1/ressource2'

This is because of the line 142 and 143 in request_common.py :

# Raise error is bad status code is returned
response.raise_for_status()

I believe an 404 error (HTTP 404 : ressource not found) should not raise an error in this function. This is not an integration error, this is just the client trying to fetch a ressource that does not exists. The client could handle this error code itself, rather than stopping the workflow. I believe this line should be removed.

Describe How to Reproduce

To reproduce this error, make a resilient workflow with the function Utilities: Call Rest API and try to get a resource that does not exists on the server. The workflow stops with an error, and there is nothing to be done to recover from this state without manual intervention on the workflow. If an error was not raised, the client could handle this error itself and the workflow would not stop in this case.

[sventec]

Echoing this, it would be beneficial to support returning results from RequestsCommon's execute (and thus execute_call_v2) with a non-2XX status code in some cases.

I experienced a similar issue to that reported in this issue originally. There are instances where a REST API returns a 404, but it's not an error with the client's request. It simply indicates that no results were found. In cases such as this, a workflow utilizing the Utilities: Call REST API function has no way of handling a 404 response and indicating (with an incident note, for example) that no results were found. Instead, the workflow fails "silently," and a user must check the action status to determine this.

Something like a boolean switch "return_if_failed" would preserve the default behavior, while allowing user handling of HTTP errors that don't require an entire workflow to fail early. This could then be implemented in the Utilities: Call REST API function from fn_utilities as well, allowing for user-controlled handling of non-2XX status codes.

I'd be happy to submit a PR for this.

[sventec]

This may be better handled by implementing a callback function in the fn_utilities Utilities: Call REST API function, if there are no other situations where not raising a status code here is potentially useful.

[mscherfling]

Sventec, be aware that fn_utilities has been deprecated and broken into smaller apps. You are encouraged to start using fn_rest_api as we still continue supporting this app. The operation is nearly identical with several improvements.

One the improvements is a function input you're looking for. 'rest_api_allowed_status_codes' is used to list the allowed https status codes. So, "200, 202, 404" would all be considered allowed results without failing the workflow. Supporting 2XX for the entire range makes sense and we'll add that for future updates.

You should also be aware of a global setting in the app.config to ensure any function failure doesn't fail a workflow: trap_exception. See it's usage below:

[resilient]
trap_exception = True

Best of luck.

[sventec]

@mscherfling thank you for the callout on fn_rest_api. The improvement you described would exactly satisfy the need expressed in my earlier comment. Will move to migrate and implement via rest_api_allowed_status_codes instead for the use case that initially raised this.

Appreciate the note on trap_exception as well.