Looking for "[]" in evaluation result results in never succeeding evaluation

[kristofre]

Tested with Keptn 0.8.2, sli service 0.10.2 and library 4.0.

Evaluation never finishes because a check on response.content.contains("[]") is set so it waits and tries again.

keptn-jenkins-library/src/sh/keptn/Keptn.groovy

Line 680 in 621a9ff

if (response.status == 404 || response.content.contains("No Keptn sh.keptn.event.evaluation.finished event found for context") || response.content.contains("[]") ) {

Sample result below.

In my case it was due to an SLI not retrieving any data, but the evaluation does finish and gets a score.

@pcjeffmac Should it be changed to an equals check?

{"events":[{"data":{"evaluation":{"gitCommit":"","indicatorResults":[{"displayName":"Response Time 95th Percentile","keySli":false,"passTargets":[{"criteria":"\u003c=+10%","targetValue":0,"violated":false},{"criteria":"\u003c800","targetValue":800,"violated":false}],"score":1,"status":"pass","value":{"metric":"response_time_p95","success":true,"value":362.10801250000003},"warningTargets":[{"criteria":"\u003c=1000","targetValue":1000,"violated":false}]},{"displayName":"Response Time of InvokeAPI Method","keySli":false,"passTargets":[{"criteria":"\u003c=+10%","targetValue":0,"violated":true},{"criteria":"\u003c850000","targetValue":0,"violated":true}],"score":0,"status":"fail","value":{"message":"Dynatrace Metrics API returned 0 result values, expected 1 for query: https://DEDACTED/e/DEDACTED/api/v2/metrics/query/?entitySelector=tag%28keptn_project%3Asimplenodeproject%29%2Ctag%28keptn_stage%3Astaging%29%2Ctag%28keptn_service%3Asimplenodeservice%29%2Ctype%28SERVICE%29\u0026from=1620200968000\u0026metricSelector=calc%3Aservice.simplenode.staging%3Afilter%28eq%28method%2C%2Fapi%2Finvoke%29%29%3Amerge%280%29%3Apercentile%2895%29\u0026resolution=Inf\u0026to=1620201159000.\nPlease ensure the response contains exactly one value (e.g., by using :merge(0):avg for the metric). Here is the output for troubleshooting: {\"metricId\":\"calc:service.simplenode.staging:filter(eq(method,/api/invoke)):merge(0):percentile(95)\",\"data\":**[]**}","metric":"rt_invokeapi","success":false,"value":0},"warningTargets":[{"criteria":"\u003c=+20%","targetValue":0,"violated":true},{"criteria":"\u003c=1000000","targetValue":0,"violated":true}]},{"displayName":"Error Rate","keySli":false,"passTargets":[{"criteria":"\u003c=+5%","targetValue":0,"violated":false},{"criteria":"\u003c2","targetValue":2,"violated":false}],"score":1,"status":"pass","value":{"metric":"error_rate","success":true,"value":0},"warningTargets":[{"criteria":"\u003c5","targetValue":5,"violated":false}]},{"displayName":"Process Heap Suspension","keySli":false,"passTargets":null,"score":0,"status":"info","value":{"metric":"pg_heap_suspension","success":true,"value":22.83852994441986},"warningTargets":null},{"displayName":"Process CPU Usage","keySli":false,"passTargets":null,"score":0,"status":"info","value":{"metric":"pg_cpu_usage","success":true,"value":1.490294337272644},"warningTargets":null}],"result":"fail","score":50,"sloFileContent":"DEDACTED","timeEnd":"2021-05-05T07:52:39.503Z","timeStart":"2021-05-05T07:49:28.091Z"},"labels":{"DtCreds":"dynatrace","art_version":"1.0.0-df4ff8","buildId":"1","buildNumber":"1","component":"api","jobname":"ace-demo/3. Test","joburl":"http://jenkins.DEDACTEDnip.io/job/ace-demo/job/3.%20Test/1/","part_of":"simplenode-app"},"project":"simplenodeproject","result":"fail","service":"simplenodeservice","stage":"staging","status":"succeeded"},"id":"531ba48d-78b7-4558-bda9-4c733310bb1a","source":"lighthouse-service","specversion":"1.0","time":"2021-05-05T07:53:42.067Z","type":"sh.keptn.event.evaluation.finished","shkeptncontext":"d164dc02-2c19-4057-9881-4b6fec5d0b1a","triggeredid":"0fd69b3e-cded-4d63-8408-085115028a48"}],"pageSize":20,"totalCount":1}

[pcjeffmac]

@kristofre
Can you provide more details?
This part of the code, is a simple error trap, if the return does not have any data.
Can you provide the Console output from Jenkins?

[kristofre]

@pcjeffmac, currently the statement looks for [], even if it is within a larger response body. The sample response body I added in the OP highlights that. So even though I got an evaluation result, because in there it contains [] the library would not exit the loop and keep on checking.

[pcjeffmac]

The issue seems to be related to the metric result and that the API is not returning a result,

Metric= "Response Time of InvokeAPI Method".

Can you share your SLI file? Are you able to pull values for this metric from the API directly?

"Dynatrace Metrics API returned 0 result values, expected 1 for query:
https://DEDACTED/e/DEDACTED/api/v2/metrics/query/?entitySelector=tag%
28keptn_project%3Asimplenodeproject%29%2Ctag%28keptn_stage%3Astaging%29%
2Ctag%28keptn_service%3Asimplenodeservice%29%2Ctype%28SERVICE%29
u0026from=1620200968000\u0026metricSelector=calc%3Aservice.simplenode.
staging%3Afilter%28eq%28method%2C%2Fapi%2Finvoke%29%29%3Amerge%280%29%
3Apercentile%2895%29\u0026resolution=Inf\u0026to=1620201159000.\nPlease
ensure the response contains exactly one value (e.g., by using
:merge(0):avg for the metric). Here is the output for troubleshooting:
{"metricId":"calc:service.simplenode.staging:filter(eq(method,/api/
invoke)):merge(0):percentile(95)","data":[]}","metric":"".

[pcjeffmac]

I have created a fix for this issue and the PR has been submitted.
In the mean time, if you can take a look at the metric result from the API. I think you would need to correct this calculated service metric and ensuring it returns one value.

Thank you for your input.

[kristofre]

The issue seems to be related to the metric result and that the API is not returning a result,

Metric= "Response Time of InvokeAPI Method".

Can you share your SLI file? Are you able to pull values for this metric from the API directly?

"Dynatrace Metrics API returned 0 result values, expected 1 for query:
https://DEDACTED/e/DEDACTED/api/v2/metrics/query/?entitySelector=tag%
28keptn_project%3Asimplenodeproject%29%2Ctag%28keptn_stage%3Astaging%29%
2Ctag%28keptn_service%3Asimplenodeservice%29%2Ctype%28SERVICE%29
u0026from=1620200968000\u0026metricSelector=calc%3Aservice.simplenode.
staging%3Afilter%28eq%28method%2C%2Fapi%2Finvoke%29%29%3Amerge%280%29%
3Apercentile%2895%29\u0026resolution=Inf\u0026to=1620201159000.\nPlease
ensure the response contains exactly one value (e.g., by using
:merge(0):avg for the metric). Here is the output for troubleshooting:
{"metricId":"calc:service.simplenode.staging:filter(eq(method,/api/
invoke)):merge(0):percentile(95)","data":[]}","metric":"".

Yes I am aware that one of the SLIs did not return a value, but all the other ones did. So the evaluation still yielded a build score and should not be stopped.

[christian-kreuzberger-dtx]

@kristofre has the original problem been solved? Can we close this issue?