Fix faulty kserve tests #1005

bdattoma · 2023-11-02T16:05:55Z

KServe (aka LLM or WatsonX) has some failing tests. This PR wants to fix them.

As part of the test fixing, I had to do some refactoring and remove yq from poetry.

yq dependency installation should be handled in a different PR to solve #953

github-advanced-security

Robocop found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

github-actions · 2023-11-02T16:10:02Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %
373	0	0	373	100

bdattoma · 2023-11-03T10:31:43Z

PR validation

rhods-ci-pr-test/2100 a better run, 2 failures remaining ODS-2377 & ODS-2380
rhods-ci-pr-test/2101 1 failure out 16 because of a mistake in teardown. Re-running only ODS-2380 rhods-ci-pr-test/2103

Signed-off-by: bdattoma <[email protected]>

jstourac · 2023-11-03T12:01:35Z

@bdattoma please, there seem to be multiple trailing whitespace lines reported by robocop and consider adding the doc to the one keyword robocop complan. 🙂

Signed-off-by: bdattoma <[email protected]>

bdattoma · 2023-11-03T12:21:51Z

@bdattoma please, there seem to be multiple trailing whitespace lines reported by robocop and consider adding the doc to the one keyword robocop complan. 🙂

done

jstourac

Thank you, put some comments but otherwise LGTM.

ods_ci/tests/Tests/400__ods_dashboard/420__model_serving/422__model_serving_llm.robot

pyproject.toml

Signed-off-by: bdattoma <[email protected]>

sonarcloud · 2023-11-03T14:18:20Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
1 Security Hotspot
0 Code Smells

No Coverage information
0.0% Duplication

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

ods_ci/tests/Tests/400__ods_dashboard/420__model_serving/422__model_serving_llm.robot

+        ...    insecure=${TRUE}    skip_res_json=${skip_json_load_response}
+        ...    &{args}
+        Log    ${res}
+        IF    ${validate_response} == ${TRUE}


ods_ci/tests/Tests/400__ods_dashboard/420__model_serving/422__model_serving_llm.robot

+        ${status}=    Run Keyword And Return Status
+        ...    Query Model Multiple Times    isvc_name=${isvc_name}    model_name=${model_name}    n_times=1
+        ...    namespace=${namespace}
+        IF    ${status} == ${TRUE}


ods_ci/tests/Tests/400__ods_dashboard/420__model_serving/422__model_serving_llm.robot

+    [Arguments]    ${model_name}    ${namespace}    ${isvc_name}=${model_name}
+    ...            ${endpoint}=${CAIKIT_ALLTOKENS_ENDPOINT}    ${n_times}=10
+    ...            ${streamed_response}=${FALSE}    ${query_idx}=0    ${validate_response}=${TRUE}    &{args}
+    IF    ${validate_response} == ${FALSE}


* refactor query keyword Signed-off-by: bdattoma <[email protected]> * fix canary test + needed refactor Signed-off-by: bdattoma <[email protected]> * increase timeout for minreplicas change Signed-off-by: bdattoma <[email protected]> * revert erroneus edits Signed-off-by: bdattoma <[email protected]> * fix typo Signed-off-by: bdattoma <[email protected]> * fix skip json load logic Signed-off-by: bdattoma <[email protected]> * increase query number in canary test (attempt) Signed-off-by: bdattoma <[email protected]> * increase queries for autoscale stability (attempt) Signed-off-by: bdattoma <[email protected]> * fix autoscaling test Signed-off-by: bdattoma <[email protected]> * remove old yq from poetry - fix hw req test Signed-off-by: bdattoma <[email protected]> * remove unused variable Signed-off-by: bdattoma <[email protected]> * fix robocop alerts and add documentations Signed-off-by: bdattoma <[email protected]> * fix suite doc Signed-off-by: bdattoma <[email protected]> * remove unused variable Signed-off-by: bdattoma <[email protected]> * fix teardown of custom requests/limits Signed-off-by: bdattoma <[email protected]> * fix line too long alert Signed-off-by: bdattoma <[email protected]> --------- Signed-off-by: bdattoma <[email protected]>

bdattoma self-assigned this Nov 2, 2023

bdattoma added needs testing Needs to be tested in Jenkins enhancements Bugfixes, enhancements, refactoring, ... in tests or libraries (PR will be listed in release-notes) labels Nov 2, 2023

github-advanced-security bot found potential problems Nov 2, 2023

View reviewed changes

bdattoma added 10 commits November 3, 2023 12:49

refactor query keyword

2136687

Signed-off-by: bdattoma <[email protected]>

fix canary test + needed refactor

1276aa6

Signed-off-by: bdattoma <[email protected]>

increase timeout for minreplicas change

4a07b56

Signed-off-by: bdattoma <[email protected]>

revert erroneus edits

3a8fd69

Signed-off-by: bdattoma <[email protected]>

fix typo

112389f

Signed-off-by: bdattoma <[email protected]>

fix skip json load logic

e7655cf

Signed-off-by: bdattoma <[email protected]>

increase query number in canary test (attempt)

20b314e

Signed-off-by: bdattoma <[email protected]>

increase queries for autoscale stability (attempt)

f3f5fa4

Signed-off-by: bdattoma <[email protected]>

fix autoscaling test

59ae659

Signed-off-by: bdattoma <[email protected]>

remove old yq from poetry - fix hw req test

948f499

Signed-off-by: bdattoma <[email protected]>

bdattoma force-pushed the fix_kserve_tests branch from 7d86553 to 948f499 Compare November 3, 2023 11:49

bdattoma requested review from tarukumar, jgarciao, jstourac, lugi0 and FedeAlonso November 3, 2023 11:52

bdattoma added 3 commits November 3, 2023 13:04

remove unused variable

1b7993a

Signed-off-by: bdattoma <[email protected]>

fix robocop alerts and add documentations

1f2c481

Signed-off-by: bdattoma <[email protected]>

fix suite doc

02b58bc

Signed-off-by: bdattoma <[email protected]>

jstourac requested changes Nov 3, 2023

View reviewed changes

remove unused variable

f4339e2

Signed-off-by: bdattoma <[email protected]>

bdattoma requested a review from jstourac November 3, 2023 14:09

bdattoma added 2 commits November 3, 2023 15:10

fix teardown of custom requests/limits

d8e04a3

Signed-off-by: bdattoma <[email protected]>

fix line too long alert

469b51b

Signed-off-by: bdattoma <[email protected]>

github-advanced-security bot found potential problems Nov 3, 2023

View reviewed changes

jstourac approved these changes Nov 3, 2023

View reviewed changes

FedeAlonso approved these changes Nov 6, 2023

View reviewed changes

bdattoma merged commit 1233b85 into red-hat-data-services:master Nov 6, 2023
9 of 10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix faulty kserve tests #1005

Fix faulty kserve tests #1005

bdattoma commented Nov 2, 2023 •

edited

Loading

github-advanced-security bot left a comment

github-actions bot commented Nov 2, 2023

bdattoma commented Nov 3, 2023 •

edited

Loading

jstourac commented Nov 3, 2023 •

edited

Loading

bdattoma commented Nov 3, 2023

jstourac left a comment

sonarcloud bot commented Nov 3, 2023

Fix faulty kserve tests #1005

Fix faulty kserve tests #1005

Conversation

bdattoma commented Nov 2, 2023 • edited Loading

github-advanced-security bot left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 2, 2023

Robot Results

bdattoma commented Nov 3, 2023 • edited Loading

jstourac commented Nov 3, 2023 • edited Loading

bdattoma commented Nov 3, 2023

jstourac left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Nov 3, 2023

bdattoma commented Nov 2, 2023 •

edited

Loading

bdattoma commented Nov 3, 2023 •

edited

Loading

jstourac commented Nov 3, 2023 •

edited

Loading