Merge Single-Stream and HC Detector Profiling Workflows #1237
Conversation
…lows Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner. During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%. I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process. Testing: * Verified that the single-stream detector works correctly with the updated profile runner. Signed-off-by: Kaituo Li <[email protected]>
kaituo
requested review from
jmazanec15,
jngz-es,
saratvemulapalli,
ohltyler,
vamshin,
VijayanB,
ylwu-amzn,
amitgalitz,
jackiehanyang,
sean-zheng-amazon,
dbwiddis,
owaiskazi19 and
joshpalis
as code owners
|
flaky test. Will fix in the following PR due to time constraint. REPRODUCE WITH: gradlew ':test' --tests "org.opensearch.ad.ml.EntityColdStarterTests.testCacheReleaseAfterClear" -Dtests.seed=7E8B1D478E1896C1 -Dtests.security.manager=false -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ja-JP -Dtests.timezone=Australia/Victoria -Druntime.java=21 org.opensearch.ad.ml.EntityColdStarterTests > testCacheReleaseAfterClear FAILED 1> [2024-06-12T07:56:10,878][INFO ][o.o.a.m.EntityColdStarterTests] [testCacheReleaseAfterClear] before test |
…lows (#1237) Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner. During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%. I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process. Testing: * Verified that the single-stream detector works correctly with the updated profile runner. Signed-off-by: Kaituo Li <[email protected]> (cherry picked from commit 54b842b) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…lows (#1237) (#1239) Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner. During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%. I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process. Testing: * Verified that the single-stream detector works correctly with the updated profile runner. (cherry picked from commit 54b842b) Signed-off-by: Kaituo Li <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…lows (opensearch-project#1237) (opensearch-project#1239) Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner. During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%. I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process. Testing: * Verified that the single-stream detector works correctly with the updated profile runner. (cherry picked from commit 54b842b) Signed-off-by: Kaituo Li <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…lows (#1237) (#1239) (#1242) Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner. During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%. I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process. Testing: * Verified that the single-stream detector works correctly with the updated profile runner. (cherry picked from commit 54b842b) Signed-off-by: Kaituo Li <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
Since version 2.15, we have merged the single-stream and HC detector workflows. Consequently, separate logic for profiling is no longer necessary. This PR removes single-stream specific logic from the profile runner.
During a Blue/Green (B/G) deployment, if an old node communicates with a new node regarding an old model, we will not execute RCFPollingAction to determine model updates. However, we have fallback logic that checks for anomaly results. If any results are found, the initialization progress is set to 100%.
I have retained the old AnomalyDetectorProfileRunner temporarily to facilitate writing tests for the legacy logic. This includes keeping RCFPollingAction and ModelManager.getTotalUpdates to handle requests from old nodes during the B/G deployment process.
Testing:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.