Skip to content

Shutdown the executor service in KubernetesApplicationOperation and prevent NPE #6785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 3 commits into from
Closed

Shutdown the executor service in KubernetesApplicationOperation and prevent NPE #6785

wants to merge 3 commits into from

Conversation

turboFei
Copy link
Member

@turboFei turboFei commented Oct 28, 2024 •
edited
Loading

🔍 Description

Issue References 🔗

As title.

Fix NPE, because the cleanupTerminatedAppInfoTrigger will be set to null.

kyuubi/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala

Line 269 in d3520dd

cleanupTerminatedAppInfoTrigger = null

Also shutdown the ExecutorService when KubernetesApplicationOperation stoped.

Describe Your Solution 🔧

Shutdown the thread executor service and check the null.

Types of changes 🔖

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Behavior With This Pull Request 🎉

Related Unit Tests

Checklist 📝

Be nice. Be informative.

@turboFei turboFei changed the title Npe k8s Shutdown the executor service in KubernetesApplicationOperation and prevent NPE Oct 28, 2024
@turboFei turboFei requested a review from pan3793 October 28, 2024 20:02
@codecov-commenter
Copy link

codecov-commenter commented Oct 28, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (51b8e7b) to head (6afd052).
Report is 37 commits behind head on master.

Files with missing lines Patch % Lines
...kyuubi/engine/KubernetesApplicationOperation.scala 0.00% 10 Missing ⚠️
Additional details and impacted files 
@@          Coverage Diff           @@
##           master   #6785   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         688     688           
  Lines       42598   42605    +7     
  Branches     5807    5809    +2     
======================================
- Misses      42598   42605    +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

wForget
wForget reviewed Nov 1, 2024
View reviewed changes
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala
@@ -273,6 +274,16 @@ class KubernetesApplicationOperation extends ApplicationOperation with Logging {
Utils.tryLogNonFatalError(client.close())
}
kubernetesClients.clear()

if (expireCleanUpTriggerCacheExecutor != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may need to be placed before assigning cleanupTerminatedAppInfoTrigger to null.

kyuubi/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala

Line 267 in d3520dd

if (cleanupTerminatedAppInfoTrigger != null) {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we need to invalidate the cache before shutdown the executor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that, the remaining terminated application can be cleaned before server stop.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The non-null check here is not atomic, so NPE may still occur.

kyuubi/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala

Line 154 in 78c2a6c

if (cleanupTerminatedAppInfoTrigger == null) return

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about?

         Option(cleanupTerminatedAppInfoTrigger).foreach { trigger =>
            trigger.asMap().asScala.foreach {
              case (key, _) =>
                // do get to trigger cache eviction
                trigger.getIfPresent(key)
            }
          }

@github-actions GitHub Actions
Copy link

Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!

Thank you for using Kyuubi!

@github-actions github-actions bot added the Stale label Feb 14, 2025
@github-actions github-actions bot closed this Feb 14, 2025
@pan3793 pan3793 reopened this Feb 14, 2025
@pan3793 pan3793 removed the Stale label Feb 14, 2025
@turboFei turboFei requested a review from wForget February 16, 2025 19:25
wForget
wForget approved these changes Mar 20, 2025
View reviewed changes
Copy link
Member

@wForget wForget left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@wForget wForget mentioned this pull request Mar 20, 2025
Closed
4 tasks
@turboFei turboFei modified the milestones: v1.10.2, v1.9.4 Mar 23, 2025
@turboFei turboFei closed this in 338206e Mar 23, 2025
turboFei added a commit that referenced this pull request Mar 23, 2025
@turboFei
[KYUUBI #6785] Shutdown the executor service in KubernetesApplication…
e076e17 
…Operation and prevent NPE

# 🔍 Description
## Issue References 🔗

As title.

Fix NPE, because the cleanupTerminatedAppInfoTrigger will be set to `null`.
https://github.com/apache/kyuubi/blob/d3520ddbcea96ec55c525600126047c44c7adb35/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala#L269

Also shutdown the ExecutorService when KubernetesApplicationOperation stoped.
## Describe Your Solution 🔧

Shutdown the thread executor service and check the null.
## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6785 from turboFei/npe_k8s.

Closes #6785

6afd052 [Wang, Fei] comments
f0c3e31 [Wang, Fei] prevent npe
9dffe01 [Wang, Fei] shutdown

Authored-by: Wang, Fei <[email protected]>
Signed-off-by: Wang, Fei <[email protected]>
(cherry picked from commit 338206e)
Signed-off-by: Wang, Fei <[email protected]>
turboFei added a commit that referenced this pull request Mar 23, 2025
@turboFei
[KYUUBI #6785] Shutdown the executor service in KubernetesApplication…
0de8033 
…Operation and prevent NPE

# 🔍 Description
## Issue References 🔗

As title.

Fix NPE, because the cleanupTerminatedAppInfoTrigger will be set to `null`.
https://github.com/apache/kyuubi/blob/d3520ddbcea96ec55c525600126047c44c7adb35/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala#L269

Also shutdown the ExecutorService when KubernetesApplicationOperation stoped.
## Describe Your Solution 🔧

Shutdown the thread executor service and check the null.
## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6785 from turboFei/npe_k8s.

Closes #6785

6afd052 [Wang, Fei] comments
f0c3e31 [Wang, Fei] prevent npe
9dffe01 [Wang, Fei] shutdown

Authored-by: Wang, Fei <[email protected]>
Signed-off-by: Wang, Fei <[email protected]>
(cherry picked from commit 338206e)
Signed-off-by: Wang, Fei <[email protected]>
@turboFei
Copy link
Member Author

thanks, merged to master, branch-1.10 and branch-1.9

@turboFei turboFei deleted the npe_k8s branch March 23, 2025 20:20
@turboFei turboFei self-assigned this Mar 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wForget wForget wForget approved these changes

@pan3793 pan3793 Awaiting requested review from pan3793

Assignees

@turboFei turboFei

Labels
module:server
Projects
None yet
Milestone
v1.9.4
Development

Successfully merging this pull request may close these issues.
4 participants
@turboFei @codecov-commenter @wForget @pan3793