Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature] (Metric) add jni metrics for jdbc connection #41752

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

[feature] (Metric) add jni metrics for jdbc connection #41752

wants to merge 18 commits into from

Conversation

qzsee
Copy link
Contributor

@qzsee qzsee commented Oct 12, 2024 •
edited
Loading

Proposed changes

jdbc scan and java udf use jni calls in BE, but lack a monitoring interface. This pr adds a monitoring interface and adds a monitoring of the number of jdbc connections

set enable_jni_metrics = true in be.conf

when we have a query of jdbc catalog , we can get a metric

http://172.17.0.1:8040/metrics?type=json

{"tags":{"metric":"jdbc_scan_connection_percent","catalog":"35717"},"unit":"percent","value":0}

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actions bot reviewed Oct 12, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
be/src/util/jni_metrics.h Show resolved Hide resolved
be/src/util/jni_metrics.h Outdated Show resolved Hide resolved
github-actions[bot]
github-actions bot reviewed Oct 12, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.h Outdated Show resolved Hide resolved
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.42% (9681/25871)
Line Coverage: 28.68% (80303/280011)
Region Coverage: 28.11% (41526/147719)
Branch Coverage: 24.70% (21111/85484)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f9ecc888d02c5fd389fe577c669aed36c44064e5_f9ecc888d02c5fd389fe577c669aed36c44064e5/report/index.html

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.45% (9710/25925)
Line Coverage: 28.72% (80584/280569)
Region Coverage: 28.17% (41691/147990)
Branch Coverage: 24.75% (21200/85652)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f29832c88971c35c4f6b39eabd3c9d2458a4617e_f29832c88971c35c4f6b39eabd3c9d2458a4617e/report/index.html

github-actions[bot]
github-actions bot reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.cpp Show resolved Hide resolved
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.44% (9707/25925)
Line Coverage: 28.71% (80562/280578)
Region Coverage: 28.16% (41681/148000)
Branch Coverage: 24.75% (21197/85654)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f3a3809458ab7bebd172d345d0da0e7de212d822_f3a3809458ab7bebd172d345d0da0e7de212d822/report/index.html

hubgeter
hubgeter reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@hubgeter hubgeter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution

be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
be/src/util/jni_metrics.cpp Show resolved Hide resolved
be/src/util/jni_metrics.cpp Show resolved Hide resolved
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcDataSource.java Show resolved Hide resolved
github-actions[bot]
github-actions bot reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
be/src/util/jni_metrics.h Outdated Show resolved Hide resolved
github-actions[bot]
github-actions bot reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.cpp Show resolved Hide resolved
github-actions[bot]
github-actions bot reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
@qzsee qzsee force-pushed the add_jni_metrics branch 3 times, most recently from ba683eb to ff2a4ae Compare October 16, 2024 03:21
@qzsee
Copy link
Contributor Author

qzsee commented Oct 16, 2024

@hubgeter please review again,thks

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9710/25921)
Line Coverage: 28.73% (80642/280659)
Region Coverage: 28.17% (41712/148074)
Branch Coverage: 24.75% (21209/85708)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3649fb0064cfe4a8e2d2846ff75cbc904170c27a_3649fb0064cfe4a8e2d2846ff75cbc904170c27a/report/index.html

@qzsee
Copy link
Contributor Author

qzsee commented Oct 16, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9710/25923)
Line Coverage: 28.72% (80610/280705)
Region Coverage: 28.16% (41712/148113)
Branch Coverage: 24.74% (21212/85726)
Coverage Report: http://coverage.selectdb-in.cc/coverage/36f124d59c96e2b21a964d62076a1a22dfe0ba24_36f124d59c96e2b21a964d62076a1a22dfe0ba24/report/index.html

hubgeter
hubgeter reviewed Nov 3, 2024
View reviewed changes
be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
be/src/util/jni_metrics.cpp Outdated Show resolved Hide resolved
be/src/util/jni_metrics.cpp Show resolved Hide resolved
garenshi added 3 commits November 4, 2024 16:11
@qzsee
Copy link
Contributor Author

qzsee commented Nov 14, 2024

run buildall

@qzsee
Copy link
Contributor Author

qzsee commented Nov 15, 2024

run buildall

@morningman morningman closed this Nov 15, 2024
@morningman morningman reopened this Nov 15, 2024
@morningman
Copy link
Contributor

run buildall

@qzsee
Copy link
Contributor Author

qzsee commented Nov 15, 2024

run buildall

@qzsee qzsee closed this Nov 15, 2024
@qzsee qzsee reopened this Nov 15, 2024
@qzsee
Copy link
Contributor Author

qzsee commented Nov 15, 2024

run buildall

@qzsee
Copy link
Contributor Author

qzsee commented Nov 19, 2024

run buildall

1 similar comment
@hello-stephen
Copy link
Contributor

run buildall

@qzsee
Copy link
Contributor Author

qzsee commented Nov 19, 2024

run buildall

1 similar comment
@shuke987
Copy link
Collaborator

run buildall

@qzsee
Copy link
Contributor Author

qzsee commented Nov 20, 2024

run buildall

@dqz123
Copy link

dqz123 commented Dec 10, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.79% (10108/26056)
Line Coverage: 29.70% (84758/285395)
Region Coverage: 28.76% (43506/151257)
Branch Coverage: 25.32% (22101/87288)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0f4514bb3b050cc38d9221762c2fc27dbe20462d_0f4514bb3b050cc38d9221762c2fc27dbe20462d/report/index.html

@dqz123
Copy link

dqz123 commented Dec 11, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.79% (10108/26056)
Line Coverage: 29.69% (84735/285399)
Region Coverage: 28.77% (43512/151264)
Branch Coverage: 25.32% (22100/87292)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0f4514bb3b050cc38d9221762c2fc27dbe20462d_0f4514bb3b050cc38d9221762c2fc27dbe20462d/report/index.html

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.79% (10108/26056)
Line Coverage: 29.70% (84766/285399)
Region Coverage: 28.76% (43508/151264)
Branch Coverage: 25.32% (22103/87292)
Coverage Report: http://coverage.selectdb-in.cc/coverage/99955f5c33ea3013f0954fa495ef5b54f2ff68e4_99955f5c33ea3013f0954fa495ef5b54f2ff68e4/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@hubgeter hubgeter hubgeter approved these changes

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@qzsee @doris-robot @hubgeter @morningman @hello-stephen @shuke987 @dqz123