Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Update mantis 3rd party dependency specifications #349

Draft
wants to merge 26 commits into
base: master
Choose a base branch
from
Draft

[WIP] Update mantis 3rd party dependency specifications #349

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
4408fa0
Update mantis-common 3rd party dependency specifications
mabelbot Feb 28, 2023
091cd5c
Refactor mantis-common libraries.nettyEpoll
mabelbot Feb 28, 2023
05a20f3
Refactor mantis-common by removing ext variable
mabelbot Feb 28, 2023
2d27da4
Refactor mantis-connector-iceberg 3rd party dependency specifications
mabelbot Mar 1, 2023
fce4d3d
Refactor mantis-connector-job 3rd party dependency specifications
mabelbot Mar 1, 2023
ba469cc
Refactor mantis-connector-kafka 3rd party dependency specifications
mabelbot Mar 1, 2023
55fd2b8
Refactor mantis-control-plane-core 3rd party dependency specifications
mabelbot Mar 1, 2023
0140680
Refactor mantis-control-plane-server 3rd party dependency specifications
mabelbot Mar 1, 2023
8c5929d
Refactor by merging iceberg and akkaHttp deps
mabelbot Mar 1, 2023
34a352e
Refactor mantis-network 3rd party dependency specifications
mabelbot Mar 1, 2023
4926b28
Refactor mantis-publish-core 3rd party dependency specifications
mabelbot Mar 2, 2023
a5a1161
Refactor mantis-publish-netty 3rd party dependency specifications
mabelbot Mar 2, 2023
ea7a19f
Refactor mantis-publish-netty-guice 3rd party dependency specifications
mabelbot Mar 2, 2023
60d8cf2
Refactor mantis-remote-observable 3rd party dependency specifications
mabelbot Mar 2, 2023
42e0ccb
Refactor mantis-server-worker 3rd party dependency specifications
mabelbot Mar 2, 2023
1ddb23d
Refactor mantis-server-worker-client 3rd party dependency specifications
mabelbot Mar 2, 2023
3f1770b
Refactor mantis-shaded 3rd party dependency specifications
mabelbot Mar 2, 2023
ee2a3b4
Refactor mantis-source-job-kafka 3rd party dependency specifications
mabelbot Mar 2, 2023
4a44dfc
Refactor mantis-examples-groupby-sample 3rd party dependency specific…
mabelbot Mar 2, 2023
f74ba90
Refactor mantis-examples-jobconnector-sample 3rd party dependency spe…
mabelbot Mar 2, 2023
bc0ad96
Refactor mantis-examples-mantis-publish-sample 3rd party dependency s…
mabelbot Mar 2, 2023
d90bad4
Refactor mantis-examples-mantis-publish-web-sample 3rd party dependen…
mabelbot Mar 2, 2023
30a65ce
Refactor mantis-examples-sine-function 3rd party dependency specifica…
mabelbot Mar 2, 2023
720f4cd
Refactor mantis-examples-synthetic-sourcejob 3rd party dependency spe…
mabelbot Mar 2, 2023
4c82455
Refactor mantis-examples-twitter-sample 3rd party dependency specific…
mabelbot Mar 2, 2023
100f073
Refactor mantis-examples-wordcount 3rd party dependency specifications
mabelbot Mar 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ ext.libraries = [
],
hadoopCommon : "org.apache.hadoop:hadoop-common:${versions.hadoop}",
hadoopS3 : "org.apache.hadoop:hadoop-aws:${versions.hadoop}",
hamcrest : "org.hamcrest:hamcrest-core:1.3",
jcip : "net.jcip:jcip-annotations:1.0",
jctools : "org.jctools:jctools-core:${versions.jctoolsVersion}",
junitJupiter : [
"org.junit.jupiter:junit-jupiter-api:${versions.junit5}",
"org.junit.jupiter:junit-jupiter-engine:${versions.junit5}",
Expand All @@ -73,10 +76,13 @@ ext.libraries = [
mockitoCore : "org.mockito:mockito-core:${versions.mockito}",
mockitoCore3 : "org.mockito:mockito-core:${versions.mockito3}",
mockneat : "net.andreinc:mockneat:0.4.8",
nettyBuffer : "io.netty:netty-buffer:${versions.nettyVersion}",
nettyCodec : "io.netty:netty-codec-http:${versions.nettyVersion}",
rxJava : "io.reactivex:rxjava:1.3.8",
rxNettyShaded : "com.netflix:mantis-rxnetty:0.4.19.1",
slf4jApi : "org.slf4j:slf4j-api:${versions.slf4j}",
slf4jLog4j12 : "org.slf4j:slf4j-log4j12:${versions.slf4j}",
snappyJava : "org.xerial.snappy:snappy-java:${versions.snappyVersion}",
spectatorApi : "com.netflix.spectator:spectator-api:${versions.spectator}",
spotifyFutures : "com.spotify:completable-futures:0.3.1",
vavr : "io.vavr:vavr:${versions.vavr}",
Expand Down
12 changes: 6 additions & 6 deletions mantis-common/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ ext {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ext variable can be safely removed with this change.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a concern I had earlier, there were different versions of the same dependencies across different modules (so I kept this part that specifies the specific versions). Would it be better to standardize the versions and move a single version to the root build.gradle's ext.versions?

Copy link
Collaborator

@hmitnflx hmitnflx Feb 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, that'd be desired to realize the full benefits of this work.
The different versions would make it tricky and here's a good starting tip:

  1. For major version difference, Create a new library variable.
  2. For minor and patch, try to bump to the latest version for all subprojects (one variable per library and see if builds correctly).

You could call out the places where you are bumping the minor and patch version in the PR for an easier review.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you! As some other GSoC applicants are having trouble with builds failing (including the Mantis tutorial) before we made any changes, I brought this up to Sundaram yesterday and iirc he said that right now there wasn't a good way for us to validate whether these "good first issue" PRs aren't going to break something - but that it will be fixed by end of March. If needed, I can provide the exact error on Discord so maybe we can get ./gradlew build to work?

The only thing that succeeds at building correctly locally is the synthetic-sourcejob example, since I found a workaround that can allow it to build temporarily. I have been using that to test but it's not ideal.

dependencies {
api project(":mantis-common-serde")
api "io.netty:netty-codec-http:$nettyVersion"
api "io.netty:netty-buffer:$nettyVersion"
api libraries.nettyCodec
api libraries.nettyBuffer
api group: 'io.netty', name: 'netty-transport-native-epoll', classifier: 'linux-x86_64', version: nettyVersion
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would recommend moving this to libraries.nettyEpoll as well even if it's a little tricky.
It would look like this - io.netty:netty-transport-native-epoll:4.1.17.Final:linux-x86_64

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, will do ASAP! Thank you.

api "org.xerial.snappy:snappy-java:$snappyVersion"
api "org.jctools:jctools-core:$jctoolsVersion"
api libraries.snappyJava
api libraries.jctools

// spectatorApi should be packaged at entry point level to avoid version conflicts.
compileOnly libraries.spectatorApi
Expand All @@ -39,11 +39,11 @@ dependencies {
api libraries.slf4jApi
api libraries.slf4jLog4j12
implementation libraries.commonsIo
implementation 'net.jcip:jcip-annotations:1.0'
implementation libraries.jcip

testImplementation libraries.spectatorApi
testImplementation libraries.commonsLang3
testImplementation "org.hamcrest:hamcrest-core:1.3"
testImplementation libraries.hamcrest
testImplementation libraries.junit4
testImplementation libraries.mockitoCore
}