Skip to content

Use most recent/portable versions of LevelDB and Snappy dependencies #1056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Oct 18, 2019 · 7 comments · Fixed by #1184
Closed

Use most recent/portable versions of LevelDB and Snappy dependencies #1056

ghost opened this issue Oct 18, 2019 · 7 comments · Fixed by #1184
Labels
type: task Generic non-code related tasks

Comments

@ghost
Copy link

ghost commented Oct 18, 2019

Currently the most recent version of leveldb-sys crate that we use for disk buffers depends on LevelDB 1.18. The latest upstream release of LevelDB is 1.22 (changelog).

In particular, version 1.21 added native Windows support, which could be very useful for #880.

Also, as mentioned in #1054, the version 1.1.2 of Snappy used by leveldb-sys has an Autotools-based build scripts, which make it less flexible about cross-compilation (in particular, a special flag has to be passed to the configure script, and levelb-sys build script doesn't support passing it). The most recent version 1.1.7 of Snappy supports CMake, which supports reading of build configuration from environment variables, so updating it could have simplified building for ARM.

Thus, it would be helpful to have an updated version of leveldb-sys crate supporting most recent versions of LevelDB and Snappy in order to support more targets.

@ghost ghost added domain: operations needs: approval Needs review & approval before work can begin. labels Oct 18, 2019
@LucioFranco
Copy link
Contributor

Just to note the author of leveldb has offered to give us access to the repo and apply these changes so that we are not blocked.

@binarylogic
Copy link
Contributor

It sounds like we should definitely do this.

@LucioFranco can you have him give us access? Is there any we didn't take him up on that?

@binarylogic binarylogic added type: task Generic non-code related tasks and removed needs: approval Needs review & approval before work can begin. labels Oct 18, 2019
@LucioFranco
Copy link
Contributor

@binarylogic I say let us discuss this more on Monday but I was not sure if we wanted to take up more work with leveldb yet so I wanted to have a discussion with the rest of the team before we commit to something like this. If we agree I will shoot an email over to the author.

@binarylogic
Copy link
Contributor

What is the alternative? I do not want to block this issue to move off of leveldb. Moving off of leveldb will likely be a larger project addressed in https://github.com/timberio/vector/milestone/23; it is not something I want to rush or do haphazardly.

@binarylogic
Copy link
Contributor

binarylogic commented Oct 24, 2019

@a-rodin just documenting a discussion in Slack. We'd like to fork the leveldb-sys repo for now and make the changes. We will get the team added as contributors so that we have commit rights to the repo.

@ghost
Copy link
Author

ghost commented Oct 25, 2019

Note that we need to add CMake to list of our development requirements with this upgrade, as newer versions of LevelDB use it as the build system.

@ghost
Copy link
Author

ghost commented Nov 14, 2019

I think that for now we can keep the same version of leveldb-sys crate to not require everyone to install CMake, but use updated leveldb crate from our fork which can be compiled on ARM.

@ghost ghost changed the title Use most recent versions of LevelDB and Snappy dependencies Use most recent/portable versions of LevelDB and Snappy dependencies Nov 14, 2019
@ghost ghost closed this as completed in #1184 Nov 14, 2019
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: task Generic non-code related tasks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants