Need x86/mips/armeabi/armv8 binaries for Android #54

karthikrg · 2016-01-03T08:44:05Z

Only armv7-eabi is present in the published artifact. Also, I can't see any easy way to build it myself.

krisskross · 2016-01-03T13:56:09Z

There are instructions on how to build Android from source here. If you need to build a different binary you must modify the agcc file before building.

krisskross · 2016-01-03T15:40:48Z

To be honest, i'm not sure what the most common platform is. We could have multiple platform releases if that make sense.

What's your experience and thoughts? Should we start building for another platform?

karthikrg · 2016-01-03T22:15:38Z

I can't figure out how to modify the agcc file to build for other archs. Can you help me here?

Normally Android NDK projects are built by specifying archs in Android.mk. Android supports the following architectures:

arm64-v8a
armeabi-v7a
armeabi
mips
mips64
x86
x86_64

The app I'm planning to integrate this library in has users with devices on all these architectures.

krisskross · 2016-01-04T08:59:59Z

I noticed that a new flag is needed for building LMDB for android, -DMDB_USE_ROBUST=0.

Here's a x86_64 configuration.

$NDK/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin/x86_64-linux-android-gcc --sysroot=$NDK/platforms/android-21/arch-x86_64 -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

Here's a ARM configuration.

$NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc --sysroot=$NDK/platforms/android-21/arch-arm -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

I was expecting this MIPS configuration to work, but there are compilation errors in hawtjni that i'm not sure how to fix.

$NDK/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/bin/mips64el-linux-android-gcc --sysroot=$NDK/platforms/android-21/arch-mips64 -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

karthikrg · 2016-01-04T09:07:08Z

This is awesome. Let me try and see if i can fix the mips issues. Thanks!

On Monday, January 4, 2016, Kristoffer Sjögren [email protected]
wrote:

I noticed that a new flag is needed for building LMDB for android,
-DMDB_USE_ROBUST=0.

Here's a x86_64 configuration.

$NDK/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin/x86_64-linux-android-gcc --sysroot=$NDK/platforms/android-21/arch-x86_64 -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

Here's a ARM configuration.

$NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc --sysroot=$NDK/platforms/android-21/arch-arm -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

I was expecting this MIPS configuration to work, but there are compilation
errors in hawtjni that i'm not sure how to fix.

$NDK/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/bin/mips64el-linux-android-gcc --sysroot=$NDK/platforms/android-21/arch-mips64 -DMDB_DSYNC=O_SYNC -DMDB_USE_ROBUST=0 $@

—
Reply to this email directly or view it on GitHub
#54 (comment).

krisskross · 2016-01-04T19:45:23Z

Did you find the cause of the MIPS compilation errors?

karthikrg · 2016-01-04T23:21:04Z

Didn't even get so far. Configure fails for me:

[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ lmdbjni-android ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ lmdbjni-android ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-hawtjni-plugin:1.11:build (default) @ lmdbjni-android ---
[INFO] Extracting /Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni/target/lmdbjni-0.4.6-SNAPSHOT-native-src.zip to /Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/target/native-build-extracted
[INFO] executing: /bin/sh -c ./configure --disable-ccache --prefix=/Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/target/native-build/target --with-lmdb=/Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/../lmdbjni/src/main/native-package/src CC=/Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/../lmdbjni/src/main/native-package/agcc --host=x86_64-unknown-linux-gnu
[INFO] configure: WARNING: unrecognized options: --disable-ccache
[INFO] checking build system type... i386-apple-darwin14.5.0
[INFO] checking host system type... x86_64-unknown-linux-gnu
[INFO] checking target system type... x86_64-unknown-linux-gnu
[INFO] checking for x86_64-unknown-linux-gnu-gcc... /Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/../lmdbjni/src/main/native-package/agcc
[INFO] checking whether the C compiler works... no
[INFO] configure: error: in /Users/kramgopa/projects/lmdbjni/lmdbjni/lmdbjni-android/target/native-build': [INFO] configure: error: C compiler cannot create executables [INFO] Seeconfig.log' for more details
[INFO] rc: 77
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] lmdbjni-project .................................... SUCCESS [ 3.535 s]
[INFO] lmdbjni ............................................ SUCCESS [ 9.611 s]
[INFO] lmdbjni-android .................................... FAILURE [ 1.807 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.062 s
[INFO] Finished at: 2016-01-04T15:19:35-08:00
[INFO] Final Memory: 30M/311M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.fusesource.hawtjni:maven-hawtjni-plugin:1.11:build (default) on project lmdbjni-android: build failed: org.apache.maven.plugin.MojoExecutionException: ./configure failed with exit code: 77 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :lmdbjni-android

karthikrg · 2016-01-05T02:53:49Z

I think the reason is that android defines cacheflush differently than standard linux for MIPS. mdb.c uses this.

https://android.googlesource.com/platform/bionic/+/1e980b6/libc/arch-mips/bionic/cacheflush.c

I think redefining cacheflush to use the GNU __clear_cache should do the trick.
#define CACHEFLUSH(start, length, flags) __clear_cache(start, (char *) (start + length))

karthikrg · 2016-01-05T02:54:29Z

Also FWIW, I hated hawtjni so I pulled out the code, edited it a bit and made it a regular Android NDK project. I'll post the source once its tested and ready on github.

krisskross · 2016-01-05T13:00:59Z

Yes, I also ran into the clear_cache issue.

Glad to hear you made it work. Interesting to see the final result when you're done. If possible, maybe we can include the changes into lmdbjni?

krisskross · 2016-02-02T17:14:36Z

I was just curious if you managed to get this working?

karthikrg · 2016-02-02T17:46:33Z

Yep. I did. It's involved significant code changes though. I've also added
code to pool buffers in java and avoid byte array garbage; and a zero copy
dma buffer implementation that works on Android.

I'm running it through my company's open source review team before getting
it out there. Due to the nature of these processes may take about a month
or so to get it out there.

Thanks,
Karthik

On Tuesday, February 2, 2016, Kristoffer Sjögren [email protected]
wrote:

I was just curious if you managed to get this working?

—
Reply to this email directly or view it on GitHub
#54 (comment).

krisskross · 2016-02-02T19:45:12Z

Cool, I had some problems earlier to get direct byte buffers working on android due to (I think) memory alignment #21. Would be interesting to hear how you solved that.

I'll keep this issue open until later.

karthikrg · 2016-02-02T19:46:41Z

Yeah unsafe doesn't work on Android. I implemented it by providing a java.nio.Buffer implementation backed by a JNI allocated byte array.

phraktle · 2016-02-03T12:52:14Z

Using ByteBuffers could also resolve my other request, #42.

krisskross · 2016-02-03T13:44:33Z

Did you use NewByteArray or NewDirectByteBuffer? As far as I understand, byte arrays are allocated on the heap?

karthikrg · 2016-02-03T16:32:37Z

My buffer implementation uses malloc directly. Direct byte buffer alloc is
very (50-100x compared to new byte[]) slow in Android, and dalvik/art still
track their allocations via a associated pointer.

On Wednesday, February 3, 2016, Kristoffer Sjögren [email protected]
wrote:

Did you use NewByteArray or NewDirectByteBuffer? As far as I understand,
byte arrays are allocated on the heap?

—
Reply to this email directly or view it on GitHub
#54 (comment).

kittinunf · 2016-07-18T05:21:31Z

Do you guys have any trick to make compilation successful on mips?

../../../vendors/lmdb/libraries/liblmdb/mdb.c:76:12: error: conflicting types for 'cacheflush'
extern int cacheflush(char *addr, int nbytes, int cache);
           ^
/Users/kittinunf/Library/Android/sdk/ndk-bundle/platforms/android-16/arch-mips/usr/include/unistd.h:183:12: note: previous declaration is here
extern int cacheflush(long start, long end, long flags);

karthikrg · 2016-07-18T05:23:12Z

Make it a macro as I described above and use it!

kittinunf · 2016-07-18T06:26:10Z

@karthikrg Sorry I must have missed that last line. Yeah, thanks it works!

krisskross added a commit that referenced this issue Mar 21, 2016

Dispose comparator on database close, #54

9a29e34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need x86/mips/armeabi/armv8 binaries for Android #54

Need x86/mips/armeabi/armv8 binaries for Android #54

karthikrg commented Jan 3, 2016

krisskross commented Jan 3, 2016

krisskross commented Jan 3, 2016

karthikrg commented Jan 3, 2016

krisskross commented Jan 4, 2016

karthikrg commented Jan 4, 2016

krisskross commented Jan 4, 2016

karthikrg commented Jan 4, 2016

karthikrg commented Jan 5, 2016

karthikrg commented Jan 5, 2016

krisskross commented Jan 5, 2016

krisskross commented Feb 2, 2016

karthikrg commented Feb 2, 2016

krisskross commented Feb 2, 2016

karthikrg commented Feb 2, 2016

phraktle commented Feb 3, 2016

krisskross commented Feb 3, 2016

karthikrg commented Feb 3, 2016

kittinunf commented Jul 18, 2016 •

edited

Loading

karthikrg commented Jul 18, 2016

kittinunf commented Jul 18, 2016 •

edited

Loading

Need x86/mips/armeabi/armv8 binaries for Android #54

Need x86/mips/armeabi/armv8 binaries for Android #54

Comments

karthikrg commented Jan 3, 2016

krisskross commented Jan 3, 2016

krisskross commented Jan 3, 2016

karthikrg commented Jan 3, 2016

krisskross commented Jan 4, 2016

karthikrg commented Jan 4, 2016

krisskross commented Jan 4, 2016

karthikrg commented Jan 4, 2016

karthikrg commented Jan 5, 2016

karthikrg commented Jan 5, 2016

krisskross commented Jan 5, 2016

krisskross commented Feb 2, 2016

karthikrg commented Feb 2, 2016

krisskross commented Feb 2, 2016

karthikrg commented Feb 2, 2016

phraktle commented Feb 3, 2016

krisskross commented Feb 3, 2016

karthikrg commented Feb 3, 2016

kittinunf commented Jul 18, 2016 • edited Loading

karthikrg commented Jul 18, 2016

kittinunf commented Jul 18, 2016 • edited Loading

kittinunf commented Jul 18, 2016 •

edited

Loading

kittinunf commented Jul 18, 2016 •

edited

Loading