[FR] make it possible to exec from an APK similar to how we can dlopen from an APK?

[DanAlbert]

Not the right bug tracker, but we don't have a public buganizer component for the loader atm so filing this here for visibility.

dlopen will automatically search the app's install directory, but I don't think it will search within the APK. Given that it would be best for app developers to move toward loading libraries directly from the APK, we should make this as painless as possible.

[alexcohn]

My experiments show that dlopen("libQQ", RTLD_NOW) simply works:

019-08-18 13:26:16.397 22073-22073/? E/linker: "/data/app/com.example.noextract/base.apk!/lib/armeabi-v7a/libQQ.so": ignoring DT_PREINIT_ARRAY in shared library!
2019-08-18 13:26:16.397 22073-22073/? I/dl: libQQ = 0x4da764b

The log above comes from an Android N device. I have also a proof that this works on Android Q x86-64 emulator.

[alexcohn]

What would be nice in the same direction, is to make it possible to exec() from /data/app/com.example.noextract/base.apk!/assets/armeabi-v7a/ffmpeg.

Since Android Q, the rules for exec() do not allow to extract the binary to files folder, as has traditionally been done in many apps, including a popular https://github.com/bravobit/FFmpeg-Android.

The workaround is to rename ffmpeg binary to lib..ffmpeg..so and move it from assets to jniLibs. But this is not compatible with extractNativeLibs="false", and you cannot selectively exclude only the executable: all native libs must be extracted.

[DanAlbert]

I suppose a way to do that would be to have the system mount the APK as a file system. That helps with using all types of assets too, not just exectuables.

That's definitely not a small task, but it's an interesting idea and I can see the utility, so I'll retitle this bug to track that (since apparently I was wrong about the original problem, thanks for correcting me!), but it's going to go in the unplanned bucket and it's not going to be something we'll have the resources for any time soon.

[jcgoble3]

@DanAlbert Has there been any update on this? The ability to execute binaries without changing the filename is essential for an app I am producing at work. Our app is dependent on a third-party program that consists of numerous "independent" executables that call one another by name. While this third-party code is open source, and so we can modify this to call libprogname.so instead of progname, the codebase is rather complex and I'd prefer to leave the code unmodified as much as possible. But we can't do so and still target API 29 to my understanding (or is that understanding wrong?).

[enh-google]

yeah, no-one is working on this, and API 30 is finished at this point too, so you're going to need a different approach.

[DanAlbert]

I'd recommend that different approach being to file a bug against studio to get the naming restriction lifted. I don't think it would be difficult to do.

[alexcohn]

I'd recommend that different approach being to file a bug against studio to get the naming restriction lifted. I don't think it would be difficult to do.

Last time I tried this, my request was rejected.

[jcgoble3]

That's disappointing that they don't care about a legitimate use case. Looks like we'll have to go the renaming route.

What is the deal with the double dots (e.g. lib..ffmpeg..so)? They are used both in the comment above and on the issue linked by @alexcohn. Is there something special about using double dots like that? I've never seen a filename like that before.

[DanAlbert]

Thanks for the link, @alexcohn. I'm on that thread now and will try to get it fixed. @jcgoble3 if you +1 that bug (there's a button up at the top) that helps too.

[jcgoble3]

I did +1 it. Hopefully we can get a few more to do so.

Can someone answer my question about the filenames with double dots? Does that have a special meaning to the system or is that just somebody's naming convention?

[enh-google]

i think that's just alex's way of implying that it doesn't actually matter what you have there.

[alexcohn]

I invented the .. convention to emphasize the purpose of such rename. You can use arbitrary names as long as they begin with the correct 3 letters lib and end with the correct 3 letters .so

[alexcohn]

@jcgoble3 There is one additional issue to consider when you choose to keep the executable names intact: even if you succeed by crook or by hook to have your executables, it may happen that the target device will not extract them to the libs directory, because the older platforms had filtering of extracted file names of their own. This limitation seems to have been lifted at API 24 or 25, but there are still many active devices where these files must be extracted manually.

[jcgoble3]

@alexcohn Thanks for the info on the naming convention and pitfalls. Our app has a minimum API level of 28, so I don't think that will be an issue for us.

[DanAlbert]

@alexcohn That's pretty concerning and makes me think we need to be more careful in allowing this. Do you have more info? I can go digging through the packagemanager history if not. I think if it does alter our plans at all it would be to restrict (or at least warn) for contents like that when the minSdkVersion is below whatever API level started supporting this. I wasn't aware of any restriction at all.

[alexcohn]

@DanAlbert I looked it up again. Unfortunately, even in current master, the 'wrong' names are filtered out. I was wrong that the restriction had been lifted: actually, in Nov'2016 the change was made to allow non-lib….so files to be extracted, but only for debuggable apps.

My mistake can easily be explained: I found that my debug app could work on newer devices, but because it still failed on older ones, I never pushed the change to use 'true' executable names to production.

TL;NR: @jcgoble3, unfortunately, you still must rename all these executables, even if you only care about API 28+.

[jcgoble3]

Well, bleep. :P Thanks for the digging, @alexcohn. I'll get to work on renaming them and adjusting the code that calls them.

[alexcohn]

Finally, 5.5 years later, this has been fixed!

[DanAlbert]

https://issuetracker.google.com/152645643 is still open though, so I think AGP will still make it difficult to package even if the OS will not longer ignore it?

And unfortunately it's an OS change, so it'll be a while before you can rely on the behavior without a fallback :(

Still, thanks @enh-google!

[jcgoble3]

Good to hear, even though I left that internship four and a half years ago and have since graduated from college and am now three years into my full-time career. I had actually forgotten about this! Hopefully it will be helpful to others.