-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support 16 KB page sizes #1171
Comments
This will be fixed in Chaquopy 16 for all Python versions. However, existing wheels will still not be 16 KB-compatible, so unless your project is pure-Python, it may be unable to run on 16 KB devices with Python 3.12 and older. Python 3.13 and newer should be fine. The current error message from loading a 4 KB binary on a 16 KB device is quite misleading:
I expect they'll improve that before 16 KB devices go mainstream. |
As mentioned at https://developer.android.com/guide/practices/page-sizes, full 16 KB compatibility would also require us to provide a 16 KB version of libc++_shared.so. This is included in NDK version 27 and later. I updated the chaquopy-libcxx recipe and tested it on Python 3.8, ARM64, API level 34, with all packages that require chaquopy-libcxx. I made this list as follows
Then do the same with the non-Python
And add any Python packages which use these, and haven't already been listed:
All of them pass except for dlib and greenlet, which crash as follows:
It's possible that this would be fixed by rebuilding dlib and greenlet against the new libc++. But since Android is obviously in the very early stages of this transition, it's not worth spending any more time on it right now. So I've saved the chaquopy-libcxx recipe updates in #1262, and haven't released any new wheels for it. |
Greenlet 3.1.1 also crashed when I first built it for Python 3.13 (#1244). However, this was probably caused by building it against a newer libc++ version from NDK 27.1.12297006, and running it with an older one, which is not safe in general. So it looks like we'll have to deal with this now anyway. Running with libc++ version 180000 from #1262, the tests passed. Running with the existing libc++ version, 11000, it crashed as follows:
```
13:24:43.184 DEBUG A Build fingerprint: 'google/sdk_gphone64_arm64/emu64a:15/AE3A.240806.005/12228598:userdebug/dev-keys'
13:24:43.184 DEBUG A Revision: '0'
13:24:43.184 DEBUG A ABI: 'arm64'
13:24:43.184 DEBUG A Timestamp: 2024-10-22 13:24:43.018613290+0100
13:24:43.184 DEBUG A Process uptime: 25s
13:24:43.184 DEBUG A Cmdline: com.chaquo.python.pkgtest3
13:24:43.184 DEBUG A pid: 1658, tid: 1749, name: Thread-3 >>> com.chaquo.python.pkgtest3 <<<
13:24:43.184 DEBUG A uid: 10213
13:24:43.184 DEBUG A tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
13:24:43.184 DEBUG A pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
13:24:43.184 DEBUG A signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
13:24:43.184 DEBUG A x0 0000000000000000 x1 00000000000006d5 x2 0000000000000006 x3 0000007ce68e21e0
13:24:43.184 DEBUG A x4 0000007ce3e2f0e8 x5 0000007ce3e2f0e8 x6 0000007ce3e2f0e8 x7 0000007ce3e2f0e8
13:24:43.184 DEBUG A x8 00000000000000f0 x9 0000007f99229468 x10 ffffff80fffffb9f x11 0000000000000000
13:24:43.184 DEBUG A x12 0000000000000004 x13 0000000000000000 x14 0000000000000008 x15 0000a655afe79f43
13:24:43.184 DEBUG A x16 0000007f992d8ff8 x17 0000007f992c31c0 x18 0000007ce526c000 x19 000000000000067a
13:24:43.184 DEBUG A x20 00000000000006d5 x21 00000000ffffffff x22 ffffff806560ff7f x23 0000007ce68e2460
13:24:43.184 DEBUG A x24 0000007f9a9f0080 x25 b400007dddcd4d90 x26 0000000000000001 x27 b400007dddcd4d90
13:24:43.184 DEBUG A x28 0000007f9a9f0080 x29 0000007ce68e2260
13:24:43.184 DEBUG A lr 0000007f992616a4 sp 0000007ce68e21c0 pc 0000007f992616d4 pst 0000000000001000
13:24:43.184 DEBUG A 16 total frames
13:24:43.184 DEBUG A backtrace:
13:24:43.184 DEBUG A NOTE: Function names and BuildId information is missing for some frames due
NOTE: to unreadable libraries. For unwinds of apps, only shared libraries
NOTE: found under the lib/ directory are readable.
NOTE: On this device, run setenforce 0 to make the libraries readable.
NOTE: Unreadable libraries:
NOTE: /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/chaquopy/lib/libc++_shared.so
NOTE: /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #00 pc 000000000005b6d4 /apex/com.android.runtime/lib64/bionic/libc.so (abort+168) (BuildId: 1b9fecf834d610f77e641f026ca7269b)
13:24:43.184 DEBUG A #1 pc 00000000000e00e0 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/chaquopy/lib/libc++_shared.so
13:24:43.184 DEBUG A #2 pc 00000000000df6dc /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/chaquopy/lib/libc++_shared.so
13:24:43.184 DEBUG A #3 pc 00000000000df4fc /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/chaquopy/lib/libc++_shared.so
13:24:43.184 DEBUG A #4 pc 00000000000358dc /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #5 pc 00000000000359a8 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #6 pc 0000000000023148 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #7 pc 0000000000024c2c /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #8 pc 000000000002801c /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #9 pc 000000000024a68c /data/app/~~Edc06gT3338cYi7yq9tbrQ==/com.chaquo.python.pkgtest3-UYNbYWAlBlWE_TpMWjHaIw==/base.apk!libpython3.13.so (offset 0x2a98000) (BuildId: d1b6455d9469909b72e8d94b5ce0ab5b3b8e0a0f)
13:24:43.184 DEBUG A #10 pc 000000000023efdc /data/app/~~Edc06gT3338cYi7yq9tbrQ==/com.chaquo.python.pkgtest3-UYNbYWAlBlWE_TpMWjHaIw==/base.apk!libpython3.13.so (offset 0x2a98000) (PyObject_Vectorcall+92) (BuildId: d1b6455d9469909b72e8d94b5ce0ab5b3b8e0a0f)
13:24:43.184 DEBUG A #11 pc 000000000036afa0 /data/app/~~Edc06gT3338cYi7yq9tbrQ==/com.chaquo.python.pkgtest3-UYNbYWAlBlWE_TpMWjHaIw==/base.apk!libpython3.13.so (offset 0x2a98000) (_PyEval_EvalFrameDefault+21448) (BuildId: d1b6455d9469909b72e8d94b5ce0ab5b3b8e0a0f)
13:24:43.184 DEBUG A #12 pc 0000000000025724 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #13 pc 0000000000024f84 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #14 pc 0000000000024ac8 /data/data/com.chaquo.python.pkgtest3/files/chaquopy/AssetFinder/requirements/greenlet/_greenlet.so
13:24:43.184 DEBUG A #15 pc 00000000001002b4 [anon:stack_and_tls:1749]
```
|
I've now made new builds of the existing versions of dlib and greenlet, and tested all C++-based packages on all Python versions, so it's safe to release chaquopy-libcxx version 180000. In theory this means we've now done everything necessary to support 16 KB pages. But I haven't been testing on a 16 KB device, so I'll leave this issue open and do that as part of the next Chaquopy release. |
Sources:
I don't expect physical devices to start requiring this until at least 2 years after Google Play makes it compulsory. But since our binaries have a lifetime equal to that of a Python version (5 years), we should start building in 16 KB mode as soon as possible. This applies to Python, its supporting libraries in cpython-android-source-deps, Chaquopy, and all wheels.
For Python 3.13 and later, we should fix this upstream.
The text was updated successfully, but these errors were encountered: