Provide a system(3) replacement on Android (+ build OCaml on Android)

[MisterDA]

This would fix #13380, where an user reported a failure to build OCaml on Android with Termux. I found out that OCaml internally calls system() (bionic source) in caml_sys_system_command via Ccomp.command and Pparse.call_external_preprocessor. The problem is that system seems to have been blocked for security reasons, and thus ocamlc fails at invoking gawk as an external preprocessor (but note that gawk can still be called from the shell):

$ make -j1 V=1
/data/data/com.termux/files/usr/bin/make coldstart
make[1]: Entering directory '/data/data/com.termux/files/home/ocaml'
/data/data/com.termux/files/usr/bin/make -C stdlib OCAMLRUN='$(ROOTDIR)/boot/ocamlrun' USE_BOOT_OCAMLC=true all
make[2]: Entering directory '/data/data/com.termux/files/home/ocaml/stdlib'
../boot/ocamlrun ../boot/ocamlc -strict-sequence -absname -w +a-4-9-41-42-44-45-48 -g -warn-error +A -bin-annot -nostdlib -principal  -nopervasives -no-alias-deps -w -49  -pp "$AWK -f ./expand_module_aliases.awk" -c stdlib.mli
sh: /data/data/com.termux/files/usr/bin/gawk: Permission denied
File "/data/data/com.termux/files/home/ocaml/stdlib/stdlib.mli", line 1:
Error: Error while running external preprocessor
Command line: gawk -f ./expand_module_aliases.awk 'stdlib.mli' > /data/data/com.termux/files/usr/tmp/ocamlpp102f05

make[2]: *** [Makefile:118: stdlib.cmi] Error 2
make[2]: Leaving directory '/data/data/com.termux/files/home/ocaml/stdlib'
make[1]: *** [Makefile:687: coldstart] Error 2
make[1]: Leaving directory '/data/data/com.termux/files/home/ocaml'
make: *** [Makefile:846: world.opt] Error 2

$ gawk -f ./expand_module_aliases.awk stdlib.mli > /data/data/com.termux/files/usr/tmp/ocamlpp102f05
$ ls -l /data/data/com.termux/files/usr/bin/gawk
-rwx------ 1 u0_a199 u0_a199 557672 Aug 23 12:15 /data/data/com.termux/files/usr/bin/gawk

Android hasn't disabled classic ways of spawning processes, and we can re-implement system(3), which this PR does, with code directly taken from the POSIX spec (are there licensing issues?) the musl library (MIT-licensed).
Do let me know if supporting Android is not an interesting target.
I'm not sure if this implementation of caml_sys_system should do smarter things, like unload the runtime after the fork or other cleanups.
Is there another way to circumvent this problem?

[sidkshatriya]

with code directly taken from the POSIX spec (are there licensing issues?)

Not sure what license the code snippets from the POSIX spec are in.

However, as an alternative, the code in musl libc is licensed liberally (see https://git.musl-libc.org/cgit/musl/tree/COPYRIGHT?h=v1.2.5 ) (I checked version v1.2.5 which is the latest tagged version as of this GitHub comment )

As an idea one could adapt the code for system() from musl. See https://git.musl-libc.org/cgit/musl/tree/src/process/system.c?h=v1.2.5 . You would probably need add a code attribution and license text of musl in some way within the OCaml source code... please ask OCaml maintainers how to do this, if this turns out to be a viable approach.

[dustanddreams]

Not sure what license the code snippets from the POSIX spec are in.

Theoretically, you need to ask IEEE for permission to use parts of the specification in your work (even code examples), and you would need to add the IEEE copyright.

I agree borrowing code for other projects with clear open source licences should be preferred.

[MisterDA]

I've taken the code from musl, it's MIT-licensed.

[dustanddreams]

Have you been able to check that posix_spawn is available on Android? Note that configure checks for its availability.

[MisterDA]

Yes, it's been available since Android 8 Oreo.

[gasche]

We already have code that calls /bin/sh in C, for example in others/unix/execvp.c, and in OCaml, for example in otherlibs/unix/unix_unix.ml. Can we use/adapt this instead of adding more code?

[MisterDA]

We already have code that calls /bin/sh in C, for example in others/unix/execvp.c, and in OCaml, for example in otherlibs/unix/unix_unix.ml. Can we use/adapt this instead of adding more code?

I'm not sure how. It would mostly require moving parts of caml_unix_spawn from the unix library to the runtime. I'm not sure the custom system re-implementation and the unix library caml_unix_spawn would share much, apart the call to posix_spawn.

Interestingly Sys.system wraps libc's system, Unix.system wraps Windows' system, but Unix.system on Unix calls caml_unix_spawn and not system directly.

[MisterDA]

A breaking but simplifying change could be to stop calling the shell with the preprocessor arguments, and rather invoke the process directly.
If I'm not mistaken the preprocessor invocation can be moved to a separate shell script if a shell is really needed.