Cross-platform replacement for c_compiler.lua?

[hugomg]

If we ever want to make Pallene more cross-platform, the c_compiler.lua part will be one of the sticking points. Currently, windows support is completely missing and the macOS support is not actively tested. (None of the devs still use macOS and it is not tested in our continuous integration). This is all very unfortunate, and it is a problem area that is out of our area of expertise. We should consider delegating this functionality to someone else. Perhaps Luarocks? It knows how to compile C code in a cross-platform way...

[Mehgugs]

You could use zig's c compiler frontend to do the cross compilation. You can invoke zig cc with the same options as clang/gcc and compile C code on any platform zig supports (targeting any platform zig supports).

I personally think using luarocks is not the solution because the onus is on the user to teach luarocks how to compile C modules on their system, for windows users this often requires a lot of extra work.

[hugomg]

Could you talk more about why Luarocks would be more difficult on Windows? I thought Luarocks also knew how to compile on Windows?

Also, another reason we were thinking about Luarocks is that it knows where to find the Lua interpreters and header files. Even if we used zig cc, maybe we'd still need some help from Luarocks to sort out the compilation flags.

[Mehgugs]

Well given that pallene wants you to build a customized version of lua from source why not just have it be built as part of the set up?
Then you can make sure all the headers (plus luacore.h) are known to the pallene system.

EDIT:

In regards to setup on windows my main sticking point is having someone compile lua from source sourcing their own compiler toolchain, rather than specifically running the luarocks installer (which happens after you've got lua + a compiler ready).

[hugomg]

Sorry, I'm not sure I understand 100%. Currently we have to install the custom Lua by hand and the Pallene compiler using Luarocks. Are you suggesting that the whole thing (custom Lua + pallene compiler) should be packaged as a Luarocks rock? Or that we should ditch Luarocks and install the compiler with a custom installation script? Or something else?

[Mehgugs]

Okay so what I'm saying is:

zig cc is a good compiler frontend for cross platform c compilation. It is also very easy to use on windows.

I would like this project to be usable without luarocks since I – and many other prospective users I imagine – already have lua environments set up. Zig's c compiler seems like the best way to achieve cross platform c compilation without assuming luarocks has been set up by the user beforehand.

[hugomg]

I see. I suppose we can consider Zig if at some point we decide that we should bundle a C compiler into the project.

In the mean time, one escape valve that already exists is that you can compile Pallene to C using --emit-c. This is 100% portable and then you can compile the C code however you want.

[srijan-paul]

None of the devs still use macOS and it is not tested in our continuous integration

I haven't actively used Pallene in quite some time, but continue to follow activity on the repository.
I'm currently using MacOS (M1) for work.
Attempting to install lua-internals throws the following error on apple clang:

gcc -std=gnu99 -o lua   lua.o liblua.a -lm -Wl,-E -ldl -lreadline
gcc -std=gnu99 -o luac   luac.o liblua.a -lm -Wl,-E -ldl -lreadline
ld: unknown option: -E
ld: unknown option: -E
clangclang: : error: linker command failed with exit code 1 (use -v to see invocation)
error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [luac] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [lua] Error 1
make[1]: *** [linux-readline] Error 2
make: *** [linux-readline] Error 2

I guess a good place to start would be to make the installation process cross-platform.

[hugomg]

Hi Srijan, nice to hear from you again! How are you doing? :)

By any chance, did you compile with make linux-readline? If so, then make macos might work. (And then I presume we should update the README to tell people about that).

[srijan-paul]

I'm doing fine, thanks for asking! :D
And yeah, compiling with macosx did work.
I was able to compile Pallene as well.

However ./run-test throws the following error:

binarytrees / pallene
./benchmarks/benchlib.lua:33: command failed: make --quiet -f benchmarks/Makefile 'benchmarks/binarytrees/pallene.so'

stack traceback:
	./benchmarks/benchlib.lua:33: in function 'benchmarks.benchlib.prepare_benchmark'
	./benchmarks/benchlib.lua:226: in function 'benchmarks.benchlib.run_with_impl_name'
	spec/benchmarks_spec.lua:34: in function <spec/benchmarks_spec.lua:33>

--- Lua Lint ---
/opt/homebrew/opt/lua/bin/lua5.4: cannot open /Users/srijanpaul/.luarocks/lib/luarocks/rocks-5.4/luacheck/0.25.0-1/bin/luacheck: No such file or directory

If I try to compile only 1 file however, it does work:

I just posted it here as soon as I ran into the error, so I might be able to resolve this with some digging.

[Nomarian]

I see. I suppose we can consider Zig if at some point we decide that we should bundle a C compiler into the project.

In the mean time, one escape valve that already exists is that you can compile Pallene to C using --emit-c. This is 100% portable and then you can compile the C code however you want.

Not that portable, currently luacore.h contains non C code, (gnu awk script), so not only is it nonportable, its only usable on gnu systems.

[hugomg]

Is that AWK script using something that only works on GNU? The plan was to work with any AWK.

In any case, the AWK script is only used to generate luacore.h. If we distribute that in the tarball then the end user wouldn't need AWK. To be honest, when it comes to portability I'm more worried about the C99 code we're using here and there.

[Nomarian]

Yes, BEGINFILE is gawk specific, it can probably be replaced by a NR==1 {}.

Also, now I see why there was awk code in my luacore.h, it probably emitted itself because the awk code was incorrect?

[hugomg]

Interesting. Let's move the AWK discussion to pallene-lang/lua-internals#1