Feature suggestion: @cinclude macro.

[notinaboat]

In some cases, especially for standard C library APIs, it is convenient to just #include a header without having to generate a wrapper first.

I've made a little prototype @cinclude macro here: https://github.com/notinaboat/CInclude.jl

e.g.

julia> using CInclude
julia> @cinclude "termios.h"

[ Info: @cinclude "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/termios.h"
...

julia> term = termios()
julia> settings.c_lflag = ICANON
julia> settings.c_cflag = CS8
julia> cfsetspeed(Ref(settings), B38400)
julia> tcsetattr(fd, TCSANOW, Ref(settings))

[Gnimuc]

Better to rebuild the package upon the latest Clang.jl#master and use cross-platform support for loading those system headers. There might be compatibility issues between the stdlibs used in Julia and those on the users local machine.

[notinaboat]

Thanks for the tip @Gnimuc. In my current project I'm only using released packages, so I'll hold off using Clang.jl#master until the next release.

[Gnimuc]

@notinaboat I've been thinking of making a Julia package like rust's libc to provide similar functionalities in your package, which doesn't need libclang as a runtime dependency. But I don't have enough bandwidth to work on it.

[notinaboat]

Hi @Gnimuc,
Out of interest, in CInclude.jl I've implemented a tiny inline C++ program to get macro-values:
https://github.com/notinaboat/CInclude.jl/blob/master/src/CInclude.jl#L86-L92
This avoids having to parse and interpret C macro syntax.

Also of interest is this C REPL I found today:
https://root.cern/cling/

[Gnimuc]

Yeah. Clang is a full-featured compiler library with which you can do what a compiler can do, for example, implementing a C++ interpreter. https://github.com/JuliaInterop/Cxx.jl is another example.

[notinaboat]

Yes, Cxx.jl is awesome! I played with it a little many years ago, but I have never been able to use it in a project.
It is very nice to see that you are working on it again :) I look forward to it being in-sync with latest Julia.

[krrutkow]

I've been thinking of making a Julia package like rust's libc to provide similar functionalities in your package...

A proof-of-concept of this is already demonstrated with https://github.com/analytech-solutions/System.jl using https://github.com/analytech-solutions/CBinding.jl to expose LibC and Linux API's, so it is possible to do in Julia.

[notinaboat]

Hi @Gnimuc,
I'm in the process of updating my UnixIO.jl package to use the latest Clang.jl
A few small changes to Clang.jl were needed to deal with wrapping system libraries.
See: #398

[notinaboat]

in CInclude.jl I've implemented a tiny inline C++ program to get macro-values.
[...]
Also of interest is this C REPL I found today: https://root.cern/cling/

The problem with trying to interpret the #define values from Julia in generators/macro.jl is that macro expressions can be arbitrarily complex and you'd end up having to build a complete C interpreter (libclang only supports parsing C code to and AST, it does not provide an API for interpreting C expressions).

The problem with building an inline C++ program to evaluate#define values is that if a single #define resolves to something that does not have a value, the program will fail to compile.

I've now implemented a basic macro-value resolver using the C-REPL cling.
(It would be nice to package cling as a _jll and call it from Clang.jl, but for now I'm just doing brew install cling).

See: https://github.com/notinaboat/UnixIO.jl/blob/julia18/packages/UnixIOHeaders/src/UnixIOHeaders.jl#L65
This works by sending commands to the C-REPL on stdin, then parsing the results from stdout.

First the relevant headers are #include-ed, then each #define-ed symbol is entered into the REPL.
If the symbol resolves to something sensible the REPL prints out the C type and the value. e.g...

% cling -isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include
[cling]$ #include <signal.h>
[cling]$ SIGSEGV
(int) 11
[cling]$ SA_RESTART
(int) 2
[cling]$ SIG_DFL
(void (*)(int)) Function @0x0
[cling]$ BADSIG
(void (*)(int)) Function @0xffffffffffffffff
[cling]$ UINT8_MAX
(int) 255
[cling]$ UINT16_MAX
(int) 65535
[cling]$ UINT32_MAX
(unsigned int) 4294967295
[cling]$ UINT64_MAX
(unsigned long long) 18446744073709551615

In Julia we end up with:

const SIGSEGV = Cint(11)
const SA_RESTART = Cint(2)
const SIG_DFL = bitcast(Ptr{Cvoid}, UInt(0x00))
const BADSIG = bitcast(Ptr{Cvoid}, UInt(0xffffffffffffffff))
const UINT8_MAX = Cint(255)
const UINT16_MAX = Cint(65535)
const UINT32_MAX = Cuint(4294967295)
const UINT64_MAX = Culonglong(18446744073709551615)

The existing generators/macro.jl produces this:

const SIGSEGV = 11
const SA_RESTART = 0x0002
const SIG_DFL = ((Cvoid(*))(Cint))(0)
const SIG_ERR = (Cvoid(*))(Cint) - 1
const BADSIG = SIG_ERR
const UINT8_MAX = 255
const UINT16_MAX = 65535
const UINT32_MAX = Cuint(4294967295)
const UINT64_MAX = Culonglong(18446744073709551615)

SIGSEGV and SA_RESTART have the wrong integer types (Int64 and UInt16 respectively)
SIG_DFL and SIG_ERR don't compile at all.
UINT8_MAX and UINT16_MAX are both Int64 where they should be Int32

In the case where the symbol does not resolve to a value, the cling REPL prints and error message.
The output parser simply ignores symbols that result in an error message. e.g. ...

[cling]$ sa_handler
input_line_11:2:2: error: cannot use dot operator on a type
 sa_handler
 ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/sys/signal.h:295:38: note: expanded from macro 'sa_handler'
#define sa_handler      __sigaction_u.__sa_handler
                                     ^
[cling]$ PTHREAD_ONCE_INIT
input_line_19:2:2: error: expected ';' after expression
 PTHREAD_ONCE_INIT
 ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/pthread.h:220:50: note: expanded from macro 'PTHREAD_ONCE_INIT'
#define PTHREAD_ONCE_INIT {_PTHREAD_ONCE_SIG_init, {0}}
                                                 ^
input_line_19:2:2: error: expected ';' after expression
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/pthread.h:220:54: note: expanded from macro 'PTHREAD_ONCE_INIT'
#define PTHREAD_ONCE_INIT {_PTHREAD_ONCE_SIG_init, {0}}
                                                     ^
[cling]$ 

[Gnimuc]

and you'd end up having to build a complete C interpreter (libclang only supports parsing C code to and AST, it does not provide an API for interpreting C expressions).

As for the interpreter , some work has been done in Gnimuc/ClangCompiler.jl#16, but not finished.

I'm still waiting for the integration of Cling and LLVM/Clang. There was a proposal for exposing a generic C++ interop interface from LLVM/Clang.

[notinaboat]

As for the interpreter , some work has been done in Gnimuc/ClangCompiler.jl#16, but not finished.

Looks good, I will look forward to this!

[Gnimuc]

UINT8_MAX and UINT16_MAX are both Int64 where they should be Int32

In Clang.jl, integer literals(e.g. 42) use Int as the default type. For other cases(e.g. 42LL, 42ULL), the type should be correctly matched. I don't remember why I did this though. 😅

[notinaboat]

Clang.jl also inherits Julia's behaviour of converting hexadecimal constants to the smallest possible unsigned type (In C they are int). e.g. SA_RESTART in the example above.

[Gnimuc]

I'm wondering how bad this would be when doing interop-ing. I guess it's OK for most of the cases as long as the type is large enough to cover the value. But indeed this should be fixed in the future.

[notinaboat]

Cling is now available as a jll: https://github.com/JuliaBinaryWrappers/Cling_jll.jl
Currently working on intel macOS and linux. Detail here: https://github.com/JuliaPackaging/Yggdrasil/blob/master/C/Cling/build_tarballs.jl#L102-L117