Revise generator for open62541 version 1.3.10

[thomvet]

Starting with version 1.4 open62541 discourages using the amalgamation option when (i.e., creating a single header file - it disallows "make install" when the option is on, see here: JuliaPackaging/Yggdrasil#7889 (comment)).

This PR updates the generator to be able to deal with a non-amalgamated build of open62541. It uses open62541 version 1.3.10 and not 1.4.0, because I wanted to keep the version step as small as possible.

Will now work on updating things for 1.4.0 and would like to include encryption in that build then.

[thomvet]

Tests are passing locally on my macbook, but not on Windows (segfaults when doing UA_Server_new()). I'll investigate why.

[thomvet]

MacOS X tests are now passing also on the CI server (which runs on M1 processors, my old macbook is on Intel).

When running with the wrappers generated from the non-amalgamated version (here), but with [email protected], the tests are passing on Windows as well.

Does it mean that something is wrong with the binary (open62541_jll) for 1.3.10 on Windows and Ubuntu?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.32%. Comparing base (ef35205) to head (45ba588).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #22      +/-   ##
==========================================
+ Coverage   44.19%   44.32%   +0.12%     
==========================================
  Files          13       13              
  Lines        3905     3912       +7     
==========================================
+ Hits         1726     1734       +8     
+ Misses       2179     2178       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[martinkosch]

I found some hints on where to search next. The automatically generated shared library for open62541 v.1.3.10 seems to be partially broken on Linux.

Calling the automatically built shared library manually with

using Libdl
libopen62541 = Libdl.dlopen("(...)/.julia/artifacts/af5206a25b8c611f2aed7b9bc3a3cc468a46af17/lib/libopen62541.so")

mutable struct UA_Server end
fun = Libdl.dlsym(libopen62541, :UA_Server_new) 
@ccall $fun()::Ptr{UA_Server} 

leads to the observed segfault.

When I instead call the same code with a self-built version of the shared library, UA_Server_new works as expected. Interestingly, this is also true if I build open62541 with your build script and the corresponding open62541 commit (2405c658). There seems to be a difference between both files, as the self-built shared library is 930136 bytes in size, while the jll version is 902992 bytes.

I also tried a simple overwrite of the shared library in the artifact folder with my self-built version. For this configuration, all package tests passed except for the memory leak tests (Expression: mem_end - mem_start < 50.0: Evaluated: 62.23828125 < 50.0), which seems to be reparable.

[martinkosch]

Some differences are visible when comparing the Yggdrasil build log with my local build log.
The most relevant difference is that I use a higher GCC version (11.4.0) locally, while Yggdrasil uses version 4.8.5 by default.

In addition, Yggdrasil's compiler throws multiple warnings complaining about the bit fields used by open62541. This one, for example:

/workspace/srcdir/open62541/include/open62541/types.h: At top level:
/workspace/srcdir/open62541/include/open62541/types.h:990:5: warning: type of bit-field ‘padding’ is a GCC extension [-Wpedantic]
     UA_Byte padding    : 6;       /* How much padding is there before this

I suspect that both phenomena are related. If I remember correctly, bit fields were also the main cause of trouble when starting this package a few years ago, certainly using an older GCC version. I also only moved to the amalgamated version to mitigate some of these (or at least very similar) instabilities.

Maybe requesting a higher GCC version is all that is needed to solve this issue. This can potentially be changed by setting preferred_gcc_version as explained here.

Could you give that a try?

[thomvet]

This turned out to be a good way of looking into it!

I made a few compiles using BinaryBuilder.jl with different versions of GCC by running the build script under Mac OS X and building it for Windows. Then replace the built artifacts and used the main branch with these 1.3.10 binaries (i.e., haven't run generator yet using the non-amalgamated files, just straight up used the 1.3.9 wrapper made for the main branch with the 1.3.10 binaries)

Here's the result:

• v4: currently on Yggdrasil, crashes on UA_Server_new() as mentioned above.
• v5: same as v4
• v6: crashes upon "using Open62541" (worse than the above!)
• v7: passes all tests
• v8: passes all tests
• v9: passes all tests
• v10: passes all tests
• didn't test newer versions, but I am assuming that they are also be fine.

So, it seems anything >= v7 is fine.

Todo list:

• try to generate the wrapper based on these non-amalgamated header files, run tests locally and make sure they pass.
• update 1.3.10 yggdrasil build
• run tests on CI with the new build, bump version.

Subsequently, I'll look into open62541 v1.4 (separate PR).

[thomvet]

Ok, seems like this works now! Before merging this, I would like to have a check whether we can be a little more flexible with the compat bound on the jll package (probably the current wrapper works with all 1.3.x versions of the jll?).

[thomvet]

Ok, made a few alterations in the package to allow for more flexibility, also with future patch builds of the open62541_jll package. I also made the package compatible again with Julia 1.6 (which still is the long term support release). This is only on Windows; I had to exclude mac os x, because of . Since 1.11 has hit release client status now, I have also tested the package against this upcoming version.

All passes. :)

@martinkosch, this is now ready for a review, I think. Once merged, I would suggest that this becomes version 0.1.1 of the package.

[martinkosch]

I'm glad it's working now. For me it looks ready to merge.