Errors trying to build simple header+loader binary #58
Comments
|
@juliohm Thanks for reporting this. The issue isn't that the build is using C99 for some reason. The issue is that it isn't using C99, but uses the default C standard, which I guess is C89. It was a conscious decision that the OpenCL ecosystem going forward won't support 30 year old language standards. Unfortunately our OpenCL-SDK CI coverage isn't quite the same breadth yet as the rest of the ecosystem, and this failure is triggered in the utility libraries. We'll discuss this issue, my educated guess is that the utility libraries will target C11 and onward, maybe C99. We'd need to improve our testing in CI to make sure such fiascos don't happen in the future. I also saw that you build a full blows SDK, however my guess is that Julia is only using the OpenCL headers and libraries. If you can pass extra arguments to CMake, I'd highly suggest adding |
|
Thank you @MathiasMagnus I will try to build it again with the proposed option. Will also try |
|
You beat me to, I was just about to amend my message with the same idea. Let me know how it turns out. |
|
Ok, the option Should I edit these hpp files to include some missing stdlib? |
|
It is been a while since the last time I touched C/C++, all my work nowadays is in Julia, I don't know the latest trends in standards, stdlibs, etc. Any help is appreciated. I think we are almost there 🙏🏽 |
|
Yeah, the issue at hand is this: I would ask what operating system this is, but GCC 4.8.5 shares the feature set of 4.8.0, which was released March 2013, so it's 9 years old snapshot of C++ and as such pre-dates C++14. I don't think we will back-port the utility libraries to C++11, as we would have to reimplement quite some metaprogramming utilities. Our baseline of testing on Linux are the compilers found in oldest supported Ubuntu LTS, which is GCC 7. We don't (currently) have the assets to have CI runners sporting older systems/toolchains. Support below that is on a best-effort basis. I see two short-term solutions:
|
|
Thank you @MathiasMagnus , I am following all your suggestions. I requested GCC 7 in the cross-compilation script and followed the instructions in the Are we required to clone |
juliohm
changed the title
|
I'm the one responsible for these changes: KhronosGroup/OpenCL-Headers#140 I thought I might have messed something up but it looks like I'm including the right header ( I can try to dig up a gcc7 system to repro if needed. In the meantime, can you please check that the intended version if inttypes.h is being included and that it really does not have these format macro constants defined? Thanks! |
|
I'm happy with any build that works. If you have another preferred
compiler, please document it and make sure that the CI build is up to date
🙏🏼 Also, it is very good practice to set CMAKE_C_STANDARD and
CMAKE_CXX_STANDARD to avoid these build issues that are hard to fix by
non-maintainers.
Em ter., 27 de set. de 2022 12:02, Ben Ashbaugh ***@***.***>
escreveu:
… I'm the one responsible for these changes: KhronosGroup/OpenCL-Headers#140
<KhronosGroup/OpenCL-Headers#140>
I thought I might have messed something up but it looks like I'm including
the right header (<inttypes.h>), which is required since C99.
I can try to dig up a gcc7 system to repro if needed. In the meantime, can
you please check that the intended version if inttypes.h is being included
and that it really does not have these format macro constants defined?
Thanks!
—
Reply to this email directly, view it on GitHub
<#58 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZQW3PS3SNTZ447MHPSFT3WAMD6ZANCNFSM6AAAAAAQV65VSM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Yes, agreed. We've done this in some places (e.g. OpenCL-ICD-Loader) but not in the headers. See KhronosGroup/OpenCL-Headers#208, I'm not sure if it might help to solve this problem? On my system it adds |
|
I've incorporated the diff but the error is still there with GCC 7. I will try with GCC 9 now. |
|
Come to think of it, we have gcc 7 in our CI for OpenCL-Headers. Example: https://github.com/KhronosGroup/OpenCL-Headers/actions/runs/3116481007/jobs/5054348256 I can't explain why this isn't working.... 😕 |
|
If you want to reproduce the exact environment where I am trying to cross-compile OpenCL, it should take a few simple steps:
When you are done selecting the GCC compiler and commit from the repository that you want to build, it will show a prompt where you can type the CMAKE commands in the README. You should get the same error message I am getting. |
|
Thanks, I have been able to reproduce the error you are seeing. Stay tuned... |
|
Thank you for taking a look into it. For completeness, this is the full process recorded in a video in case you want to finish the cross-compilation yourself: https://binarybuilder.org |
|
I did a visual diff of the inttypes.h when the error occurs and my inttypes.h and noticed a difference: /* The ISO C99 standard specifies that these macros must only be
defined if explicitly requested. */
#if !defined __cplusplus || defined __STDC_FORMAT_MACROSSo it seems that these headers at least require There's some interesting information in a note on cppreference as well:
Regardless, we should fix this in the OpenCL-Header tests and check to see if a similar problem occurs in any of the OpenCL repos. Here are two additions to the test_headers.c file that seemed to fix the problem:
#if defined(__cplusplus)
#define __STDC_FORMAT_MACROS
#endif
#if defined(__cplusplus)
#include <cinttypes>
#else
#include <inttypes.h>
#endifAny preference? If not, I'll probably make a PR with (2). Thanks! |
|
Thank you @bashbaug , any option that works is fine. As soon as the changes are merged into the main branch we can retry the cross-compilation again with the specific commit. The video I linked above shows how the binaries can be generated for all 13 platforms automatically after we define a single build script that works. |
|
PR for (2) is here: KhronosGroup/OpenCL-Headers#209 |
|
Thank you @bashbaug , can you confirm that the build was successful for the OpenCL-ICD-Loader as well after the OpenCL-Headers? Should we wait for the PR to get merged into the main branch? |
|
@bashbaug @MathiasMagnus I've successfully built OpenCL for 11 different platforms with this script: https://github.com/juliohm/cross-platform-opencl Can you help adding support for the remaining platforms? You can run the script for a specific platform by passing the name of the platform in the command line: This is an example build that is failing due to a compilation error in OpenCL-ICD-Loader on FreeBSD. If we can solve all these tiny issues, we will be able to provide a more robust experience in future releases. |
|
The missing platforms are: with corresponding identifier: If we can fix at least the build on Windows platforms, we will be able to provide OpenCL for the majority of Julia users. |
|
Most errors I am getting trying to build and install OpenCL are actually errors that occur in the test folders. Ideally the CMake build instructions wouldn't attempt to test the code during installation targets to avoid additional dependencies that are only required at test time. Can the OpenCL maintainers take a look at the current CMake files for OpenCL-Headers and OpenCL-ICD-Loader to make sure that nothing else is being done other than building the library and copying the files to the prefix? I could then try to help fixing possible compilation errors on FreeBSD, Windows and MacOS. |
If it helps, all of our CMake files check for the standard BUILD_TESTING variable, so to bypass testing you can simply set BUILD_TESTING=0. |
|
Thank you @bashbaug , I've updated the cross-compilation script with this option turned off and the build is now successful in MacOS, totaling 13 platforms. We are only missing Windows and FreeBSD now. Do you think you can take a look at the error messages produced on Windows? |
|
Excellent, that's encouraging. I started looking at the Windows build issues yesterday. I didn't get too far but my hunch is that they are related to this issue: KhronosGroup/OpenCL-ICD-Loader#130 (comment) Can you say what this package is being used for? We may have some different options if it's only providing an import library to link to vs. building a DLL for deployment, but let's see if we can get things building without special options first. BTW - somebody else was trying to get FreeBSD working here, with some success: #55 (comment) |
We are trying to revive the OpenCL.jl package, which enables opencl calls in familiar Julia syntax: https://github.com/JuliaGPU/OpenCL.jl The package simply loads the binaries that were built by robots on GitHub for the 13 platforms and makes direct C calls to the opencl api with Julia's At the moment the OpenCL.jl package is assuming that the user installed opencl beforehand in the target system. We are changing that to the BinaryBuilder.jl approach which downloads the appropriate precompiled binary depending on the target system. |
|
@bashbaug I am a bit lost regarding the organization of OpenCL libraries. Say I want to compile a library to load later in Julia applications and make direct C calls with this library. Is it correct to say that I only need to complie OpenCL-Headers and OpenCL-ICD-Loader to get the library of interest and load it somewhere? What is ICD? I am finding some material online saying that there are two ways of working with OpenCL, but I am not an expert in GPUs, so everything is very confusing to me. Besides installing the library, do we need anything else on the target machine? Do we also need to install custom drivers depending on the hardware we have? I am following this page by Beignet for Intel GPUs and many questions started to pop: https://www.freedesktop.org/wiki/Software/Beignet I loaded the library on a Julia session, but any command is giving me the error: I wonder what it means exactly. I wonder if the compiled library was loaded correctly because the error seems be something that was thrown by the library. Is the error occurring because I don't have a opencl-compatible driver installed? I have a NVIDA GPU with a proprietary driver installed. |
|
I understood that the ICD is a system that loads different OpenCL driver implementations. And that we still need to have at least one driver installed manually on the target system to be able to use the library functions. Is that correct? |
|
I think I'm overdue to write a full page for the OpenCL-Guide to describe how this works, so consider this a short preview draft for now, and I'll try to write up something more comprehensive shortly. The "two methods" you mentioned may have come from my article describing how OpenCL works on Linux. It's correct, but for the purposes of this discussion we'll assume that we're using the OpenCL ICD loader, which is by far the most common mechanism for client usages and the one provided by the OpenCL SDK. TerminologyHere are a few useful bits of terminology - see also the cl_khr_icd specification:
UsageOpenCL applications link with the OpenCL ICD loader when building to provide definitions for the OpenCL APIs. Because the OpenCL ICD loader is a shared library (either You only need an OpenCL ICD installed to run an OpenCL application on an OpenCL device, and you do not need any OpenCL ICDs installed purely to build an OpenCL application (this is useful for testing, though!). TroubleshootingLet's look at a couple of potential OpenCL ICD loader-related problems: Missing OpenCL ICD LoaderIf your target system does not have an OpenCL ICD loader installed then your application will fail to start because the run-time dependencies are not satisfied. This is not OpenCL-specific, and similar behavior will also be seen if any other shared libraries are missing. Example: $ ldd ./a.out
libOpenCL.so.1 => not found
...
$ ./a.out
./a.out: error while loading shared libraries: libOpenCL.so.1: cannot open shared object file: No such file or directoryTo fix this problem, either ensure that an existing OpenCL ICD loader can be found (may involve setting environment variables like Out-of-Date OpenCL ICD LoaderIf your target system has an OpenCL ICD loader installed but it is too old then your application will still fail to start because some run-time dependencies will still be unsatisfied. This is also not OpenCL-specific, and similar behavior will also be seen if any other shared libraries are too old. Example: $ nm -u ./specconst
...
U clSetProgramSpecializationConstant@@OPENCL_2.2
$ ./specconst
./specconst: /usr/lib/x86_64-linux-gnu/libOpenCL.so.1: version `OPENCL_2.2' not found (required by ./specconst)To fix this problem, either install an updated OpenCL ICD loader package, or build and install a newer OpenCL ICD loader. No OpenCL PlatformsIf your application is able to start but is unable to find any OpenCL platforms - either The fix for this problem is a bit more open-ended but the article linked previously describes some troubleshooting steps to check to ensure that an OpenCL ICD exists and that it can be found by the OpenCL ICD loader. Note that the Khronos OpenCL ICD loader added several environment variables that were not available when the article was written that can be helpful for debug, and the ocl-icd OpenCL ICD loader also has similar environment variables. |
|
Since you mentioned Intel GPUs, if you have an Intel GPU that is Broadwell or newer I'd strongly recommend using the "NEO" compute runtime rather than Beignet. NEO is our open source production driver stack and it is much more up-to-date and well-maintained. We also ship a CPU OpenCL implementation as part of our oneAPI basekit. If you are looking for an open source CPU implementation I'd recommend having a look at PoCL, which provides an up-to-date CPU OpenCL device and more. |
|
Thank you @bashbaug for the very detailed explanation ❤️ It helped a lot. We did some experiments yesterday and learned about ICD in practice by installing and removing drivers from different vendors. I wonder if this explanation could go in the README of this repository as well, to educate outsiders about these terms. I myself had a lot of trouble finding OpenCL-ICD-Loader and understanding that it is the library needed as a shared library in OpenCL programs. I started my journey here in the SDK repository thinking it was the main project to compile as a OpenCL programmer. On the topic of drivers, do you know if there exists a list of recommended drivers for each device out there? We are trying to automate this installation process for end-users in Julia. Currently we already install the ICD shared library automatically depending on the platform. So whenever users decide that they will use OpenCL.jl, the Julia package manager downloads the correct ICD library as an artifact: using Pkg
# installs OpenCL.jl, which in turn installs the appropriate
# shared library for the platform, we call it OpenCL_jll
Pkg.add("OpenCL")However, the user still needs to manually install a driver, which is a bit painful. If we had a compiled list of drivers somewhere, we could try to download them at installation time, to make sure that at least one "recommended" driver is available. People who know what they are doing can always choose their drivers at runtime I suppose, but beginners will get an error message and will not know what is happening. |
|
Coming back to the original topic of the issue, we are only missing Windows now in our cross-compilation 🎉 Do you know a fix for the following error? |
|
I opened #61 to continue the discussion of drivers. |
|
@bashbaug if you want to test the build in a different platform where an error occurs, you can pass the option In this attempt with Windows for example, you will get a compilation error. You can then edit the files normally and re-run the CMake and build instructions until everything works. |
|
Disclaimer: I don't develop on Windows, so @jenatali or @MathiasMagnus would know more. Just a couple of facts, and maybe a solution:
Maybe we could test for the presence of Maybe MSYS2 need to update their I hope this helps. |
Thank you @Kerilk , that is very helpful. From the warning I understand that we should enable the option as we are distributing the ICD loader library in Julia programs, correct?
That would be wonderful. Do you think you could cook a small PR with the proposed fix? I can easily grab the diff from the PR to update the build instructions in the script. |
|
When I try to disable the option in the command line I get additional build errors. I really need some help from Windows developers 🙏🏽 |
|
I started looking at the Windows build issues and they do appear to be due to the issue I linked previously: In short, the included version of cfgmgr32.h is a little too old - it's missing the commit from July 2020 that adds the symbols the ICD loader requires - see the history for this file. Is there any chance we could use a slightly newer version of mingw? The one we have is awfully close (it has the previous commit, from April 2020) but not quite new enough. If we can't move to a newer version, the next best option I can think of is to add a CMake control to disable enumerating drivers via HKR, similar to the control to disable OpenCLon12. This is REALLY something we shouldn't do if we're building an OpenCL ICD loader that will actually be used to enumerate drivers, but it would be OK if all we need is an OpenCL.lib import library (or in the case of mingw, a libOpenCL.dll.a import library) to link with. If I bypass the HKR enumeration and disable OpenCLon12 I am able to build successfully: Install the project...
/usr/bin/cmake -P cmake_install.cmake
-- Install configuration: "Release"
-- Installing: /workspace/destdir/lib/libOpenCL.dll.a
-- Installing: /workspace/destdir/bin/libOpenCL.dll
-- Installing: /workspace/destdir/share/cmake/OpenCLICDLoader/OpenCLICDLoaderTargets.cmake
-- Installing: /workspace/destdir/share/cmake/OpenCLICDLoader/OpenCLICDLoaderTargets-release.cmake
-- Installing: /workspace/destdir/share/cmake/OpenCLICDLoader/OpenCLICDLoaderConfig.cmake
-- Installing: /workspace/destdir/share/cmake/OpenCLICDLoader/OpenCLICDLoaderConfigVersion.cmake
-- Up-to-date: /workspace/destdir/lib/libOpenCL.dll.a
-- Up-to-date: /workspace/destdir/bin/libOpenCL.dll
sandbox:${WORKSPACE}/srcdir # |
|
Another thought: we could just use magic numbers in the case that the headers are tool old and the newer symbols are not defined. This appears to be what the Vulkan loader is doing: |
|
PR KhronosGroup/OpenCL-ICD-Loader#186 let me get farther and I can build I'm getting a different error for We'll also need to decide if disabling OpenCLon12 is OK in this use-case or if we need to find an alternate solution. |
|
That is awesome @bashbaug ! 🎉 Thank you for helping!
I will investigate this with the BinaryBuilder.jl maintainers. I am currently specifying GCC 6.1 in the build script as the oldest version for which the cross-compilation worked. Any version that is more recent should work fine, but I am not sure about the mingw story as I never developed on Windows myself. Will keep you posted about any update 👍🏽 |
|
Good news, I got There are still a few linker warnings for this configuration that look a little scary though: This appears to have something to do with |
|
The linker warnings do seem to be a problem. This is a good description what is going on: Specifically, for stdcall functions (the calling convention used by the 32-bit Windows ICD loader) mingw is generating decorated exports (with an |
|
Hi @juliohm, I think the last of the ICD loader fixes for mingw were merged today. Would you mind giving the latest code a try to see if any issues still remain? Thanks! |
|
Thank you @bashbaug for the update. I will try to rerun the build script today with Windows in it. I am assuming that I can just clone the master branch of the ICD loader and everything should work fine 🙏🏽 |
|
@bashbaug I've updated the build script at https://github.com/juliohm/cross-platform-opencl with the latest commits from the Headers and ICD-Loader repositories. The build is successful on all platforms except on Windows as before. Any additional patch that I should consider? |
|
Hi, I grabbed the latest build script - thanks! Can you confirm this is the build error you are seeing on Windows? This is from x86_64, but I see a similar error with i686 also: bashbaug@bashbaug-nuc:~/git/cross-platform-opencl$ julia --project build_tarballs.jl --debug x86_64-w64-mingw32
[ Info: Building for x86_64-w64-mingw32
┌ Warning: Linked library CFGMGR32.dll could not be resolved and could not be auto-mapped
└ @ BinaryBuilder.Auditor ~/.julia/packages/BinaryBuilder/wohhx/src/Auditor.jl:383
[ Info: Found a valid dl path OpenCL.dll while looking for libOpenCL
[ Info: Could not locate libOpenCL inside ["/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir/bin"]
┌ Error: Built OpenCL but libopencl still unsatisfied:
└ @ BinaryBuilder ~/.julia/packages/BinaryBuilder/wohhx/src/AutoBuild.jl:906
ERROR: LoadError: Cannot continue with unsatisfied build products!If so, I think we're close, but maybe the build script is looking for the wrong files or in the wrong directory? It looks like everything was built correctly, and I see OpenCL.dll, libOpenCL.dll.a, and all of the right headers: bashbaug@bashbaug-nuc:~/git/cross-platform-opencl$ ls -R /home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir
/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir:
bin include lib logs share
/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir/bin:
OpenCL.dll
/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir/include:
CL
/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir/include/CL:
cl_d3d10.h cl_dx9_media_sharing.h cl_egl.h cl_ext_intel.h cl_gl.h cl_half.h cl_layer.h cl_va_api_media_sharing_intel.h opencl.h
cl_d3d11.h cl_dx9_media_sharing_intel.h cl_ext.h cl_gl_ext.h cl.h cl_icd.h cl_platform.h cl_version.h
/home/bashbaug/git/cross-platform-opencl/build/x86_64-w64-mingw32/m7Tomo8T/x86_64-w64-mingw32-libgfortran3-cxx11/destdir/lib:
libOpenCL.dll.a
...Any ideas? |
Yep, the exact same error. This cross-compilation environment is supposed to be reproducible 🙏🏽
That is the same guess, but I am not familiar with Windows builds to hunt the source of the issue. It is probably specific to mingw compiler flags? |
OK great, just wanted to be 100% sure. I think we need to understand what is going on here:
From my end everything looks like it is working as expected:
We must still be doing something wrong though, or at least not what the BinaryBuilder is expecting. I'll poke around at this but any insight you could provide would be much appreciated. Thank you again for all of your help so far! |
|
Thank you all for the help here 👍🏽 I believe that most issues were solved. Feel free to open a separate issue to track the Windows build if it is still necessary. |
I am trying to cross-compile OpenCL for different applications in Julia, but the build is giving this error as if the CMAKE_CXX_STANDARD at the top level was ignored. Can you please take a look at the following build record?
https://pastebin.com/V5CCYxDP
You can see that the C99 standard is being used for some reason in the build.
The text was updated successfully, but these errors were encountered: