When building in docker container, headers are not copied, so indexer can't use them

[ChristianS99]

Describe the bug
Using cmake4eclipse project and building inside docker container, Indexer is not working correctly.
I think, the underlying Problem is, that headers aren't copied from inside the container to the workspace, so that the indexer can find them.
CMake compilation DB is select as provider in Preprocesser Include Paths, and it shows pathes like <workspace>/.metadata/.plugins/org.eclipse.cdt.docker.launcher/HEADERS/unix_var_run_docker_sock/<imagename>/usr/include

Problem is, <workspace>/.metadata/.plugins/org.eclipse.cdt.docker.launcher/ is empty, no HEADERS dir.

To Reproduce

1. new workspace, new cmake4eclipse project
2. create CMakeLists.txt, source files
3. change project properties to build in docker container
4. build
5. select a systemheader (eg iostream...) in your sources, hit F3

Expected behavior
correct header file opens

Version Information (please complete the following information):
cmake4eclipse version: 4.0.2
Which OS do you use: Linux
Cmake version: 3.27.4
Eclipse Version: 2023-06 (4.28.0)
C/C++ Development Tools: 11.2.0.202306051917

Additional context
when opening the workspace, there is an error message:

eclipse.buildId=4.28.0.20230608-1200
java.version=17.0.7
java.vendor=Gentoo
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.cpp.product
Command-line arguments:  -os linux -ws gtk -arch x86_64 -product org.eclipse.epp.package.cpp.product

org.eclipse.core.jobs
Error
Fri Sep 15 16:52:23 CEST 2023
An internal error occurred during: "Parsing compilation database of project test".

java.lang.NullPointerException: Cannot invoke "org.mandas.docker.client.DockerClient.pull(String, org.mandas.docker.client.ProgressHandler)" because "clt" is null
	at org.eclipse.linuxtools.internal.docker.core.DockerConnection.pullImage(DockerConnection.java:1241)
	at org.eclipse.linuxtools.internal.docker.core.DockerConnection.pullImage(DockerConnection.java:1256)
	at org.eclipse.cdt.internal.docker.launcher.ContainerLaunchUtils.provideDockerImage(ContainerLaunchUtils.java:134)
	at org.eclipse.cdt.docker.launcher.ContainerCommandLauncher.execute(ContainerCommandLauncher.java:311)
	at de.marw.cmake4eclipse.mbs.internal.JsonCompilationDatabaseParser$EnvCommandlauncher.execute(JsonCompilationDatabaseParser.java:597)
	at org.eclipse.cdt.jsoncdb.core.internal.builtins.CompilerBuiltinsDetector.detectBuiltins(CompilerBuiltinsDetector.java:107)
	at org.eclipse.cdt.jsoncdb.core.CompileCommandsJsonParser.detectBuiltins(CompileCommandsJsonParser.java:290)
	at org.eclipse.cdt.jsoncdb.core.CompileCommandsJsonParser.processJsonFile(CompileCommandsJsonParser.java:193)
	at org.eclipse.cdt.jsoncdb.core.CompileCommandsJsonParser.parse(CompileCommandsJsonParser.java:455)
	at de.marw.cmake4eclipse.mbs.internal.JsonCompilationDatabaseParser.parseAndSetEntries(JsonCompilationDatabaseParser.java:316)
	at de.marw.cmake4eclipse.mbs.internal.JsonCompilationDatabaseParser$1.runInWorkspace(JsonCompilationDatabaseParser.java:407)
	at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

See 15knots/cmake4eclipse#187

[jonahgraham]

Thanks @ChristianS99 for the report.

@ChristianS99 can you test creating a simple C/C++ project on your machine with latest Eclipse released this week (2023-09) - for clarity, I mean a project created with the wizard like this, then built on your docker image:

cc: @jjohnstn any chance you can look once @ChristianS99 reports back, @15knots has already done some analysis in 15knots/cmake4eclipse#187

[ChristianS99]

With new eclipse and project without cmake (plain eclipse), the headers are copied and indexer works.

After this, in the same workspace, I created a new project with cmake4eclipse, and use the same build image.
Then the indexer finds and uses the headers, that were already there and indexer is able to bring me to the right header.

Then, in the cmake4eclipse project, I changed the build image, where the headers haven't been copied, they are not copied and indexer is broken.

So, it seems, that the copying headers step is missing when it is a cmake4eclipse project

[jonahgraham]

Thanks @ChristianS99 for reporting back.

I hope that is sufficient information for @jjohnstn and @15knots to find a solution. I can also help, but not sure where to start on this one.

[15knots]

Then, in the cmake4eclipse project, I changed the build image, where the headers haven't been copied, they are not copied and indexer is broken.

linuxtools is the part that copies the headers into the workspace, but only when a build is run. If you just switch the build image, no build is run, no headers get copied.
What happens if you switch the build image in plain CDT with re-building?

[ChristianS99]

linuxtools is the part that copies the headers into the workspace, but only when a build is run. If you just switch the build image, no build is run, no headers get copied. What happens if you switch the build image in plain CDT with re-building?

In plain CDT, if I switch the build image and click "Apply", I see "Discover compiler builtin settings" followed by "Copying from Docker" in the right part of the status bar. I don't have to build the project.

On the other hand, in a project with cmake4eclipse, the copying step isn't run after building.

[15knots]

It works in plain CDT, because GCCBuiltinSpecsDetector gets some special treatment by org.eclipse.cdt.internal.docker.launcher.ContainerPropertyTab which calls GCCBuiltinSpecsDetector#handleEvent().
So this is not a bug in CDT (apart from the NPE above).

But I think the code in ContainerPropertyTab#recalculateSpecs() looks quirky to me. It does an instance check if (provider instanceof GCCBuiltinSpecsDetector) but method handleEvent() is specified for interface org.eclipse.cdt.utils.envvar.IEnvironmentChangeListener. Checking for the concrete class instead of the interface makes is hard to solve 15knots/cmake4eclipse#187.

[ChristianS99]

Hm, so this won't be fixed, as far as I understand?

[15knots]

Changing the instanceof check from GCCBuiltinSpecsDetector to IEnvironmentChangeListener could break existing IEnvironmentChangeListener implementations because ContainerPropertyTab passes null as the argument to handleEvent(). I doubt any existing IEnvironmentChangeListener is prepared to get a null event object.

Anyway, if the container image gets switched anything in the build directory should be considered to be potentially stale. So a clean rebuild is required to make it non-stale. This will also copy the header files.

In plain CDT (we call it Managed Build System MBS), the headers get copied back as a side effect of notifying the IEnvironmentChangeListener: The GCCBuiltinSpecsDetector runs a command in the container (gcc in this case) and the docker launcher then copies back the headers, regardless which command is run.

[ilario-gottardello]

Hello, I have this exact same problem, using cmake4eclipse and docker headers files are not copied. Could you please provide me a workaround? Thank you