Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[c2cpg] Fixed logging via CCorePlugin #5175

Closed
wants to merge 1 commit into from

Conversation

max-leuthaeuser
Copy link
Contributor

This PR adds a stripped-down version of the original org.eclipse.cdt.core.CCorePlugin shadowing it and providing only the functionality to get it running without all the Eclipse OSGI context.

Sadly, some parser internal components (e.g., the ambiguity resolving) log via this class. Without a proper OSGI instantiation (which we do not have/want in Joern) we are running into all kind of exceptions due to non-initialized entities (e.g., the said logging utils).

Discovered during testing c2cpg on https://github.com/chromium/chromium where we have a lot of ambiguous nodes during parsing that can't be resolved and are logged causing the said exceptions.

This PR adds a stripped-down version of the original `org.eclipse.cdt.core.CCorePlugin` shadowing it and
providing only the functionality to get it running without all the Eclipse OSGI context.

Sadly, some parser internal components (e.g., the ambiguity resolving) log via this class.
Without a proper OSGI instantiation (which we do not have/want in Joern) we are running into
all kind of exceptions due to non-initialized entities (e.g., the said logging utils).

Discovered during testing c2cpg on https://github.com/chromium/chromium where we have a lot of
ambiguous nodes during parsing that can't be resolved and are logged causing the said exceptions.
@mpollmeier
Copy link
Contributor

I have some doubts about the shadowing... with the proposed setup we get two versions of org.eclipse.cdt.core.CCorePlugin.class in our classpath, and some unforseen magic in the classloader decides at runtime which one is being loaded... Am I missing something?

Not sure if this works with cdt, but have you considered shading instead of shadowing? I.e., copy the source to our repo and put it into a different namespace, similar to what I did in the repl: https://github.com/mpollmeier/scala-repl-pp/tree/main/shaded-libs/src/main/scala/replpp/shaded

Another option: given that we already ship a redeployed version of cdt, we could also fiddle with the jar before publishing it, i.e. remove the original .class file. Doesn't sound very nice though.

@max-leuthaeuser
Copy link
Contributor Author

max-leuthaeuser commented Dec 12, 2024

and some unforseen magic in the classloader decides at runtime which one is being loaded

Hm, might be true. I was hoping the class files from c2cpg always take precedence.

Not sure if this works with cdt, but have you considered shading instead of shadowing?

Yes, but I wasn't able to get it running. If you want to try it feel free to alter this PR here directly. Note: CCorePlugin.log is called from within org.eclipse.cdt.foo (e.g., org.eclipse.cdt.internal.core.dom.parser) and not our code in reality.

@mpollmeier
Copy link
Contributor

I was hoping the class files from c2cpg always take precedence.

No, there's nothing special about the project's sources once it's all packaged up...
See e.g. after sbt stage:

➜  lib git:(max/fixCCorePluginLogging) unzip -l io.joern.c2cpg-4.0.182+1-d55c4699.jar|grep CCorePlugin
     3049  2010-01-01 00:00   org/eclipse/cdt/core/CCorePlugin.class
➜  lib git:(max/fixCCorePluginLogging) unzip -l io.joern.eclipse-cdt-core-8.5.0.202410191453+2.jar |grep CCorePlugin
    39710  2024-12-09 10:25   org/eclipse/cdt/core/CCorePlugin.class
     4077  2024-12-09 10:25   org/eclipse/cdt/core/CCorePlugin$2.class
     1104  2024-12-09 10:25   org/eclipse/cdt/core/CCorePlugin$NullConsole$1.class
     2472  2024-12-09 10:25   org/eclipse/cdt/core/CCorePlugin$1.class
     1290  2024-12-09 10:25   org/eclipse/cdt/core/CCorePlugin$NullConsole.class
     5944  2024-12-09 10:25   org/eclipse/cdt/internal/core/CCorePluginResources.properties

@mpollmeier
Copy link
Contributor

Note: CCorePlugin.log is called from within org.eclipse.cdt.foo (e.g., org.eclipse.cdt.internal.core.dom.parser) and not our code in reality.

In that case shading doesn't seem to be a good option either. We'd need to build the entire cdt project and publish it separately.

Another option would be to remove the class from the original jar on sbt stage, but that's not a good idea either because it needs to be repeated for every stage/assembly config downstream as well, and it means we're lying about the project's dependencies.

Sounds like there's no good solution here, and out of the bad solutions I lean towards removing the class from the jar when we republish it... thoughts?

@max-leuthaeuser
Copy link
Contributor Author

Sounds like there's no good solution here, and out of the bad solutions I lean towards removing the class from the jar when we republish it... thoughts?

I was thinking about this in first place, yes. Only downside: without a custom CCorePlugin like we do in c2cpg this republished jar is useless.

mpollmeier added a commit that referenced this pull request Dec 12, 2024
Adapted from #5175

This PR adds a stripped-down version of the original `org.eclipse.cdt.core.CCorePlugin` and providing only the functionality to get it running without all the Eclipse OSGI context.
To avoid classpath issues we removes the original class when we repulish the jar...

Sadly, some parser internal components (e.g., the ambiguity resolving) log via this class. Without a proper OSGI instantiation (which we do not have/want in Joern) we are running into all kind of exceptions due to non-initialized entities (e.g., the said logging utils).

Discovered during testing c2cpg on https://github.com/chromium/chromium where we have a lot of ambiguous nodes during parsing that can't be resolved and are logged causing the said exceptions.
@mpollmeier
Copy link
Contributor

That's true, although maybe not the end of the world.

That being said, we could move the simplified CCorePlugin into the custom build that we create anyway for republishing. Like so: #5178

@max-leuthaeuser
Copy link
Contributor Author

Ok, let's do it that way.

@max-leuthaeuser max-leuthaeuser deleted the max/fixCCorePluginLogging branch December 12, 2024 09:18
mpollmeier added a commit that referenced this pull request Dec 12, 2024
* [c2cpg] Fixed logging via CCorePlugin

This PR adds a stripped-down version of the original `org.eclipse.cdt.core.CCorePlugin` shadowing it and
providing only the functionality to get it running without all the Eclipse OSGI context.

Sadly, some parser internal components (e.g., the ambiguity resolving) log via this class.
Without a proper OSGI instantiation (which we do not have/want in Joern) we are running into
all kind of exceptions due to non-initialized entities (e.g., the said logging utils).

Discovered during testing c2cpg on https://github.com/chromium/chromium where we have a lot of
ambiguous nodes during parsing that can't be resolved and are logged causing the said exceptions.

* [c2cpg] Fixed logging via CCorePlugin

Adapted from #5175

This PR adds a stripped-down version of the original `org.eclipse.cdt.core.CCorePlugin` and providing only the functionality to get it running without all the Eclipse OSGI context.
To avoid classpath issues we removes the original class when we repulish the jar...

Sadly, some parser internal components (e.g., the ambiguity resolving) log via this class. Without a proper OSGI instantiation (which we do not have/want in Joern) we are running into all kind of exceptions due to non-initialized entities (e.g., the said logging utils).

Discovered during testing c2cpg on https://github.com/chromium/chromium where we have a lot of ambiguous nodes during parsing that can't be resolved and are logged causing the said exceptions.

* upgrade eclipse cdt (release pending)

* works now apparently

---------

Co-authored-by: Max Leuthäuser <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants