log4j-layout-template-json Add support to Compact Log Event Format

[amosshi]

Warning!
It is highly recommended to discuss feature requests in the mailing lists first.

[A clear and concise description of the feature requested.]

The Problem

Currently log4j2 provides several JSON templates at:

• https://github.com/apache/logging-log4j2/tree/2.x/log4j-layout-template-json/src/main/resources

All of current template will generate log files with long key name, which produces bigger log output.

The Request

We deliver a compact JSON format, like CLEF or similar, to make the log output smaller.

Example: CLEF format could be 47% smaller than the default EcsLayout.json.

Reference

• https://clef-json.org
• https://github.com/serilog/serilog-formatting-compact

[jaykataria1111]

Seems interesting,

I would be curious to know, how this is implemented, or looked into. I am getting familiar with the repoo, this looks like an interesting feature for people.

[vy]

@amosshi, @jaykataria1111, would you mind providing a JSON Template Layout event template for CLEF and publishing it as a part of the CLEF project with a Apache 2 License compatible license, please? If the CLEF community provides such an integration, we would be happy to point to it in the Log4j manual.

[amosshi]

Hi @vy , serilog/serilog-formatting-compact#69 created for CLEF side.

[TiloGit]

@amosshi I really like your idea and suggestion. Did you get a chance to add a PR to https://github.com/clef-json/clef-json.github.io?

[amosshi]

Hi @TiloGit ,

I have connected with serilog/serilog-formatting-compact#69 - since it is a .NET library cannot used by Java application, so it is not proper to add a PR to it.

So the current idea is

• Add the config file to an open source project at https://github.com/sap-contributions/log4j-layout-template-json-compact

For now,

• we have created the compact file in an SAP Internal repo, and being validated in pipeline and will be used in Production environment in 2025 (I hope)
• Once validated, and going through some open-source protocol, we will contribute it to https://github.com/sap-contributions/log4j-layout-template-json-compact
• Then we will raise a PR to log4j2 document site, as @vy suggested

More info

• The format https://github.com/clef-json/clef-json.github.io is still big than needed
• Suppose we have more than 1 billion transaction per day, then every un-necessary extra character in log counts
• So we need an even smaller format than clef-json

[vy]

@amosshi, really liked the idea of releasing the template in a dedicated artifact! 🤩

1. The provided sap-contributions/log4j-layout-template-json-compact link returns 404 for me, I can neither find a sap-contributions repository containing log4j in its name. Any ideas?
2. Once you publish your project (with sufficient README.adoc meat) and validate it works, would you mind submitting PRs to Log4j's log4j-json-template-layout.adoc and clef-json.github.io, please?