Allow logging of None type values on logging context via a config value

[CaptainDriftwood]

Use case

The built in logging formatter provides very good defaults, however is there a potential work around to avoid stripping logging context whose value is None? This method call appears to remove any key value pairs on the log context whose value is None:

powertools-lambda-python/aws_lambda_powertools/logging/formatter.py

Line 181 in e87b2b0

formatted_log = self._strip_none_records(records=formatted_log)

This would allow log messages to show context that is set to None which would be valuable in certain circumstances.

Solution/User Experience

Allow the passing in of a keyword argument that allows None type values to be logged, as well as an environment variable that configures this behavior.

Alternative solutions

No response

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

[heitorlessa]

giving you an answer until tomorrow as to why; we've been heads down on putting controls to prevent future regressions due to optional dependencies we had in the last release, hence why we missed.

[heitorlessa]

hey @CaptainDriftwood appreciate your patience while we resolved a higher priority - created a complete workaround for you at the bottom.

I've added help wanted and need-customer-feedback to help attract +1, and anyone interested in making a PR for this -- we didn't do it earlier to not make assumptions about certain scenarios (please share yours to help out :)).

To work around it, you can provide your own Formatter and overriding _strip_none_records static method.

The side effect of allowing a None value is that it will include other possibly unwanted keys like exception: null, polluting your logs (and $ however small depending on log volume).

To save you time, I've covered that side effect in the example at the bottom along with the option for you to extend whatever key you don't want to be present if None too.

Hope that that still helps

Side effect example

{
   "level":"INFO",
   "location":"<module>:19",
   "message":"Key with None will not be dropped",
   "timestamp":"2024-06-21 09:21:22,482+0200",
   "service":"service_undefined",
   "sampling_rate":null,
   "customer_id":null,
   "exception":null,
   "exception_name":null,
   "stack_trace":null,
   "xray_trace_id":null
}

---

from typing import Dict, Any

from aws_lambda_powertools.logging import Logger
from aws_lambda_powertools.logging.formatter import LambdaPowertoolsFormatter


class AllowNoneFormatter(LambdaPowertoolsFormatter):
    POWERTOOLS_OPTIONAL_FIELDS = (
        "exception",
        "exception_name",
        "stack_trace",
        "xray_trace_id",
        "sampling_rate",
    )

    @staticmethod
    def _strip_none_records(records: Dict[str, Any]) -> Dict[str, Any]:
        return {
            k: v
            for k, v in records.items()
            if k not in AllowNoneFormatter.POWERTOOLS_OPTIONAL_FIELDS or v is not None
        }

logger = Logger(logger_formatter=AllowNoneFormatter())

logger.info("Key with None will not be dropped", customer_id=None)