Noisey Log Messages #9
Comments
|
Biggest red flag from this output is that it's missing the logger name and log level that's creating that message: meltano/sdk#1232. Those messages must be emitted at the INFO level, since that's what we configure by default for the root logger in https://github.com/meltano/sdk/blob/cf51acc6debf608da12ffbeef99aacc0a43e5a99/singer_sdk/default_logging.yml#L1-L15. Once we figure out a better log format than |
|
@pnadolny13 can I ask you a favor? Can you run the tap with SINGER_SDK_LOG_CONFIG=sdk_logging.yaml tap-snowflake ...where version: 1
disable_existing_loggers: no
formatters:
default:
format: "{asctime} {levelname} {name} {message}"
style: "{"
handlers:
console:
class: logging.StreamHandler
formatter: default
stream: ext://sys.stderr
loggers:
"":
level: INFO
propagate: no
handlers: [console]That should tell us which logger is emitting those messages, so we can make it quieter by default. |
@edgarrmondragon Sure! Here it is: |
|
@pnadolny13 thanks! So, those are coming from https://github.com/snowflakedb/snowflake-connector-python. You should be able to quiet them for the moment with version: 1
disable_existing_loggers: no
handlers:
console:
class: logging.StreamHandler
stream: ext://sys.stderr
level: WARNING
loggers:
snowflake.connector.cursor:
handlers: [console]
propagate: no |
|
@edgarrmondragon do you have an idea of how to fix this? Is it a case of shipping a log config Ref: SO question. |
|
@kgpayne There's no easy way to do this at the moment so I created meltano/sdk#1373 |
|
I find this problem quite annoying and performance detrimental. version: 1
disable_existing_loggers: false
formatters:
simple:
format: "{levelname} | {name} | {pathname} | {message}"
style: "{"
default:
format: "%(message)s"
json:
(): meltano.core.logging.json_formatter
key_value:
(): meltano.core.logging.key_value_formatter
sort_keys: False
handlers:
console:
class: logging.StreamHandler
formatter: default
stream: "ext://sys.stderr"
null_handler:
class: logging.NullHandler
root:
level: WARNING
handlers: [console]
loggers:
snowflake.connector.cursor:
level: CRITICAL
handlers: [null_handler]
propagate: no
singer_sdk.metrics:
level: INFO
handlers: [console]
propagate: no
I want the Its all or nothing with this config. I can only control the root logger but other logger configuration is ineffective |
Did you use the You can set that in # meltano.yml
env:
SINGER_SDK_LOG_CONFIG: logging/singer_logging.yaml |
Its possible that this is part of the SDK in which case I can transfer it over to the SDK repo but I'm getting a ton of logs that seem like theyre debug level. Context around queries isnt all that relevant for the tap user.
An example of part of my log:
cc @kgpayne @edgarrmondragon
The text was updated successfully, but these errors were encountered: