Log level consistency (for discussion)

[candlerb]

snmp_exporter version

snmp_exporter, version 0.24.0 (branch: HEAD, revision: c9fd025309572b81eaca65100de4ca38ea406da9)
  build user:       root@fb4e1a1f31ed
  build date:       20230829-12:20:02
  go version:       go1.21.0
  platform:         linux/amd64
  tags:             netgo

What did you do that produced an error?

Some things which intentionally cause errors:

• Make a scrape with a missing target parameter: curl -sS 'localhost:9116/snmp'
• Make a scrape which times out: curl -sS 'localhost:9116/snmp?module=if_mib&target=1.2.3.4'

These both return meaningful error messages in the HTTP response, via http.Error()

What did you expect to see?

These errors also to be consistently available in logs.

What did you see instead?

The missing target error is not logged at all, not even at "debug" level.

The target scrape error is logged at "info" level:

Sep 01 10:09:28 prometheus snmp_exporter[188994]: ts=2023-09-01T09:09:28.788Z caller=collector.go:393 level=info auth=public_v2 target=1.2.3.4 module=if_mib msg="Error scraping target" err="error getting target 1.2.3.4: request timeout (after 3 retries)"

For comparison: blackbox_exporter logs some parameter errors at "debug" level (e.g. "Unknown module"), but not others (e.g. "Target parameter is missing"). In any case I think these should be at "info" or higher, otherwise you never see them unless you turn on debug, which gives you a lot of chatter for successful probes as well.

Other notes

These errors will be detected by prometheus itself as up==0. However, under Status > Targets the HTTP response body isn't visible. Instead you get:

server returned HTTP status 400 Bad Request

or

Get "http://127.0.0.1:9116/snmp?auth=xxxx&module=yyyy&target=z.z.z.z": context deadline exceeded

Hence having the actual error appear in logs is helpful for debugging; otherwise you need to make a manual scrape (e.g. with curl) to see the error, or tcpdump the traffic to the exporter.