no metric on File source on max_byte_line exceeded

[filipmnowak]

hello there!

as in the title: seems like File source is not emitting any useful metric signifying this event? or perhaps i am missing something?

perhaps rejection should count into component_discarded_events_total?

For following source configuration:

  test_source:
    type: file
    include:
      - logs/*.txt
    read_from: beginning
    data_dir: vector_work_dir
    max_line_bytes: 2

and following log:

a
b
c
abc

i am getting:

log_collector_component_received_events_total (...) 5
log_collector_component_sent_events_total (...) 5
log_collector_events_in_total (...) 5
log_collector_events_out_total (...) 5

thank you,
Filip

[bencord0]

Is the component_errors_total metric good enough? for your purpose?

https://vector.dev/docs/reference/configuration/sources/file/#component_errors_total

[bruceg]

You are correct. That condition does not translate into a metric at this point. Rejected log lines haven't been turned into events, so they don't count as discarded_events, however it would be useful to have visibility on that discard happening in another form. I have opened an issue to track this: #14321

[filipmnowak]

thank you @bruceg; i am not sure if #14321 covers kubernetes_logs too, but it would be even more useful to have ability to count such events for that source also.

[bruceg]

I have made note of that. It would likely fall out naturally, but it's good to be explicit about it.