parse_grok! expression can't fail even though it's marked fallible

[kvberge]

Hello, it seems like there is some confusion (maybe on my part) when trying to handle a failure in parse_grok! - I'm getting this error when I attemp to remap using parse_grok!:

error[E651]: unnecessary error coalescing operation
  ┌─ :1:17
  │
1 │   structured =  parse_grok!(.message, "%{TIME:time}%{SPACE}%{POSINT:pid}%{SPACE}%{BASE16NUM:thread}%{SPACE}%{LOGLEVEL:level}%{SPACE}%{WORD:category}%{SPACE}%{PROG:prog}%{SPACE}%{GREEDYDATA:message}") ?? {"message":.message, "level": "info" }
  │                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- -------------------------------------- this expression never resolves
  │                 │                                                                                                                                                                                       │
  │                 │                                                                                                                                                                                       remove this error coalescing operation
  │                 this expression can't fail
  │
  = see language documentation at https://vrl.dev
  = try your code in the VRL REPL, learn more at https://vrl.dev/examples

parse_grok is marked as fallible, so I'm not sure why this expression can't fail. We have messages that are not formatted properly and failing this grok parsing (which is currently expected as our app has a mix of 'formatted' and 'un-formatted' logging causing the failures), causing logs in vector like this:

2024-12-31T16:33:05.610160Z ERROR transform{component_kind="transform" component_id=cust_apps component_type=remap}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"parse_grok\" at (8:191): unable to parse input with grok pattern" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-12-31T16:33:07.419820Z ERROR transform{component_kind="transform" component_id=cust_apps component_type=remap}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being suppressed to avoid flooding.

For now I was just trying to naively transform the level to info to avoid other processing we have later on which would ship these messages with an incorrect level downstream.

I am running vector in a docker container:

vector 0.39.0 (aarch64-unknown-linux-gnu 73da9bb 2024-06-17 16:00:23.791735272)

Using timberio/vector:0.39.0-debian as the base image.

[jszwedko]

Hi @kvberge !

The issue here is that you are using a ! when calling parse_grok(). That is, you are using parse_grok!() rather than parse_grok(). The ! tells VRL to abort if the function call fails and so the ?? expression you have here will never be executed. Instead you likely want:

1 │   structured =  parse_grok(.message, "%{TIME:time}%{SPACE}%{POSINT:pid}%{SPACE}%{BASE16NUM:thread}%{SPACE}%{LOGLEVEL:level}%{SPACE}%{WORD:category}%{SPACE}%{PROG:prog}%{SPACE}%{GREEDYDATA:message}") ?? {"message":.message, "level": "info" }

[kvberge]

Ah, thank you for the response. I hadn't realized the difference between a ! and not using the !.

[jszwedko]

No worries, it is subtle. There are some more details about error handling in VRL here: https://vector.dev/docs/reference/vrl/errors/#handling