Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for the gNMI extension field #503

Open
Ichabond opened this issue Aug 8, 2024 · 3 comments
Open

Support for the gNMI extension field #503

Ichabond opened this issue Aug 8, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@Ichabond
Copy link

Ichabond commented Aug 8, 2024

gNMI extensions (https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-extensions.md) allow for extended features in the gRPC messages sent back.
The extension field is present at the top-level of the gNMI RPCs.
Juniper as a vendor is registering an extensionID for their telemetry: openconfig/gnmi#76
Right now, their prototext looks like this:

extension: {
  registered_ext: {
    id: 1
    msg: "\n\x0cedge01.lis01\x10\xff\xff\x03\"\x0fsensor_1007_2_1*d/interfaces/interface[name='irb']/subinterfaces/subinterface[index='100']/state/counters/out-octets/2d/interfaces/interface[name='irb']/subinterfaces/subinterface[index='100']/state/counters/out-octets/:\x05mib2d@\x01H\xff\x89\x99\x97\x932P\xff\x89\x99\x97\x932`\x81\x8a\x99\x97\x932p\x01\x80\x01\x02"
  }
}

which contains a lot of helpful data, which should be exported as tags on the events.

The ask:
Add generic support for the extension field, allowing for arbitrary protobuf files to be used to decode extension IDs (a config map of ID -> protobuf file maybe?). Then use the field in processor to generate tags from it.

The specific reason for this ask:
This extension field contains linecard information of the linecard generating the response. This is necessary data to deduplicate certain events (logical interfaces spanning multiple linecards will generate an event per linecard, which will be clobbered, given they look like duplicates right now). If the extension data was exported as tags, it would be a unique event.

@karimra karimra added the enhancement New feature or request label Aug 8, 2024
@Ichabond
Copy link
Author

Specifically, this is an extension of the existing --proto-file support, but which is locked to the Nokia SROS protofiles.

I'll have a look if this could be done by just letting the user define a mapping of:
extensionID -> protofile + proto root.

This would then get decoded properly, I think, given I'm seeing Unmarshall errors on the JSON side which are related to the binary-marshalled protobuf in the msg field inside extension.
So I think if the code here: https://github.com/openconfig/gnmic/blob/main/pkg/api/target/subscribe.go#L290 is made more generic, at least decoding would work.
From there, I think it being converted to tags would "just work"?
Will try to get some PoC going to prove/disprove this understanding.

@Ichabond
Copy link
Author

@karimra I have a working prototype.
Before I make a PR though, a couple questions I'd appreciate your input on:

  1. The current protobuf reflection is fo SROS-only, and relies on a fully compiled-in Protobuf that can decode whatever is in the Protobytes. As users might be relying on this feature for SROS, I would add it as a new distinct feature, specifically geared at the extenions field?
  2. Configuration location: I'd add it as a subsection of the targets dictionary, given this is where the existing proto-parsing lives:
    func (a *App) parseProtoFiles(t *target.Target) error {

@karimra
Copy link
Collaborator

karimra commented Aug 12, 2024

  1. The current protobuf reflection is fo SROS-only, and relies on a fully compiled-in Protobuf that can decode whatever is in the Protobytes. As users might be relying on this feature for SROS, I would add it as a new distinct feature, specifically geared at the extenions field?
  2. Configuration location: I'd add it as a subsection of the targets dictionary, given this is where the existing proto-parsing lives:
    func (a *App) parseProtoFiles(t *target.Target) error {

Yes for both, something like:

targets:
  router1:
    address: x.x.x.x
    #
    # other fields
    #
    registered-extensions:
      - id: 42
        proto-file: /path/to/protofile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants