Exemplar sketch as the first step for OpenMetrics support

[jeremiahishere]

I posted a few months volunteering to work on OpenMetrics support in this gem on this issue page: #189

I have just posted a change list on my fork of the repo with a potential exemplar storage and interface plan: jeremiahishere#2. This sketch was requested by @dmagliola and I was told @Sinjo may be interested as well.

The general idea is to get exemplar support in place without making the breaking changes required for OpenMetrics around _total, _created, and unit types.

I have deliberately kept features to a minimum around exemplar ingestion and storage. I want to make sure the data structures look ok and figure out what to do with the direct file store. Could I get some feedback on this before I start working on a fully featured version?

[Sinjo]

Thank you for sending this over! I'm a little busy with some travel this week, but I'll try to find time to look over it.

[Sinjo]

Sorry for the radio silence. I should have some time to give this a look over towards the end of the week or over the weekend.

[Sinjo]

Hi @jeremiahishere,

Sorry for the slow reply. I've taken a couple of reads over the sketch and have some initial thoughts:

• At an API level this feels broadly right. Introducing a new optional parameter to metrics and stores feels like the right choice, though is still a breaking change to the store interface (I'll touch on this later).
• The introduction of the new media type looks right to me.
• We definitely need to think hard about how we store these values in DirectFileStore. I'll need to sit down with the OpenMetrics spec open and think the proposed approach through.

So, onto the breaking change. While I'm not aware of any third-party data stores in the wild, we did commit to this interface way back in the 1.0 release and have avoided breaking it since. I'm not opposed to breaking it, but we'll need a major version bump to do so.

Since we're making breaking changes for OpenMetrics support, I would ideally like to make them in one major version. Looking at #189, it calls out the enforcement of the _total suffix on counters as a breaking change. I'd like us to start emitting warnings for this in the current major version so that we can enforce it in the next one.

There may be other breaking changes I'm not aware of. I'd like to try and land them in one major version if we can. I think this approach will let us temporarily skip things like the new metric types (StateSet and Info) without resulting in another breaking release down the line.

In an ideal world I'll find an afternoon to brew a coffee and look through the OpenMetrics spec in its entirety, but free time has been hard to find recently.

@dmagliola I'm curious to hear your thoughts. Maybe this is something we could chat about next time we see each other and then summarise here?