Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zipkin tracing duration precision too high for Jaeger/Zipkin #5

Open
t3hmrman opened this issue Apr 12, 2018 · 0 comments
Open

Zipkin tracing duration precision too high for Jaeger/Zipkin #5

t3hmrman opened this issue Apr 12, 2018 · 0 comments

Comments

@t3hmrman
Copy link

t3hmrman commented Apr 12, 2018

Hey I just got the library working and am pretty grateful!

tl;dr - duration is too high precision for jaeger/zipkin by default for spans (maybe it should be abstracted into another type to ensure max precision?), also current calculation using toMicros might be wrong

I noticed one small issue (that I just fixed) -- Span durations that are getting sent to Jaeger are actually too precise for it to parse, you end up getting an response like this:

[2018-04-12 01:52:40 JST : App.Tracing : DEBUG] Zipkin Resp: Just (Response {
responseStatus = Status {statusCode = 400, statusMessage = "Bad Request"}, 
responseVersion = HTTP/1.1, 
responseHeaders = [("Content-Type","text/plain; charset=utf-8"),("X-Content-Type-Options","nos
niff"),("Date","Wed, 11 Apr 2018 16:52:40 GMT"),("Content-Length","120")], 
responseBody = "Unable to process request body: json: cannot unmarshal number 1500.525 into Go struct field Span.duration of type int64\n", 
responseCookieJar = CJ {expose = []}, 
responseClose' = ResponseClose
}) 

(I spaced out the response for readability)

Zipkin does the same thing:

[2018-04-12 12:21:13 JST : App.Tracing : DEBUG] Finishing request span (OpName "api/v1/me"), duration = 0.001499978s
[2018-04-12 12:21:13 JST : App.Tracing : DEBUG] Publishing 1 saved spans...
[2018-04-12 12:21:13 JST : App.Tracing : DEBUG] Zipkin Resp: Just (Response {responseStatus = Status {statusCode = 400, statusMessage = "Bad Request"}, responseVersion = HTTP/1.1, responseHeaders = [("Connection","keep-alive"),("vary","origin"),("Content-Length","19"),("Date","Thu, 12 Apr 2018 03:21:13 GMT")], responseBody = "Cannot decode spans", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose})

(I included more output here, but you can see Zipkin has a problem too it's just less descriptive about it)

So what I did was implement a method that reduced the precision (yikes?):

-- ^ Reduce the precision of the NominalDiffTime objects that are being sent
-- this is necessary because the following error gets sent from Jaeger (when sending a NominalDiffTime of "0.001500525s" ):
-- "Unable to process request body: json: cannot unmarshal number 1500.525 into Go struct field Span.duration of type int64\n"
-- (running regular zipkin will just produce a "failed to parse" error with no details)
reducePrecision :: (Num a, RealFrac a) => Int -> a -> a
reducePrecision places = (/ factor) . fromInteger . round . (* factor)
    where
      factor = 10 ^ places

That said, I'm actually generating my Spans manually (not using the monad transformers provided by the library), but I find that when I copy your approach in publishZipkin w/ toMicros, I get the wrong result in Jaeger I think:

traces-with-different-methods

The most recent trace was using toMicros instead of the reducePrecision I wrote. As I'm somewhat outside of the usual case (not using the Monad transformer you provided, building the Span myself), please feel free to let me know if you're not seeing this issue, maybe I misread the code using toMicros and it's doing something more.

PS: I also tried with zipkin and see both things happening (duration used as-is not working and the wrong timing when using toMicros as I saw it being used):

zipkin-traces-with-different-methods

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant