Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using X-Ray Agent in Spring Boot & Zuul Proxy API Gateway #111

Open
yukselcodingwithyou opened this issue Dec 1, 2021 · 1 comment
Open

Comments

@yukselcodingwithyou
Copy link

We are trying to instrument our microservices using x-ray java agent. Our microservices are implemented using Spring Boot framework. For other microservices, we accomplished the auto instrumentation using the disco plugins by adding environment variables to our applications as below.

-Dcom.amazonaws.xray.strategy.tracingName=serviceName
-Dcom.amazonaws.xray.strategy.contextMissingStrategy=IGNORE_ERROR
-Dcom.amazonaws.xray.emitter.daemonAddress=127.0.0.1:2000
-Dcom.amazonaws.xray.tracingEnabled=True
-javaagent:src/main/resources/disco/disco-java-agent.jar=pluginPath=src/main/resources/disco/disco-plugins

But in our API Gateway, we failed to do so. While we try to trace any request from our API Gateway, we came across an error like this below:

{
"error": "Exception while serializing entity.",
"stack_trace":"com.fasterxml.jackson.databind.JsonMappingException: Failed to load class
'com.amazonaws.xray.entities.SegmentImpl$Access4JacksonSerializerdb8acfce':
com.fasterxml.jackson.module.afterburner.ser.BeanPropertyAccessor
com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:295)
}
full-stack-trace.json.zip

Also in our x-ray daemon running locally, we encountered with this warning below:

[Warn] Missing header or segment: {"format": "json", "version": 1}

To resolve this error, we tried to exclude plugin path from command like below:

-Dcom.amazonaws.xray.strategy.tracingName=bid-service-local
-Dcom.amazonaws.xray.strategy.contextMissingStrategy=IGNORE_ERROR
-Dcom.amazonaws.xray.emitter.daemonAddress=127.0.0.1:2000
-Dcom.amazonaws.xray.tracingEnabled=True
-javaagent:src/main/resources/disco/disco-java-agent.jar

But there was no segment sent to the daemon that is running locally. Then we tried to add the plugins to its path one by one,

aws-xray-agent-plugin.jar
disco-java-agent-aws-plugin.jar

First, we added the two jars above to disco-plugins path above, there was no error, but segment also not sent to our daemon .

Then, we added the jar below and, we saw that this error is related to this plugin, with some dependencies that we use in our API Gateway application.

disco-java-agent-web-plugin.jar

Dependencies & Technologies

Spring Boot version is 2.3.8.

 - implementation "org.springframework.boot:spring-boot-configuration-processor"
 - implementation "org.springframework.boot:spring-boot-starter-actuator"
 - implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
 - implementation 'org.springframework.cloud:spring-cloud-starter-netflix-zuul'
 - implementation 'org.springframework.cloud:spring-cloud-starter-netflix-hystrix-dashboard'
 - implementation 'org.springframework.cloud:spring-cloud-starter-netflix-hystrix'
 - implementation 'org.springframework.cloud:spring-cloud-starter-netflix-ribbon'
 - implementation "org.springframework.cloud:spring-cloud-starter-sleuth:2.2.8.RELEASE"

Also we are using ZuulFilter to add custom headers to incoming requests' header. Could this also be an issue for our error that we encounter, how can we move forward from here? 🙏

@yukselcodingwithyou yukselcodingwithyou changed the title Using X-Ray Agent in Spring Boot API Gateway Using X-Ray Agent in Spring Boot & Zuul Proxy API Gateway Dec 1, 2021
@willarmiros
Copy link
Contributor

willarmiros commented Dec 20, 2021

Hi @yukselcodingwithyou,

Apologies for the delayed response. Just to confirm, when you mention your Zuul proxy API Gateway, you're referring to a service that you own and not AWS's API Gateway service right?

Regardless, I'm not too sure about this issue as I haven't seen it before but perhaps it is related to FasterXML/jackson-modules-base#131 based on a quick google search. Currently the agent pulls in v2.11.0, so it could be that upgrading the agent's version would fix this. Is there any way you could provide some reproduction code for this, so we can verify the fix?

Also, just as an FYI, if you're still having the X-Ray java agent feel free to try out the AWS Distro for OpenTelemetry Java Agent. It natively supports instrumenting many more libraries/frameworks, and has 1st-class support for X-Ray.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants