error extracting the metadata for a Quay.io image #5885

actraiser2 · 2024-08-04T12:05:31Z

Description:
I'm obtaining the following error when trying to get the application metadata related to quay.io image:

org.springframework.web.client.UnknownContentTypeException: Could not extract response: no suitable HttpMessageConverter found for response type [interface java.util.Map] and content type [binary/octet-stream]

Release versions:
Latest version: 2.11.4

Steps to reproduce:
This is the configuration associated to the quay.io registry in the application.yaml:
.....
dataflow
container:
registry-configurations:
quay:
registry-host: quay.io
authorization-type: dockeroauth2
user: ${QUAY_USERNAME}
secret: ${QUAY_PASSWORD}
extra:
registryAuthUri: "https://quay.io/v2/auth?service=quay.io&scope=repository:{repository}:pull"_

Screenshots:

*Additional context:
The error produces after spring cloud data flow has redirected to the following ur.l:

https://cdn03.quay.io/quayio-production-s3/sha256/dd/dd1561170543fb2b2fa23c0ff7b880ebd34db620d4a38faf2822cef0ce6cc12c?X-Amz-Algorithm=AWS4-HMAC-SHA256&....................................

If i curl the previous url, i can get all the image metadata with a response header Content-Type: binary/octet-stream, that Spring doesn't like:

**{
"architecture": "amd64",
"created": "2024-06-11T07:53:55.512358385Z",
"history": [
..........
}
"config": {
"Entrypoint": [
.....
"Env": [
"......
],
"Labels": {
......
},

}
}**

corneil · 2024-08-05T09:05:21Z

@actraiser2 Thank you for raising the issue.
Can you please attach a more detailed stacktrace in text format?

actraiser2 · 2024-08-05T10:12:41Z

Here is the detailed stacktrace:

2024-08-05 12:05:37.336 DEBUG 1 --- [io-8080-exec-10] thorizationHeaderRequestRedirectStrategy : [!dt dt.entity.process_group=PROCESS_GROUP-A4C0D936FBECC05C, dt.trace_sampled=true, dt.trace_id=a18236f5615737891e2ba12d36df8aec, dt.entity.host_group=HOST_GROUP-7D678375C12400F3, dt.entity.process_group_instance=PROCESS_GROUP_INSTANCE-E027DA69DBB9DB9C, dt.host_group.id=HOST-USU, dt.span_id=b993d068e7e61a80, dt.entity.host=HOST-7B2BCC248E339F00] Redirect requested to location 'https://cdn03.quay.io/quayio-production-s3/sha256/3c/3c6a219e7df6ab6a8a7d27869b650409bfce69b51ff2b47ad2b4e4a336b53a43?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA%2F20240805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240805T100537Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=680adb9ff6bf7d002e1743a2c3cb4a47f5059cd4f109405d3744a34c3ddad0ec&cf_sign=RLQVmD3GBXY1k8Yh8I7J59IF6BLVJnoACVZ3D4bB72GpDz4Fsm9rhx7BL9nwWjaGwxfyhImhHXaFfYE8gwSbvixruJiaznEZBRL3oZWSIGdNriGfCVrtO53nKFniK%2BFflmzjv9auijMZhEqMjxrbSEITr9hVR2XHwIImo0inq8blhCCUsK%2FnJkmxMfIqrwQQ5nKTbSzKWcQAKlbk7tnqQmMkM%2FrPwbn2BMjr%2FqsD8xw4nWkgFUErT9zdtIhNfd3ANIRQBG8H1n7VrToLIuKMAUcxJncaeny67xDU0oG8QcpdIue6L437UEgwTjh5L%2FjjEj7DfKWhr3KQC5l2Xz0eYg%3D%3D&cf_expiry=1722852937&region=us-east-1&namespace=actraiser&username=actraiser%2Bcicd_pull&repo_name=sl-financiero-financiero-batch-apertura-tramites-sf-recibos'
2024-08-05 12:05:37.354 WARN 1 --- [io-8080-exec-10] ApplicationConfigurationMetadataResolver : [!dt dt.entity.process_group=PROCESS_GROUP-A4C0D936FBECC05C, dt.trace_sampled=true, dt.trace_id=a18236f5615737891e2ba12d36df8aec, dt.entity.host_group=HOST_GROUP-7D678375C12400F3, dt.entity.process_group_instance=PROCESS_GROUP_INSTANCE-E027DA69DBB9DB9C, dt.host_group.id=HOST-USU, dt.span_id=7d52d5ef61e11c71, dt.entity.host=HOST-7B2BCC248E339F00] Failed to retrieve port names for resource Docker Resource [docker:quay.io/actraiser/sl-financiero-financiero-batch-apertura-tramites-sf-recibos:1.0.7-SNAPSHOT] because of UnknownContentTypeException: Could not extract response: no suitable HttpMessageConverter found for response type [interface java.util.Map] and content type [binary/octet-stream]
2024-08-05 12:05:37.354 DEBUG 1 --- [io-8080-exec-10] ApplicationConfigurationMetadataResolver : [!dt dt.entity.process_group=PROCESS_GROUP-A4C0D936FBECC05C, dt.trace_sampled=true, dt.trace_id=a18236f5615737891e2ba12d36df8aec, dt.entity.host_group=HOST_GROUP-7D678375C12400F3, dt.entity.process_group_instance=PROCESS_GROUP_INSTANCE-E027DA69DBB9DB9C, dt.host_group.id=HOST-USU, dt.span_id=7d52d5ef61e11c71, dt.entity.host=HOST-7B2BCC248E339F00] (Details) for failed to retrieve port names for resource:Docker Resource [docker:quay.io/actraiser/sl-financiero-financiero-batch-apertura-tramites-sf-recibos:1.0.7-SNAPSHOT]

org.springframework.web.client.UnknownContentTypeException: Could not extract response: no suitable HttpMessageConverter found for response type [interface java.util.Map] and content type [binary/octet-stream]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:126)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1043)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1026)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:784)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:757)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:627)
at org.springframework.cloud.dataflow.container.registry.ContainerRegistryService.getImageBlob(ContainerRegistryService.java:231)
at org.springframework.cloud.dataflow.configuration.metadata.container.DefaultContainerImageMetadataResolver.getImageLabels(DefaultContainerImageMetadataResolver.java:81)
at org.springframework.cloud.dataflow.configuration.metadata.BootApplicationConfigurationMetadataResolver.resolveOptionGroupsFromContainerImage(BootApplicationConfigurationMetadataResolver.java:278)
at org.springframework.cloud.dataflow.configuration.metadata.BootApplicationConfigurationMetadataResolver.listOptionGroups(BootApplicationConfigurationMetadataResolver.java:199)
at org.springframework.cloud.dataflow.server.controller.AppRegistryController.getInfo(AppRegistryController.java:216)
at org.springframework.cloud.dataflow.server.controller.AppRegistryController.info(AppRegistryController.java:170)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:529)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:156)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:164)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:164)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:125)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter.doFilterInternal(DefaultLogoutPageGeneratingFilter.java:58)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:237)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:223)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:223)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:178)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:761)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:388)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:936)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
at java.base/java.lang.Thread.run(Thread.java:1583)

Quay.io responds with binary/octet-stream when reading imageBlob. application/octet-stream is identical to binary/octet-stream. Fixes spring-cloud#5885

corneil · 2024-08-23T10:46:30Z

@actraiser2 Would you mind checking out PR #5892 and letting us know if the problem is resolved when using quay.io?

git checkout https://github.com/spring-cloud/spring-cloud-dataflow.git
cd spring-cloud-dataflow
gh pr checkout 5892
./mvnw clean package -am -pl spring-cloud-dataflow-server -DskipTests
# if using k8s test
./mvnw spring-boot:build-image -pl spring-cloud-dataflow-server

cppwfs · 2024-09-05T21:15:25Z

@actraiser2 Have you had a chance to kick the tires on this PR?

github-actions bot added the status/need-triage Team needs to triage and take a first look label Aug 4, 2024

corneil added status/need-feedback Calling participant to provide feedback and removed status/need-triage Team needs to triage and take a first look labels Aug 5, 2024

github-actions bot added for/team-attention For team attention and removed status/need-feedback Calling participant to provide feedback labels Aug 5, 2024

corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Aug 7, 2024

Add support for binary/octet-stream.

4883a34

Quay.io responds with binary/octet-stream when reading imageBlob. application/octet-stream is identical to binary/octet-stream. Fixes spring-cloud#5885

corneil linked a pull request Aug 7, 2024 that will close this issue

Add support for binary/octet-stream. #5892

Open

github-actions bot added status/need-feedback Calling participant to provide feedback and removed for/team-attention For team attention labels Aug 23, 2024

corneil added for/team-attention For team attention status/in-progress Something is happening and removed for/team-attention For team attention labels Aug 23, 2024

cppwfs added this to the 2.11.x milestone Sep 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

error extracting the metadata for a Quay.io image #5885

error extracting the metadata for a Quay.io image #5885

actraiser2 commented Aug 4, 2024

corneil commented Aug 5, 2024

actraiser2 commented Aug 5, 2024

corneil commented Aug 23, 2024 •

edited

Loading

cppwfs commented Sep 5, 2024 •

edited

Loading

error extracting the metadata for a Quay.io image #5885

error extracting the metadata for a Quay.io image #5885

Comments

actraiser2 commented Aug 4, 2024

corneil commented Aug 5, 2024

actraiser2 commented Aug 5, 2024

corneil commented Aug 23, 2024 • edited Loading

cppwfs commented Sep 5, 2024 • edited Loading

corneil commented Aug 23, 2024 •

edited

Loading

cppwfs commented Sep 5, 2024 •

edited

Loading