Containerise Olog-ES #1805

willrogers · 2021-03-08T14:45:56Z

willrogers
Mar 8, 2021
Collaborator

We want three containers and docker compose configuration to coordinate them.

Can someone describe the three required containers?
Where should the code live?
What does ES stand for? ElasticSearch?

shroffk · 2021-03-08T15:03:15Z

shroffk
Mar 8, 2021
Maintainer

A simple example for docker compose using kafka is available here:
https://github.com/ControlSystemStudio/phoebus/tree/master/services/alarm-server/src/test/resources/docker

0 replies

shroffk · 2021-03-08T15:05:26Z

shroffk
Mar 8, 2021
Maintainer

The 3 required containers are:

elastic 6.4
mongoDB
both images for 1 and 2 are already published and we simply need to thread them together in a docker compose
olog-es
we will have to create a dockerfile to build an image which hosts the olog-es REST service.

3 replies

willrogers Mar 8, 2021
Collaborator Author

How do you build the olog-es REST service?

georgweiss Mar 8, 2021
Maintainer

It's a different repo: https://github.com/Olog/olog-es

georgweiss Mar 8, 2021
Maintainer

Sorry, should have clarified that earlier...

willrogers · 2021-03-09T10:50:37Z

willrogers
Mar 9, 2021
Collaborator Author

I think I am making pretty good progress on this but I am not sure how to debug this problem:

olog_1     | 2021-03-09 10:46:18.538  INFO 9 --- [l'}-mongo:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:1, serverValue:2}] to mongo:27017
olog_1     | 2021-03-09 10:46:18.815  INFO 9 --- [l'}-mongo:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=mongo:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 4, 4]}, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=190528900}
olog_1     | 2021-03-09 10:46:24.609  WARN 9 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'authenticationResource': Unsatisfied dependency expressed through field 'authenticationManager'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.security.authentication.AuthenticationManager' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

@shroffk @georgweiss any ideas?

6 replies

willrogers Mar 9, 2021
Collaborator Author

I think I have got past that one. Now I cannot connect to the elastic container, but this is more of a Docker problem than an olog one:

olog_1     | 2021-03-09 11:09:27.516  WARN 9 --- [           main] gov.bnl.olog.ElasticConfig               : Failed to create template for index olog_log
olog_1     |
olog_1     | java.net.ConnectException: Connection refused
olog_1     |    at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:959) ~[elasticsearch-rest-client-6.8.6.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233) ~[elasticsearch-rest-client-6.8.6.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1749) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1708) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.IndicesClient.getIndexTemplate(IndicesClient.java:1618) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at gov.bnl.olog.ElasticConfig.elasticIndexValidation(ElasticConfig.java:185) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig.getIndexClient(ElasticConfig.java:92) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853.CGLIB$getIndexClient$0(<generated>) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853$$FastClassBySpringCGLIB$$8703823c.invoke(<generated>) ~[classes!/:na]
olog_1     |    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853.getIndexClient(<generated>) ~[classes!/:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
olog_1     |    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
olog_1     |    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:484) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at gov.bnl.olog.Application.main(Application.java:38) ~[classes!/:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
olog_1     |    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
olog_1     |    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[olog-es-2.0.jar:na]
olog_1     | Caused by: java.net.ConnectException: Connection refused
olog_1     |    at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:na]
olog_1     |    at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[na:na]
olog_1     |    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
olog_1     |    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
olog_1     |    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

georgweiss Mar 9, 2021
Maintainer

Indeed seems like a Docker thing...

georgweiss Mar 9, 2021
Maintainer

By default Elastic binds to 127.0.0.1:9200. Not sure how Docker treats that....

willrogers Mar 9, 2021
Collaborator Author

Aha I think I have got it working. Here is the response to https://localhost:8181/Olog/logs:

[]

georgweiss Mar 9, 2021
Maintainer

Right, empty JSON array...

willrogers · 2021-03-09T13:58:35Z

willrogers
Mar 9, 2021
Collaborator Author

A minimal version is here: https://github.com/willrogers/olog-es-docker

0 replies

shroffk · 2021-03-10T14:06:26Z

shroffk
Mar 10, 2021
Maintainer

Documentation of how to run Olog-es with a user specified application.properties

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Containerise Olog-ES #1805

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 9 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Containerise Olog-ES #1805

willrogers Mar 8, 2021 Collaborator

Replies: 5 comments · 9 replies

shroffk Mar 8, 2021 Maintainer

shroffk Mar 8, 2021 Maintainer

willrogers Mar 8, 2021 Collaborator Author

georgweiss Mar 8, 2021 Maintainer

georgweiss Mar 8, 2021 Maintainer

willrogers Mar 9, 2021 Collaborator Author

willrogers Mar 9, 2021 Collaborator Author

georgweiss Mar 9, 2021 Maintainer

georgweiss Mar 9, 2021 Maintainer

willrogers Mar 9, 2021 Collaborator Author

georgweiss Mar 9, 2021 Maintainer

willrogers Mar 9, 2021 Collaborator Author

shroffk Mar 10, 2021 Maintainer

willrogers
Mar 8, 2021
Collaborator

Replies: 5 comments 9 replies

shroffk
Mar 8, 2021
Maintainer

shroffk
Mar 8, 2021
Maintainer

willrogers Mar 8, 2021
Collaborator Author

georgweiss Mar 8, 2021
Maintainer

georgweiss Mar 8, 2021
Maintainer

willrogers
Mar 9, 2021
Collaborator Author

willrogers Mar 9, 2021
Collaborator Author

georgweiss Mar 9, 2021
Maintainer

georgweiss Mar 9, 2021
Maintainer

willrogers Mar 9, 2021
Collaborator Author

georgweiss Mar 9, 2021
Maintainer

willrogers
Mar 9, 2021
Collaborator Author

shroffk
Mar 10, 2021
Maintainer