Deployment error in one Siddhi application breaks the deployment of rest of the Siddhi applications.

[janithcmw]

Description

Hi All,

In an HA set-up, when there is an error in the Siddhi application deployment process the rest of the Siddhi Applications will not be deployed.

This can be seen only when the Siddhi Application deployment process is triggered over an event relevant to the 'coordination change'. This behaviour is not observed during the Siddhi app deployment process during the server start-up. As per the analysis, this behaviour is due to an issue with error handling.

Following is the corresponding stack trace.

WARN {org.wso2.carbon.cluster.coordinator.rdbms.RDBMSMemberEventListenerTask} - Error occurred while reading membership events. io.siddhi.core.exception.SiddhiAppCreationException: Error on 'edp-filestats-kafka-source' @ Line: 23. Position: 24, near '@source(ref='kafka-filestat-events',trace='enable',
@map(type='json',
 fail.on.missing.attribute='false',
 @attributes(canonicalName = 
 lifeCycle = 
 fileName = 
 rejectedTotalCount = 
 succeedTotalCount = 
 missingCount = 
 totalFileSize = 
 lineNumber = 
 isEnd = 
 header = 
 receiverHost = 
 kafkaTopic = 
 ingestTimeStamp = 
 topicName= 
 partition= 
 offset= 
	at io.siddhi.core.util.parser.helper.DefinitionParserHelper.updateAnnotationRef(DefinitionParserHelper.java:905)
	at io.siddhi.core.util.parser.helper.DefinitionParserHelper.addEventSource(DefinitionParserHelper.java:316)
	at io.siddhi.core.util.SiddhiAppRuntimeBuilder.defineStream(SiddhiAppRuntimeBuilder.java:117)
	at io.siddhi.core.util.parser.SiddhiAppParser.defineStreamDefinitions(SiddhiAppParser.java:375)
	at io.siddhi.core.util.parser.SiddhiAppParser.parse(SiddhiAppParser.java:231)
	at io.siddhi.core.SiddhiManager.createSiddhiAppRuntime(SiddhiManager.java:87)
	at io.siddhi.core.SiddhiManager.createSiddhiAppRuntime(SiddhiManager.java:97)
	at org.wso2.carbon.streaming.integrator.core.ha.HAManager.lambda$createSiddhiAppRuntimes$0(HAManager.java:374)
	at java.base/java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)
	at org.wso2.carbon.streaming.integrator.core.ha.HAManager.createSiddhiAppRuntimes(HAManager.java:372)
	at org.wso2.carbon.streaming.integrator.core.ha.HAManager.changeToActive(HAManager.java:189)
	at org.wso2.carbon.streaming.integrator.core.ha.HAEventListener.coordinatorChanged(HAEventListener.java:165)
	at org.wso2.carbon.cluster.coordinator.rdbms.RDBMSMemberEventListenerTask.notifyCoordinatorChangeEvent(RDBMSMemberEventListenerTask.java:135)
	at org.wso2.carbon.cluster.coordinator.rdbms.RDBMSMemberEventListenerTask.run(RDBMSMemberEventListenerTask.java:100)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
I have tried to handle the corresponding error, from the code base but it seems it is difficult to find a suitable place to handle the error.

Best Regards,
Janith.

Steps to Reproduce

Steps to Reproduce.

1. Add a malformed Siddhi application(which will not be deployed successfully) to a HA cluster.
2. Push some load to a working Siddhi application so that you can observe the event processing with a log line.
3. Then allow to change the state of the HA cluster. This can be done by deleting the entry in the database table 'LEADER_STATUS_TABLE' in the database 'WSO2_CLUSTER_DB'.
4. After the state change the Siddhi applications will not be deployed and you will not be able to observe the log which you have added over the load even if the load is there.
5. Try to use a message broker as a load so that events will be in the queue for sure and once restart the servers this issue will be resolved.

Affected Component

SI

Version

1.1.0

Environment Details (with versions)

No response

Relevant Log Output

No response

Related Issues

No response

Suggested Labels

No response