[CAPPL-536] Workflow update not picked up by Syncer

[agparadiso]

Description

we were seeing these errors:

{"level":"error","ts":"2025-02-12T18:04:12.571Z","logger":"WorkflowRegistrySyncer","caller":"syncer/workflow_registry.go:335","msg":"failed to handle event","version":"2.20.0@6ca1531","err":"failed to close workflow engine: rpc error: code = Unknown desc = error unregistering trigger: triggerId wf_005bd99697e7dfe1acd89a6a5a9a1907f53654355a671d0bd658514066e4ae54_trigger_0 not found","type":"WorkflowUpdatedV1","stacktrace":"github.com/smartcontractkit/chainlink/v2/core/services/workflows/syncer.(*workflowRegistry).readRegistryEvents\n\tgithub.com/smartcontractkit/chainlink/v2/core/services/workflows/syncer/workflow_registry.go:335\ngithub.com/smartcontractkit/chainlink/v2/core/services/workflows/syncer.(*workflowRegistry).Start.func1.1\n\tgithub.com/smartcontractkit/chainlink/v2/core/services/workflows/syncer/workflow_registry.go:234"}

In order to reproduce the issue:

• Register a workflow with a cron schedule that executes more often than every 30s (eg: every 15s)
• Observe that we see errors registering the trigger in the engine like this one:

 [ERROR] trigger event was an error rpc error: code = Unknown desc = error registering trigger: maximum fastest cron schedule is 30s; not executing workflows/engine.go:744          logger=WorkflowRegistrySyncer.WorkflowEngine stacktrace=github.com/smartcontractkit/chainlink/v2/core/services/workflows.(*Engine).worker

• Update the workflow
• Observe that we fail to update the workflow

Why is this happening?

communication between client and server is not synchronous, if the trigger failed to be registered (eg: when registering a workflow with a cron schedule lower than 30s) this error was being dragged to the forwardTriggerResponsesToChannel, the trigger was never registered and we were trying to Unregister a non existing trigger .

The fix

In order to avoid this Im sending a first ACK/ERROR message that will block until this is received. if there is an error registering the trigger in the first place this will be handled properly

How was this tested?

Added a unit test reproducing the error + tested on a local DON

CAPPL-536

[cedric-cordenier]

@agparadiso This code is a bit ambiguous I think: what happens if the server sends a response without an ack? We should clearly error in that case as that would mean the protocol has been violated.

[agparadiso]

changed to send: TriggerResponseMessage_Ack and fail if ackMsg.GetAck() == nil much cleaner, thanks🙇🏼

[cedric-cordenier]

No need to return an error in this case, you already returned it back to the caller successfully, just return nil.

[cedric-cordenier]

Better yet, I think return server.Send(msg) is enough

[agparadiso]

true, done 👍🏼