CTR always times out waiting for boot3 tasks to start

[jnt0r]

Description:
When starting boot3 tasks as composed tasks the CTR fails waiting for the tasks to start because the boot3 tasks only write their status into the BOOT3_TASK_* tables but the CTR checks the TASK_* tables. As I found in documentation and other comments it should be possible to run boot3 tasks with CTR. Am I right with this?

I found that CTR adds following parameter to the launched tasks "SPRING_CLOUD_TASK_PARENT-SCHEMA-TARGET". But I do not find this property documented in spring cloud task documentation. What is this about? Is there some additional work necessary on the custom task side? Is it possible to configure the CTR or the launched tasks to write/watch in both schemas?

Release versions:
SCDF 2.11.3 and CTR 2.11.3

[corneil]

CTR should have no issue launch boot 3 tasks, we have acceptance tests where we orchestrate multiple boot 2.x and boot 3.x tasks without issue.

The property --spring.cloud.task.parent-schema-target ensure that task parents can properly be identified across the boot 2.x and boot 3.x task executions which have separate tables that we cannot mess with from data flow side. It has no specific meaning to the child task.

Do you have an more information on if the boot 3.x task is actually running?
What platform are you using?
Do you see any errors in the task logs?

[jnt0r]

I will try to deploy a composed task using the boot2 and boot3 samples (timestamp) and try to verify the behavior. For my understanding, we are using spring batch and spring cloud task as described in the documentation.

I was just wondering that I could not find the Property --spring.cloud.task.parent-schema-target described anywhere in the documentation of spring cloud task.

I am running SCDF on Kubernetes.
The tasks complete successfully and I therefore see no errors. I also see the results of the actual task and batch job in SCDF, but the composed task runner fails to get the status. It just seems that my tasks ignore the parent-schema-target property.

[jnt0r]

Is there any specialty or important point to configure in my tasks to make them use the parent-schema-target, or should it work out of the box? Is there any possibility I deactivated that functionality by mistake?

[jnt0r]

I just tested it with the timestamp-batch-task in Version 3.0.0 and the CTR also failed to check if the task started correctly. Again, the timestamp task completed correctly and I found no obvious log with an error. The task did not log anything related with "parent", "target schema" or so on.

I had to build the docker image by myself as I had to add another database driver but did not change the configuration of the task.

[corneil]

Is there any specialty or important point to configure in my tasks to make them use the parent-schema-target, or should it work out of the box? Is there any possibility I deactivated that functionality by mistake?

The task doesn't need to do anything about the property in question. Since the arguments are logged we can find the combination of parent id and parent schema to find all of the ctr process children.

[corneil]

I just tested it with the timestamp-batch-task in Version 3.0.0 and the CTR also failed to check if the task started correctly. Again, the timestamp task completed correctly and I found no obvious log with an error. The task did not log anything related with "parent", "target schema" or so on.

I had to build the docker image by myself as I had to add another database driver but did not change the configuration of the task.

You can follow these instructions for adding a database driver to an existing jar and then build the container using Paketo pack.

[corneil]

I will try to deploy a composed task using the boot2 and boot3 samples (timestamp) and try to verify the behavior. For my understanding, we are using spring batch and spring cloud task as described in the documentation.

I was just wondering that I could not find the Property --spring.cloud.task.parent-schema-target described anywhere in the documentation of spring cloud task.

I am running SCDF on Kubernetes. The tasks complete successfully and I therefore see no errors. I also see the results of the actual task and batch job in SCDF, but the composed task runner fails to get the status. It just seems that my tasks ignore the parent-schema-target property.

Can you use kail to retrieve logs from all pods using kail -n <scdf-namespace> | tee scdf.log from just before you deploy the task?

[jnt0r]

I just tested it with the timestamp-batch-task in Version 3.0.0 and the CTR also failed to check if the task started correctly. Again, the timestamp task completed correctly and I found no obvious log with an error. The task did not log anything related with "parent", "target schema" or so on.
I had to build the docker image by myself as I had to add another database driver but did not change the configuration of the task.

You can follow these instructions for adding a database driver to an existing jar and then build the container using Paketo pack.

Is there any special configuration when building with Paketo buildpacks that I am missing? Our build process depends on Dockerfiles for now, and I did not test if I am able to build using pack.

[corneil]

I just tested it with the timestamp-batch-task in Version 3.0.0 and the CTR also failed to check if the task started correctly. Again, the timestamp task completed correctly and I found no obvious log with an error. The task did not log anything related with "parent", "target schema" or so on.
I had to build the docker image by myself as I had to add another database driver but did not change the configuration of the task.

You can follow these instructions for adding a database driver to an existing jar and then build the container using Paketo pack.

Is there any special configuration when building with Paketo buildpacks that I am missing? Our build process depends on Dockerfiles for now, and I did not test if I am able to build using pack.

Install pack using these instructions
https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/#pack-cli
and follow the guide in the docs.

[cppwfs]

Closing this issue due to inactivity. If this issue needs to be reopened, please leave a comment. Thank you!

[jnt0r]

I think this has been fixed for us by switching the entryPointStyle from shell back to exec. It may be only a problem with the shell entryPointStyle.