[RoadMap][Year 2024] Community RoadMap

[zhongjiajie]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• User Friendly
  • Audit log improvement design [DSIP-26][Audit log] Audit log improvement design #15423
  • [TriggerPlugin] Introduce Trigger Plugin [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330
• New Important Task or Task Related
  • Dynamic Task Plugin DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526
  • Independent Release of Task Plugin
  • Support Streaming Task [DSIP-16][Task] Support stream task #11352
  • Task [Feature] DS can support task running on remote host, not just worker server. #11652
  • Lifecycle method to TaskPlugin [DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin  #14877
  • Unify the task spi about logic task and worker task [DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task  #14823
• Test Improvement
  • API Test [DSIP-14][Improvement] Add api tests and integration tests for Apache DolphinScheduler  #10411
  • UT Improvement [DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573
• Data Ops
  • Git Ops Support
  • Git CICD related

[github-actions]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[zhongjiajie]

Y2022Q4 roadmap in #12436

[rickchengx]

Hi, @zhongjiajie , Here are the features I'd like to implement in Y2023Q1 :)

1. [Feature] Remote Logging on more targets (Google Cloud Storage, S3, Azure Blob Storage, Etc.)

• [DSIP-18][Remote Logging] Add support for writing task logs to remote storage #13017

2. [Feature] Extending the Storage Backend for Resource Center (Google Cloud Storage, Azure Blob Storage)

• [Feature][Resource Center] Add support for Azure Blob Storage in the resource center #13481
• [Feature][Resource Center] Add support for Google Cloud Storage in the resource center #13428

[davidzollo]

if you'd like to contribute any feature to the Apache DolphinScheduler, please leave a message.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[EricGao888]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

No stale.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[EricGao888]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

no stale

[zhongjiajie]

Summary Roadmap Finish in 2023

• Cloud Native
  • Remote Logging [DSIP-18][Remote Logging] Add support for writing task logs to remote storage #13017
  • K8S Operator https://github.com/apache/dolphinscheduler-operator
• New Important Task or Task related
  • Remote Task
    • [Feature][Task Plugin] RemoteShell Task Plugin #13794
  • Dynamic Worklfows Generate [Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters #14125

[gzusgw]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• Cloud Native

  • K8S Executor [Feature][Executor] Add K8S Executor for task dispatching #13316

• User Friendly

  • Dynamic Task Plugin DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526
  • Task Parameters Sharing Optimization
  • Independent Release of Task Plugin
  • Audit log improvement design [Improvement][Audit log] Audit log improvement design #15423

• New Important Task or Task Related

  • Support Streaming Task [DSIP-16][Task] Support stream task #11352

  • Remote Task

    • [Feature] DS can support task running on remote host, not just worker server. #11652

  • [TriggerPlugin] Introduce Trigger Plugin [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

  • Task Sharing Storage

  • Trigger Workflow by Event

  • Lifecycle method to TaskPlugin [DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin  #14877

  • Unify the task spi about logic task and worker task [DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task  #14823

• Test Improvement

  • API Test
  • UT Improvement [DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573
  • [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

• Data Ops

  • Git Ops Support
  • Git CICD related

could add loop task？ #14619

[zhongjiajie]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• Cloud Native

  • K8S Executor [Feature][Executor] Add K8S Executor for task dispatching #13316

• User Friendly

  • Dynamic Task Plugin DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526
  • Task Parameters Sharing Optimization
  • Independent Release of Task Plugin
  • Audit log improvement design [Improvement][Audit log] Audit log improvement design #15423

• New Important Task or Task Related

  • Support Streaming Task [DSIP-16][Task] Support stream task #11352

  • Remote Task

    • [Feature] DS can support task running on remote host, not just worker server. #11652

  • [TriggerPlugin] Introduce Trigger Plugin [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

  • Task Sharing Storage

  • Trigger Workflow by Event

  • Lifecycle method to TaskPlugin [DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin  #14877

  • Unify the task spi about logic task and worker task [DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task  #14823

• Test Improvement

  • API Test
  • UT Improvement [DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573
  • [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

• Data Ops

  • Git Ops Support
  • Git CICD related

could add loop task？ #14619

not sure whether is like dynamic task, so I add some comment on it, which you can see in #14619 (comment)

[Dyqer]

Hi, Team, How about to create a Java SDK?

[lenboo]

put it back

• Cloud Native
  • K8S Executor [Feature][Executor] Add K8S Executor for task dispatching #13316