dbt cli snapshot plugin fails with UnsupportedOperationException

[craigbloodworth]

Describe the issue

Since the latest update to 0.21.4 a task to clone a repo from Github and then run the code in a Docker runner immediately fails with the error: java.lang.UnsupportedOperationException.

Job:

 - id: finalSnapshot_clone_repository
    type: io.kestra.plugin.git.Clone
    url: https://github.com/[...repo...]
    branch: main

  - id: finalSnapshot_dbt_snapshot
    type: io.kestra.plugin.dbt.cli.Snapshot
    debug: true
    dbtPath: /usr/local/bin/dbt
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
      fileHandlingStrategy: MOUNT
      delete: true
      networkMode: host
    containerImage: docker.io/craigbloodworth/dbt-sqlserver:latest
    profiles: |

finalSnapshot_dbt_snapshot log:

2025-03-06T07:20:30.928Z TRACE java.lang.UnsupportedOperationException
	at java.base/java.util.AbstractMap.put(Unknown Source)
	at java.base/java.util.AbstractMap.putAll(Unknown Source)
	at io.kestra.plugin.scripts.exec.scripts.runners.CommandsWrapper.addEnv(CommandsWrapper.java:147)
	at io.kestra.plugin.dbt.cli.AbstractDbt.run(AbstractDbt.java:178)
	at io.kestra.plugin.dbt.cli.AbstractDbt.run(AbstractDbt.java:34)
	at io.kestra.core.runners.WorkerTaskCallable.doCall(WorkerTaskCallable.java:78)
	at io.kestra.core.runners.AbstractWorkerCallable.call(AbstractWorkerCallable.java:62)
	at io.kestra.core.runners.WorkerSecurityService.callInSecurityContext(WorkerSecurityService.java:10)
	at io.kestra.core.runners.Worker.lambda$callJob$19(Worker.java:837)
	at io.kestra.core.trace.NoopTracer.inCurrentContext(NoopTracer.java:15)
	at io.kestra.core.runners.Worker.callJob(Worker.java:833)
	at io.kestra.core.runners.Worker.runAttempt(Worker.java:792)
	at io.kestra.core.runners.Worker.run(Worker.java:640)
	at io.kestra.core.runners.Worker.handleTask(Worker.java:372)
	at io.kestra.core.runners.Worker.lambda$run$8(Worker.java:261)
	at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Environment

• Kestra Version: 0.21.4

[Ben8t]

Still investigating, but I'm not able to reproduce with this flow on our develop version:

id: dbt_snapshot
namespace: company.team

tasks:
  - id: working_directory
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-demo
        branch: main
      
      - id: dbt_snapshot
        type: io.kestra.plugin.dbt.cli.Snapshot
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
        dbtPath: /usr/local/bin/dbt
        containerImage: ghcr.io/kestra-io/dbt-duckdb
        profiles: |
          jaffle_shop:
            outputs:
              dev:
                type: duckdb
                path: ':memory:'
                extensions:
                  - parquet
            target: dev

[Ben8t]

Update: I'm able to reproduce on 0.21.5 and with other dbt tasks

[Ben8t]

This works though (using dbt cli task):

id: dbt_snapshot
namespace: company.team

tasks:
  - id: dbt
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: cloneRepository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: main
      - id: dbt-build
        type: io.kestra.plugin.dbt.cli.DbtCLI
        containerImage: ghcr.io/kestra-io/dbt-duckdb:latest
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
        commands:
          - dbt snapshot
        profiles: |
          my_dbt_project:
            outputs:
              dev:
                type: duckdb
                path: ":memory:"
            target: dev

so it seems an issue related to wrapper tasks of dbt (Run, Build, Snapshot, etc.)

[craigbloodworth]

Yes, I can confirm switching to DbtCLI works for me too. Thanks so much for the work around @Ben8t, really appreciate you jumping on it so quickly

[mgabelle]

Issue fixed in : PR
This will be available in the next bugfix release.

@Ben8t Closing the issue.