[Feature] Get dbt retry working even when there are compilation errors

[aranke]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

If we have a compilation error during dbt run, dbt retry has no nodes to pick up.

Expected Behavior

dbt retry should retry all nodes that were failed/skipped.
The fix here might be marking nodes as failed/skipped during compilation errors.

Steps To Reproduce

Step 1: Remove closing }} from one of the Jinja templates

❯ dbt run
21:58:09  Running with dbt=1.8.3
21:58:09  Registered adapter: duckdb=1.8.1
21:58:09  Encountered an error:
Compilation Error in model stg_customers (models/staging/stg_customers.sql)
  unexpected ')'
    line 11
      ),

Step 2: Fix model and retry

❯ dbt retry
21:58:24  Running with dbt=1.8.3
21:58:24  Registered adapter: duckdb=1.8.1
21:58:24  Found 5 models, 3 seeds, 20 data tests, 409 macros
21:58:24  
21:58:24  Nothing to do. Try checking your model configs and model specification args

Relevant log output

No response

Environment

No response

Which database adapter are you using with dbt?

No response

Additional Context

No response

[dbeatty10]

Let's say that you have the following model:

models/my_model.sql

select {{ dbt.string_literal("A")  as letter

Then you do the following:

dbt clean
dbt run -s my_model

None of dbt run, dbt build, etc. will produce target folder (and associated artifacts) due to the compilation error, so how could we possibly do a dbt retry?

[aranke]

None of dbt run, dbt build, etc. will produce target folder (and associated artifacts) due to the compilation error

I guess the question we should answer here is whether they should?
I could see a case either way, will defer to yourself, @jtcohen6, @graciegoheen to answer this question.

If we decide not to do this work, it feels like a gotcha worth documenting.

[dbeatty10]

@aranke Thanks for discovering this gotcha!

I relabeled this as a feature request, since I'd expect a successful parse to be a prerequisite for retry. (Put another way, I wouldn't expect dbt retry to work in the case where the project can't be parsed.)

It seems like it would be both a big shift and significant work to generate target artifacts even when parsing doesn't succeed, so I'm going to close this in favor of updating the documentation: dbt-labs/docs.getdbt.com#5964.

We can always re-open if @jtcohen6 or @graciegoheen end up wanting this.

[graciegoheen]

Put another way, I wouldn't expect dbt retry to work in the case where the project can't be parsed.

Is this because nothing would ever be built in the run since the parse would fail? In that case should we just retry the prior command? Or have a clearer error message?

From a user perspective, I can understand it would be unexpected for dbt retry not to work when the prior command failed.