add depends_on to schema object or CLI option to control deployment sequence

[minghungcho]

I understand Dependency Management can be tricky. However, I think it would be great if there is a more deterministic way to control the build/deploy sequence.

Since it is a common practice to segregate different functional areas by SCHEMA, it would be great if there is a way to control deploy sequence so that for instance staging schema and its objects are deployed before public schema objects.

Otherwise, a CLI option to specify which schema to build would solve the problem too.

[littleK0i]

Objects in schema A might depend on objects in schema B, and the other way around. Circular dependency would break the config application process.

Also, it would considerably affect the performance, especially if you have a large number of objects. Snowflake DDL operations may take up to a few seconds sometimes. It is not very noticeable, since SnowDDL runs all resolvers in parallel. But it would be very apparent with no parallelism.

You may simulate it now by setting --max-workers=1.

[minghungcho]

The only way I could come up to solve this problem is by setting up 2 separate database entry-points and using 2 CLI to deploy them in the right order. Do you think there is a better way to accomplish this?

sample_db
+--sample_db_public
+--public
+--view
+-- employee_list.yaml
+--sample_db_staging
+--staging
+--table
+-- employee.yaml

snowddl-singledb -c sample_db --config-db sample_db_staging --target-db sample_db --apply-unsafe apply
snowddl-singledb -c sample_db --config-db sample_db_public --target-db sample_db --apply-unsafe apply

[bchin3]

I am seeing some inconsistencies with depends_on where the objects are not always created in an order that will prevent issues but have not yet been able to pinpoint the exact scenarios that cause the problem. I have already checked that there are no circular dependencies in my project.

Are there any known limitations of "depends_on" that we should be aware of? Does it work across schemas and databases? Are there object types that it doesn't handle (e.g. depends_on: function_x)? Are there limits to the levels of nesting dependencies?

Apologies in advance for reviving an older thread

[littleK0i]

1. Explicit dependencies currently work only for VIEW (depends_on) and TASK (after) object types.
2. Explicit dependencies work across databases and schemas. Dependency name should be fully qualified (db.schema.view_name).
3. Explicit dependencies work only across one object type. VIEW cannot depend on FUNCTION. But it should not a problem, since all views are resolved after all functions by design.
4. There is no limit for nesting of dependencies.

Please check this manual page for a bit more info: https://docs.snowddl.com/guides/other/dependency-management

If some view occasionally fails due to missing dependency on another view, please double check depends_on config option. Probably some object names are incorrect or missing.