[Feature] Package prefix to overcome model naming issue over distinct projects

[lander-tropos]

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

Situation

We would like to combine our distinct projects into one great 'super-project'. This 'super-project' will then be used to group the documentation from distinct projects. In order to group distinct projects into a one project, we should insert them as packages. When using distinct projects as a package in a new project, we should be aware of the unique naming convention of models in dbt. However, because distinct projects are managed by distinct development groups, we can't be aware of unique naming conventions without having a management group to manage the distinct projects.

Problem

Those distinct projects are developed on different repositories. This means that, without having a management group over those distinct projects, we are not able to check whether or not dbt models have unique names over those distinct projects.

However, in order to group those projects, we should take into consideration the uniqueness of the model names along distinct projects. When distinct teams work on distinct projects, the managemend of this can be a true bottleneck to manage.

Solution

Therefor, we would like to see some sort of 'package prefixing' solution such that a package can be prefixed and therefore models can have the same name in distinct projects.

Describe alternatives you've considered

An alternative for this is that we change the generate_alias_name macro in order to overwrite all aliases in a project based on the project name. This is a possible workaround, however, we should take into consideration that projects using this datasets as sources are infected by this change. We should manually go through every project, check whether are not they are using those datasets as sources, and change them if necessary.

Who will this benefit?

This is usefull for companies that have multiple dbt projects running and want to make use of dbt docs feature. Combining distinct dbt projects that uses datasets from different dbt projects as a source in one greater project, will result in one documentation page instead of hosting multiple ones.

Another benefit is that different teams working on different dbt projects shouldn't be aware anymore of the naming conventions used in an other project. Adding a prefix to a package will make the adaption of the package easier.

Are you interested in contributing this feature?

For sure, but I don't have any expierence in contributing to opensource projects.

Anything else?

More information about the dbt project setup can be found under option 4
https://discourse.getdbt.com/t/how-to-configure-your-dbt-repository-one-or-many/2121

[gshank]

This is a very reasonable thing to want, and Jeremy and I have discussed this particular issue on a number of occasions -- i.e. allowing duplicate model names that are differentiated by project or directory name. It's something that would be really nice to have, but there are quite a few code implications since the assumption that the model name alone differentiates the node is buried in a lot of different places. So this is on the roadmap somewhere -- I'll leave it to Jeremy to say where.

[lander-tropos]

Cool, thanks! Feel free to get in touch if I can help with something. Happy to help :)

[jtcohen6]

While this is not on our near-term roadmap, we are seeing more and more organizations scale their dbt deployments across multiple packages/projects, and I see this as one of the bigger quality-of-life improvements we could make to help.

I do see this as a duplicate of a (much older) request: #1269. It's a totally reasonable thing to want, and a conversation worth re-opening. We made an initial attempt in #3053, just to see how big a lift it is. As Gerda mentions, there are some tricky code implications; the biggest one is around matching up node "patches" (properties defined in .yml files) if there can exist multiple nodes by the same name across packages. I think the right approach here would go hand-in-hand with the change proposed in #4157, i.e. an ability to specify which node you're defining / overriding properties for, assumed by default to be the node in the same package.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.

[sdebruyn]

These automatic stale issue things on feature requests are just annoying. This has not been solved properly yet AFAIK.