feat(auth) - New privilege for Associate tags

[mkamalas]

New privilege to restrict association of tags. We have some use-cases where only a specific group users are authorized to associate certain tags. Also added a resource exclusion capability so that policy can be created for all resources excluding a certain set of resources using NOT_EQUALS policy condition.

To mimic behavior before this change, we need to create a policy which authorizes all users to the new privilege ASSOCIATE TAGS all tag resources.

Checklist

• [ x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• [ x] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Summary by CodeRabbit

• New Features

  • Introduced enhanced authorization checks for tag association and removal, ensuring only authorized users can perform these actions.
  • Added a new enumeration value NOT_EQUALS to improve filtering capabilities in policy conditions.
  • Introduced a new privilege "Associate Tags" for more granular permission management related to tag associations.

• Bug Fixes

  • Improved error handling for permission-related issues during bulk edits of tags, providing clearer feedback to users.

• Documentation

  • Updated documentation to reflect new privileges and conditions related to tag management and policy definitions.

[anshbansal]

HTTP 405 is well defined in general tech. Let's not override the standard HTTP codes

[mkamalas]

Reverted back the error code to 403 with different error message

[mkamalas]

Is this an urgent ask from your side?

We've not heard this request from anyone else yet, so it's unclear to me that we will deliver broad, generalizable value to the community in exchange the non-trivial complexity that this change introduces for the system.

Perhaps the use case behind the change can be elaborated upon so we can better understand? I understand WHAT the change is, but I don't fully understand the WHY. My hunch: There is a group of users who are able to apply specific tags across all entities - data stewards - in which case I'd recommend granting data stewards the ability to edit ANY tags for ANY asset - why does this not work?

We have some use-cases where only a specific group users are authorized to associate certain tags. For all other tags, we want to allow all users to be able to associate the tags to ANY entity. Edit Tag privilege is a privilege which allows ANY tag to be associated to an entity. The entity can be specified or restricted by ownership. But, there is no way to restrict association by tags.

Also, there is only ways to specify policy by including resources but no way to specify policies by excluding resources.
So, added a resource exclusion capability so that policy can be created for all resources excluding a certain set of resources using NOT_EQUALS policy condition.

[rtekal]

@jjoyce0510 and @chriscollins3456

Meenakshi synced up the code with master again after validation. Could one of you please review and merge the PR. Thanks in advance.

[rtekal]

@anshbansal , @chriscollins3456 and @jjoyce0510
Can anyone review the PR please. The PR has been sync'ed to the main and re-tested.

Thanks and Regards.

[RyanHolstien]

Auth calls should come first before sending validation operations to the entity service, ideally done as soon as possible before any business logic occurs to short circuit.

[jjoyce0510]

This is introducing a non-trivial amount of new risk into this system by adding a lot of new complexity.

Is there consideration about priority of exclusion inclusion rules and conflicting rules?

[jjoyce0510]

Thanks for providing the detailed overview of the rationale behind associate Tags. I think it's safe to add this change, with the one caveat that the OpenAPI endpoints are not currently honoring this privilege - either that would need to be solved in this PR or a followup, I'll allow @david-leifker to chime in.

As for the exclusionary rules, I think this is a bit more concerning with larger implications. To date, we've fought long and hard to keep the policy system as simple as possible, to minimize the chance of user error when assigning privileges, by maintaining ONLY an inclusionary access model.

Exclusionary is a lot more risk because it means folks can do things by default. It is also not clear how to restrict privileges once it's been granted via an exclusionary policy has been defined - it's quite easy to accidentally grant broad access to many people without deeply understanding that you've done this.

I'm wondering if you folks are willing to drop this requirement, or to better understand why inclusionary access is not possible for your situation. This change will have security implications ALL users of DataHub across the globe, so it's quite important we get this one right.

[mkamalas]

Thanks for providing the detailed overview of the rationale behind associate Tags. I think it's safe to add this change, with the one caveat that the OpenAPI endpoints are not currently honoring this privilege - either that would need to be solved in this PR or a followup, I'll allow @david-leifker to chime in.

As for the exclusionary rules, I think this is a bit more concerning with larger implications. To date, we've fought long and hard to keep the policy system as simple as possible, to minimize the chance of user error when assigning privileges, by maintaining ONLY an inclusionary access model.

Exclusionary is a lot more risk because it means folks can do things by default. It is also not clear how to restrict privileges once it's been granted via an exclusionary policy has been defined - it's quite easy to accidentally grant broad access to many people without deeply understanding that you've done this.

I'm wondering if you folks are willing to drop this requirement, or to better understand why inclusionary access is not possible for your situation. This change will have security implications ALL users of DataHub across the globe, so it's quite important we get this one right.

In our organization, we want to allow all users to be able to create and associate tags for their own use cases. But, there are certain tags which are restricted. That is the reason we want to add this Exclusionary logic so that we can restrict access for these special tags alone. Would it be okay if we add this with a feature flag to reduce impact ?

[david-leifker]

I think we definitely need to add a test around the policy and new privilege added to the existing tests around the policy engine. Note that the policy engine will grant access if any of the policies match, which means the NOT_EQUAL condition would not be given precedence. It is not a deny condition, but the naming might be a bit confusing based on the NOT_EQUAL name.

Specifically, I am not sure this logic allows this use case

That is the reason we want to add this Exclusionary logic so that we can restrict access for these special tags alone. Would it be okay if we add this with a feature flag to reduce impact ?

[rtekal]

Comments from John Joyce:
If we do this, we'd want to be consistent and do it for terms, structured properties, domains as well. I think it does make sense.

The defaults also have to be great