Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Google CloudDNS): add routing policy support #4928

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

Dadeos-Menlo
Copy link

Description

Google's CloudDNS service supports routing policies (i.e. geolocation and weighted-round-robin based query responses) but the current implementation of the "google" provider does not support managing such routing policy enabled resource record sets.

The proposed changes include:

  • Introduction of a plan.Changes.All() function, and associated plan.RRSetChange structure, for representing changes to be applied to a managed domain as an ordered sequence of changes grouped by resource record set
  • Refactoring of the "google" provider unit-tests in order to express test expectations in terms of the state of managed domains' resource record sets
  • Modification of the "google" provider to introduce support for handling routing policy enabled resource record sets

Notes:

  • Routing policies introduce additional dimensions to resource records sets (e.g. the regular dimensions of DNS name and record type are augmented by location or weighted set entry) the Endpoint representation of resource records uses the, somewhat AWS Route53 centric, SetIdentifier field to support any additional dimension[s]
  • AWS' Route53 service maintains distinct objects for each routing policy enabled resource record whereas Google's CloudDNS service maintains all of the routing policy configuration within a single ResourceRecordSet object, this data-structure necessitates a read-modify-write approach when attempting to alter a routing policy enabled sub-record
  • The proposed changes introduce a number of "google" provider specific annotations, the getProviderSpecificAnnotations(…) function is modified accordingly but the changes proposed in pull-request 4875 are considered to be a preferable approach

Checklist

  • Unit tests updated
  • End user documentation updated

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 4, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @Dadeos-Menlo. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Dec 4, 2024
@mloiseleur
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 22, 2024
@mloiseleur
Copy link
Contributor

/retitle feat(Google CloudDNS): add routing policy support

@k8s-ci-robot k8s-ci-robot changed the title Add routing policy support to Google CloudDNS provider feat(Google CloudDNS): add routing policy support Dec 22, 2024
@Dadeos-Menlo Dadeos-Menlo force-pushed the google_routing_policy branch 2 times, most recently from 4cfd960 to 15dea04 Compare January 2, 2025 16:34
@mloiseleur
Copy link
Contributor

/ok-to-test

@mloiseleur
Copy link
Contributor

@Dadeos-Menlo You'll need to run make generate-flags-documentation and it should be good

BTW, do you think you can join the kubernetes slack ?

@Dadeos-Menlo Dadeos-Menlo force-pushed the google_routing_policy branch from 15dea04 to cd12389 Compare January 23, 2025 10:45
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign raffo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Dadeos-Menlo
Copy link
Author

@Dadeos-Menlo You'll need to run make generate-flags-documentation and it should be good

Thanks; I've rebased the proposed changes, which introduced the generate-flags-documentation Makefile target, and updated docs/flags.md accordingly.

BTW, do you think you can join the kubernetes slack ?

Sure; I've joined the #external-dns channel.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 30, 2025
@Dadeos-Menlo Dadeos-Menlo force-pushed the google_routing_policy branch from cd12389 to 867a948 Compare January 31, 2025 15:50
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 31, 2025
@Dadeos-Menlo Dadeos-Menlo force-pushed the google_routing_policy branch from 867a948 to dc82175 Compare February 1, 2025 14:06
relying upon the implementation under test to establish the initial test
conditions. Also make mock interface implementations more representative
of real behaviour.
@mloiseleur
Copy link
Contributor

cc @ivankatliarchuk for review

@ivankatliarchuk
Copy link
Contributor

I think we should not proceed with proposed approach.

More information here #4875 (comment)

But I'll try to be structured, with what is wrong to attach annotations on CRD of the same vendor.

Like from example

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: geo-example
  annotations:
    external-dns.alpha.kubernetes.io/google-routing-policy: "geo"
    external-dns.alpha.kubernetes.io/google-location: "us-east1"

When defining a CRD, the spec section should be the authoritative source of configuration. It breaks Kubernetes convention → Annotations should not replace spec, but augment.

  1. Annotations Are Meant for Metadata, Not Core Spec Configuration

    • Annotations in Kubernetes are intended for metadata and hints, not primary configuration.
    • By design, they are unstructured and should not be relied upon for essential logic.
    • This goes against best practices in Kubernetes API design.
  2. Reduces API Consistency

    • Users expect structured fields when interacting with CRDs.
    • Mixing annotations for key settings makes it confusing:
      • Some configs are in spec, others in metadata.annotations.
      • Inconsistent API experience for developers and operators.
  3. Annotations Have a size Limit

    • While this is not an immediate concern for small configurations, if more DNS settings are added in the future, it could hit this limit.
  • Structured CRD fields are a more scalable approach in this scenario
  1. Reduced Readability & Maintainability

    • If more provider-specific configurations are added, the annotations section will become cluttered and unmanageable. So before making this decision we should look in ways to review annotations across whole project
  2. Loss of Validation & Schema Enforcement (Soft not really a problem)

  • In the original CRD, Kubernetes validates the fields under spec against the CRD schema.
  • Moving everything to annotations makes it free-form text, losing validation and type safety. Typos or incorrect values won't be caught until runtime (or may silently fail).

This approach is generally not recommended by the Kubernetes community. The community prefers:

- Using spec for structured configuration.
- Using annotations for optional metadata, not core settings.
- Maintaining clear API contracts with validation.

When Is It Okay to Mix CRDs & and attach Annotations from same vendor?

  • Annotations Provide Extra, Non-Essential Information

    • Example: Storing metadata such as timestamps, debugging info, or tracking data.
    • Not critical to the function of the CRD.
  • Annotations Are Used for Temporary Backward Compatibility

    • Example: If migrating from annotation-based configs to CRDs, both can exist temporarily.
    • But eventually, all structured settings should move to spec.

Alternative Approach

  • I'm unsure what are the alternatives. Example moving this configuration to other sources, is nice but

@Dadeos-Menlo
Copy link
Author

Dadeos-Menlo commented Feb 28, 2025

I think we should not proceed with proposed approach.

More information here #4875 (comment)

But I'll try to be structured, with what is wrong to attach annotations on CRD of the same vendor.

The changes proposed in this pull-request are entirely unrelated to CRDs; so I do not understand your objections?

The only reference the changes proposed here have to CRDs is the fact that I happen to have used a DNSEndpoint object as the example source object upon which to demonstrate use of the proposed new provider-specific annotations associated with the proposed new provider-specific functionality.

Ironically, #4875 also has very little to do with CRDs; that pull-request relates to the consistent representation of provider-specific properties throughout "external-dns".

The usage of a DNSEndpoint object in the example offered in the proposed documentation for this pull-request demonstrates a, minor, point associated with the changes proposed in #4875; namely:

  • Is it reasonable to expect that a DNSEndpoint object honours provider-specific annotations (just like every other source type does)?

I would argue "yes"; and therefore that is a bug I propose addressing as a small part of the changes proposed under #4875.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants