Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: security scanner gh token #22060

Draft
wants to merge 201 commits into
base: main
Choose a base branch
from

Conversation

abhishek-hashicorp
Copy link
Contributor

Description

The security scan check is failing due to expired gh token. Using the org level GH token which rotates and is maintained
Failed scan: https://github.com/hashicorp/consul/actions/runs/12657463329/job/35272531206?pr=22056

Testing & Reproduction steps

Links

PR Checklist

  • updated test coverage
  • external facing docs updated
  • appropriate backport labels added
  • not a security concern

* backport of commit b76447f

* backport of commit 395984c

* backport of commit cc1246d

* backport of commit 43170a5

---------

Co-authored-by: Xinyi Wang <[email protected]>
…re first use into release/1.18.x (#20462)

v2: ensure the controller caches are fully populated before first use (#20421)

The new controller caches are initialized before the DependencyMappers or the 
Reconciler run, but importantly they are not populated. The expectation is that 
when the WatchList call is made to the resource service it will send an initial 
snapshot of all resources matching a single type, and then perpetually send 
UPSERT/DELETE events afterward. This initial snapshot will cycle through the 
caching layer and will catch it up to reflect the stored data.

Critically the dependency mappers and reconcilers will race against the restoration 
of the caches on server startup or leader election. During this time it is possible a
 mapper or reconciler will use the cache to lookup a specific relationship and 
not find it. That very same reconciler may choose to then recompute some 
persisted resource and in effect rewind it to a prior computed state.

Change

- Since we are updating the behavior of the WatchList RPC, it was aligned to 
  match that of pbsubscribe and pbpeerstream using a protobuf oneof instead of the enum+fields option.

- The WatchList rpc now has 3 alternating response events: Upsert, Delete, 
  EndOfSnapshot. When set the initial batch of "snapshot" Upserts sent on a new 
  watch, those operations will be followed by an EndOfSnapshot event before beginning 
  the never-ending sequence of Upsert/Delete events.

- Within the Controller startup code we will launch N+1 goroutines to execute WatchList 
  queries for the watched types. The UPSERTs will be applied to the nascent cache
   only (no mappers will execute).

- Upon witnessing the END operation, those goroutines will terminate.

- When all cache priming routines complete, then the normal set of N+1 long lived 
watch routines will launch to officially witness all events in the system using the 
primed cached.

Co-authored-by: R.B. Boyer <[email protected]>
…services into release/1.18.x (#20467)

catalog: improve the bound workload identity encoding on services (#20458)

The endpoints controller currently encodes the list of unique workload identities 
referenced by all workload matched by a Service into a special data-bearing 
status condition on that Service. This allows a downstream controller to avoid an 
expensive watch on the ServiceEndpoints type just to get this data.

The current encoding does not lend itself well to machine parsing, which is what 
the field is meant for, so this PR simplifies the encoding from:

    "blah blah: " + strings.Join(ids, ",") + "."

to

    strings.Join(ids, ",")

It also provides an exported utility function to easily extract this data.

Co-authored-by: R.B. Boyer <[email protected]>
…ce updates into release/1.18.x (#20465)

* backport of commit 2069bd1

* backport of commit c0446fd

* backport of commit 5227cc2

---------

Co-authored-by: Nick Ethier <[email protected]>
…8.x (#20489)

* backport of commit b5c6c1d

* backport of commit 1b7dbd7

---------

Co-authored-by: Derek Menteer <[email protected]>
…#20331) (#20493)

Exported services CLI and docs (#20331)

* Exported services CLI and docs

* Changelog added

* Added format option for pretty print

* Update command/exportedservices/exported_services.go



* Addressing PR comments, moving the command under services category

* Add consumer peer and partition filter

* Adding bexpr filter, change format of data

---------

Co-authored-by: Ashesh Vidyut <[email protected]>
* backport of commit 392b8d7

* backport of commit b471659

* backport of commit a03cb97

* backport of commit 73b277c

* backport of commit e53b979

---------

Co-authored-by: Matt Keeler <[email protected]>
…P addresses into release/1.18.x (#20502)

NET-7631 - Fix Node records that point to external/ non-IP addresses (#20491)

* NET-7630 - Fix TXT record creation on node queries

* NET-7631 - Fix Node records that point to external/ non-IP addresses

* NET-7630 - Fix TXT record creation on node queries

Co-authored-by: John Murret <[email protected]>
…elease/1.18.x (#20500)

NET-7630 - Fix TXT record creation on node queries (#20483)

Co-authored-by: John Murret <[email protected]>
* backport of commit 0047b38

* backport of commit 7057901

---------

Co-authored-by: Xinyi Wang <[email protected]>
… release/1.18.x (#20518)

* V1 Compat Exported Services Controller Optimizations (#20517)

V1 compat exported services controller optimizations

* Don't start the v2 exported services controller in v1 mode.
* Use the controller cache.

* Trigger the V1 Compat exported-services Controller when V1 Config Entries are Updated (#20456)

* Trigger the v1 compat exported-services controller when the v1 config entry is modified.

* Hook up exported-services config entries to the event publisher.
* Add tests to the v2 exported services shim.
* Use the local materializer trigger updates on the v1 compat exported services controller when exported-services config entries are modified.

* stop sleeping when context is cancelled

---------

Co-authored-by: Eric Haberkorn <[email protected]>
….18.x (#20520)

* backport of commit 356d7a9

* backport of commit 96bf768

* backport of commit f3e0535

---------

Co-authored-by: skpratt <[email protected]>
…tewayClassConfig into release/1.18.x (#20524)

backport of commit 33fc2f9

Co-authored-by: Nathan Coleman <[email protected]>
…ode into release/1.18.x (#20529)

* backport of commit 5d591bb

* backport of commit cdfb10b

---------

Co-authored-by: Melissa Kam <[email protected]>
* no-op commit due to failed cherry-picking

* DNS v2 Multiple fixes. (#20525)

* DNS v2 Multiple fixes.

* add license header

* get rid of DefaultIntentionPolicy change that was not supposed to be there.

---------

Co-authored-by: temp <[email protected]>
Co-authored-by: John Murret <[email protected]>
…nto release/1.18.x (#20541)

* backport of commit 47c5c8b

* backport of commit 291bbf3

---------

Co-authored-by: Joshua Timmons <[email protected]>
* backport of commit 78411b5

* backport of commit 0da74f7

---------

Co-authored-by: Chris S. Kim <[email protected]>
…ET-7649/NET-7645 - Multiple DNS v2 fixes into release/1.18.x (#20561)

* no-op commit due to failed cherry-picking

* Do not parallelize DNS tests because they consume too many ports (#20482)

---------

Co-authored-by: temp <[email protected]>
Co-authored-by: John Murret <[email protected]>
…release/1.18.x (#20551)

* backport of commit 1668a31

* backport of commit d33eda9

---------

Co-authored-by: Derek Menteer <[email protected]>
…e use into release/1.18.x (#20572)

mesh: add ComputedImplicitDestinations resource for future use (#20547)

Creates a new controller to create ComputedImplicitDestinations resources by
composing ComputedRoutes, Services, and ComputedTrafficPermissions to
infer all ParentRef services that could possibly send some portion of traffic to a
Service that has at least one accessible Workload Identity. A followup PR will
rewire the sidecar controller to make use of this new resource.

As this is a performance optimization, rather than a security feature the following
aspects of traffic permissions have been ignored:

- DENY rules
- port rules (all ports are allowed)

Also:

- Add some v2 TestController machinery to help test complex dependency mappers.

Co-authored-by: R.B. Boyer <[email protected]>
…e notes into release/1.18.x (#20576)

* backport of commit 610892e

* backport of commit a14018b

* backport of commit aebdd6b

---------

Co-authored-by: David Yu <[email protected]>
jmurret and others added 16 commits August 26, 2024 17:18
* troubleshoot/v0.6.5

* updating rootgo.mod
docs/WAF: failure zones refresh (#21545)

* failure zones initial commit

* Apply suggestions from code review




* Update improving-consul-resilience.mdx



* typo

* adding diagrams

* fixing inline bulletpoint image

* adding light and dark diagrams

* Apply suggestions from code review



* fix links in article

* fix inline alert render

---------

Co-authored-by: Krastin Krastev <[email protected]>
Co-authored-by: Aimee Ukasick <[email protected]>
Co-authored-by: danielehc <[email protected]>
…release/1.18.x (#21668)

run integration tests on push in main and release/* (#21666)

* run integration tests on push in main and release/*

* Update .github/workflows/test-integrations.yml



---------

Co-authored-by: Michael Zalimeni <[email protected]>
* update changelog to include 1.18.4

* update version

* Apply suggestions from code review

Co-authored-by: Michael Zalimeni <[email protected]>

---------

Co-authored-by: Michael Zalimeni <[email protected]>
…1.18.x (#21690)

Docs CE-709: Remove circular links (#21685)

Docs CE-70: Remove circular links

Remove links to tutorials that no longer exist and redirect back
to the ACL overview page.
…nd L7 intentions header additions (1.18) (#21841)

api: Add fields for HTTP request normalization and L7 intentions header additions

This feature is available in Consul CE 1.20.1 and Consul Enterprise 1.19.3, 1.18.4, and 1.15.15.
…and grpc into release/1.18.x (#21847)

backport of commit 1986c55

Co-authored-by: R.B. Boyer <[email protected]>
…ons, and security guidance to release/1.18.x (#21857)

backport of commit a9d70fe and 9e7757d
…ntos:7 since it is EOL (#21892)

backport centos logic to point to vault.centos.org for centos:7 since it is EOL

Co-authored-by: github-team-consul-core <[email protected]>
@github-actions github-actions bot added type/docs Documentation needs to be created/updated/clarified theme/api Relating to the HTTP API interface theme/health-checks Health Check functionality theme/acls ACL and token generation theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/ui Anything related to the UI theme/telemetry Anything related to telemetry or observability type/ci Relating to continuous integration (CI) tooling for testing or releases pr/dependencies PR specifically updates dependencies of project theme/envoy/xds Related to Envoy support theme/contributing Additions and enhancements to community contributing materials theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation labels Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified
Projects
None yet
Development

Successfully merging this pull request may close these issues.