Add steps to configure a PGP key for agent upgrade #984

kilfoyle · 2024-03-26T20:35:18Z

This adds instructions for air-gapped environments on how to set server.pgp.upstream_url so that Elastic Agents can access the PGP/GPG key required for upgrade.

See docs preview page.

Please let me know if I've missed anything, especially in the "Note that" section under the steps.

Closes: #980
Rel: https://github.com/elastic/sdh-beats/issues/4496

github-actions · 2024-03-26T20:35:31Z

A documentation preview will be available soon.

🔨 Buildkite builds
📚 HTML diff
📙 Preview page

Request a new doc build by commenting

Rebuild this PR: run docs-build
Rebuild this PR and all Elastic docs: run docs-build rebuild

_{run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.}

_{If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.}

kilfoyle · 2024-04-03T18:11:16Z

@lucabelluccini When you have a chance, please let me know if this new section looks okay. Thanks!

kilfoyle · 2024-04-12T02:32:07Z

@lucabelluccini trying again. Please have a look if you have a chance. Thanks!

kilfoyle · 2024-04-23T13:54:33Z

@michel-laterman , @michalpristas , @lucabelluccini
Can any of you please review and confirm that these added Configure Elastic Agents to download a PGP/GPG key from Fleet Server steps are correct?

kilfoyle · 2024-05-29T18:10:37Z

@elasticmachine run docs-build

lucabelluccini · 2024-07-29T16:03:05Z

I still think a developer should review this.

kilfoyle · 2024-07-29T16:21:18Z

Thanks @lucabelluccini

I still think a developer should review this.

@michel-laterman or @michalpristas Would either of you mind taking a look at this? Please let me know if you're not the right people to ask.

michel-laterman · 2024-07-29T19:28:12Z

I think it looks good, do we want to add anything to explain how to use the fleet-server endpoint as an alternative when distributing gpg keys (issue elastic/fleet-server#2887)?

kilfoyle · 2024-07-29T19:49:00Z

do we want to add anything to explain how to use the fleet-server endpoint as an alternative when distributing gpg keys (issue elastic/fleet-server#2887)?

@michel-laterman I've add the steps below, but I'm way out of my depth here so please let me know whatever needs fixing up.

michel-laterman · 2024-07-29T20:29:12Z

docs/en/ingest-management/fleet/air-gapped.asciidoc

+
+In an air-gapped environment, the agent won't be able to download the remote key and therefore cannot be upgraded. For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation.
+
+Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from a custom URL specified in {fleet-server}:


I think my reading comprehension on my 1st pass wasn't great.

Just to be clear fleet-server serves PGP keys at GET /api/agents/upgrades/{major}.{minor}.{patch}/pgp-public-key
It will serve the key in FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp by default, if there is no default.pgp key in the dir it will look upstream to retrieve a key; server.pgp.upstream_url can be used to configure to a custom URL for fleet-server to use.

IIRC agents will try to get the upstream key, then the fleet-server endpoint (@michalpristas please confirm), they do not use the url we add to fleet-server config

Thanks @michel-laterman! That's really clear. Here's my attempt to fix things up:

docs/en/ingest-management/fleet/air-gapped.asciidoc

lucabelluccini · 2024-08-23T17:21:29Z

docs/en/ingest-management/fleet/air-gapped.asciidoc

+
+By default, {fleet-server} serves {agents} with the key located in `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`.
+The key is served through the endpoint `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`.
+Accessing the endpoint requires a valid API key and the endpoint is rate limited.


Suggested change

Accessing the endpoint requires a valid API key and the endpoint is rate limited.

Accessing the endpoint requires a valid API key and the endpoint is rate limited.

I am not sure we're protecting the key via API key - can we confirm?
If yes, are Elastic Agent adding the API Key to the requests?

lucabelluccini · 2024-08-23T17:32:28Z

docs/en/ingest-management/fleet/air-gapped.asciidoc

+The key is served through the endpoint `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`.
+Accessing the endpoint requires a valid API key and the endpoint is rate limited.
+
+If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting.


Suggested change

If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting.

If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting.

Few points to clarify:

Is the default.pgp bundled with Fleet Server?

If it's bundled, air-gapped users are good to go with it.

Is the default.pgp downloaded lazily from https://artifacts.elastic.co/GPG-KEY-elastic-agent by default?

If it is the case, then air-gapped users can grant access to it (HTTP Proxy or open the firewall).

Is the default.php downloaded lazily from the URL provided via server.pgp.upstream_url setting?

If it is the case, then air-gapped users can specify a URL so that it will be downloaded from there.

It's a lazy download from server.pgp.upstream_url

To be very clear:

Is the default.pgp bundled with Fleet Server?

No, on install this file does not exist; but a user can add it manually write the file on the instance if there is 0 internet access in order to distribute it to the agents

Is the default.pgp downloaded lazily from https://artifacts.elastic.co/GPG-KEY-elastic-agent by default?

Yes, the default value of server.pgp.upstream_url is https://artifacts.elastic.co/GPG-KEY-elastic-agent; if default.pgp is not on-disk when the key is requested, fleet-server will download the file from the upstream URL, save it to disk and serve it

Is the default.php downloaded lazily from the URL provided via server.pgp.upstream_url setting?

yes, see above 😄

lucabelluccini · 2024-08-23T17:33:38Z

Hello @kilfoyle - I've left some comments. We might need clarifications by the developers.
I've not taken the time to setup an environment to try it out yet.

kilfoyle · 2024-08-29T13:43:30Z

Thanks @lucabelluccini I've added in your suggestions. @michel-laterman when you have time I'd appreciate if you can help with Luca's questions.

I've updated the screen capture here to show the latest version.

shainaraskas · 2024-10-04T20:23:04Z

Does this still need a review, or should it be closed?

kilfoyle · 2024-10-05T01:22:49Z

@shainaraskas thanks, this can stay open and I plan to address comments.

Add steps to configure a PGP key for agent upgrade

08188fc

kilfoyle requested a review from a team as a code owner March 26, 2024 20:35

kilfoyle added the backport-8.13 label Mar 26, 2024

kilfoyle requested review from lucabelluccini, michalpristas and michel-laterman March 26, 2024 21:12

lucabelluccini previously approved these changes Jul 29, 2024

View reviewed changes

Rebuild

80e6075

Add alternative of using fleet-server endpoint

ab5616f

fixup

ceadbd6

michel-laterman reviewed Jul 29, 2024

View reviewed changes

Update logic based on Michel's input

bcb1d38

kilfoyle dismissed lucabelluccini’s stale review via bcb1d38 August 2, 2024 19:43

lucabelluccini reviewed Aug 23, 2024

View reviewed changes

docs/en/ingest-management/fleet/air-gapped.asciidoc Outdated Show resolved Hide resolved

lucabelluccini reviewed Aug 23, 2024

View reviewed changes

docs/en/ingest-management/fleet/air-gapped.asciidoc Show resolved Hide resolved

lucabelluccini reviewed Aug 23, 2024

View reviewed changes

docs/en/ingest-management/fleet/air-gapped.asciidoc Outdated Show resolved Hide resolved

lucabelluccini reviewed Aug 23, 2024

View reviewed changes

kilfoyle removed the request for review from michalpristas August 23, 2024 17:54

Add Luca's suggestions

f698c5e

kilfoyle requested a review from michel-laterman August 29, 2024 13:43

kilfoyle added backport-8.x backport-8.16 and removed backport-8.13 labels Oct 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add steps to configure a PGP key for agent upgrade #984

Add steps to configure a PGP key for agent upgrade #984

kilfoyle commented Mar 26, 2024 •

edited

Loading

github-actions bot commented Mar 26, 2024

kilfoyle commented Apr 3, 2024

kilfoyle commented Apr 12, 2024

kilfoyle commented Apr 23, 2024

kilfoyle commented May 29, 2024

lucabelluccini commented Jul 29, 2024

kilfoyle commented Jul 29, 2024

michel-laterman commented Jul 29, 2024

kilfoyle commented Jul 29, 2024

michel-laterman Jul 29, 2024

kilfoyle Jul 30, 2024 •

edited

Loading

lucabelluccini Aug 23, 2024 •

edited

Loading

lucabelluccini Aug 23, 2024

michel-laterman Aug 29, 2024

michel-laterman Sep 18, 2024

lucabelluccini commented Aug 23, 2024

kilfoyle commented Aug 29, 2024

shainaraskas commented Oct 4, 2024

kilfoyle commented Oct 5, 2024


		In an air-gapped environment, the agent won't be able to download the remote key and therefore cannot be upgraded. For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation.

		Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from a custom URL specified in {fleet-server}:

	Accessing the endpoint requires a valid API key and the endpoint is rate limited.
	Accessing the endpoint requires a valid API key and the endpoint is rate limited.

	If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting.
	If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting.

Add steps to configure a PGP key for agent upgrade #984

Are you sure you want to change the base?

Add steps to configure a PGP key for agent upgrade #984

Conversation

kilfoyle commented Mar 26, 2024 • edited Loading

github-actions bot commented Mar 26, 2024

kilfoyle commented Apr 3, 2024

kilfoyle commented Apr 12, 2024

kilfoyle commented Apr 23, 2024

kilfoyle commented May 29, 2024

lucabelluccini commented Jul 29, 2024

kilfoyle commented Jul 29, 2024

michel-laterman commented Jul 29, 2024

kilfoyle commented Jul 29, 2024

michel-laterman Jul 29, 2024

Choose a reason for hiding this comment

kilfoyle Jul 30, 2024 • edited Loading

Choose a reason for hiding this comment

lucabelluccini Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

lucabelluccini Aug 23, 2024

Choose a reason for hiding this comment

michel-laterman Aug 29, 2024

Choose a reason for hiding this comment

michel-laterman Sep 18, 2024

Choose a reason for hiding this comment

lucabelluccini commented Aug 23, 2024

kilfoyle commented Aug 29, 2024

shainaraskas commented Oct 4, 2024

kilfoyle commented Oct 5, 2024

kilfoyle commented Mar 26, 2024 •

edited

Loading

kilfoyle Jul 30, 2024 •

edited

Loading

lucabelluccini Aug 23, 2024 •

edited

Loading