From 08188fc8df00e7f03ed441cf0802f5ea2a6f2055 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Tue, 26 Mar 2024 16:33:06 -0400 Subject: [PATCH 01/10] Add steps to configure a PGP key for agent upgrade --- .../upgrade-standalone-elastic-agent.asciidoc | 2 ++ .../fleet/air-gapped.asciidoc | 30 +++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/docs/en/ingest-management/elastic-agent/upgrade-standalone-elastic-agent.asciidoc b/docs/en/ingest-management/elastic-agent/upgrade-standalone-elastic-agent.asciidoc index a1788de62..4fe6e2402 100644 --- a/docs/en/ingest-management/elastic-agent/upgrade-standalone-elastic-agent.asciidoc +++ b/docs/en/ingest-management/elastic-agent/upgrade-standalone-elastic-agent.asciidoc @@ -34,6 +34,8 @@ As an alterative, you can do one of the following: * <> for standalone {agent} to access the {artifact-registry}. * <> for standalone {agent} to access binary downloads. +As well, starting from version 8.9.0, during the upgrade process {agent} needs to download a PGP/GPG key. Refer to <> for the steps to configure the key download location in an air-gapped environment. + Refer to <> for more details. [[upgrade-standalone-verify-package]] diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index e543cdd33..8f2be4799 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -32,6 +32,36 @@ Set the following property in {kib} to enable air-gapped mode in {fleet}. This a xpack.fleet.isAirGapped: true ---- +[discrete] +[[air-gapped-pgp-fleet]] +== Configure {agents} to download a PGP/GPG key from {fleet-server} + +Starting from version 8.9.0, when {agent} tries to perform an upgrade, it first verifies the binary signature with the key bundled in the agent. This process has a backup mechanism that will use the key coming from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` instead of the one it already has. + +In an air-gapped environment, the agent won't be able to download the remote key and therefore cannot be upgraded. For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation. + +Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from a custom URL specified in {fleet-server}: + +To set a custom URL for {agents} to download a PGP/GPG key: + +. In {kib}, go to *Management > {fleet} > Agent policies*. +. Select a policy for the agents that you want to upgrade. +. On the policy page, in the **Actions** menu for the {fleet-server} integration, select **Edit integration**. +. In the {fleet-server} settings section expand **Change defaults** and **Advanced options**. +. In the **Custom fleet-server configurations** field, add the setting `server.pgp.upstream_url` with the URL where the PGP/GPG key can be downloaded. For example: + +[source,yaml] +---- +server.pgp.upstream_url: +---- + +The setting `server.pgp.upstream_url` must point to a web server hosting the PGP/GPG key, which must be reachable by the host where {fleet-server} is installed. + +Note that: + + * `server.pgp.upstream_url` may be specified as an `http` endpoint (instead of `https`). + * For an `https` endpoint, the CA must be trusted by {fleet-server}. + [discrete] [[air-gapped-proxy-server]] == Use a proxy server to access the {package-registry} From 80e60755bb0456d3ac6fbc1dd4aee8b3e2d6f9a4 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Mon, 29 Jul 2024 15:21:22 -0400 Subject: [PATCH 02/10] Rebuild From ab5616f6483147cf2dc0ec3d514503628e3f3738 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Mon, 29 Jul 2024 15:46:36 -0400 Subject: [PATCH 03/10] Add alternative of using fleet-server endpoint --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index 8f2be4799..a33700780 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -62,6 +62,18 @@ Note that: * `server.pgp.upstream_url` may be specified as an `http` endpoint (instead of `https`). * For an `https` endpoint, the CA must be trusted by {fleet-server}. +As an alternative, you can use a {fleet-server} endpoint to retrieve a PGP key: + +[source,shell] +---- +GET /api/agents/upgrades/{major}.{minor}.{patch}/pgp-public-key +---- + +Accessing the endpoint requires a valid API key and the endpoint is rate limited. + +The endpoint will serve a PGP key from it's cache if one exists, otherwise it will serve the contents of a PGP key found on disk at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. Note that the directory can be changed through configuration. +The key value is added to cache if it exists. + [discrete] [[air-gapped-proxy-server]] == Use a proxy server to access the {package-registry} From ceadbd6f753af5999949d4461613e06d81204dd8 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Mon, 29 Jul 2024 16:03:11 -0400 Subject: [PATCH 04/10] fixup --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index a33700780..dbcd77211 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -72,7 +72,7 @@ GET /api/agents/upgrades/{major}.{minor}.{patch}/pgp-public-key Accessing the endpoint requires a valid API key and the endpoint is rate limited. The endpoint will serve a PGP key from it's cache if one exists, otherwise it will serve the contents of a PGP key found on disk at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. Note that the directory can be changed through configuration. -The key value is added to cache if it exists. +The key value is added to the cache if it exists. [discrete] [[air-gapped-proxy-server]] From bcb1d38d661bf0dc90780cdb8d7458798cfbca87 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Tue, 30 Jul 2024 12:38:06 -0400 Subject: [PATCH 05/10] Update logic based on Michel's input --- .../fleet/air-gapped.asciidoc | 25 ++++++++----------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index dbcd77211..d6a37b1d0 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -40,15 +40,22 @@ Starting from version 8.9.0, when {agent} tries to perform an upgrade, it first In an air-gapped environment, the agent won't be able to download the remote key and therefore cannot be upgraded. For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation. -Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from a custom URL specified in {fleet-server}: -To set a custom URL for {agents} to download a PGP/GPG key: +Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from {fleet-server}. + +By default, {fleet-server} serves {agents} with the key located in `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. +The key is served through the endpoint `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. +Accessing the endpoint requires a valid API key and the endpoint is rate limited. + +If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting. + +To set a custom URL for {fleet-server} to access a PGP/GPG key and make it available to {agents}: . In {kib}, go to *Management > {fleet} > Agent policies*. . Select a policy for the agents that you want to upgrade. . On the policy page, in the **Actions** menu for the {fleet-server} integration, select **Edit integration**. . In the {fleet-server} settings section expand **Change defaults** and **Advanced options**. -. In the **Custom fleet-server configurations** field, add the setting `server.pgp.upstream_url` with the URL where the PGP/GPG key can be downloaded. For example: +. In the **Custom fleet-server configurations** field, add the setting `server.pgp.upstream_url` with the URL where the PGP/GPG key can be accessed. For example: [source,yaml] ---- @@ -62,18 +69,6 @@ Note that: * `server.pgp.upstream_url` may be specified as an `http` endpoint (instead of `https`). * For an `https` endpoint, the CA must be trusted by {fleet-server}. -As an alternative, you can use a {fleet-server} endpoint to retrieve a PGP key: - -[source,shell] ----- -GET /api/agents/upgrades/{major}.{minor}.{patch}/pgp-public-key ----- - -Accessing the endpoint requires a valid API key and the endpoint is rate limited. - -The endpoint will serve a PGP key from it's cache if one exists, otherwise it will serve the contents of a PGP key found on disk at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. Note that the directory can be changed through configuration. -The key value is added to the cache if it exists. - [discrete] [[air-gapped-proxy-server]] == Use a proxy server to access the {package-registry} From f698c5e0a0d5281f279d95865d0c3dd62019841e Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Thu, 29 Aug 2024 09:40:16 -0400 Subject: [PATCH 06/10] Add Luca's suggestions --- .../en/ingest-management/fleet/air-gapped.asciidoc | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index d6a37b1d0..1953e4262 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -38,14 +38,20 @@ xpack.fleet.isAirGapped: true Starting from version 8.9.0, when {agent} tries to perform an upgrade, it first verifies the binary signature with the key bundled in the agent. This process has a backup mechanism that will use the key coming from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` instead of the one it already has. -In an air-gapped environment, the agent won't be able to download the remote key and therefore cannot be upgraded. For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation. - +In an air-gapped environment, an {agent} which doesn't have access to a PGP/GPG key from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` would fail to be upgraded. +For versions 8.9.0 to 8.10.3, you can resolve this problem following the steps described in the associated link:https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html#known-issues-8.9.0[known issue] documentation. Starting in version 8.10.4, you can resolve this problem by configuring {agents} to download the PGP/GPG key from {fleet-server}. +Starting in version 8.10.4, {agent} will: + +. Verify the binary signature with the key bundled in the agent. +. If the verification doesn't pass, the agent will download the PGP/GPG key from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` and verify it. +. If that verification doesn't pass, the agent will download the PGP/GPG key from {fleet-server} and verify it. +. If that verification doesn't pass, the upgrade is blocked. + By default, {fleet-server} serves {agents} with the key located in `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. -The key is served through the endpoint `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. -Accessing the endpoint requires a valid API key and the endpoint is rate limited. +The key is served through the {fleet-server} endpoint `GET /api/agents/upgrades/{major}.{minor}.{patch}/pgp-public-key`. If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting. From 3d7e3d83fe23659b91d66625abf2b74b7cb7796a Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Tue, 28 Jan 2025 12:41:59 -0500 Subject: [PATCH 07/10] Update docs/en/ingest-management/fleet/air-gapped.asciidoc Co-authored-by: Luca Belluccini --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index 1953e4262..bada30eeb 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -55,6 +55,8 @@ The key is served through the {fleet-server} endpoint `GET /api/agents/upgrades/ If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting. +You can prevent {fleet} to download the PGP/GPG key from `server.pgp.upstream_url` by manually downloading it from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` and storing it at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. + To set a custom URL for {fleet-server} to access a PGP/GPG key and make it available to {agents}: . In {kib}, go to *Management > {fleet} > Agent policies*. From f221a6e68f5efc17c2b29f4f860eebb69eee28f9 Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Tue, 28 Jan 2025 12:48:01 -0500 Subject: [PATCH 08/10] Update docs/en/ingest-management/fleet/air-gapped.asciidoc --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index bada30eeb..f01c1fe6d 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -55,7 +55,7 @@ The key is served through the {fleet-server} endpoint `GET /api/agents/upgrades/ If there isn't a `default.pgp` key in the `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp` directory, {fleet-server} instead will attempt to retrieve a PGP/GPG key from the URL that you can specify with the `server.pgp.upstream_url` setting. -You can prevent {fleet} to download the PGP/GPG key from `server.pgp.upstream_url` by manually downloading it from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` and storing it at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. +You can prevent {fleet} from downloading the PGP/GPG key from `server.pgp.upstream_url` by manually downloading it from `https://artifacts.elastic.co/GPG-KEY-elastic-agent` and storing it at `FLEETSERVER_BINARY_DIR/elastic-agent-upgrade-keys/default.pgp`. To set a custom URL for {fleet-server} to access a PGP/GPG key and make it available to {agents}: From 09bbffe89dda5b989f8a635603a91990ef1c893e Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Tue, 28 Jan 2025 13:15:55 -0500 Subject: [PATCH 09/10] Update docs/en/ingest-management/fleet/air-gapped.asciidoc --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index f01c1fe6d..c102871cb 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -63,11 +63,11 @@ To set a custom URL for {fleet-server} to access a PGP/GPG key and make it avail . Select a policy for the agents that you want to upgrade. . On the policy page, in the **Actions** menu for the {fleet-server} integration, select **Edit integration**. . In the {fleet-server} settings section expand **Change defaults** and **Advanced options**. -. In the **Custom fleet-server configurations** field, add the setting `server.pgp.upstream_url` with the URL where the PGP/GPG key can be accessed. For example: +. In the **Custom fleet-server configurations** field, add the setting `server.pgp.upstream_url` with the full URL where the PGP/GPG key can be accessed. For example: [source,yaml] ---- -server.pgp.upstream_url: +server.pgp.upstream_url: ---- The setting `server.pgp.upstream_url` must point to a web server hosting the PGP/GPG key, which must be reachable by the host where {fleet-server} is installed. From c2cb2d08021748a285acd335399c5e9922c55d03 Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Tue, 28 Jan 2025 17:56:29 -0500 Subject: [PATCH 10/10] Update docs/en/ingest-management/fleet/air-gapped.asciidoc --- docs/en/ingest-management/fleet/air-gapped.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/ingest-management/fleet/air-gapped.asciidoc b/docs/en/ingest-management/fleet/air-gapped.asciidoc index c102871cb..3a0edd3c9 100644 --- a/docs/en/ingest-management/fleet/air-gapped.asciidoc +++ b/docs/en/ingest-management/fleet/air-gapped.asciidoc @@ -75,7 +75,7 @@ The setting `server.pgp.upstream_url` must point to a web server hosting the PGP Note that: * `server.pgp.upstream_url` may be specified as an `http` endpoint (instead of `https`). - * For an `https` endpoint, the CA must be trusted by {fleet-server}. + * For an `https` endpoint, the CA for {fleet-server} to connect to `server.pgp.upstream_url` must be trusted by {fleet-server} using the `--certificate-authorities` setting that is used globally for {agent}. [discrete] [[air-gapped-proxy-server]]