Skip to content

Commit

Permalink
Add tutorial for standalone agent on serverless monitoring nginx (#906)…
Browse files Browse the repository at this point in the history
… (#917)

* Add tutorial for standalone agent on serverless monitoring nginx

* fixup

* fixup

* Change original guide to use API key auth

* Change IDs to -ess; small fixes

(cherry picked from commit 4db4aa2)

Co-authored-by: David Kilfoyle <[email protected]>
  • Loading branch information
mergify[bot] and kilfoyle authored Feb 13, 2024
1 parent c1d9a12 commit 65ac696
Show file tree
Hide file tree
Showing 3 changed files with 362 additions and 40 deletions.
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
[[example-standalone-monitor-nginx]]
= Example: Use standalone {agent} to monitor nginx
= Example: Use standalone {agent} with {ess} to monitor nginx

This guide walks you through a simple monitoring scenario so you can learn the basics of setting up standalone {agent}, using it to work with {ecloud} and an Elastic integration.
This guide walks you through a simple monitoring scenario so you can learn the basics of setting up standalone {agent}, using it to work with {ess} and an Elastic integration.

Following these steps, you'll deploy the {stack}, install a standalone {agent} on a host to monitor an nginx web server instance, and access visualizations based on the collected logs.

. <<nginx-guide-install-nginx,Install nginx>>.
. <<nginx-guide-sign-up,Create an {ecloud} deployment>>.
. <<nginx-guide-create-policy,Create an {agent} policy>>.
. <<nginx-guide-add-integration,Add the Nginx Integration>>.
. <<nginx-guide-configure-standalone-agent,Configure standalone {agent}>>.
. <<nginx-guide-confirm-agent-data,Confirm that your {agent} data is flowing>>.
. <<nginx-guide-view-system-data,View your system data>>.
. <<nginx-guide-view-nginx-data,View your nginx logging data>>.
. <<nginx-guide-install-nginx-ess,Install nginx>>.
. <<nginx-guide-sign-up-ess,Create an {ecloud} deployment>>.
. <<nginx-guide-create-api-key-ess,Create an {ecloud} API key.>>
. <<nginx-guide-create-policy-ess,Create an {agent} policy>>.
. <<nginx-guide-add-integration-ess,Add the Nginx Integration>>.
. <<nginx-guide-configure-standalone-agent-ess,Configure standalone {agent}>>.
. <<nginx-guide-confirm-agent-data-ess,Confirm that your {agent} data is flowing>>.
. <<nginx-guide-view-system-data-ess,View your system data>>.
. <<nginx-guide-view-nginx-data-ess,View your nginx logging data>>.

[discrete]
[[nginx-guide-prereqs]]
[[nginx-guide-prereqs-ess]]
=== Prerequisites

To get started, you need:
Expand All @@ -24,7 +25,7 @@ To get started, you need:
. A Linux host machine on which you'll install an nginx web server. The commands in this guide use an Ubuntu image but any Linux distribution should be fine.

[discrete]
[[nginx-guide-install-nginx]]
[[nginx-guide-install-nginx-ess]]
=== Step 1: Install nginx

To start, we'll set up a basic link:https://docs.nginx.com/nginx/admin-guide/web-server/[nginx web server].
Expand All @@ -42,12 +43,12 @@ sudo apt install nginx
image::images/guide-nginx-welcome.png["Browser window showing Welcome to nginx!"]

[discrete]
[[nginx-guide-sign-up]]
[[nginx-guide-sign-up-ess]]
=== Step 2: Create an {ecloud} deployment

NOTE: If you've already signed up for a trial deployment you can skip this step.

Now that your web server is running, let's get set up to monitor it in {ecloud}. An {ecloud} deployment offers you all of the features of the {stack} as a hosted service. To test drive your first deployment, sign up for a free {ecloud} trial:
Now that your web server is running, let's get set up to monitor it in {ecloud}. An {ecloud} {ess} deployment offers you all of the features of the {stack} as a hosted service. To test drive your first deployment, sign up for a free {ecloud} trial:

. Go to our link:https://cloud.elastic.co/registration?elektra=guide-welcome-cta[{ecloud} Trial] page.

Expand All @@ -56,18 +57,33 @@ Now that your web server is running, let's get set up to monitor it in {ecloud}.
[role="screenshot"]
image::images/guide-sign-up-trial.png["Start your free Elastic Cloud trial",width="50%"]

. After you've link:https://cloud.elastic.co/login[logged in], you can create a deployment. Give your deployment a name and select *Create deployment*.
+
[role="screenshot"]
image::images/guide-create-first-deployment.png["Create your first deployment",width="70%"]
. After you've link:https://cloud.elastic.co/login[logged in], select *Create deployment* and give your deployment a name. You can leave the default options or select a different cloud provider, region, hardware profile, or version.

. Select *Create deployment*.

. While the deployment sets up, make a note of your `elastic` superuser password and keep it in a safe place.

. Once the deployment is ready, select *Continue*. At this point, you access {kib} and a selection of setup guides.

[discrete]
[[nginx-guide-create-policy]]
=== Step 3: Create an {agent} policy
[[nginx-guide-create-api-key-ess]]
=== Step 3: Create an {ecloud} API key

. From the {kib} menu and go to *Stack Management* -> *API keys*.

. Select *Create API key*.

. Give the key a name, for example `nginx example API key`.

. Leave the other default options and select *Create API key*.

. In the *Create API key* confirmation dialog, change the dropdown menu setting from `Encoded` to `Beats`. This sets the API key format for communication between {agent} (which is based on {beats}) and {es}.

. Copy the generated API key and store it in a safe place. You'll use it in a later step.

[discrete]
[[nginx-guide-create-policy-ess]]
=== Step 4: Create an {agent} policy

{agent} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, and more. A single agent makes it easy and fast to deploy monitoring across your infrastructure. Each agent has a single policy (a collection of input settings) that you can update to add integrations for new data sources, security protections, and more.

Expand All @@ -82,15 +98,15 @@ image::images/guide-agent-policies.png["Agent policies tab in Fleet"]
image::images/guide-create-agent-policy.png["Create agent policy UI"]

[discrete]
[[nginx-guide-add-integration]]
=== Step 4: Add the Nginx Integration
[[nginx-guide-add-integration-ess]]
=== Step 5: Add the Nginx Integration

Elastic integrations are a streamlined way to connect your data from popular services and platforms to the {stack}, including nginx.

. From the **{fleet} -> Agent policies** tab, click the link for your new `nginx-policy`.
+
image::images/guide-nginx-policy.png["The nginx-policy UI with integrations tab selected"]
. Note that `system-` integration is included because you opted earlier to collect system logs and metrics.
. Note that the System integration (`system-1`) is included because you opted earlier to collect system logs and metrics.
. Click **Add integration**.
. On the Integrations page search for "nginx".
+
Expand All @@ -100,16 +116,16 @@ image::images/guide-integrations-page.png["Integrations page with nginx in the s
. Click the link to **Add integration only (skip agent installation)**. You'll install standalone {agent} in a later step.
. Here, you can select options such as the paths to where your nginx logs are stored, whether or not to collect metrics data, and various other settings.
+
For now, leave all of the default settings and click **Save and continue** to add the Nginx Integration to your `nginx-policy` policy.
For now, leave all of the default settings and click **Save and continue** to add the Nginx integration to your `nginx-policy` policy.
+
image::images/guide-add-nginx-integration.png["Add Nginx Integration UI"]
. In the confirmation dialog, select to **Add {agent} later**.
+
image::images/guide-nginx-integration-added.png["Nginx Integration added confirmation UI with Add {agent} later selected."]

[discrete]
[[nginx-guide-configure-standalone-agent]]
=== Step 5: Configure standalone {agent}
[[nginx-guide-configure-standalone-agent-ess]]
=== Step 6: Configure standalone {agent}

Rather than opt for {fleet} to centrally manage {agent}, you'll configure an agent to run in standalone mode, so it will be managed by hand.

Expand All @@ -120,7 +136,7 @@ Rather than opt for {fleet} to centrally manage {agent}, you'll configure an age
image::images/guide-add-agent-standalone01.png["Add agent UI with nginx-policy and Run-standalone selected."]
. For the **Configure the agent** step, choose **Download Policy**. Save the `elastic-agent.yml` file to a directory on the host where you'll install nginx for monitoring.
+
Notice that the policy contains all of the input, output, and other settings for the Nginx and System Integrations. If you already have a standalone agent installed on a host with an existing {agent} policy, you can use the method described here to add a new integration. Just add the settings from the **Configure the agent** step to your existing `elastic-agent.yml` file.
Have a look inside the policy file and notice that it contains all of the input, output, and other settings for the Nginx and System integrations. If you already have a standalone agent installed on a host with an existing {agent} policy, you can use the method described here to add a new integration. Just add the settings from the **Configure the agent** step to your existing `elastic-agent.yml` file.
. For the **Install {agent} on your host** step, select the tab for your host operating system and run the commands on your host.
+
image::images/guide-install-agent-on-host.png["Install {agent} on your host step, showing tabs with the commands for different operating systems."]
Expand Down Expand Up @@ -148,8 +164,23 @@ elastic-agent status
+
Since you're running the agent in standalone mode the `Not enrolled into Fleet` message is expected.
. Open the `elastic-agent.yml` policy file that you saved.
. Near the top of the file, replace `${ES_USERNAME}` with the username for the {ecloud} deployment that you created in <<nginx-guide-sign-up,Step 2>>. The default is `elastic`.
. Similarly, replace `${ES_PASSWORD}` with your superuser password.
. Near the top of the file, replace:
+
[source,yaml]
----
username: '${ES_USERNAME}'
password: '${ES_PASSWORD}'
----
+
with:
+
[source,yaml]
----
api_key: '<your-api-key>'
----
+
where `your-api-key` is the API key that you generated in <<nginx-guide-create-api-key-ess>>.

. Find the location of the default `elastic-agent.yml` policy file that is included in your {agent} install. Install directories for each platform are described in <<installation-layout,Installation layout>>. In our example Ubuntu image the default policy file can be found in `/etc/elastic-agent/elastic-agent.yml`.
. Replace the default policy file with the version that you downloaded and updated. For example:
+
Expand Down Expand Up @@ -187,8 +218,8 @@ message: Running
----

[discrete]
[[nginx-guide-confirm-agent-data]]
=== Step 6: Confirm that your {agent} data is flowing
[[nginx-guide-confirm-agent-data-ess]]
=== Step 7: Confirm that your {agent} data is flowing

Now that {agent} is running, it's time to confirm that the agent data is flowing into {es}.

Expand All @@ -208,26 +239,28 @@ like the agent logs, the agent metrics should be flowing into {es} and visible i
image::images/guide-agent-metrics-flowing.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]

[discrete]
[[nginx-guide-view-system-data]]
=== Step 7: View your system data
[[nginx-guide-view-system-data-ess]]
=== Step 8: View your system data

In the step to <<nginx-guide-create-policy,create an {agent} policy>> you chose to collect system logs and metrics, so you can access those now.
In the step to <<nginx-guide-create-policy-ess,create an {agent} policy>> you chose to collect system logs and metrics, so you can access those now.

. View your system logs.
.. Open the {kib} menu and go to **Management -> Integrations -> Installed integrations**.
.. Select the **System** card and open the **Assets** tab. This is a quick way to access all of the dashboards, saved searches, and visualizations that come with each integration.
.. Select `[Logs System] Syslog dashboard`. The {kib} Dashboard opens with visualizations of Syslog events, hostnames and processes, and more.
.. Select `[Logs System] Syslog dashboard`.
.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of Syslog events, hostnames and processes, and more.
. View your system metrics.

.. Return to **Management -> Integrations -> Installed integrations**.
.. Select the **System** card and open the **Assets** tab.
.. Select `[Metrics System] Host overview`. The {kib} Dashboard opens with visualizations of host metrics including CPU usage, memory usage, running processes, and others.
.. This time, select `[Metrics System] Host overview`.
.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of host metrics including CPU usage, memory usage, running processes, and others.
+
image::images/guide-system-metrics-dashboard.png["The System metrics host overview showing CPU usage, memory usage, and other visualizations"]

[discrete]
[[nginx-guide-view-nginx-data]]
=== Step 8: View your nginx logging data
[[nginx-guide-view-nginx-data-ess]]
=== Step 9: View your nginx logging data

Now let's view your nginx logging data.

Expand All @@ -240,7 +273,7 @@ image::images/guide-nginx-logs-dashboard.png["The nginx logs dashboard shows var
+
image::images/guide-nginx-browser-breakdown.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]

You have now successfully set up monitoring for nginx logs using standalone {agent} and an {ecloud} deployment.
Congratulations! You have successfully set up monitoring for nginx using standalone {agent} and an {ecloud} deployment.

[discrete]
=== What's next?
Expand Down
Loading

0 comments on commit 65ac696

Please sign in to comment.