Add tutorial for standalone agent on serverless monitoring nginx

docs/en/ingest-management/elastic-agent/example-standalone-monitor-nginx-serverless.asciidoc

@@ -0,0 +1,287 @@
+[[example-standalone-monitor-nginx-serverless]]
+= Example: Use standalone {agent} to monitor nginx ({serverless-full} version)
+
+This guide walks you through a simple monitoring scenario so you can learn the basics of setting up standalone {agent}, using it to work with {serverless-full} and an Elastic integration. 
+
+Following these steps, you'll deploy the {stack}, install a standalone {agent} on a host to monitor an nginx web server instance, and access visualizations based on the collected logs.
+
+. <<nginx-guide-install-nginx-serverless,Install nginx>>.
+. <<nginx-guide-sign-up-serverless,Create an {serverless-full} project>>.
+. <<nginx-guide-create-api-key-serverless,Create an API key>>.
+. <<nginx-guide-create-policy-serverless,Create an {agent} policy>>.
+. <<nginx-guide-add-integration-serverless,Add the Nginx Integration>>.
+. <<nginx-guide-configure-standalone-agent-serverless,Configure standalone {agent}>>.
+. <<nginx-guide-confirm-agent-data-serverless,Confirm that your {agent} data is flowing>>.
+. <<nginx-guide-view-system-data-serverless,View your system data>>.
+. <<nginx-guide-view-nginx-data-serverless,View your nginx logging data>>.
+
+[discrete]
+[[nginx-guide-prereqs-serverless]]
+=== Prerequisites
+
+To get started, you need:
+
+. An internet connection and an email address for your {ecloud} trial.
+. A Linux host machine on which you'll install an nginx web server. The commands in this guide use an Ubuntu image but any Linux distribution should be fine.
+
+[discrete]
+[[nginx-guide-install-nginx-serverless]]
+=== Step 1: Install nginx
+
+To start, we'll set up a basic link:https://docs.nginx.com/nginx/admin-guide/web-server/[nginx web server].
+
+. Run the following command on an Ubuntu Linux host, or refer to the link:https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-open-source/[nginx install documentation] for the command appropriate to your operating system.
++
+[source,sh]
+----
+sudo apt install nginx
+----
++
+. Open a web browser and visit your host machine's external URL, for example `http://192.168.64.17/`. You should see the nginx welcome message.
++
+[role="screenshot"]
+image::images/guide-nginx-welcome.png["Browser window showing Welcome to nginx!"]
+
+[discrete]
+[[nginx-guide-sign-up-serverless]]
+=== Step 2: Create an {serverless-full} project
+
+NOTE: If you've already signed up for a trial deployment you can skip this step.
+
+Now that your web server is running, let's get set up to monitor it in {ecloud}. An {ecloud} project offers you all of the features of the {stack} as a hosted service. To test drive your first deployment, sign up for a free {ecloud} trial:
+
+. Go to our link:https://cloud.elastic.co/registration?elektra=guide-welcome-cta[{ecloud} Trial] page.
+
+. Enter your email address and a password.
++
+[role="screenshot"]
+image::images/guide-sign-up-trial.png["Start your free Elastic Cloud trial",width="50%"]
+
+. After you've link:https://cloud.elastic.co/login[logged in], select *Create project*.
+
+. On the *Observability* tab, select *Next*. The *Observability* and *Security* projects both include {fleet}, which you can use to create a policy for the {agent} that will monitor your nginx installation.
+
+. Give your project a name. You can leave the default options or select a different cloud provider and region.
+
+. Select *Create project*, and then wait a few minutes for the new project to set up.
+
+. Once the project is ready, select *Continue*. At this point, you access {kib} and a selection of setup guides.
+
+
+
+[discrete]
+[[nginx-guide-create-api-key-serverless]]
+=== Step 3: Create an {ecloud} API key
+
+. When your {serverless-short} project is ready, open the {kib} menu and go to **Project settings** -> **Management -> API keys**.
+
+. Select *Create API key*.
+
+. Give the key a name, for example `nginx example API key`.
+
+. Leave the other default options and select *Create API key*.
+
+. In the *Create API key* confirmation dialog, select change the dropdown menu from `Encoded` to `Beats`. This sets the API key format for communication between {agent} (which is based on {beats}) and {es}. 
+
+. Copy the generated API key and store it in a safe place. You'll use it in a later step.
+
+[discrete]
+[[nginx-guide-create-policy-serverless]]
+=== Step 4: Create an {agent} policy
+
+{agent} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, and more. A single agent makes it easy and fast to deploy monitoring across your infrastructure. Each agent has a single policy (a collection of input settings) that you can update to add integrations for new data sources, security protections, and more.
+
+. Open the {kib} menu and go to **Project settings** -> **{fleet} -> Agent policies**.
++
+image::images/guide-agent-policies.png["Agent policies tab in Fleet"]
+. Click *Create agent policy*.
+. Give your policy a name. For this example we'll call it `nginx-policy`.
+. Leave *Collect system logs and metrics* selected.
+. Click *Create agent policy*.
++
+image::images/guide-create-agent-policy.png["Create agent policy UI"]
+
+[discrete]
+[[nginx-guide-add-integration-serverless]]
+=== Step 5: Add the Nginx Integration
+
+Elastic integrations are a streamlined way to connect your data from popular services and platforms to the {stack}, including nginx.
+
+. From the **{fleet} -> Agent policies** tab, click the link for your new `nginx-policy`.
++
+image::images/guide-nginx-policy.png["The nginx-policy UI with integrations tab selected"]
+. Note that the System integration (`system-1`) is included because you opted earlier to collect system logs and metrics.
+. Click **Add integration**.
+. On the Integrations page search for "nginx".
++
+image::images/guide-integrations-page.png["Integrations page with nginx in the search bar"]
+. Select the **Nginx** card.
+. Click **Add Nginx**.
+. Click the link to **Add integration only (skip agent installation)**. You'll install standalone {agent} in a later step.
+. Here, you can select options such as the paths to where your nginx logs are stored, whether or not to collect metrics data, and various other settings.
++
+For now, leave all of the default settings and click **Save and continue** to add the Nginx integration to your `nginx-policy` policy.
++
+image::images/guide-add-nginx-integration.png["Add Nginx Integration UI"]
+. In the confirmation dialog, select to **Add {agent} later**.
++
+image::images/guide-nginx-integration-added.png["Nginx Integration added confirmation UI with Add {agent} later selected."]
+
+[discrete]
+[[nginx-guide-configure-standalone-agent-serverless]]
+=== Step 6: Configure standalone {agent}
+
+Rather than opt for {fleet} to centrally manage {agent}, you'll configure an agent to run in standalone mode, so it will be managed by hand.
+
+. Open the {kib} menu and go to **{fleet} -> Agents** and click **Add agent**.
+. For the **What type of host are you adding?** step, select `nginx-policy` from the drop-down menu if it's not already selected.
+. For the **Enroll in {fleet}?** step, select **Run standalone**.
++
+image::images/guide-add-agent-standalone01.png["Add agent UI with nginx-policy and Run-standalone selected."]
+. For the **Configure the agent** step, choose **Download Policy**. Save the `elastic-agent.yml` file to a directory on the host where you'll install nginx for monitoring.
++
+Have a look inside the policy file and notice that it contains all of the input, output, and other settings for the Nginx and System integrations. If you already have a standalone agent installed on a host with an existing {agent} policy, you can use the method described here to add a new integration. Just add the settings from the **Configure the agent** step to your existing `elastic-agent.yml` file.
+. For the **Install {agent} on your host** step, select the tab for your host operating system and run the commands on your host.
++
+image::images/guide-install-agent-on-host.png["Install {agent} on your host step, showing tabs with the commands for different operating systems."]
++
+[NOTE] 
+==== 
+{agent} commands need to be run as `root`. You can prefix each agent command with `sudo` or you can start a new shell as `root` by running `sudo su`. 
+====
++
+If you're prompted with `Elastic Agent will be installed at {installation location} and will run as a service. Do you want to continue?` answer `Yes`.
++
+If you're prompted with `Do you want to enroll this Agent into Fleet?` answer `no`.
++
+. You can run the `status` command to confirm that {agent} is running.
++
+[source,yaml]
+----
+elastic-agent status
+
+┌─ fleet
+│  └─ status: (STOPPED) Not enrolled into Fleet
+└─ elastic-agent
+   └─ status: (HEALTHY) Running
+----
++
+Since you're running the agent in standalone mode the `Not enrolled into Fleet` message is expected.
+. Open the `elastic-agent.yml` policy file that you saved.
+
+. Near the top of the file, replace: 
++
+[source,yaml]
+----
+    username: '${ES_USERNAME}'
+    password: '${ES_PASSWORD}'
+----
++
+with:
++
+[source,yaml]
+----
+    api_key: '<your-api-key>'
+----
++
+where `your-api-key` is the API key that you generated in <<nginx-guide-create-api-key-serverless>>.
+
+. Find the location of the default `elastic-agent.yml` policy file that is included in your {agent} install. Install directories for each platform are described in <<installation-layout,Installation layout>>. In our example Ubuntu image the default policy file can be found in `/etc/elastic-agent/elastic-agent.yml`.
+. Replace the default policy file with the version that you downloaded and updated. For example:
++
+[source,sh]
+----
+cp /home/ubuntu/homedir/downloads/elastic-agent.yml /etc/elastic-agent/elastic-agent.yml
+----
++
+NOTE: You may need to prefix the `cp` command with `sudo` for the permission required to replace the default file.
++
+By default, {agent} monitors the configuration file and reloads the configuration automatically when `elastic-agent.yml` is updated.
+
+. Run the `status` command again, this time with the `--output yaml` option which provides structured and much more detailed output. See the <<elastic-agent-status-command,`elastic-agent status`>> command documentation for more details.
++
+[source,shell]
+----
+elastic-agent status --output yaml
+----
++
+The results show you the agent status together with details about the running components, which correspond to the inputs and outputs defined for the integrations that have been added to the {agent} policy, in this case the System and Nginx Integrations.
+. At the top of the command output, the `info` section contains details about the agent instance. Make a note of the agent ID. In this example the ID is `4779b439-1130-4841-a878-e3d7d1a457d0`. You'll use that ID in the next section.
++
+[source,yaml]
+----
+elastic-agent status --output yaml
+
+info:
+  id: 4779b439-1130-4841-a878-e3d7d1a457d0
+  version: 8.9.1
+  commit: 5640f50143410fe33b292c9f8b584117c7c8f188
+  build_time: 2023-08-10 17:04:04 +0000 UTC
+  snapshot: false
+state: 2
+message: Running
+----
+
+[discrete]
+[[nginx-guide-confirm-agent-data-serverless]]
+=== Step 7: Confirm that your {agent} data is flowing
+
+Now that {agent} is running, it's time to confirm that the agent data is flowing into {es}.
+
+. Check that {agent} logs are flowing.
+.. Open the {kib} menu and go to **Observability -> Discover**.
+.. In the KQL query bar, enter the query `agent.id : "{agent-id}"` where `{agent-id}` is the ID you retrieved from the `elastic-agent status --output yaml` command. For example: `agent.id : "4779b439-1130-4841-a878-e3d7d1a457d0"`.
++
+If {agent} has connected successfully with your {ecloud} deployment, the agent logs should be flowing into {es} and visible in {kib} Discover.
++
+image::images/guide-agent-logs-flowing.png["Kibana Discover shows agent logs are flowing into Elasticsearch."]
+. Check that {agent} metrics are flowing.
+.. Open the {kib} menu and go to **Observability -> Dashboards**.
+.. In the search field, search for `Elastic Agent` and select `[Elastic Agent] Agent metrics` in the results.
++
+like the agent logs, the agent metrics should be flowing into {es} and visible in {kib} Dashboard. You can view metrics on CPU usage, memory usage, open handles, events rate, and more.
++
+image::images/guide-agent-metrics-flowing.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]
+
+[discrete]
+[[nginx-guide-view-system-data-serverless]]
+=== Step 8: View your system data
+
+In the step to <<nginx-guide-create-policy-serverless,create an {agent} policy>> you chose to collect system logs and metrics, so you can access those now.
+
+. View your system logs.
+.. Open the {kib} menu and go to **Project settings -> Integrations -> Installed integrations**.
+.. Select the **System** card and open the **Assets** tab. This is a quick way to access all of the dashboards, saved searches, and visualizations that come with each integration.
+.. Select `[Logs System] Syslog dashboard`. 
+.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of Syslog events, hostnames and processes, and more.
+. View your system metrics.
+
+.. Return to **Project settings -> Integrations -> Installed integrations**.
+.. Select the **System** card and open the **Assets** tab.
+.. This time, select `[Metrics System] Host overview`. 
+.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of host metrics including CPU usage, memory usage, running processes, and others.
++
+image::images/guide-system-metrics-dashboard.png["The System metrics host overview showing CPU usage, memory usage, and other visualizations"]
+
+[discrete]
+[[nginx-guide-view-nginx-data-serverless]]
+=== Step 9: View your nginx logging data
+
+Now let's view your nginx logging data.
+
+. Open the {kib} menu and go to **Management -> Integrations -> Installed integrations**.
+. Select the **Nginx** card and open the **Assets** tab.
+. Select `[Logs Nginx] Overview`. The {kib} Dashboard opens with geographical log details, response codes and errors over time, top pages, and more.
++
+image::images/guide-nginx-logs-dashboard.png["The nginx logs dashboard shows various visualizations on the nginx logs."]
+. Refresh your nginx web page several times to update the logging data. You can also try accessing the nginx page from different web browsers. After a minute or so, the `Browsers breakdown` visualization shows the respective volume of requests from the different browser types.
++
+image::images/guide-nginx-browser-breakdown.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]
+
+You have now successfully set up monitoring for nginx logs using standalone {agent} and an {ecloud} deployment.
+
+[discrete]
+=== What's next?
+
+* Learn more about <<fleet-overview,{fleet} and {agent}>>.
+* Learn more about {integrations-docs}[{integrations}].