Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting started tutorial #923

Merged
merged 29 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
945d582
add missing instruction to enable UI for linux (optional)
moxious May 24, 2024
148795e
make note of hot reloading
moxious May 24, 2024
8df108a
get started tutorial first commit
moxious May 24, 2024
5280e95
fix shortcodes
moxious May 24, 2024
64b9672
fix bad pastes on shortcodes
moxious May 24, 2024
2c2d064
yet more shortcode fixes
moxious May 24, 2024
38d2cb8
shortcodes
moxious May 24, 2024
06188fb
iterate
moxious May 24, 2024
e44a93f
modifications from build inspection
moxious May 24, 2024
b2d364e
today I learned that macos is a programming language but macOS isn't :)
moxious May 24, 2024
16641be
loki deserves an upgrade
moxious May 24, 2024
2665e34
upgrade grafana to 11.0.0 as use of :latest is considered antipattern
moxious May 24, 2024
4e7d92f
Apply suggestions from code review
moxious May 28, 2024
fbfd14e
link at the bottom
moxious May 28, 2024
1339b76
need to use sudo in order to edit that file
moxious Jun 3, 2024
7746125
add docker compose alteration
moxious Jun 3, 2024
fc81207
applying Lisa's PR feedback
moxious Jun 4, 2024
51f20ea
Apply suggestions from code review
moxious Jun 4, 2024
86d2dc7
clayton comments on debuging UI
moxious Jun 5, 2024
f54bc24
clayton comments on docker-compose
moxious Jun 5, 2024
2c4efba
Apply suggestions from code review
moxious Jun 5, 2024
007f18d
Merge branch 'getting-started-tutorial' of https://github.com/grafana…
moxious Jun 5, 2024
7756874
Apply suggestions from code review
moxious Jun 10, 2024
ee4e799
cleanup issues related to the debug UI
moxious Jun 10, 2024
32d5dd3
Update docs/sources/tutorials/get-started.md
moxious Jun 10, 2024
cdf4f67
Apply suggestions from code review
moxious Jun 11, 2024
1f9ce83
apply some rfratto suggestions
moxious Jun 11, 2024
9087460
Merge branch 'main' into getting-started-tutorial
clayton-cornell Jun 12, 2024
a456a70
PR feedback
moxious Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/sources/get-started/install/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,17 @@ sslcacert=/etc/pki/tls/certs/ca-bundle.crt' | sudo tee /etc/yum.repos.d/grafana.
```
{{< /code >}}

1. For beginners, those following tutorials, and debugging: Edit the run configuration to enable the {{< param "PRODUCT_NAME" >}} UI
moxious marked this conversation as resolved.
Show resolved Hide resolved

```bash
sudo sed -i -e 's/CUSTOM_ARGS=""/CUSTOM_ARGS="--server.http.listen-addr=0.0.0.0:12345"/' /etc/default/alloy
```

The {{< param "PRODUCT_NAME" >}} UI is a web application that is available at `http://localhost:12345/` and allows for
[debugging your configuration] and viewing it in a visual format. Enabling this UI is recommended
for beginners and is not necessary for network isolated, headless, or production installs. It also enables hot-reloading
of the {{< param "PRODUCT_NAME" >}} configuration without restarting the system service.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Uninstall

To uninstall {{< param "PRODUCT_NAME" >}} on Linux, run the following commands in a terminal window.
Expand Down Expand Up @@ -127,3 +138,4 @@ To uninstall {{< param "PRODUCT_NAME" >}} on Linux, run the following commands i

[Run]: ../../run/linux/
[Configure]: ../../../tasks/configure/configure-linux/
[debugging your configuration]: ../../tasks/debug/
191 changes: 175 additions & 16 deletions docs/sources/tutorials/get-started.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,49 @@
---
canonical: https://grafana.com/docs/alloy/latest/tutorials/get-started/
description: Getting started with the tutorials
title: Get started
description: Getting started with Alloy
title: Get started with Alloy
weight: 10
---

moxious marked this conversation as resolved.
Show resolved Hide resolved
## Get started with {{% param "PRODUCT_NAME" %}}

This set of tutorials contains a collection of examples that build on each other to demonstrate how to configure and use [{{< param "PRODUCT_NAME" >}}][alloy].
To follow these tutorials, you need to have a basic understanding of what {{< param "PRODUCT_NAME" >}} is and telemetry collection in general.
You should also be familiar with with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation.
You don't need to know about the {{< param "PRODUCT_NAME" >}} [configuration syntax][configuration] concepts.
This tutorial will walk you through configuring Alloy to collect logs from your local machine and report logs to Loki running in the local Grafana Stack. This process will enable you to query and visualize the logs sent to Loki using the Grafana dashboard.

To follow this tutorial, you must have a basic understanding of Alloy and telemetry collection in general. You should also be familiar with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation. You don't need to know about the
{{< param "PRODUCT_NAME" >}} [configuration syntax][configuration] concepts.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

The tutorials require a Linux or Unix environment with Docker installed.
The examples run on a single host so that you can run them on your laptop or in a Virtual Machine.
You are encouraged to try the examples using a `config.alloy` file and experiment with the examples yourself.
This tutorial requires a Linux, Unix, or Mac environment with Docker installed.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Install {{% param "PRODUCT_NAME" %}} and start the service

### Linux

moxious marked this conversation as resolved.
Show resolved Hide resolved
Follow the instructions on the [Linux Install] page for the steps for several popular
Linux distributions.
moxious marked this conversation as resolved.
Show resolved Hide resolved

To run the examples, you should have an {{< param "PRODUCT_NAME" >}} binary available. You can follow the instructions on how to [Install {{< param "PRODUCT_NAME" >}} as a Standalone Binary][install] to get a binary.
{{< admonition type="tip" >}}
Make sure to follow the optional install step to enable the UI, we will be referring to it in this tutorial.
{{< /admonition >}}
moxious marked this conversation as resolved.
Show resolved Hide resolved

Once you have completed this, follow the instructions to [Run on Linux][] using `systemctl`.
moxious marked this conversation as resolved.
Show resolved Hide resolved

### macOS
moxious marked this conversation as resolved.
Show resolved Hide resolved

Follow the instructions on the [macOS Install] page for Homebrew instructions. Once you have
completed this, follow the instructions to [Run on macOS] which will start
{{< param "PRODUCT_NAME" >}} as a Homebrew service.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Set up a local Grafana instance

You can use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus which are pre-configured as datasources. You can run and experiment with the examples on your local system.
To enable Grafana Alloy to write data to Loki running in the local Grafana Stack, we will use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus which are pre-configured as data sources.
moxious marked this conversation as resolved.
Show resolved Hide resolved

```yaml
version: '3'
services:
loki:
image: grafana/loki:2.9.0
image: grafana/loki:3.0.0
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
Expand Down Expand Up @@ -71,17 +86,161 @@ services:
editable: false
EOF
/run.sh
image: grafana/grafana:latest
image: grafana/grafana:11.0.0
ports:
- "3000:3000"
```

After running `docker-compose up`, open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI.
After running `docker-compose up`, open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI. In some newer
docker installs, this command may be run as `docker compose up` without the dash character.
moxious marked this conversation as resolved.
Show resolved Hide resolved
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Configure {{< param "PRODUCT_NAME" >}}
moxious marked this conversation as resolved.
Show resolved Hide resolved

Once the local Grafana instance has been set up, the next step is to give detailed instructions to Alloy on which logs we want to scrape, how we want to process that data, and where we want the data sent. This is done through `config.alloy` file which contains components with instructions for Alloy to execute. Our configuration will connect components into a workflow.
moxious marked this conversation as resolved.
Show resolved Hide resolved

The examples run on a single host so that you can run them on your laptop or in a Virtual Machine. You are encouraged to try the examples using a `config.alloy` file and experiment with the examples yourself.
moxious marked this conversation as resolved.
Show resolved Hide resolved

For the following steps, create a file called `config.alloy` in your current working directory.
{{< param "PRODUCT_NAME" >}} has a feature that allows us to "hot reload" a configuration from a file. In a later
step, we will copy this file to where {{< param "PRODUCT_NAME" >}} will pick it up, and be able to reload without restarting
the system service.
moxious marked this conversation as resolved.
Show resolved Hide resolved

### First Component: Log files
moxious marked this conversation as resolved.
Show resolved Hide resolved

Put this component into the top of the `config.alloy` file:
moxious marked this conversation as resolved.
Show resolved Hide resolved

```alloy
local.file_match "local_files" {
path_targets = [{"__path__" = "/var/log/*.log"}]
sync_period = "5s"
}
```

In {{< param "PRODUCT_NAME" >}}'s configuration language, this creates a [local.file_match] component named `local_files` with an attribute that tells {{< param "PRODUCT_NAME" >}} which files to source, and to check every 5 seconds.
moxious marked this conversation as resolved.
Show resolved Hide resolved

### Second Component: Scraping
moxious marked this conversation as resolved.
Show resolved Hide resolved

Put this component next in the `config.alloy` file:
moxious marked this conversation as resolved.
Show resolved Hide resolved

```alloy
loki.source.file "log_scrape" {
targets = local.file_match.local_files.targets
forward_to = [loki.write.grafana_loki.receiver]
tail_from_end = true
}
```

This configuration creates a [loki.source.file] component named `log_scrape`, and
shows the pipeline concept of {{< param "PRODUCT_NAME" >}} in action:
moxious marked this conversation as resolved.
Show resolved Hide resolved

1. It applies to the `local_files` component (its "source" or target)
2. It forwards the logs it scrapes to the "receiver" of another component called `grafana_loki` that we will define next
3. It provides extra attributes and options, in this case, we will tail log files from the end and not ingest the entire
past history
moxious marked this conversation as resolved.
Show resolved Hide resolved

### Third Component: Write Logs to Loki
moxious marked this conversation as resolved.
Show resolved Hide resolved

Place this component last in your configuration file:
moxious marked this conversation as resolved.
Show resolved Hide resolved

```alloy
loki.write "grafana_loki" {
endpoint {
url = "http://localhost:3100/loki/api/v1/push"

// basic_auth {
// username = "admin"
// password = "admin"
// }
}
}
```

We create a [loki.write] component named `grafana_loki` that points to `http://localhost:3100/loki/api/v1/push`. This completes a simple configuration pipeline.
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< admonition type="tip" >}}
Notice that the `basic_auth` is commented out. Our local `docker-compose` stack does not require it; we include it in this example
to show how you can configure auth for other environments. For further auth options, consult the [loki.write] component reference.
moxious marked this conversation as resolved.
Show resolved Hide resolved
{{< /admonition >}}

This connects directly to the Loki instance running via `docker-compose` from the earlier step.

## Reload the Configuration
moxious marked this conversation as resolved.
Show resolved Hide resolved

Copy your local `config.alloy` file into the default configuration file location, which depends on your OS.

{{< code >}}

```macos
sudo cp config.alloy $(brew --prefix)/etc/alloy/config.alloy
```

```linux
sudo cp config.alloy /etc/alloy/config.alloy
```

{{< /code >}}

Finally, call the reload endpoint to alert {{< param "PRODUCT_NAME" >}} to the configuration change without the need
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
for restarting the system service.

```bash
curl -X POST http://localhost:12345/-/reload
```

{{< admonition type="tip" >}}
If this step does not work for you, please note that in the install instructions, enabling it requires
one extra optional step for Linux, while this is enabled by default on MacOS.
{{< /admonition >}}

The alternative to using this endpoint is to reload the {{< param "PRODUCT_NAME" >}} configuration, which can
be done as follows:
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```macos
brew services restart alloy
```

```linux
sudo systemctl reload alloy
```

{{< /code >}}

## Inspect your Configuration in the {{< param "PRODUCT_NAME" >}} UI
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:12345] and click the Graph tab at the top, which will show
something similar to the following:
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< figure src="/media/docs/alloy/tutorial/healthy-config.png" alt="Logs reported by Alloy in Grafana" >}}

The UI allows us to see a visual representation of the pipeline we are building with our {{< param "PRODUCT_NAME" >}}
component configuration. We can further see that the components are healthy, and we are ready to go.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Log into Grafana and Explore Loki Logs
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:3000/explore] to access Grafana's Explore feature. Select Loki as
the data source, and click the "Label Browser" button to select a file that {{< param "PRODUCT_NAME" >}} as sent to Loki.
moxious marked this conversation as resolved.
Show resolved Hide resolved

Here we can see that logs are flowing through to Loki as expected, and the end-to-end configuration was successful!
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< figure src="/media/docs/alloy/tutorial/loki-logs.png" alt="Logs reported by Alloy in Grafana" >}}

The tutorials are designed to be followed in order and generally build on each other. Each example explains what it does and how it works.
## Conclusion

The Recommended Reading sections in each tutorial provide a list of documentation topics. Read the recommended topics in the order given to help you understand the concepts used in the example.
Congratulations, you have fully installed and configured {{< param "PRODUCT_NAME" >}}, and shipped logs from your local
host to a Grafana stack. In the following tutorials, you will learn more about configuration concepts, metrics, and more
advanced log scraping.
moxious marked this conversation as resolved.
Show resolved Hide resolved

[http://localhost:3000/explore]: http://localhost:3000/explore
[http://localhost:12345]: http://localhost:12345
[MacOS Install]: ../../get-started/install/macos/
[Linux Install]: ../../get-started/install/linux/
[Run on Linux]: ../../get-started/run/linux/
[Run on MacOS]: ../../get-started/run/macos/
[local.file_match]: ../../reference/components/local.file_match/
[loki.write]: ../../reference/components/loki.write/
[loki.source.file]: ../../reference/components/loki.source.file/
[alloy]: https://grafana.com/docs/alloy/latest/
[configuration]: ../../concepts/configuration-syntax/
[install]: ../../get-started/install/binary/#install-alloy-as-a-standalone-binary
Loading