Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting started tutorial #923

Merged
merged 29 commits into from
Jun 12, 2024
Merged
Changes from 25 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
945d582
add missing instruction to enable UI for linux (optional)
moxious May 24, 2024
148795e
make note of hot reloading
moxious May 24, 2024
8df108a
get started tutorial first commit
moxious May 24, 2024
5280e95
fix shortcodes
moxious May 24, 2024
64b9672
fix bad pastes on shortcodes
moxious May 24, 2024
2c2d064
yet more shortcode fixes
moxious May 24, 2024
38d2cb8
shortcodes
moxious May 24, 2024
06188fb
iterate
moxious May 24, 2024
e44a93f
modifications from build inspection
moxious May 24, 2024
b2d364e
today I learned that macos is a programming language but macOS isn't :)
moxious May 24, 2024
16641be
loki deserves an upgrade
moxious May 24, 2024
2665e34
upgrade grafana to 11.0.0 as use of :latest is considered antipattern
moxious May 24, 2024
4e7d92f
Apply suggestions from code review
moxious May 28, 2024
fbfd14e
link at the bottom
moxious May 28, 2024
1339b76
need to use sudo in order to edit that file
moxious Jun 3, 2024
7746125
add docker compose alteration
moxious Jun 3, 2024
fc81207
applying Lisa's PR feedback
moxious Jun 4, 2024
51f20ea
Apply suggestions from code review
moxious Jun 4, 2024
86d2dc7
clayton comments on debuging UI
moxious Jun 5, 2024
f54bc24
clayton comments on docker-compose
moxious Jun 5, 2024
2c4efba
Apply suggestions from code review
moxious Jun 5, 2024
007f18d
Merge branch 'getting-started-tutorial' of https://github.com/grafana…
moxious Jun 5, 2024
7756874
Apply suggestions from code review
moxious Jun 10, 2024
ee4e799
cleanup issues related to the debug UI
moxious Jun 10, 2024
32d5dd3
Update docs/sources/tutorials/get-started.md
moxious Jun 10, 2024
cdf4f67
Apply suggestions from code review
moxious Jun 11, 2024
1f9ce83
apply some rfratto suggestions
moxious Jun 11, 2024
9087460
Merge branch 'main' into getting-started-tutorial
clayton-cornell Jun 12, 2024
a456a70
PR feedback
moxious Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 183 additions & 15 deletions docs/sources/tutorials/get-started.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,50 @@
---
canonical: https://grafana.com/docs/alloy/latest/tutorials/get-started/
description: Getting started with the tutorials
title: Get started
description: Getting started with Alloy
title: Get started with Alloy
weight: 10
---

moxious marked this conversation as resolved.
Show resolved Hide resolved
## Get started with {{% param "PRODUCT_NAME" %}}

This set of tutorials contains a collection of examples that build on each other to demonstrate how to configure and use [{{< param "PRODUCT_NAME" >}}][alloy].
To follow these tutorials, you need to have a basic understanding of what {{< param "PRODUCT_NAME" >}} is and telemetry collection in general.
You should also be familiar with with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation.
This tutorial shows you how to configure Alloy to collect logs from your local machine and send them to Loki, running in a local Grafana stack.
This process will enable you to query and visualize the logs sent to Loki using the Grafana dashboard.

To follow this tutorial, you must have a basic understanding of Alloy and telemetry collection in general.
You should also be familiar with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation.
You don't need to know about the {{< param "PRODUCT_NAME" >}} [configuration syntax][configuration] concepts.

## Prerequisites

The tutorials require a Linux or Unix environment with Docker installed.
The examples run on a single host so that you can run them on your laptop or in a Virtual Machine.
You are encouraged to try the examples using a `config.alloy` file and experiment with the examples yourself.
This tutorial requires a Linux or macOS environment with Docker installed.

## Install {{% param "PRODUCT_NAME" %}} and start the service

### Linux

moxious marked this conversation as resolved.
Show resolved Hide resolved
Install and run {{< param "PRODUCT_NAME" >}} on Linux.

1. [Install {{< param "PRODUCT_NAME" >}}][Linux Install].

1. [Run {{< param "PRODUCT_NAME" >}}][Run on Linux].

### macOS
moxious marked this conversation as resolved.
Show resolved Hide resolved

To run the examples, you should have an {{< param "PRODUCT_NAME" >}} binary available. You can follow the instructions on how to [Install {{< param "PRODUCT_NAME" >}} as a Standalone Binary][install] to get a binary.
Install and run {{< param "PRODUCT_NAME" >}} on macOS.

1. [Install {{< param "PRODUCT_NAME" >}}][macOS Install].

1. [Run {{< param "PRODUCT_NAME" >}}][Run on macOS].

## Set up a local Grafana instance

You can use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus which are pre-configured as datasources. You can run and experiment with the examples on your local system.
To allow {{< param "PRODUCT_NAME" >}} to write data to Loki running in the local Grafana stack, you can use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus, which are pre-configured as data sources.

```yaml
version: '3'
services:
loki:
image: grafana/loki:2.9.0
image: grafana/loki:3.0.0
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
Expand Down Expand Up @@ -71,17 +87,169 @@ services:
editable: false
EOF
/run.sh
image: grafana/grafana:latest
image: grafana/grafana:11.0.0
ports:
- "3000:3000"
```

After running `docker-compose up`, open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI.
Run `docker compose up` to start your Docker container and open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI.

{{< admonition type="note" >}}
If you the following error when you start your Docker container, `docker: 'compose' is not a docker command`, use the command `docker-compose up` to start your Docker container.
{{< /admonition >}}

## Configure {{< param "PRODUCT_NAME" >}}
moxious marked this conversation as resolved.
Show resolved Hide resolved

Once the local Grafana instance is set up, the next step is to configure {{< param "PRODUCT_NAME" >}}.
You use components in the `config.alloy` file to tell {{< param "PRODUCT_NAME" >}} which logs you want to scrape, how you want to process that data, and where you want the data sent.

The examples run on a single host so that you can run them on your laptop or in a Virtual Machine.
You can try the examples using a `config.alloy` file and experiment with the examples yourself.

For the following steps, create a file called `config.alloy` in your current working directory.
If you have enabled the {{< param "PRODUCT_NAME" >}} UI, you can "hot reload" a configuration from a file.
In a later step, you will copy this file to where {{< param "PRODUCT_NAME" >}} will pick it up, and be able to reload without restarting the system service.

### First Component: Log files
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component into the top of the `config.alloy` file:

```alloy
local.file_match "local_files" {
path_targets = [{"__path__" = "/var/log/*.log"}]
sync_period = "5s"
}
```

This component creates a [local.file_match][] component named `local_files` with an attribute that tells {{< param "PRODUCT_NAME" >}} which files to source, and to check every 5 seconds.
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved

### Second Component: Scraping
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component next in the `config.alloy` file:

```alloy
loki.source.file "log_scrape" {
targets = local.file_match.local_files.targets
forward_to = [loki.write.grafana_loki.receiver]
tail_from_end = true
}
```

This configuration creates a [loki.source.file][] component named `log_scrape`, and shows the pipeline concept of {{< param "PRODUCT_NAME" >}} in action. The `log.scrape` component does the following:
moxious marked this conversation as resolved.
Show resolved Hide resolved

1. It applies to the `local_files` component (its "source" or target).
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
1. It forwards the logs it scrapes to the "receiver" of another component called `grafana_loki` that you will define next.
1. It provides extra attributes and options, in this case, you will tail log files from the end and not ingest the entire past history.

### Third Component: Write Logs to Loki
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component last in your configuration file:

```alloy
loki.write "grafana_loki" {
endpoint {
url = "http://localhost:3100/loki/api/v1/push"

// basic_auth {
// username = "admin"
// password = "admin"
// }
}
}
```

This last component creates a [loki.write][] component named `grafana_loki` that points to `http://localhost:3100/loki/api/v1/push`.
This completes the simple configuration pipeline.

{{< admonition type="tip" >}}
The `basic_auth` is commented out because the local `docker compose` stack doesn't require it.
It is included in this example to show how you can configure authorization for other environments.
For further authorization options, refer to the [loki.write][] component reference.

{{< /admonition >}}

This connects directly to the Loki instance running in the Docker container.

## Reload the Configuration
moxious marked this conversation as resolved.
Show resolved Hide resolved

Copy your local `config.alloy` file into the default configuration file location, which depends on your OS.

{{< code >}}

```macos
sudo cp config.alloy $(brew --prefix)/etc/alloy/config.alloy
```

```linux
sudo cp config.alloy /etc/alloy/config.alloy
```

{{< /code >}}

Finally, call the reload endpoint to alert {{< param "PRODUCT_NAME" >}} to the configuration change without the need
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
for restarting the system service.

```bash
curl -X POST http://localhost:12345/-/reload
```

{{< admonition type="tip" >}}
This step uses the Alloy UI, which is exposed on `localhost` port `12345`.
If you chose to run Alloy in a Docker container, make sure you use the `--server.http.listen-addr=0.0.0.0:12345` argument.
If you don’t use this argument, the [debugging UI][debug] won’t be available outside of the Docker container.

[debug]: ../../tasks/debug/#alloy-ui
{{< /admonition >}}

The alternative to using this endpoint is to reload the {{< param "PRODUCT_NAME" >}} configuration, which can
be done as follows:
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```macos
brew services restart alloy
```

```linux
sudo systemctl reload alloy
```

{{< /code >}}

## Inspect your Configuration in the {{< param "PRODUCT_NAME" >}} UI
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:12345] and click the Graph tab at the top.
The graph should look similar to the following:

{{< figure src="/media/docs/alloy/tutorial/healthy-config.png" alt="Logs reported by Alloy in Grafana" >}}

The UI allows you to see a visual representation of the pipeline you built with your {{< param "PRODUCT_NAME" >}} component configuration.
We can see that the components are healthy, and you are ready to go.

## Log into Grafana and Explore Loki Logs
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:3000/explore] to access Explore feature in Grafana.
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
Select Loki as the data source and click the "Label Browser" button to select a file that {{< param "PRODUCT_NAME" >}} has sent to Loki.

Here you can see that logs are flowing through to Loki as expected, and the end-to-end configuration was successful!

{{< figure src="/media/docs/alloy/tutorial/loki-logs.png" alt="Logs reported by Alloy in Grafana" >}}

The tutorials are designed to be followed in order and generally build on each other. Each example explains what it does and how it works.
## Conclusion

The Recommended Reading sections in each tutorial provide a list of documentation topics. Read the recommended topics in the order given to help you understand the concepts used in the example.
Congratulations, you have fully installed and configured {{< param "PRODUCT_NAME" >}}, and sent logs from your local host to a Grafana stack.
In the following tutorials, you will learn more about configuration concepts, metrics, and more advanced log scraping.

[http://localhost:3000/explore]: http://localhost:3000/explore
[http://localhost:12345]: http://localhost:12345
[MacOS Install]: ../../get-started/install/macos/
[Linux Install]: ../../get-started/install/linux/
[Run on Linux]: ../../get-started/run/linux/
[Run on MacOS]: ../../get-started/run/macos/
[local.file_match]: ../../reference/components/local.file_match/
[loki.write]: ../../reference/components/loki.write/
[loki.source.file]: ../../reference/components/loki.source.file/
[alloy]: https://grafana.com/docs/alloy/latest/
[configuration]: ../../concepts/configuration-syntax/
[install]: ../../get-started/install/binary/#install-alloy-as-a-standalone-binary
[debugging your configuration]: ../../tasks/debug/
Loading