Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting started tutorial #923

Merged
merged 29 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
945d582
add missing instruction to enable UI for linux (optional)
moxious May 24, 2024
148795e
make note of hot reloading
moxious May 24, 2024
8df108a
get started tutorial first commit
moxious May 24, 2024
5280e95
fix shortcodes
moxious May 24, 2024
64b9672
fix bad pastes on shortcodes
moxious May 24, 2024
2c2d064
yet more shortcode fixes
moxious May 24, 2024
38d2cb8
shortcodes
moxious May 24, 2024
06188fb
iterate
moxious May 24, 2024
e44a93f
modifications from build inspection
moxious May 24, 2024
b2d364e
today I learned that macos is a programming language but macOS isn't :)
moxious May 24, 2024
16641be
loki deserves an upgrade
moxious May 24, 2024
2665e34
upgrade grafana to 11.0.0 as use of :latest is considered antipattern
moxious May 24, 2024
4e7d92f
Apply suggestions from code review
moxious May 28, 2024
fbfd14e
link at the bottom
moxious May 28, 2024
1339b76
need to use sudo in order to edit that file
moxious Jun 3, 2024
7746125
add docker compose alteration
moxious Jun 3, 2024
fc81207
applying Lisa's PR feedback
moxious Jun 4, 2024
51f20ea
Apply suggestions from code review
moxious Jun 4, 2024
86d2dc7
clayton comments on debuging UI
moxious Jun 5, 2024
f54bc24
clayton comments on docker-compose
moxious Jun 5, 2024
2c4efba
Apply suggestions from code review
moxious Jun 5, 2024
007f18d
Merge branch 'getting-started-tutorial' of https://github.com/grafana…
moxious Jun 5, 2024
7756874
Apply suggestions from code review
moxious Jun 10, 2024
ee4e799
cleanup issues related to the debug UI
moxious Jun 10, 2024
32d5dd3
Update docs/sources/tutorials/get-started.md
moxious Jun 10, 2024
cdf4f67
Apply suggestions from code review
moxious Jun 11, 2024
1f9ce83
apply some rfratto suggestions
moxious Jun 11, 2024
9087460
Merge branch 'main' into getting-started-tutorial
clayton-cornell Jun 12, 2024
a456a70
PR feedback
moxious Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/sources/get-started/install/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,17 @@ sslcacert=/etc/pki/tls/certs/ca-bundle.crt' | sudo tee /etc/yum.repos.d/grafana.
```
{{< /code >}}

1. For beginners, those following tutorials, and debugging: Edit the run configuration to enable the {{< param "PRODUCT_NAME" >}} UI
moxious marked this conversation as resolved.
Show resolved Hide resolved

```bash
sudo sed -i -e 's/CUSTOM_ARGS=""/CUSTOM_ARGS="--server.http.listen-addr=0.0.0.0:12345"/' /etc/default/alloy
```

The {{< param "PRODUCT_NAME" >}} UI is a web application that is available at `http://localhost:12345/`.
The {{< param "PRODUCT_NAME" >}} UI allows you to [debug your configuration] and view it in a visual format.
It also allows you to hot-reload the {{< param "PRODUCT_NAME" >}} configuration without restarting the system service.
This UI is not necessary for network isolated, headless, or production installs.

## Uninstall

To uninstall {{< param "PRODUCT_NAME" >}} on Linux, run the following commands in a terminal window.
Expand Down Expand Up @@ -127,3 +138,4 @@ To uninstall {{< param "PRODUCT_NAME" >}} on Linux, run the following commands i

[Run]: ../../run/linux/
[Configure]: ../../../tasks/configure/configure-linux/
[debugging your configuration]: ../../tasks/debug/
197 changes: 182 additions & 15 deletions docs/sources/tutorials/get-started.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,54 @@
---
canonical: https://grafana.com/docs/alloy/latest/tutorials/get-started/
description: Getting started with the tutorials
title: Get started
description: Getting started with Alloy
title: Get started with Alloy
weight: 10
---

moxious marked this conversation as resolved.
Show resolved Hide resolved
## Get started with {{% param "PRODUCT_NAME" %}}

This set of tutorials contains a collection of examples that build on each other to demonstrate how to configure and use [{{< param "PRODUCT_NAME" >}}][alloy].
To follow these tutorials, you need to have a basic understanding of what {{< param "PRODUCT_NAME" >}} is and telemetry collection in general.
You should also be familiar with with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation.
This tutorial shows you how to configure Alloy to collect logs from your local machine and send them to Loki, running in a local Grafana stack.
This process will enable you to query and visualize the logs sent to Loki using the Grafana dashboard.

To follow this tutorial, you must have a basic understanding of Alloy and telemetry collection in general.
You should also be familiar with Prometheus and PromQL, Loki and LogQL, and basic Grafana navigation.
You don't need to know about the {{< param "PRODUCT_NAME" >}} [configuration syntax][configuration] concepts.

## Prerequisites

The tutorials require a Linux or Unix environment with Docker installed.
The examples run on a single host so that you can run them on your laptop or in a Virtual Machine.
You are encouraged to try the examples using a `config.alloy` file and experiment with the examples yourself.
This tutorial requires a Linux, Unix, or Mac environment with Docker installed.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Install {{% param "PRODUCT_NAME" %}} and start the service

### Linux

moxious marked this conversation as resolved.
Show resolved Hide resolved
1. [Install {{< param "PRODUCT_NAME" >}}][Linux Install].
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< admonition type="tip" >}}
Make sure you enable the {{< param "PRODUCT_NAME" >}} UI.
{{< /admonition >}}

1. [Run {{< param "PRODUCT_NAME" >}}][Run on Linux].

### macOS
moxious marked this conversation as resolved.
Show resolved Hide resolved

To run the examples, you should have an {{< param "PRODUCT_NAME" >}} binary available. You can follow the instructions on how to [Install {{< param "PRODUCT_NAME" >}} as a Standalone Binary][install] to get a binary.
1. [Install {{< param "PRODUCT_NAME" >}} on macOS][macOS Install].
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< admonition type="tip" >}}
The {{< param "PRODUCT_NAME" >}} UI is enabled by default in {{< param "PRODUCT_NAME" >}} on macOS.
{{< /admonition >}}

1. [Run {{< param "PRODUCT_NAME" >}}][Run on macOS].

## Set up a local Grafana instance

You can use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus which are pre-configured as datasources. You can run and experiment with the examples on your local system.
To allow {{< param "PRODUCT__NAME" >}} to write data to Loki running in the local Grafana stack, you can use the following Docker Compose file to set up a local Grafana instance alongside Loki and Prometheus, which are pre-configured as data sources.
moxious marked this conversation as resolved.
Show resolved Hide resolved

```yaml
version: '3'
services:
loki:
image: grafana/loki:2.9.0
image: grafana/loki:3.0.0
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
Expand Down Expand Up @@ -71,17 +91,164 @@ services:
editable: false
EOF
/run.sh
image: grafana/grafana:latest
image: grafana/grafana:11.0.0
ports:
- "3000:3000"
```

After running `docker-compose up`, open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI.
After running `docker-compose up`, open [http://localhost:3000](http://localhost:3000) in your browser to view the Grafana UI. In some newer
docker installs, this command may be run as `docker compose up` without the dash character.
moxious marked this conversation as resolved.
Show resolved Hide resolved
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Configure {{< param "PRODUCT_NAME" >}}
moxious marked this conversation as resolved.
Show resolved Hide resolved

Once the local Grafana instance is set up, the next step is to configure {{< param "PRODUCT_NAME" >}}.
You use components in the `config.alloy` file to tell {{< param "PRODUCT_NAME" >}} which logs you want to scrape, how you want to process that data, and where you want the data sent.

The examples run on a single host so that you can run them on your laptop or in a Virtual Machine.
You can try the examples using a `config.alloy` file and experiment with the examples yourself.

For the following steps, create a file called `config.alloy` in your current working directory.
If you have enabled the {{< param "PRODUCT_NAME" >}} UI, you can "hot reload" a configuration from a file.
In a later step, you will copy this file to where {{< param "PRODUCT_NAME" >}} will pick it up, and be able to reload without restarting the system service.

### First Component: Log files
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component into the top of the `config.alloy` file:

```alloy
local.file_match "local_files" {
path_targets = [{"__path__" = "/var/log/*.log"}]
sync_period = "5s"
}
```

This component creates a [local.file_match][] component named `local_files` with an attribute that tells {{< param "PRODUCT_NAME" >}} which files to source, and to check every 5 seconds.
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved

### Second Component: Scraping
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component next in the `config.alloy` file:

```alloy
loki.source.file "log_scrape" {
targets = local.file_match.local_files.targets
forward_to = [loki.write.grafana_loki.receiver]
tail_from_end = true
}
```

This configuration creates a [loki.source.file][] component named `log_scrape`, and shows the pipeline concept of {{< param "PRODUCT_NAME" >}} in action. The `log.scrape` component does the following:
moxious marked this conversation as resolved.
Show resolved Hide resolved

1. It applies to the `local_files` component (its "source" or target).
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
1. It forwards the logs it scrapes to the "receiver" of another component called `grafana_loki` that you will define next.
1. It provides extra attributes and options, in this case, you will tail log files from the end and not ingest the entire past history.

### Third Component: Write Logs to Loki
moxious marked this conversation as resolved.
Show resolved Hide resolved

Paste this component last in your configuration file:

```alloy
loki.write "grafana_loki" {
endpoint {
url = "http://localhost:3100/loki/api/v1/push"

// basic_auth {
// username = "admin"
// password = "admin"
// }
}
}
```

This last component creates a [loki.write][] component named `grafana_loki` that points to `http://localhost:3100/loki/api/v1/push`.
This completes the simple configuration pipeline.

{{< admonition type="tip" >}}
The `basic_auth` is commented out because the local `docker-compose` stack doesn't require it.
It is included in this example to show how you can configure authorization for other environments.
For further authorization options, refer to the [loki.write][] component reference.

[loki.write]: ../../reference/components/loki.write/
{{< /admonition >}}

This connects directly to the Loki instance running via `docker-compose` from the earlier step.

## Reload the Configuration
moxious marked this conversation as resolved.
Show resolved Hide resolved

Copy your local `config.alloy` file into the default configuration file location, which depends on your OS.

{{< code >}}

```macos
sudo cp config.alloy $(brew --prefix)/etc/alloy/config.alloy
```

```linux
sudo cp config.alloy /etc/alloy/config.alloy
```

{{< /code >}}

Finally, call the reload endpoint to alert {{< param "PRODUCT_NAME" >}} to the configuration change without the need
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
for restarting the system service.

```bash
curl -X POST http://localhost:12345/-/reload
```

{{< admonition type="tip" >}}
If this step does not work for you, please note that in the install instructions, enabling it requires
one extra optional step for Linux, while this is enabled by default on MacOS.
{{< /admonition >}}

The alternative to using this endpoint is to reload the {{< param "PRODUCT_NAME" >}} configuration, which can
be done as follows:
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved

{{< code >}}

```macos
brew services restart alloy
```

```linux
sudo systemctl reload alloy
```

{{< /code >}}

## Inspect your Configuration in the {{< param "PRODUCT_NAME" >}} UI
clayton-cornell marked this conversation as resolved.
Show resolved Hide resolved
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:12345] and click the Graph tab at the top, which will show
something similar to the following:
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< figure src="/media/docs/alloy/tutorial/healthy-config.png" alt="Logs reported by Alloy in Grafana" >}}

The UI allows us to see a visual representation of the pipeline we are building with our {{< param "PRODUCT_NAME" >}}
component configuration. We can further see that the components are healthy, and we are ready to go.
moxious marked this conversation as resolved.
Show resolved Hide resolved

## Log into Grafana and Explore Loki Logs
moxious marked this conversation as resolved.
Show resolved Hide resolved

Open [http://localhost:3000/explore] to access Grafana's Explore feature. Select Loki as
the data source, and click the "Label Browser" button to select a file that {{< param "PRODUCT_NAME" >}} as sent to Loki.
moxious marked this conversation as resolved.
Show resolved Hide resolved

Here we can see that logs are flowing through to Loki as expected, and the end-to-end configuration was successful!
moxious marked this conversation as resolved.
Show resolved Hide resolved

{{< figure src="/media/docs/alloy/tutorial/loki-logs.png" alt="Logs reported by Alloy in Grafana" >}}

The tutorials are designed to be followed in order and generally build on each other. Each example explains what it does and how it works.
## Conclusion

The Recommended Reading sections in each tutorial provide a list of documentation topics. Read the recommended topics in the order given to help you understand the concepts used in the example.
Congratulations, you have fully installed and configured {{< param "PRODUCT_NAME" >}}, and shipped logs from your local
host to a Grafana stack. In the following tutorials, you will learn more about configuration concepts, metrics, and more
advanced log scraping.
moxious marked this conversation as resolved.
Show resolved Hide resolved

[http://localhost:3000/explore]: http://localhost:3000/explore
[http://localhost:12345]: http://localhost:12345
[MacOS Install]: ../../get-started/install/macos/
[Linux Install]: ../../get-started/install/linux/
[Run on Linux]: ../../get-started/run/linux/
[Run on MacOS]: ../../get-started/run/macos/
[local.file_match]: ../../reference/components/local.file_match/
[loki.write]: ../../reference/components/loki.write/
[loki.source.file]: ../../reference/components/loki.source.file/
[alloy]: https://grafana.com/docs/alloy/latest/
[configuration]: ../../concepts/configuration-syntax/
[install]: ../../get-started/install/binary/#install-alloy-as-a-standalone-binary
Loading