Skip to content

Commit 402d9b6

Browse files
authored
Merge pull request #1701 from fluent/lynettemiles/sc-136163/update-fluent-bit-docs-pipeline-filters-throttle
2 parents 012ea74 + d39ce42 commit 402d9b6

File tree

1 file changed

+21
-23
lines changed

1 file changed

+21
-23
lines changed

pipeline/filters/throttle.md

Lines changed: 21 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
# Throttle
22

3-
The _Throttle Filter_ plugin sets the average _Rate_ of messages per _Interval_, based on leaky bucket and sliding window algorithm. In case of overflood, it will leak within certain rate.
3+
The _Throttle_ filter sets the average `Rate` of messages per `Interval`, based on the leaky bucket and sliding window algorithm. In case of flooding, it will leak at a certain rate.
44

5-
## Configuration Parameters
5+
## Configuration parameters
66

77
The plugin supports the following configuration parameters:
88

99
| Key | Value Format | Description |
1010
| :--- | :--- | :--- |
11-
| Rate | Integer | Amount of messages for the time. |
12-
| Window | Integer | Amount of intervals to calculate average over. Default 5. |
13-
| Interval | String | Time interval, expressed in "sleep" format. e.g 3s, 1.5m, 0.5h etc |
14-
| Print\_Status | Bool | Whether to print status messages with current rate and the limits to information logs |
11+
| `Rate` | `Integer` | Amount of messages for the time. |
12+
| `Window` | `Integer` | Amount of intervals to calculate average over. Default: `5`. |
13+
| `Interval` | `String` | Time interval, expressed in `sleep` format. For example, `3s`, `1.5m`, `0.5h`. |
14+
| `Print_Status` | `Bool` | Whether to print status messages with current rate and the limits to information logs. |
1515

1616
## Functional description
1717

@@ -23,9 +23,7 @@ Window 5
2323
Interval 1s
2424
```
2525

26-
You would received 1 message in the first second, 3 messages second, and 5 third.
27-
Disregard that Window is actually 5, because the configuration uses `slow` start
28-
to prevent flooding during the startup.
26+
You would receive 1 message in the first second, 3 messages second, and 5 third. Disregard that Window is actually 5, because the configuration uses `slow` start to prevent flooding during the startup.
2927

3028
```text
3129
+-------+-+-+-+
@@ -35,7 +33,7 @@ to prevent flooding during the startup.
3533
+-----+
3634
```
3735

38-
But as soon as we reached Window size \* Interval, we will have true sliding window with aggregation over complete window.
36+
But as soon as you reach `Window size * Interval`, you will have true sliding window with aggregation over complete window.
3937

4038
```text
4139
+-------------+
@@ -45,7 +43,7 @@ But as soon as we reached Window size \* Interval, we will have true sliding win
4543
----------+
4644
```
4745

48-
When we have average over window is more than Rate, we will start dropping messages, so that
46+
When the average over window is more than `Rate`, Fluent Bit starts dropping messages, so the following:
4947

5048
```text
5149
+-------------+
@@ -65,11 +63,11 @@ will become:
6563
+---------+
6664
```
6765

68-
As you can see, last pane of the window was overwritten and 1 message was dropped.
66+
The last pane of the window was overwritten and 1 message was dropped.
6967

70-
### Interval vs Window size
68+
### Interval versus Window size
7169

72-
You might noticed possibility to configure _Interval_ of the _Window_ shift. It is counter intuitive, but there is a difference between two examples above:
70+
You might notice it's possible to configure the `Interval` of the `Window` shift. It's counter intuitive, but there is a difference between the two previous examples:
7371

7472
```text
7573
Rate 60
@@ -85,7 +83,7 @@ Window 300
8583
Interval 1s
8684
```
8785

88-
Even though both examples will allow maximum Rate of 60 messages per minute, first example may get all 60 messages within first second, and will drop all the rest for the entire minute:
86+
Even though both examples will allow maximum `Rate` of 60 messages per minute, the first example might get all 60 messages within first second, and will drop all the rest for the entire minute:
8987

9088
```text
9189
XX XX XX
@@ -97,24 +95,24 @@ XX XX XX
9795
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
9896
```
9997

100-
While the second example will not allow more than 1 message per second every second, making output rate more smooth:
98+
While the second example won't allow more than 1 message per second every second, making output rate more smooth:
10199

102100
```text
103101
X X X X X X
104102
XXXX XXXX XXXX XXXX XXXX XXXX
105103
+-+-+-+-+-+--+-+-+-+-+-+-+-+-+-+
106104
```
107105

108-
It may drop some data if the rate is ragged. I would recommend to use bigger interval and rate for streams of rare but important events, while keep _Window_ bigger and _Interval_ small for constantly intensive inputs.
106+
Fluent Bit might drop some data if the rate is ragged. Use bigger intervals and rates for streams of rare but important events, while keeping `Window` bigger and `Interval` smaller for constantly intensive inputs.
109107

110-
### Command Line
108+
### Command line
111109

112-
> Note: It's suggested to use a configuration file.
110+
It's suggested to use a configuration file.
113111

114-
The following command will load the _tail_ plugin and read the content of _lines.txt_ file. Then the _throttle_ filter will apply a rate limit and only _pass_ the records which are read below the certain _rate_:
112+
The following command will load the Tail plugin and read the content of the `lines.txt` file. Then, the Throttle filter will apply a rate limit and only pass the records which are read below the `rate`:
115113

116-
```text
117-
$ bin/fluent-bit -i tail -p 'path=lines.txt' -F throttle -p 'rate=1' -m '*' -o stdout
114+
```shell
115+
bin/fluent-bit -i tail -p 'path=lines.txt' -F throttle -p 'rate=1' -m '*' -o stdout
118116
```
119117

120118
### Configuration File
@@ -136,4 +134,4 @@ $ bin/fluent-bit -i tail -p 'path=lines.txt' -F throttle -p 'rate=1' -m '*' -o s
136134
Match *
137135
```
138136

139-
The example above will pass 1000 messages per second in average over 300 seconds.
137+
This example will pass 1000 messages per second in average over 300 seconds.

0 commit comments

Comments
 (0)