You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/filters/throttle.md
+21-23Lines changed: 21 additions & 23 deletions
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,17 @@
1
1
# Throttle
2
2
3
-
The _Throttle Filter_ plugin sets the average _Rate_ of messages per _Interval_, based on leaky bucket and sliding window algorithm. In case of overflood, it will leak within certain rate.
3
+
The _Throttle_ filter sets the average `Rate` of messages per `Interval`, based on the leaky bucket and sliding window algorithm. In case of flooding, it will leak at a certain rate.
4
4
5
-
## Configuration Parameters
5
+
## Configuration parameters
6
6
7
7
The plugin supports the following configuration parameters:
8
8
9
9
| Key | Value Format | Description |
10
10
| :--- | :--- | :--- |
11
-
| Rate | Integer | Amount of messages for the time. |
12
-
| Window | Integer | Amount of intervals to calculate average over. Default 5. |
13
-
| Interval | String | Time interval, expressed in "sleep" format. e.g 3s, 1.5m, 0.5h etc|
14
-
|Print\_Status| Bool | Whether to print status messages with current rate and the limits to information logs |
11
+
|`Rate`|`Integer`| Amount of messages for the time. |
12
+
|`Window`|`Integer`| Amount of intervals to calculate average over. Default: `5`. |
13
+
|`Interval`|`String`| Time interval, expressed in `sleep` format. For example, `3s`, `1.5m`, `0.5h`.|
14
+
|`Print_Status`|`Bool`| Whether to print status messages with current rate and the limits to information logs.|
15
15
16
16
## Functional description
17
17
@@ -23,9 +23,7 @@ Window 5
23
23
Interval 1s
24
24
```
25
25
26
-
You would received 1 message in the first second, 3 messages second, and 5 third.
27
-
Disregard that Window is actually 5, because the configuration uses `slow` start
28
-
to prevent flooding during the startup.
26
+
You would receive 1 message in the first second, 3 messages second, and 5 third. Disregard that Window is actually 5, because the configuration uses `slow` start to prevent flooding during the startup.
29
27
30
28
```text
31
29
+-------+-+-+-+
@@ -35,7 +33,7 @@ to prevent flooding during the startup.
35
33
+-----+
36
34
```
37
35
38
-
But as soon as we reached Window size \* Interval, we will have true sliding window with aggregation over complete window.
36
+
But as soon as you reach `Window size * Interval`, you will have true sliding window with aggregation over complete window.
39
37
40
38
```text
41
39
+-------------+
@@ -45,7 +43,7 @@ But as soon as we reached Window size \* Interval, we will have true sliding win
45
43
----------+
46
44
```
47
45
48
-
When we have average over window is more than Rate, we will start dropping messages, so that
46
+
When the average over window is more than `Rate`, Fluent Bit starts dropping messages, so the following:
49
47
50
48
```text
51
49
+-------------+
@@ -65,11 +63,11 @@ will become:
65
63
+---------+
66
64
```
67
65
68
-
As you can see, last pane of the window was overwritten and 1 message was dropped.
66
+
The last pane of the window was overwritten and 1 message was dropped.
69
67
70
-
### Interval vs Window size
68
+
### Interval versus Window size
71
69
72
-
You might noticed possibility to configure _Interval_of the _Window_ shift. It is counter intuitive, but there is a difference between two examples above:
70
+
You might notice it's possible to configure the `Interval`of the `Window` shift. It's counter intuitive, but there is a difference between the two previous examples:
73
71
74
72
```text
75
73
Rate 60
@@ -85,7 +83,7 @@ Window 300
85
83
Interval 1s
86
84
```
87
85
88
-
Even though both examples will allow maximum Rate of 60 messages per minute, first example may get all 60 messages within first second, and will drop all the rest for the entire minute:
86
+
Even though both examples will allow maximum `Rate` of 60 messages per minute, the first example might get all 60 messages within first second, and will drop all the rest for the entire minute:
89
87
90
88
```text
91
89
XX XX XX
@@ -97,24 +95,24 @@ XX XX XX
97
95
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
98
96
```
99
97
100
-
While the second example will not allow more than 1 message per second every second, making output rate more smooth:
98
+
While the second example won't allow more than 1 message per second every second, making output rate more smooth:
101
99
102
100
```text
103
101
X X X X X X
104
102
XXXX XXXX XXXX XXXX XXXX XXXX
105
103
+-+-+-+-+-+--+-+-+-+-+-+-+-+-+-+
106
104
```
107
105
108
-
It may drop some data if the rate is ragged. I would recommend to use bigger interval and rate for streams of rare but important events, while keep _Window_ bigger and _Interval_ small for constantly intensive inputs.
106
+
Fluent Bit might drop some data if the rate is ragged. Use bigger intervals and rates for streams of rare but important events, while keeping `Window` bigger and `Interval` smaller for constantly intensive inputs.
109
107
110
-
### Command Line
108
+
### Command line
111
109
112
-
> Note: It's suggested to use a configuration file.
110
+
It's suggested to use a configuration file.
113
111
114
-
The following command will load the _tail_ plugin and read the content of _lines.txt_ file. Then the _throttle_ filter will apply a rate limit and only _pass_ the records which are read below the certain _rate_:
112
+
The following command will load the Tail plugin and read the content of the `lines.txt` file. Then, the Throttle filter will apply a rate limit and only pass the records which are read below the `rate`:
0 commit comments