diff --git a/news/bugfix-296.md b/news/bugfix-296.md new file mode 100644 index 000000000..a85755c05 --- /dev/null +++ b/news/bugfix-296.md @@ -0,0 +1,2 @@ +`stats`: Fixed `free_window` counters. + diff --git a/news/feature-326.md b/news/feature-326.md new file mode 100644 index 000000000..10fc54427 --- /dev/null +++ b/news/feature-326.md @@ -0,0 +1,14 @@ +`syslog-ng-ctl`: Added `attach` subcommand. + +With `attach`, it is possible to attach to the +standard IO of the `syslog-ng` proccess. + +Example usage: +``` +# takes the stdio fds for 10 seconds and displays syslog-ng output in that time period +$ syslog-ng-ctl attach stdio --seconds 10 +``` +``` +# steal trace level log messages for 10 seconds +$ syslog-ng-ctl attach logs --seconds 10 --level trace +``` diff --git a/news/feature-351.md b/news/feature-351.md new file mode 100644 index 000000000..588276d67 --- /dev/null +++ b/news/feature-351.md @@ -0,0 +1,4 @@ +socked based sources: Added new flag, called `exit-on-eof` + +Setting this flag to a source makes AxoSyslog stop, +when EOF is received. diff --git a/news/feature-355.md b/news/feature-355.md new file mode 100644 index 000000000..df2ac4003 --- /dev/null +++ b/news/feature-355.md @@ -0,0 +1,4 @@ +socked based sources: Added a new option called `idle-timeout()`. + +Setting this option makes AxoSyslog close the client connection +if no data is received for the set amount of seconds. diff --git a/news/filterx-203.md b/news/filterx-203.md new file mode 100644 index 000000000..24fbccc35 --- /dev/null +++ b/news/filterx-203.md @@ -0,0 +1,19 @@ +`regexp_subst()`: Added various pcre flags. + +* `jit`: + * enables or disables JIT compliling + * default: `true` +* `global`: + * sets whether all found matches should be replaced + * default: `false` +* `utf8`: + * enables or disables UTF-8 validation + * default: `false` +* `ignorecase` + * sets case sensitivity + * default: `false` (case-sensitive) +* `newline` + * configures the behavior of end of line finding + * `false` returns end of line when CR, LF and CRLF characters are found + * `true` makes the matcher process CR, LF, CRLF characters + * default: `false` diff --git a/news/filterx-215.md b/news/filterx-215.md new file mode 100644 index 000000000..cb33010be --- /dev/null +++ b/news/filterx-215.md @@ -0,0 +1,2 @@ +`unset()`: Now accepts any number of variables to unset. + diff --git a/news/filterx-217.md b/news/filterx-217.md new file mode 100644 index 000000000..c14b1b21a --- /dev/null +++ b/news/filterx-217.md @@ -0,0 +1,2 @@ +Added `+` operator. + diff --git a/news/filterx-218.md b/news/filterx-218.md new file mode 100644 index 000000000..3a5b30536 --- /dev/null +++ b/news/filterx-218.md @@ -0,0 +1,2 @@ +`format_csv()`: Fixed delimiter formatting. + diff --git a/news/filterx-219-1.md b/news/filterx-219-1.md new file mode 100644 index 000000000..0f091ac3c --- /dev/null +++ b/news/filterx-219-1.md @@ -0,0 +1,2 @@ +`parse_csv()`: Changed strip whitespace default to `false`. + diff --git a/news/filterx-219-2.md b/news/filterx-219-2.md new file mode 100644 index 000000000..ae3dfc700 --- /dev/null +++ b/news/filterx-219-2.md @@ -0,0 +1,2 @@ +`parse_csv()`: Renamed `strip_whitespaces` argument to `strip_whitespace`. + diff --git a/news/filterx-220.md b/news/filterx-220.md new file mode 100644 index 000000000..4dd4f7a51 --- /dev/null +++ b/news/filterx-220.md @@ -0,0 +1,6 @@ +`update_metric()`: Added a new function similar to `metrics-probe` parser. + +Example usage: +``` +update_metric("filterx_metric", labels={"msg": $MSG, "foo": "foovalue"}, level=1, increment=$INCREMENT); +``` diff --git a/news/filterx-221.md b/news/filterx-221.md new file mode 100644 index 000000000..e8ffdfd81 --- /dev/null +++ b/news/filterx-221.md @@ -0,0 +1,10 @@ +`flatten()`: Added new function to flatten dicts and lists. + +The function modifies the object in-place. +The separator can be set with the `separator` argument, +which is `.` by default. + +Example usage: +``` +flatten(my_dict_or_list, separator="->"); +``` diff --git a/news/filterx-230.md b/news/filterx-230.md new file mode 100644 index 000000000..eb72b29f2 --- /dev/null +++ b/news/filterx-230.md @@ -0,0 +1,2 @@ +`json`: Fixed a crash that occured when doubles were stored and accessed. + diff --git a/news/filterx-238.md b/news/filterx-238.md new file mode 100644 index 000000000..005f27596 --- /dev/null +++ b/news/filterx-238.md @@ -0,0 +1,2 @@ +Added `!~` operator as the negated `=~` operator. + diff --git a/news/filterx-242.md b/news/filterx-242.md new file mode 100644 index 000000000..4cfff482a --- /dev/null +++ b/news/filterx-242.md @@ -0,0 +1,15 @@ +Added new RFC5424 SDATA related functions. + +All of the functions require traditional syslog parsing beforehand. + +* `has_sdata()` + * Returns whether the current log has SDATA information. + * Example: `sdata_avail = has_sdata(;)` +* `is_sdata_from_enterprise()` + * Checks if there is SDATA that corresponds to the given enterprise ID. + * Example: `sdata_from_6876 = is_sdata_from_enterprise("6876");` +* `get_sdata()` + * Returns a 2 level dict of the available SDATAs. + * Example: `sdata = get_sdata();` + * Returns: `{"Originator@6876": {"sub": "Vimsvc.ha-eventmgr", "opID": "esxui-13c6-6b16"}}` + diff --git a/news/filterx-249.md b/news/filterx-249.md new file mode 100644 index 000000000..4eefef2a5 --- /dev/null +++ b/news/filterx-249.md @@ -0,0 +1,2 @@ +`parse_csv()`: Fixed a race condition. + diff --git a/news/filterx-251.md b/news/filterx-251.md new file mode 100644 index 000000000..141c93dc7 --- /dev/null +++ b/news/filterx-251.md @@ -0,0 +1,52 @@ +`parse_xml()`: Added new function to parse XMLs. + +Example usage: +``` +my_structured_data = parse_xml(raw_xml); +``` + +Converting XML to a dict is not standardized. + +Our intention is to create the most compact dict as possible, +which means certain nodes will have different types and +structures based on a number of different qualities of the +input XML element. + +The following points will demonstrate the choices we made in our parser. +In the examples we will use the JSON dict implementation. + +1. Empty XML elements become empty strings. +``` + XML: + JSON: {"foo": ""} +``` + +2. Attributions are stored in `@attr` key-value pairs, + similarly to some other converters (e.g.: python xmltodict). +``` + XML: + JSON: {"foo": {"@bar": "123", "@baz": "bad"}} +``` + +3. If an XML element has both attributes and a value, + we need to store them in a dict, and the value needs a key. + We store the text value under the #text key. +``` + XML: baz + JSON: {"foo": {"@bar": "123", "#text": "baz"}} +``` + +4. An XML element can have both a value and inner elements. + We use the `#text` key here, too. +``` + XML: bar123 + JSON: {"foo": {"#text": "bar", "baz": "123"}} +``` + +5. An XML element can have multiple values separated by inner elements. + In that case we concatenate the values. +``` + XML: barbaz + JSON: {"foo": {"#text": "barbaz", "a": ""}} +``` + diff --git a/news/filterx-255.md b/news/filterx-255.md new file mode 100644 index 000000000..483ed4b7e --- /dev/null +++ b/news/filterx-255.md @@ -0,0 +1,7 @@ +Use `json` and `json_array` as default types for dict and list literals. + +This is now a valid config and creates `json` and `json_array` objects: +``` +my_json_object = {"foo": "bar"}; +my_json_array = ["foo", "bar"]; +``` diff --git a/news/filterx-257.md b/news/filterx-257.md new file mode 100644 index 000000000..02c6eebda --- /dev/null +++ b/news/filterx-257.md @@ -0,0 +1,2 @@ +Fixed race conditions in several functions. + diff --git a/news/filterx-258.md b/news/filterx-258.md new file mode 100644 index 000000000..0792acdb7 --- /dev/null +++ b/news/filterx-258.md @@ -0,0 +1,2 @@ +`json`: Fixed race condition in marshalling. + diff --git a/news/filterx-269.md b/news/filterx-269.md new file mode 100644 index 000000000..231912d65 --- /dev/null +++ b/news/filterx-269.md @@ -0,0 +1,5 @@ +Added new filterx control flow controls. + +* `drop`: Drops the currently processed message and returns success. +* `done`: Stops the processing and returns success. + diff --git a/news/filterx-273.md b/news/filterx-273.md new file mode 100644 index 000000000..e511d1728 --- /dev/null +++ b/news/filterx-273.md @@ -0,0 +1,2 @@ +`json_array`: Fixed failing to return `null` values. + diff --git a/news/filterx-275.md b/news/filterx-275.md new file mode 100644 index 000000000..c07e43e1d --- /dev/null +++ b/news/filterx-275.md @@ -0,0 +1,21 @@ +`unset_empties()`: Added advanced options. + +`unset_empties` removes elements from the given dictionary or list that match +the empties set. If the `recursive` argument is provided, the function will +process nested dictionaries as well. The `replacement` argument allows +replacing target elements with a specified object, and the targets +argument customizes which elements are removed or replaced, overriding +the default empties set. + +* Optional named arguments: + * recursive: Enables recursive processing of nested dictionaries. default: `true` + * ignorecase: Enables case-insensitive matching. default: `true` + * replacement: Specifies an object to replace target elements instead of removing them. + default: nothing (remove) + * targets: A list of elements to identify for removal or replacement, clearing the default empty set. + default: `["", null, [], {}]` + +Example usage: +``` +unset_empties(js1, targets=["foo", "bar", null, "", [], {}], ignorecase=false, replacement="N/A", recursive=false); +``` diff --git a/news/filterx-282.md b/news/filterx-282.md new file mode 100644 index 000000000..815cda5da --- /dev/null +++ b/news/filterx-282.md @@ -0,0 +1,8 @@ +`parse_windows_eventlog_xml()`: Added a new function to parse Windows EventLog XMLs. + +This parser is really similar to `parse_xml()` with +a couple of small differences: + +1. There is a quick schema validation. +2. The `Event`->`EventData` field automatically handles named `Data` elements. + diff --git a/news/filterx-283.md b/news/filterx-283.md new file mode 100644 index 000000000..f2cb6519e --- /dev/null +++ b/news/filterx-283.md @@ -0,0 +1,2 @@ +`datetime`: 0 valued `datetime` objects are now falsy. + diff --git a/news/filterx-284.md b/news/filterx-284.md new file mode 100644 index 000000000..fae9f2a79 --- /dev/null +++ b/news/filterx-284.md @@ -0,0 +1,2 @@ +`datetime`: `datetime` objects can now be cased to `integer` and `double`. + diff --git a/news/filterx-287-1.md b/news/filterx-287-1.md new file mode 100644 index 000000000..e6620e01e --- /dev/null +++ b/news/filterx-287-1.md @@ -0,0 +1,2 @@ +Declared variables now can be set with dict and list literals. + diff --git a/news/filterx-287-2.md b/news/filterx-287-2.md new file mode 100644 index 000000000..d0d64dfe9 --- /dev/null +++ b/news/filterx-287-2.md @@ -0,0 +1,2 @@ +`parse_csv()`: Fixed an invalid read. + diff --git a/news/filterx-297.md b/news/filterx-297.md new file mode 100644 index 000000000..3203db933 --- /dev/null +++ b/news/filterx-297.md @@ -0,0 +1,18 @@ +`startswith()`, `endswith()`, `includes()`: Added string matching functions. + +* First argument is the string that is being matched. +* Second argument is either a single substring or a list of substrings. +* Optionally the `ignorecase` argument can be set to configure case sensitivity + * default: `false` + +Example usage: +``` +startswith(string, prefix, ignorecase=false); +startswith(string, [prefix_1, prefix_2], ignorecase=true); + +endswith(string, suffix, ignorecase=false); +endswith(string, [suffix_1, suffix_2], ignorecase=true); + +includes(string, substring, ignorecase=false); +includes(string, [substring_1, substring_2], ignorecase=true); +``` diff --git a/news/filterx-324.md b/news/filterx-324.md new file mode 100644 index 000000000..6ac994153 --- /dev/null +++ b/news/filterx-324.md @@ -0,0 +1,11 @@ +`parse_cef()`, `parse_leef()`: Added CEF and LEEF parsers. + +* The first argument is the raw message. +* Optionally `pair_separator` and `value_separator` arguments + can be set to override the respective extension parsing behavior. + +Example usage: +``` +my_structured_leef = parse_leef(leef_message); +my_structured_cef = parse_cef(cef_message); +``` diff --git a/news/filterx-333.md b/news/filterx-333.md new file mode 100644 index 000000000..795dcbc35 --- /dev/null +++ b/news/filterx-333.md @@ -0,0 +1,2 @@ +Fixed LogMessage -> FilterX variable synchronization. +