Regex support of add_resource_metadata (#834)

* adding page for adavnce agent with k8s

* adding kubernetes provider example

* fxing typos and indentation

* fxing typos and indentation

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* udpating code yaml box and adding parameter of manifest

* updating number in list

* updating number in list and iamge for kibana UI

* adding initial doc for metadata enrichment

* adding initial doc for metadata enrichment

* adding initial doc for metadata enrichment

* adding examples

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/processors/processor-add_kubernetes_metadata.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* fixing conflicts

* fixing part of add_resource

* Fix kubernetes provider documentation (#817)

* fix namespace fields; add information about the deployment and cronjob settings in add_resource_metadata

Signed-off-by: Tetiana Kravchenko <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: Andrew Gizas <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: Andrew Gizas <[email protected]>

---------

Signed-off-by: Tetiana Kravchenko <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
Co-authored-by: Andrew Gizas <[email protected]>

* squashing

* fixing newline

---------

Signed-off-by: Tetiana Kravchenko <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
Co-authored-by: Tetiana Kravchenko <[email protected]>

docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

@@ -74,8 +74,8 @@ Configuration parameters:
  * `node` or `namespace`: Specify labels and annotations filters for the extra metadata coming from node and namespace. By default
  all labels are included while annotations are not. To change the default behaviour `include_labels`, `exclude_labels` and `include_annotations`
  can be defined. These settings are useful when storing labels and annotations that require special handling to avoid overloading the storage output.
- Note: wildcards are not supported for those settings.
  The enrichment of `node` or `namespace` metadata can be individually disabled by setting `enabled: false`.
+ Wildcards are supported in these settings by using `use_regex_include: true` in combination with `include_labels`, and respectively by setting `use_regex_exclude: true` in combination with `exclude_labels`.
  * `deployment`: If resource is `pod` and it is created from a `deployment`, by default the deployment name isn't added, this can be enabled by setting `deployment: true`.
  * `cronjob`: If resource is `pod` and it is created from a `cronjob`, by default the cronjob name isn't added, this can be enabled by setting `cronjob: true`.
  Example:
@@ -84,10 +84,16 @@ Configuration parameters:
 -------------------------------------------------------------------------------------
       add_resource_metadata:
         namespace:
+          #use_regex_include: false
           include_labels: ["namespacelabel1"]
+          #use_regex_exclude: false
+          #exclude_labels: ["namespacelabel2"]
         node:
+          #use_regex_include: false
           include_labels: ["nodelabel2"]
           include_annotations: ["nodeannotation1"]
+          #use_regex_exclude: false
+          #exclude_labels: ["nodelabel3"]
         #deployment: false
         #cronjob: false
 -------------------------------------------------------------------------------------

docs/en/ingest-management/processors/processor-add_kubernetes_metadata.asciidoc

@@ -137,7 +137,7 @@ a| Filters and configuration for adding extra metadata to the event. This settin
 By default all labels are included, but annotations are not.
 To change the default behavior, you can set `include_labels`, `exclude_labels`, and `include_annotations`.
 These settings are useful when storing labels and annotations that require special handling to avoid overloading the storage output.
-Note that wildcards are not supported in these settings.
+Wildcards are supported in these settings by using `use_regex_include: true` in combination with `include_labels`, and respectively by setting `use_regex_exclude: true` in combination with `exclude_labels`.
 To turn off enrichment of `node` or `namespace` metadata individually, set `enabled: false`.
 * `deployment`: If the resource is `pod` and it is created from a `deployment`, the deployment name is not added by default. To enable this behavior, set `deployment: true`.
 * `cronjob`: If the resource is `pod` and it is created from a `cronjob`, the cronjob name is not added by default. To enable this behavior, set `cronjob: true`.
@@ -151,11 +151,17 @@ To turn off enrichment of `node` or `namespace` metadata individually, set `enab
       add_resource_metadata:
         namespace:
           include_labels: ["namespacelabel1"]
+          # use_regex_include: false
+          # use_regex_exclude: false
+          # exclude_labels: ["namespacelabel2"]
           #labels.dedot: true
           #annotations.dedot: true
         node:
+          # use_regex_include: false
           include_labels: ["nodelabel2"]
           include_annotations: ["nodeannotation1"]
+          # use_regex_exclude: false
+          # exclude_annotations: ["nodeannotation2"]
           #labels.dedot: true
           #annotations.dedot: true
         deployment: true