diff --git a/raw-migrated-files/docs-content/serverless/security-advanced-behavioral-detections.md b/raw-migrated-files/docs-content/serverless/security-advanced-behavioral-detections.md deleted file mode 100644 index 04879ba5c..000000000 --- a/raw-migrated-files/docs-content/serverless/security-advanced-behavioral-detections.md +++ /dev/null @@ -1,14 +0,0 @@ -# Advanced behavioral detections [security-advanced-behavioral-detections] - -Elastic’s {{ml}} capabilities and advanced correlation, scoring, and visualization techniques can help you identify potential behavioral threats that may be associated with security incidents. - -Advanced behavioral detections includes two key capabilities: - -* [Anomaly detection](../../../solutions/security/advanced-entity-analytics/anomaly-detection.md) -* [Behavioral detection use cases](../../../solutions/security/advanced-entity-analytics/behavioral-detection-use-cases.md) - - - - - - diff --git a/raw-migrated-files/docs-content/serverless/security-tuning-anomaly-results.md b/raw-migrated-files/docs-content/serverless/security-tuning-anomaly-results.md deleted file mode 100644 index 5bf1d924b..000000000 --- a/raw-migrated-files/docs-content/serverless/security-tuning-anomaly-results.md +++ /dev/null @@ -1,144 +0,0 @@ -# Optimizing anomaly results [security-tuning-anomaly-results] - -To gain clearer insights into real threats, you can tune the anomaly results. The following procedures help to reduce the number of false positives: - -* [Tune results for rare applications and processes](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#rarely-used-processes) -* [Define an anomaly threshold for a job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#define-rule-threshold) - - -## Filter out anomalies from rarely used applications and processes [rarely-used-processes] - -When anomalies include results from a known process that only runs occasionally, you can filter out the unwanted results. - -For example, to filter out results from a housekeeping process, named `maintenanceservice.exe`, that only executes occasionally you need to: - -1. [Create a filter list](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#create-fiter-list) -2. [Add the filter to the relevant job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#add-job-filter) -3. [Clone and rerun the job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#clone-job) (optional) - - -### Create a filter list [create-fiter-list] - -1. Go to **Machine learning** → **Anomaly Detection** → **Settings**. -2. Click **Filter Lists** and then **Create**. - - The **Create new filter list** pane is displayed. - -3. Enter a filter list ID. -4. Enter a description for the filter list (optional). -5. Click **Add item**. -6. In the **Items** textbox, enter the name of the process for which you want to filter out anomaly results (`maintenanceservice.exe` in our example). - - :::{image} ../../../images/serverless--detections-machine-learning-filter-add-item.png - :alt: detections machine learning filter add item - :class: screenshot - ::: - -7. Click **Add** and then **Save**. - - The new filter appears in the Filter List and can be added to relevant jobs. - - - -### Add the filter to the relevant job [add-job-filter] - -1. Go to **Machine learning** → **Anomaly Detection** → **Anomaly Explorer**. -2. Navigate to the job results for which the filter is required. If the job results are not listed, click **Edit job selection** and select the relevant job. -3. In the **actions** column, click the gear icon and then select *Configure rules*. - - The **Create Rule** window is displayed. - - :::{image} ../../../images/serverless--detections-machine-learning-rule-scope.png - :alt: detections machine learning rule scope - :class: screenshot - ::: - -4. Select: - - 1. *Add a filter list to limit where the rule applies*. - 2. The *WHEN* statement for the relevant detector (`process.name` in our example). - 3. The *IS IN* statement. - 4. The filter you created as part of the [Create a filter list](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#create-fiter-list) procedure. - - ::::{tip} - For more information, see [Customizing detectors with custom rules](../../../explore-analyze/machine-learning/anomaly-detection/ml-configuring-detector-custom-rules.md). - - :::: - -5. Click **Save**. - -::::{note} -Changes to rules only affect new results. All anomalies found by the job before the filter was added are still displayed. - -:::: - - - -### Clone and rerun the job [clone-job] - -If you want to remove all the previously detected results for the process, you must clone and run the cloned job. - -::::{important} -Running the cloned job can take some time. Only run the job after you have completed all job rule changes. - -:::: - - -1. Go to **Machine learning** → **Anomaly Detection** → **Jobs**. -2. Navigate to the job for which you configured the rule. -3. Optionally, expand the job row and click **JSON** to verify the configured filter appears under `custom rules` in the JSON code. -4. In the **actions** column, click the options menu (![Options menu](../../../images/serverless-boxesHorizontal.svg "")) and select **Clone job**. - - The **Configure datafeed** page is displayed. - -5. Click **Data Preview** and check the data is displayed without errors. -6. Click **Next** until the **Job details** page is displayed. -7. Enter a Job ID for the cloned job that indicates it is an iteration of the original one. For example, append a number or a username to the original job name, such as `windows-rare-network-process-2`. - - :::{image} ../../../images/serverless--detections-machine-learning-cloned-job-details.png - :alt: detections machine learning cloned job details - :class: screenshot - ::: - -8. Click **Next** and check the job validates without errors. You can ignore warnings about multiple influencers. -9. Click **Next** and then **Create job**. - - The **Start ** window is displayed. - - :::{image} ../../../images/serverless--detections-machine-learning-start-job-window.png - :alt: detections machine learning start job window - :class: screenshot - ::: - -10. Select the point of time from which the job will analyze anomalies. -11. Click **Start**. - - After a while, results will start to appear on the **Anomaly Explorer** page. - - - -## Define an anomaly threshold for a job [define-rule-threshold] - -Certain jobs use a high-count function to look for unusual spikes in process events. For some processes, a burst of activity is a normal, such as automation and housekeeping jobs running on server fleets. However, sometimes a high-delta event count is unlikely to be the result of routine behavior. In these cases, you can define a minimum threshold for when a high-event count is considered an anomaly. - -Depending on your anomaly detection results, you may want to set a minimum event count threshold for the `packetbeat_dns_tunneling` job: - -1. Go to **Machine learning** → **Anomaly Detection** → **Anomaly Explorer**. -2. Navigate to the job results for the `packetbeat_dns_tunneling` job. If the job results are not listed, click **Edit job selection** and select `packetbeat_dns_tunneling`. -3. In the **actions** column, click the gear icon and then select **Configure rules**. - - The **Create Rule** window is displayed. - - :::{image} ../../../images/serverless--detections-machine-learning-ml-rule-threshold.png - :alt: detections machine learning ml rule threshold - :class: screenshot - ::: - -4. Select **Add numeric conditions for when the rule applies** and the following `when` statement: - - *WHEN actual IS GREATER THAN * - - Where `` is the threshold above which anomalies are detected. - -5. Click **Save**. -6. To apply the new threshold, rerun the job (**Job Management** → **Actions** → **Start datafeed**). diff --git a/raw-migrated-files/security-docs/security/advanced-behavioral-detections.md b/raw-migrated-files/security-docs/security/advanced-behavioral-detections.md deleted file mode 100644 index 557775d4e..000000000 --- a/raw-migrated-files/security-docs/security/advanced-behavioral-detections.md +++ /dev/null @@ -1,14 +0,0 @@ -# Advanced behavioral detections [advanced-behavioral-detections] - -Elastic’s {{ml}} capabilities and advanced correlation, scoring, and visualization techniques can help you identify potential behavioral threats that may be associated with security incidents. - -Advanced behavioral detections includes two key capabilities: - -* [Anomaly detection](../../../solutions/security/advanced-entity-analytics/anomaly-detection.md) -* [Behavioral detection use cases](../../../solutions/security/advanced-entity-analytics/behavioral-detection-use-cases.md) - - - - - - diff --git a/raw-migrated-files/security-docs/security/tuning-anomaly-results.md b/raw-migrated-files/security-docs/security/tuning-anomaly-results.md deleted file mode 100644 index b621de0b6..000000000 --- a/raw-migrated-files/security-docs/security/tuning-anomaly-results.md +++ /dev/null @@ -1,145 +0,0 @@ -# Optimizing anomaly results [tuning-anomaly-results] - -To gain clearer insights into real threats, you can tune the anomaly results. The following procedures help to reduce the number of false positives: - -* [Tune results for rare applications and processes](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#rarely-used-processes) -* [Define an anomaly threshold for a job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#define-rule-threshold) - - -## Filter out anomalies from rarely used applications and processes [rarely-used-processes] - -When anomalies include results from a known process that only runs occasionally, you can filter out the unwanted results. - -For example, to filter out results from a housekeeping process, named `maintenanceservice.exe`, that only executes occasionally you need to: - -1. [Create a filter list](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#create-fiter-list) -2. [Add the filter to the relevant job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#add-job-filter) -3. [Clone and rerun the job](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#clone-job) (optional) - - -### Create a filter list [create-fiter-list] - -1. Find **Machine Learning** in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). -2. Under **Anomaly Detection**, select **Settings**. -3. Click **Filter Lists** and then **New**. - - The **Create new filter list** pane is displayed. - -4. Enter a filter list ID. -5. Enter a description for the filter list (optional). -6. Click **Add item**. -7. In the **Items** textbox, enter the name of the process for which you want to filter out anomaly results (`maintenanceservice.exe` in our example). - - :::{image} ../../../images/security-filter-add-item.png - :alt: filter add item - :class: screenshot - ::: - -8. Click **Add** and then **Save**. - - The new filter appears in the Filter List and can be added to relevant jobs. - - - -### Add the filter to the relevant job [add-job-filter] - -1. Find **Machine Learning** in the navigation menu. -2. Under **Anomaly Detection**, select **Anomaly Explorer**. -3. Navigate to the job results for which the filter is required. If the job results are not listed, click **Edit job selection** and select the relevant job. -4. In the **actions** column, click the gear icon and then select *Configure rules*. - - The **Create Rule** window is displayed. - - :::{image} ../../../images/security-rule-scope.png - :alt: rule scope - :class: screenshot - ::: - -5. Select: - - 1. *Add a filter list to limit where the rule applies*. - 2. The *WHEN* statement for the relevant detector (`process.name` in our example). - 3. The *IS IN* statement. - 4. The filter you created as part of the [Create a filter list](../../../solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md#create-fiter-list) procedure. - - ::::{tip} - For more information, see [Customizing detectors with custom rules](../../../explore-analyze/machine-learning/anomaly-detection/ml-configuring-detector-custom-rules.md). - :::: - -6. Click **Save**. - -::::{note} -Changes to rules only affect new results. All anomalies found by the job before the filter was added are still displayed. -:::: - - - -### Clone and rerun the job [clone-job] - -If you want to remove all the previously detected results for the process, you must clone and run the cloned job. - -::::{important} -Running the cloned job can take some time. Only run the job after you have completed all job rule changes. -:::: - - -1. Find **Machine Learning** in the navigation menu. -2. Under **Anomaly Detection**, select **Jobs**. -3. Navigate to the job for which you configured the rule. -4. Optionally, expand the job row and click **JSON** to verify the configured filter appears under `custom rules` in the JSON code. -5. In the **actions** column, click the more (three dots) icon and select *Clone job*. - - The **Configure datafeed** page is displayed. - -6. Click **Data Preview** and check the data is displayed without errors. -7. Click **Next** until the **Job details** page is displayed. -8. Enter a Job ID for the cloned job that indicates it is an iteration of the original one. For example, append a number or a username to the original job name, such as `windows-rare-network-process-2`. - - :::{image} ../../../images/security-cloned-job-details.png - :alt: cloned job details - :class: screenshot - ::: - -9. Click **Next** and check the job validates without errors. You can ignore warnings about multiple influencers. -10. Click **Next** and then **Create job**. - - The **Start ** window is displayed. - - :::{image} ../../../images/security-start-job-window.png - :alt: start job window - :class: screenshot - ::: - -11. Select the point of time from which the job will analyze anomalies. -12. Click **Start**. - - After a while, results will start to appear on the **Anomaly Explorer** page. - - - -## Define an anomaly threshold for a job [define-rule-threshold] - -Certain jobs use a high-count function to look for unusual spikes in process events. For some processes, a burst of activity is a normal, such as automation and housekeeping jobs running on server fleets. However, sometimes a high-delta event count is unlikely to be the result of routine behavior. In these cases, you can define a minimum threshold for when a high-event count is considered an anomaly. - -Depending on your anomaly detection results, you may want to set a minimum event count threshold for the `packetbeat_dns_tunneling` job: - -1. Find **Machine Learning** in the navigation menu. -2. Under **Anomaly Detection**, select **Anomaly Explorer**. -3. Navigate to the job results for the `packetbeat_dns_tunneling` job. If the job results are not listed, click **Edit job selection** and select `packetbeat_dns_tunneling`. -4. In the **actions** column, click the gear icon and then select *Configure rules*. - - The **Create Rule** window is displayed. - - :::{image} ../../../images/security-ml-rule-threshold.png - :alt: ml rule threshold - :class: screenshot - ::: - -5. Select *Add numeric conditions for when the rule applies* and the following `when` statement: - - *WHEN actual IS GREATER THAN * - - Where `` is the threshold above which anomalies are detected. - -6. Click **Save**. -7. To apply the new threshold, rerun the job by selecting **Actions** → **Start datafeed** on the **Anomaly Detection Jobs** page. diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index aca513f58..44c9dd8bf 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -400,7 +400,6 @@ toc: - file: docs-content/serverless/security-about-rules.md - file: docs-content/serverless/security-add-exceptions.md - file: docs-content/serverless/security-add-manage-notes.md - - file: docs-content/serverless/security-advanced-behavioral-detections.md - file: docs-content/serverless/security-advanced-settings.md - file: docs-content/serverless/security-agent-tamper-protection.md - file: docs-content/serverless/security-ai-assistant-esql-queries.md @@ -516,7 +515,6 @@ toc: - file: docs-content/serverless/security-triage-alerts-with-elastic-ai-assistant.md - file: docs-content/serverless/security-trusted-applications.md - file: docs-content/serverless/security-tune-detection-signals.md - - file: docs-content/serverless/security-tuning-anomaly-results.md - file: docs-content/serverless/security-turn-on-risk-engine.md - file: docs-content/serverless/security-ui.md - file: docs-content/serverless/security-uninstall-agent.md @@ -785,7 +783,6 @@ toc: - file: security-docs/security/add-exceptions.md - file: security-docs/security/add-manage-notes.md - file: security-docs/security/admin-page-ov.md - - file: security-docs/security/advanced-behavioral-detections.md - file: security-docs/security/advanced-settings.md - file: security-docs/security/agent-tamper-protection.md - file: security-docs/security/ai-assistant-knowledge-base.md @@ -908,7 +905,6 @@ toc: - file: security-docs/security/timeline-templates-ui.md - file: security-docs/security/timelines-ui.md - file: security-docs/security/trusted-apps-ov.md - - file: security-docs/security/tuning-anomaly-results.md - file: security-docs/security/tuning-detection-signals.md - file: security-docs/security/turn-on-risk-engine.md - file: security-docs/security/uninstall-agent.md diff --git a/solutions/security/advanced-entity-analytics/advanced-behavioral-detections.md b/solutions/security/advanced-entity-analytics/advanced-behavioral-detections.md index 1a0bde8e2..3a4beefe8 100644 --- a/solutions/security/advanced-entity-analytics/advanced-behavioral-detections.md +++ b/solutions/security/advanced-entity-analytics/advanced-behavioral-detections.md @@ -4,11 +4,12 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/security-advanced-behavioral-detections.html --- -# Advanced behavioral detections +# Advanced behavioral detections [advanced-behavioral-detections] -% What needs to be done: Lift-and-shift +Elastic’s {{ml}} capabilities and advanced correlation, scoring, and visualization techniques can help you identify potential behavioral threats that may be associated with security incidents. -% Use migrated content from existing pages that map to this page: +Advanced behavioral detections includes two key capabilities: + +* [Anomaly detection](anomaly-detection.md) +* [Behavioral detection use cases](behavioral-detection-use-cases.md) -% - [ ] ./raw-migrated-files/security-docs/security/advanced-behavioral-detections.md -% - [ ] ./raw-migrated-files/docs-content/serverless/security-advanced-behavioral-detections.md \ No newline at end of file diff --git a/solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md b/solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md index 48985ba83..8adbc0013 100644 --- a/solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md +++ b/solutions/security/advanced-entity-analytics/optimizing-anomaly-results.md @@ -4,23 +4,148 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/security-tuning-anomaly-results.html --- -# Optimizing anomaly results +# Optimizing anomaly results [tuning-anomaly-results] -% What needs to be done: Lift-and-shift +To gain clearer insights into real threats, you can tune the anomaly results. The following procedures help to reduce the number of false positives: -% Use migrated content from existing pages that map to this page: +* [Tune results for rare applications and processes](#rarely-used-processes) +* [Define an anomaly threshold for a job](#define-rule-threshold) -% - [ ] ./raw-migrated-files/security-docs/security/tuning-anomaly-results.md -% - [ ] ./raw-migrated-files/docs-content/serverless/security-tuning-anomaly-results.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +## Filter out anomalies from rarely used applications and processes [rarely-used-processes] -$$$add-job-filter$$$ +When anomalies include results from a known process that only runs occasionally, you can filter out the unwanted results. -$$$clone-job$$$ +For example, to filter out results from a housekeeping process, named `maintenanceservice.exe`, that only executes occasionally you need to: -$$$create-fiter-list$$$ +1. [Create a filter list](#create-fiter-list) +2. [Add the filter to the relevant job](#add-job-filter) +3. [Clone and rerun the job](#clone-job) (optional) -$$$define-rule-threshold$$$ -$$$rarely-used-processes$$$ \ No newline at end of file +### Create a filter list [create-fiter-list] + +1. Find **Machine Learning** in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). +2. Under **Anomaly Detection**, select **Settings**. +3. Click **Filter Lists** and then **New**. + + The **Create new filter list** pane is displayed. + +4. Enter a filter list ID. +5. Enter a description for the filter list (optional). +6. Click **Add item**. +7. In the **Items** textbox, enter the name of the process for which you want to filter out anomaly results (`maintenanceservice.exe` in our example). + + :::{image} ../../../images/security-filter-add-item.png + :alt: filter add item + :class: screenshot + ::: + +8. Click **Add** and then **Save**. + + The new filter appears in the Filter List and can be added to relevant jobs. + + + +### Add the filter to the relevant job [add-job-filter] + +1. Find **Machine Learning** in the navigation menu. +2. Under **Anomaly Detection**, select **Anomaly Explorer**. +3. Navigate to the job results for which the filter is required. If the job results are not listed, click **Edit job selection** and select the relevant job. +4. In the **actions** column, click the gear icon and then select *Configure rules*. + + The **Create Rule** window is displayed. + + :::{image} ../../../images/security-rule-scope.png + :alt: rule scope + :class: screenshot + ::: + +5. Select: + + 1. *Add a filter list to limit where the rule applies*. + 2. The *WHEN* statement for the relevant detector (`process.name` in our example). + 3. The *IS IN* statement. + 4. The filter you created as part of the [Create a filter list](#create-fiter-list) procedure. + + ::::{tip} + For more information, see [Customizing detectors with custom rules](../../../explore-analyze/machine-learning/anomaly-detection/ml-configuring-detector-custom-rules.md). + :::: + +6. Click **Save**. + +::::{note} +Changes to rules only affect new results. All anomalies found by the job before the filter was added are still displayed. +:::: + + + +### Clone and rerun the job [clone-job] + +If you want to remove all the previously detected results for the process, you must clone and run the cloned job. + +::::{important} +Running the cloned job can take some time. Only run the job after you have completed all job rule changes. +:::: + + +1. Find **Machine Learning** in the navigation menu. +2. Under **Anomaly Detection**, select **Jobs**. +3. Navigate to the job for which you configured the rule. +4. Optionally, expand the job row and click **JSON** to verify the configured filter appears under `custom rules` in the JSON code. +5. In the **actions** column, click the more (three dots) icon and select *Clone job*. + + The **Configure datafeed** page is displayed. + +6. Click **Data Preview** and check the data is displayed without errors. +7. Click **Next** until the **Job details** page is displayed. +8. Enter a Job ID for the cloned job that indicates it is an iteration of the original one. For example, append a number or a username to the original job name, such as `windows-rare-network-process-2`. + + :::{image} ../../../images/security-cloned-job-details.png + :alt: cloned job details + :class: screenshot + ::: + +9. Click **Next** and check the job validates without errors. You can ignore warnings about multiple influencers. +10. Click **Next** and then **Create job**. + + The **Start ** window is displayed. + + :::{image} ../../../images/security-start-job-window.png + :alt: start job window + :class: screenshot + ::: + +11. Select the point of time from which the job will analyze anomalies. +12. Click **Start**. + + After a while, results will start to appear on the **Anomaly Explorer** page. + + + +## Define an anomaly threshold for a job [define-rule-threshold] + +Certain jobs use a high-count function to look for unusual spikes in process events. For some processes, a burst of activity is a normal, such as automation and housekeeping jobs running on server fleets. However, sometimes a high-delta event count is unlikely to be the result of routine behavior. In these cases, you can define a minimum threshold for when a high-event count is considered an anomaly. + +Depending on your anomaly detection results, you may want to set a minimum event count threshold for the `packetbeat_dns_tunneling` job: + +1. Find **Machine Learning** in the navigation menu. +2. Under **Anomaly Detection**, select **Anomaly Explorer**. +3. Navigate to the job results for the `packetbeat_dns_tunneling` job. If the job results are not listed, click **Edit job selection** and select `packetbeat_dns_tunneling`. +4. In the **actions** column, click the gear icon and then select *Configure rules*. + + The **Create Rule** window is displayed. + + :::{image} ../../../images/security-ml-rule-threshold.png + :alt: ml rule threshold + :class: screenshot + ::: + +5. Select *Add numeric conditions for when the rule applies* and the following `when` statement: + + *WHEN actual IS GREATER THAN * + + Where `` is the threshold above which anomalies are detected. + +6. Click **Save**. +7. To apply the new threshold, rerun the job by selecting **Actions** → **Start datafeed** on the **Anomaly Detection Jobs** page.