alert-sources.md

description
Connect your third party tools to ilert.

Alert sources

An alert source represents the connection between your tools (usually a monitoring system, a ticketing tool, or an application) and ilert. We often refer to alert sources as inbound integrations.

ilert provides the following inbound integration options:

Tool integrations	These are pre-built integrations by ilert and work-out-of the box with your monitoring tools. If you're missing a tool, feel free to suggest an integration that you'd like to see in ilert.
Email integration	Forward emails to an alert source's email address to integrate with ilert.
Event API	Write your own integration using our easy-to-use Event API.
SMS integration	Send alerts to ilert via SMS.
Heartbeat monitoring	A heartbeat alert source will automatically create an alert if it does not receive a heartbeat signal from your app at regular intervals.

Create an alert source

1. Go to Alert sources -> Alert sources and click Create new alert source.\

   

2. Select your integration type in the search field and click Next.\

   

3. Give your alert source a name, optionally assign teams, and click Next.

4. Select an escalation policy by creating a new one or assigning an existing one. \

   

5. Select your Alert grouping preference and click Continue setup. You may click Do not group alerts for now and change it later.\

   

6. The next page shows additional settings, such as custom alert templates or notification priority. Click on Finish setup for now.

Event Explorer

Event Explorer provides a more detailed view of alerts received from the specific alert source. To see alert information in JSON format, choose the alert source you are interested in and navigate to the "Event Explorer" section beneath the source title. Event Explorer facilitates the search for specific events linked to alert sources using keywords or time frames.

{% hint style="info" %} Event Explorer is available for a selected list of alert sources, with event history available from March 2022 and alert correlations from December 2023 onwards. {% endhint %}

Alert template

With alert templates, you can create your own template for the alert summary and alert details using preset fields from the integration. Moreover, our templating lets you extract links from the alert payload. Extracted links will be added to the links section of an alert.

Custom alert summary and details template

1. Click on Alert sources -> Alert sources and choose an alert source to edit

2. Navigate to the section Alert template and check the boxes for Alert summary and/or Alert details

3. Create your custom template by selecting the fields you want use and entering any static text.
   The available fields are specific to the integration.\

   

{% hint style="info" %} Field colors and accessing raw fields

• Blue fields are preset fields provided by ilert.
• Orange fields are extracted from past alerts in your account that were sent from the specific integration
• Grey fields lets you extract any raw fields from the JSON payload by typing the name of the custom field, e.g. custom_field. You malso access nested fields and arrays, e.g. custom_field.array_field[5].nested_field {% endhint %}

Testing your templates before saving

Using the preview button you may try out your current template. By default, ilert will try to find one of the latest event payload's that was received by your alert source. If there is none present, we will render a fallback JSON doc, which you might alter as you like.

Using the template text syntax

Your alert source template fields will start in text mode by default. In text mode you may use the Insert data... dropdown to help you add template variables quickly (see here to understand more about variables and how ilert automatically parses event data to offer additional variables to you) - the text syntax works like this:

Type	Sample	Description
Text	Some text	You may of course add generic text content to your liking
Variable	{{var}}	Extract content of the event and insert it. Note: there is no further sanitizing of the values
Accessing nested variables	{{ var.subfield.evenMore }}	Access sub fields
Accessing fields of an array	{{ var.arrayField[0].more }}	Access array contents

See ITL below to learn more about the templating language and its features for blocks, loops and functions:

{% content-ref url="../rest-api/itl-ilert-template-language.md" %} itl-ilert-template-language.md {% endcontent-ref %}

{% embed url="https://www.youtube.com/watch?t=3s&v=RIYsmc1Uajs" %}

Alert links

ilert can extract alert links from the alert payload. Extracted links will be added to the alert's links section.

Alert link template	Alert with extracted link

Dynamic escalation policy routing

With dynamic escalation policy routing, the escalation policy to be used will be determined based on the incoming alert, instead of always using the same escalation policy that is configured on the alert source.

To extract the escalation policy routing key from the alert payload, add a routing key template in the section Escalation -> Dynamic routing.

In the above example, the field Group key from the alert payload will be used as the routing key.

Notification priority and support hours

Default notification priority

By using notification priority, you can easily customise your alert notification based on your notification rules.

1. Click on Alert sources -> Alert sources and choose an alert source to edit
2. Scroll down to the section Notification priority and set your desired Notification priority

ilert provides different priority settings to customize your alerts.

• High (with escalation): You will be notified based on your high-priority notification rules and an alert can be escalated based on escalation policy.
• Low (no escalation): You will be notified based on your low-priority notification rules and an alert cannot be escalated.

Support hours based notification priority

ilert also lets you dynamically set the notification priority based on the alert source's support hours. This lets you, for instance, use more obstrusive notification methods like phone calls outside of business hours and use not so obstrusive ones during business hours.

• High during support hours, low priority otherwise: During your support hours, you are notified based on your high priority notification rules. At all other times, you are notified based on your low priority notification rules.
• Low during support hours, high priority otherwise: During your support hours, you are notified based on your low priority notification rules. At all other times, you are notified based on your high priority notification rules.

If you select High during support hours, low priority otherwise, you can choose to Raise priority of all pending alerts by ticking the checkbox located under the support hour selection. All your pending alerts for the current alert source will be raised to "high" when your support hours begin.

If you select Low during support hours, high priority otherwise, you can choose to Raise priority of all pending alerts by ticking the checkbox located under the support hour selection. All your pending alerts for the current alert source will be raised to "high" when your support hours end.

Dynamic priority mapping

With dynamic priority mapping, you can use alert fields to extract and map notification priority. This will overwrite default priority, if enabled.

To enable dynamic priority mapping

1. Click on Alert sources -> Alert sources and choose an alert source to edit
2. Scroll down to the section Notification priority and check Enable dynamic prioriuty mapping
3. Enter template to to extract the priority field from the alert payload
4. Add priority mappings. A priority mapping maps an extracted value from the alert payload to the ilert priority

{% hint style="info" %} ilert will fallback to the alert source's default priority, if a priority could not be extracted. {% endhint %}

Event filter

Sometimes flagging alerts as low priority is not enough and it is necessary to drop events completely. e.g. Grafana DatasourceNoData This is why you can configure one or multiple event filter groups for your alert source to process only desired events into alerts.

You may choose between properties of the known ilert event payload such as priority or summary, some more advanced dynamic fields like trigger counts which allow you define your own rate limits as well as schedules and support hours to fine tune accept windows - and of course custom payload fields.

If you wish for even more customization you can switch the UI into the code editor mode and get full access to the ICL, find out more about it here:

{% content-ref url="../rest-api/icl-ilert-condition-language.md" %} icl-ilert-condition-language.md {% endcontent-ref %}

Alert grouping

Alert grouping helps you reduce noise by clustering related alerts within a defined time window or by allowing only one open alert at a time per source.

Enable alert grouping during alert source creation or in the alert source's advanced settings

There are 5 types of alert grouping available:

• Native integration based grouping
• Time-based grouping
• Grouping until accepted
• Grouping until resolved
• ilert AI similiarity based grouping

Native integration based grouping

By default every alert source attempts to offer the best experience based on the features that the corresponding third party integration tool has available. Some tools offer more e.g. resolve events or proper alertKeys to group events, some tools offer a plain webhook without any additional context. ilert shows the integration features in the creation wizard:

Integrations such as Autotask, Jira, Grafana or Prometheus provide rich payloads which ilert automatically uses to identify alertKey and eventTypes, which are used to automatically group incoming events, if an unresolved alert with the same identifier is found.

Time based grouping

An alert source with alert grouping enabled will group together alerts triggered within the defined time window and create only one alert. Grouped alerts will show up as events in the alert's timeline. You can select relative time windows e.g. 2 minutes, 5 minutes, etc.

Grouping until accepted / resolved

Besides static relative windows you can also define action-based windows. By doing so alerts will be grouped until all alerts of the same alert source are accepted or resolved, before opening a new alert. (You find these options at the end of the time-based selector)

ilert AI based similarity grouping

Check out the link below to see more about AI based grouping:

{% content-ref url="../ilert-ai/using-ilert-ai-for-alert-grouping.md" %} using-ilert-ai-for-alert-grouping.md {% endcontent-ref %}