Configurable monitor history retention per monitor

[iCaotix]

⚠️ Please verify that this feature request has NOT been suggested before.

• I checked and didn't find similar feature request

🏷️ Feature Request Type

UI Feature, Other

🔖 Feature description

The ability to configure how long the monitor history is retained on a per monitor basis.

✔️ Solution

The idea would be to have a setting on the "Edit Monitor" page where you could set a different retention time for this specific monitor. If that field is empty (on purpose or for all previously created monitors) the global fallback is used.

❓ Alternatives

I'm not sure if there would be another option to achieve this behavior. But I'm open to suggestions.

📝 Additional Context

This feature was briefly mentioned in #21 as a checkbox / bullet point but I didn't find a separate issue. And it looks like this was meant for the global setting, which is already implement, but not sure since it's not checked off yet.

Additional info, I think I even would be able to implement that feature and raise a PR for it, if it fits into the project.

[copenhaus]

....or this feature can be made available in Settings, like how Notifications is now, that there's a toggle for "Apply on all existing monitors"

[CommanderStorm]

@iCaotix why do you think this would be a good feature?
Please go further into the usecase have.

The added complexity of this feature (one more thing to configure, more difficult deletion) is substantial, while the downside of the current approach (one monitor being stored a bit longer => a few MB of storage) is negligible.

[chakflying]

Doing this would greatly increase the workload of the nightly clear-old-data job. Right now it just 1 line of SQL, if you have to retrieve settings then do it per-monitor it would take much longer and might affect the performance of the actual monitors.

[copenhaus]

does that 1 line of SQL just delete data that's older than 30 days? I think we are asking to delete anything older than "X" days, and we get to choose the "X" value. Yes, with the understanding that keeping more data will impact the performance. Because I'd like to keep the historical data to up to one year, so I can observe "seasonal trend."

[iCaotix]

@CommanderStorm The use case is to have monitors which monitor your "own" services for which I would want to keep the history for a year let's say and on the other hand I have about a dozen monitors which monitor "other people's" services (e.g. an API, a Website for a keyword) where I'm only interested in let's say the last two or four weeks.

For example on Zabbix this is already possible on a per item basis and on prometheus it's not that much of a deal in terms of data size (because of delta encoding) as it would be here.

Concrete example: I have 3 monitors (60 sec interval) I want to keep for a year and 12 monitors (also 60 sec interval) which I only need 4 weeks. Currently my setting would be to keep all for a year.

[iCaotix]

Doing this would greatly increase the workload of the nightly clear-old-data job. Right now it just 1 line of SQL, if you have to retrieve settings then do it per-monitor it would take much longer and might affect the performance of the actual monitors.

No this can still be performant when it's done with small transactions, e.g. One per monitor, and if it's a continuous cleanup job which runs every day at most 5760 entries (24 hours of 15 second intervals) would be removed from the db, which is really not much

[CommanderStorm]

In v2 (see #4500 for further detais), we have improved the performance considerably. We now only store aggregated results => configuring montiors on a per monitor level is not nessesary anymore from an performance standpoint.

Gven that heatbeat data is not PII, I cannot see a reason why this would be nessesary anyomore.

=> closing as not planned (because it does not make sense anymore)