(Stale) Reminder doesnt get deleted

[jmhofmann]

During the past weeks, we observed a stale reminder which didnt get deleted while the service is already back passing/healthy.

That means, critical notifications are continuing even if a service has already recovered and is back healthy:

`
consul-alerts@xxx-europe-xxx

System is CRITICAL
The following nodes are currently experiencing issues:
Failed: 1
Warning: 0
Passed: 0

Node: xxxx
Servicename: Servicename
Since: 2020-01-26 07:21:06.216467314 +0100 CET
Output:

HTTP GET http://localhost:xxxx/health: 200 OK Output: healthy`

The KV entry in consul-alerts/reminders/$hostname$ still exists when this happens. Once the entry is manually deleted, the errornous sent notifications stop.