Monitoring: Support SNS -> Matrix channel alerting

[ryanjjung]

I'm not sure how to do this, exactly. SNS currently notifies by email, but we could also subscribe it to a Lambda that handles messaging to Matrix for us.

[Sancus]

SNS can publish to text messages, mobile push notifications, sqs queues, arbitrary http/https webhooks, lambda, and many other things via firehose. So it's really just a pick your poison situation.

Personally I'd probably just build a standard persistent bot that gets notified by webhook, rather than trying to make a lambda that reconnects and bails every time it needs to send a message. But whatever works.

[ryanjjung]

Mel pointed me to postmoogle, which is an email server that bridges inbound messages over to Matrix channels (among other things). So for a Matrix-specific solution, that's an option. It would likely run as a container in Fargate, a separate deployment/stack. That would solve the current ask (alert to Matrix), though I will take a look at other options because Matrix-backed solutions don't generally appear, from the little bit of searching around I did the other day, very stable or well-maintained.

Email notification is what's built into SNS at its core, but yeah, you can subscribe almost anything to it. Until something more sophisticated comes along (on-call schedule/rotation, etc.), Matrix alerts seem the most immediate solution for notifying teams of alerts.