Alerts

Alerts are a way to get notified when one of your objectives or score cards goes below a threshold. In Reliably, you define Alert Policies that trigger Alert Events on an Alert Channel.

Alert Channels

Alert Channels are the way Events are sent to you when triggered by an Alert Policy when an objective or score card didn’t meet its expected threshold.

Supported Alert Channels can currently be set to a Slack Channel or a webhook, with email support on the way.

Slack Channel

To create a Slack Channel, you first need to integrate Reliably with your Slack workspace.

  • Navigate to your organization settings from the left-hand sidebar, and select the Integrations tab.
  • Click the Connect to Slack button to connect your Reliably organization with a Slack workspace.

You are now ready to create Slack Alert Channels.

  • Navigate to Explore and select the Channels tab.
  • Select Slack Channel as your channel type.
  • Enter a name for your channel.
  • Type the name of an existing channel in your connected Slack workspace.

Webhook Channel

  • Navigate to Explore and select the Channels tab.
  • Select Webhook Channel as your channel type.
  • Enter a name for your channel.
  • Type the URL of your existing webhook.

Alert Policies

To create a new Alert Policy, navigate to Explore and select the Alerts tab. Clicking the Create Alert Policy will bring a modal window to create your new policy.

  • Type the name of your alert policy.
  • Select its target type, the targets can be either objectives or score cards.
  • Select its target, which can be one or more existing objectives or score card.
  • Add one or more Slack or webhook channels that alert events will be sent to.
  • If you selected an objective target type, you must now select a threshold type. Different threshold types have different behaviour. The two types are:
    • Error Budget Threshold: allows you to define a threshold based on your error budget.
    • Indicator Threshold: this type targets the indicator’s value as the threshold, irrespective of the objective’s target.
  • Select a threshold, which will determine at what point the alert is triggered. A recovery Alert Event will be sent when the value surpasses the threshold again.
  • Enter its severity level, an integer between 1 and 3, with 3 being the most serious.
  • Add a message that will be displayed if an alert is triggered.

Error Budgets

An error budget is a simple metric that allows you to easily track how close your indicator is to breaching the target for your objective. For example, given an objective target of 98%, and an indicator value of 99%, your error budget would be 50% because you have used half of the “errors” you have available before your objective is unsuccessful.

Some teams go further and actively consume their error budgets with practices such as Chaos Engineering. If you are not currently looking to consume your error budget, it’s still a useful metric because it lets you define behaviour at different budget levels. You may want to define an alert when your error budget has reached 50%, and yet another one when it’s only at 25%, etc.