Business Event Alerting: Thresholds That Do Not Create Noise

An alert exists so somebody does something. The moment it arrives at a frequency where you cannot act on all of them, it stops serving that purpose - and worse, it trains everyone to ignore the one that did matter. That is why a setup with twenty alerts finds faults later than one with two.

What is worth alerting on in a business

Not "data". Situations requiring a decision within a short window, and only those:

• A financial exception - an invoice larger than usual, a payment that did not arrive on time.
• Stock - falling below the reorder point on a critical item.
• A commitment to a customer - a delivery that did not go out, tomorrow's meeting unconfirmed.
• A stalled process - an order stuck in the same status for more than three days.
• Technical failure - an automation that fell over or did not run.

The fourth is the least defined in most businesses, and it produces most of the angry customers. A stalled enquiry generates no event of its own, so it needs an alert based on time rather than on something that happened.

How to set a threshold

Three steps: measure for two weeks without alerting, observe the normal range, and set the threshold outside that range. Not by feel, and not by what sounds reasonable.

What happens when you set by feel is one of two things: a threshold too low that fires daily, or one too high that never fired - and both look exactly the same after a month, namely silence.

Trend, not event

Alerting on a single event almost always produces noise, because single events happen. Three errors in an hour is a signal; one error is a normal day. The same logic applies to the business: one customer complaining is an incident, three in the same week about the same thing is a broken process.

So a well-designed alert has two parameters rather than one: how many, and in what time window. That is also what turns an alert from "something happened" into "there is a pattern here", and that is the alert somebody will actually act on.

Alerts outside working hours

A decision worth making in advance: which alerts are allowed to wake somebody, and which wait for morning. Without it one of two things happens - either everything always arrives and people switch alerts off in the evening, or nothing arrives and that is discovered when something critical waited nine hours.

The simple rule: only wake somebody for something that has an available action at that hour. A fault that can only be fixed in the morning does not justify a message at night, even when it is serious - what it does justify is being first on the morning list.

That requires two levels rather than one: an urgent channel that always arrives, and a normal channel read during working hours. That separation is what lets people leave their alerts switched on.

Who receives it and what it says

One name. An alert to a group produces the bystander effect - everybody sees it, nobody handles it. Where more than one person needs to know, one receives and others are copied, and that is defined explicitly.

The body needs three things: what happened, how serious it is, and what to do. An alert saying only "anomaly detected" transfers the work to the recipient at exactly the moment they are least free to go looking. The full logic of what happens after an alert is in an automation runbook.

Alerting on what did not happen

This is the forgotten category and usually the most important. An event that happened produces a record; an event that did not happen produces nothing, so there is nothing to wake anybody.

Three practical examples: no orders received today by midday, the morning report did not send, no new enquiry recorded in a channel that normally produces five a day. Each of those could be a quiet day - and each could be a broken form, and there is no way to tell without checking.

What separates an alert from a report

The two tools look similar and serve opposite needs, and businesses that mix them get the drawbacks of both. A report says what happened over a period; an alert says something needs to happen now.

Three practical distinctions follow. Frequency: a report goes by schedule, an alert by event. Content: a report includes what is fine, an alert only the exception. And tone: a report can be read tomorrow, an alert cannot - and if it can be read tomorrow, it should have been a line in a report.

The simple test: if a recipient who got it late in the evening could wait until morning with no harm done, it should not have been an alert. What belongs in a report instead is covered in scheduled reports instead of asking.

What do you do when alerts have already flooded everyone?

Do not mute gradually - start from zero. The practical route is to switch everything off and bring back only two alerts, the ones that genuinely led to action in the last month. After those, add one at a time, only when one is missed.

What that nearly always reveals: out of twenty alerts, three or four were useful and the rest were built "just in case". The slow reintroduction is also what restores trust - after a month where every alert was real, people start opening them again.

Sources

• Business automation for small businesses in Israel
• Make - error handling in scenarios