Alerts
Watch tells you when something changes and then stays quiet. Six alerts, each sent once, and how to point them at your phone.
WatchOn this page
What alerts #
Three things alert, and each has a matching message when it clears:
| It happens | You get | When it clears |
|---|---|---|
| A box goes dark | Machine down | Machine recovered |
| An agent server stops while its box stays up | Environment down | Environment recovered |
| A session waits on you past the deadline | Session waiting | Session resumed |
That is the whole list. A session that fails does not alert: it goes near the top of the inbox and waits for you to look. A box that goes stale does not alert either. Stale is the warning Watch gives itself; down is the one it gives you.
When a whole box goes dark you get one alert for the machine, not one for every environment on it.
Why each fires once #
An alert is sent when a status changes, never while it stays the same. A box that goes down at 02:41 and is still down at breakfast has sent you one message, at 02:41. It sends one more when it comes back.
This is deliberate. A message that repeats every few minutes teaches you to swipe it away, and then the one that matters gets swiped away too. One message per change means every buzz from Watch is news. If you have had the alert and nothing since, the thing is still as it was.
The same holds for a waiting session. It alerts once, when it has waited past the deadline. It does not alert again an hour later. When you answer it, or it ends, you get one "resumed".
The console keeps a history of every alert that has fired, with whether each destination accepted it, for as long as your plan keeps history.
Add a destination #
An alert destination is where Watch sends alerts. In the console, open the alert destinations and add one:
- Give it a name you will recognise, such as "My phone".
- Give it an address. It must be an
httpsaddress on the public internet. Addresses on a private network,localhostand plainhttpare refused. - If the receiving service needs a token, add it. Watch sends it as a bearer token, stores it encrypted, and never shows it again.
- Press the test button. Watch sends a sample alert there and shows you what came back.
A destination can be switched off without deleting it. Your plan sets how many destinations an account has and which kinds of channel it can use: see Plans and limits.
What a webhook receives #
Watch makes one POST per alert. The body is plain text, and the headers carry the rest. It is the shape push services such as ntfy accept directly, and anything that can receive a request can read it:
POST /your-topic HTTP/1.1
Content-Type: text/plain; charset=utf-8
Title: agentbox has gone dark
Priority: high
Tags: rotating_light
Authorization: Bearer <your token, if you stored one>
X-Nucarium-Alert-Id: alr_5b6n2f9k4m7q1x8z
X-Nucarium-Alert-Kind: machine_down
No heartbeat from agentbox for 10 minutes. Last seen 03:12 UTC.
| Header | Carries |
|---|---|
Title |
One line saying what happened and to what. |
Priority |
high for the three alerts, default for the three recoveries. |
X-Nucarium-Alert-Kind |
machine_down, machine_recovered, environment_down, environment_recovered, session_waiting or session_resumed. |
X-Nucarium-Alert-Id |
The alert's id, the same one the console's history shows. |
Any 2xx answer counts as delivered. Watch tries each destination once per alert, waits ten seconds for an answer, and does not follow redirects. If a destination is unreachable or answers with an error, the history says so, and the destination's entry in the console shows its last delivery.
An alert names the machine, the environment and the session, because you chose to see those. It never contains anything a session said.
If you want to be told sooner, or later #
The deadlines behind the alerts are yours to change, within what your plan allows. Deadlines explains each.