Synthetic checks vs real user monitoring: use both, page on one

Real user monitoring instruments your actual application and reports what happened to actual visitors: page load times, JavaScript errors, failed requests, on their devices and their networks. Synthetic monitoring runs a scripted robot that pretends to be a customer, on a schedule, from infrastructure you control.

Most teams have one or the other, usually RUM, because it comes bundled with something they already pay for. Then they configure alerts on it, and at that point they have built something that cannot see the outages that matter most.

The 03:00 problem

Here is the failure mode, and it is the reason this article exists.

RUM is a measurement of traffic. Every metric it produces — error rate, load time, conversion — is a ratio computed over the people who showed up. Which means the signal is only as strong as the traffic, and at three in the morning for a company selling to one country, the traffic is close to zero.

This is not a subtle edge case. A total outage reduces your error count, because a browser that cannot load your application never runs the script that reports the error. The graph goes quiet, and quiet is what healthy looks like.

We have seen a client with a genuinely good RUM setup lose their entire European morning because the outage started at 04:10, the overnight traffic was six sessions an hour, and their alerting needed a statistically meaningful error rate before it would fire. It got one at 07:30, along with the first commuters.

What synthetics do differently

A synthetic check does not wait for anybody. It generates its own traffic at a fixed interval, so its signal has nothing to do with whether your customers are awake. Three in the morning looks exactly like three in the afternoon: a run every three minutes, pass or fail.

More importantly, a well-written synthetic exercises a business process rather than an endpoint. Our standard checkout journey does this:

Every individual service in that stack could be reporting itself healthy. The web tier is up, the search service is up, the basket service is up. The thing that is broken is the join between step three and step four — and there is no server-level check anywhere that would notice.

This is the single strongest argument for synthetics: they are the only check that tests the seams, and the seams are where most real outages live.

What synthetics cannot see

Being fair about the other direction, because synthetics have a genuine blind spot and it is a large one.

Your robot runs from a data centre in Frankfurt on a fast, stable connection, in one browser version you chose, with no extensions, no ad blocker, no corporate proxy, no six-year-old Android phone on a bad train connection. It will therefore report your application as fast and working in conditions no actual customer is in.

• Client-side breakage. A JavaScript error that only fires on Safari 16, or on a device with a particular locale. Your synthetic is on headless Chrome and sees nothing.
• Real-world performance. Frankfurt-to-Frankfurt in 400 ms is not the 2.8 seconds your customer in rural Portugal is living with.
• Partial failure. One payment method failing for one card issuer. Your synthetic uses one test card and it works.
• Anything you did not think to script. A synthetic only covers the journeys somebody wrote. RUM covers everything a person actually did, including the paths nobody predicted.

So: synthetics catch outages, RUM catches degradation and the long tail. They are not competing purchases.

The rule we apply

Synthetic checks — deterministic, traffic-independent, testing a defined business process — are allowed to ring a phone at night, after two consecutive failures from two locations.

RUM feeds dashboards, weekly reviews and the backlog. It answers "which of these fifteen things should we fix first", which is a genuinely valuable question and not one that needs answering at 04:00. Where RUM does alert, it alerts into a ticket queue during working hours.

The exception, and there is one: a sudden collapse in traffic volume itself is a legitimate page. Not the error rate — the raw session count. If a site that reliably does 400 sessions an hour does eleven, something is wrong upstream of your application entirely: DNS, CDN, a BGP problem, a certificate on the edge. That check is worth having and it is one of the few places RUM data earns a phone call.

Practical setup

What we deploy on a typical client, as a starting point:

• Three to five journeys, no more. The ones where failure means lost revenue: sign in, the core action, checkout, and whatever your support team gets called about most.
• Every three minutes, from three cities. Frequency costs money and detection time is roughly half the interval, so three minutes is where we land for most people. One location is not enough to distinguish your outage from a network problem near the prober.
• Assert on business outcomes, not on HTTP 200. The page returned 200 and said "payment temporarily unavailable" is a failure. Check for the confirmation text and the order ID, not the status code.
• A dedicated test account and test card, excluded from analytics and revenue reporting, with a standing note to the finance team about what those transactions are.
• Screenshot and DOM capture on failure. When the phone rings at 03:00, "step 4 failed" plus a screenshot of the actual error saves ten minutes of an engineer reproducing it.

That last one sounds like a small thing. On the timeline on our home page, it is roughly the difference between minute five and minute fifteen.

← All field notes