Why one performance score is not enough

A website can feel quick on a developer’s laptop yet load slowly for a visitor on a less capable phone or an unreliable connection. It can also perform poorly in one automated test while most visitors have a good experience. Neither observation is necessarily wrong: the two measurements describe different conditions.

Web performance teams commonly use two approaches. A lab test loads a page in a controlled environment, often with a chosen device profile, network setting and browser. Real-user monitoring, or RUM, collects measurements from visits as they happen in users’ browsers. Lab tests help answer “What happens under these test conditions?” RUM helps answer “What are visitors experiencing across the conditions they actually use?” The most useful picture comes from understanding both questions.

What lab tests do well

Because a lab test can repeat a page load under similar conditions, it is useful for investigating causes. A report may show when the browser requested an image, how long a script took to run, or whether a large stylesheet delayed rendering. A recording or timeline can help connect a visible pause to a specific resource. Teams can also test a proposed change before releasing it, then compare runs under the same settings.

Control is also the main limitation. A simulated connection cannot represent every real network, and one device profile cannot represent every visitor’s hardware. Test results may change if the page serves different content, an external resource responds differently, or the test starts with a warm rather than an empty cache. For a meaningful comparison, keep the test page, settings and cache state consistent. Treat a single run as a clue, not a verdict.

Lab tests are especially helpful when you need to reproduce a slow page, inspect its loading sequence or check whether a code change introduces extra work. They can test pages that have little or no visitor traffic, including a page still in development.

What real-user monitoring adds

RUM measures visits across the mix of devices, connections, locations and navigation paths that reach a site. Instead of asking how one configured test behaves, it shows how an experience varies. A team might find that a product page loads acceptably for desktop visitors but struggles on smaller devices, or that returning visitors see a different result from first-time visitors. Those patterns can guide where to investigate first.

RUM also has limits. It shows that a problem occurred, but it may not identify the exact script, image or server behavior responsible. Results can shift when the audience or traffic mix changes, even if the code does not. Pages with few visits may provide too little information for a reliable comparison. Browser support, collection settings and blocked measurement scripts can affect which visits are represented.

Choose measurements that match the experience you want to improve, such as when main content becomes visible, whether the layout shifts unexpectedly, and how quickly the page responds to interaction. Look at a range of experiences rather than an average alone: an acceptable average can hide a frustrating slower group. Segment carefully, since very narrow groups may be too small to interpret.

Use the two approaches as a feedback loop

Suppose RUM shows that a landing page is slow for visitors on phones. First, check whether the pattern is concentrated on a particular page type or appears across the site. Next, run a lab test using a relevant device and connection profile. Inspect the loading timeline: perhaps a large image arrives late or a script occupies the browser before the main content appears. Make one targeted change, then rerun the lab test under comparable conditions.

A better lab result is promising, but it is not the end of the investigation. After release, check RUM to see whether the affected visitors experience an improvement. If the field measurements do not move as expected, revisit the assumption: the lab profile may not match those visitors, another bottleneck may dominate, or a different traffic mix may obscure the change. This cycle turns RUM into a way to prioritize and verify work, while lab tests provide a place to diagnose and experiment.

Build a measurement practice you can trust

Start with a small set of important page types and document how each is tested. Record the lab settings used for comparisons, and annotate notable releases so changes in RUM have context. Compare like with like where possible: the same page type, device class and measurement definition. Avoid treating a score from one lab configuration as interchangeable with a RUM result.

Collect only the user data needed to understand performance. Review what page addresses, identifiers or custom events a monitoring tool records, since they may reveal more than a team intends to collect. Limit access and follow the site’s privacy requirements. Good performance measurement should help improve the visitor’s experience without unnecessarily exposing details about the visitor.

Back to category ↗