Your tests stay Playwright, Selenium and pytest. We don't replace them. We make them enterprise-ready, and you own your code, your evidence and your history.

Load testing and Web Vitals against a budget.

Nexus StudioQA runs a load scenario from a JSON file or the Load screen and judges it against budgets you wrote before the run. Each run ends PASS, OVER_BUDGET, REGRESSED or UNMEASURED. In browser tests, LCP, CLS and INP budgets can turn a passing test into PERFORMANCE_DEGRADED when strict mode is on.

By the Nexus StudioQA team, Verdict System LLC.

Last updated: .

The four load verdicts

VerdictWhen
PASSAt least one criterion could be checked and all were met
OVER_BUDGETA criterion was missed, or the open model could not start more than 1% of the requested iterations
REGRESSEDWithin budget, but the p95 is above 1.25 times the median p95 of the last passing runs, and at least 25 ms above it
UNMEASUREDNothing could be checked: no criteria, fewer than 5 attempts, fewer than 5 good samples for a p95, or a Web Vitals budget the probe could not measure

The baseline for REGRESSED is the same scenario, target, model and user count, over the last 5 passing runs, with at least 3 needed. A target that is completely down is OVER_BUDGET, not UNMEASURED, because the error rate is judged on attempts. With strict on, OVER_BUDGET and REGRESSED end the process with exit code 1, so a pipeline can stop.

What a load run does

The scenario sets the target, the steps with their assertions, the think time and either a closed model (1 to 5000 users) or an open model (a request rate). Virtual users send HTTP requests from worker threads, and the report gives percentiles, throughput and an error rate, never only an average. Two safeguards come first: you type AUTHORIZED before anything is sent, and a target that is not this machine also needs --yes. Results go to evidence/load-runs.

Web Vitals

With --vitals, a probe in Playwright Chromium loads your pages before, during and after the load and checks p75 budgets for LCP, FCP, CLS, TBT and TTFB. INP is refused there because the probe never interacts with the page. In browser tests, the engine measures LCP, CLS and INP on each run against budgets from NEXUS_PERF_LCP, NEXUS_PERF_CLS and NEXUS_PERF_INP. The default thresholds follow web.dev:

MetricGood up toPoor above
LCP2500 ms4000 ms
CLS0.10.25
INP200 ms500 ms

These are lab numbers from a scripted page load, a regression signal on a fixed profile, not field data from real users. INP needs a click or a keystroke, so a test that only loads pages has none. A budget miss only changes a test to PERFORMANCE_DEGRADED when NEXUS_PERF_STRICT is true; otherwise it is recorded and shown in the report.

In the release report

The Executive Report counts load runs that ended OVER_BUDGET or REGRESSED as failed, shows the last run, and adds a recommendation when the last load run is not PASS or UNMEASURED.

Common questions

  • Does the load generator drive real browsers?

    No. Virtual users send HTTP requests, which is how a load generator reaches thousands of users. Playwright Chromium is used only for the Web Vitals probe and for browser tests.

  • Why is a run UNMEASURED instead of PASS?

    Because nothing could be checked, for example no criteria, too few samples for a percentile, or a Web Vitals budget the probe could not read. A budget nobody checked did not pass.

  • Can I load test any site?

    Only a system you own or have written permission to test. The tool asks you to type AUTHORIZED even for localhost, and a target that is not this machine needs --yes.

  • Does a slow page fail my test?

    Only in strict mode. By default a performance budget miss is recorded as an advisory. With NEXUS_PERF_STRICT=true it makes the test PERFORMANCE_DEGRADED, which still exits 0 in CI.

Put a budget on speed.

Everything runs on your own machines. No card needed.