Your tests stay Playwright, Selenium and pytest. We don't replace them. We make them enterprise-ready, and you own your code, your evidence and your history.

AI test risk prediction from your own history.

Nexus StudioQA scores every test and browser from 0 to 100 for the risk that its next run has a problem. A logistic regression written in plain JavaScript learns from the run history on your machine. When there is too little history, an exponentially weighted average takes over. Each score lists the factors behind it.

By the Nexus StudioQA team, Verdict System LLC.

Last updated: .

What it looks at

For each test and browser, the predictor reads the chronological runs in History and builds eight features from the last 10 runs.

FeatureWhat it measures
fail_rate_5, fail_rate_10Share of problem runs in the last 5 and the last 10
flaky_rate_10Share of runs that needed a retry or were flaky
alternationHow often the result flips between pass and fail
last_problemWhether the last run was a problem
streak_signedThe current streak: positive for consecutive problems, negative for consecutive passes
duration_cv, duration_driftHow unstable the duration is, and how the last run compares with the median

A problem run is BREACHED, BREACHED_FLAKY, TEST_DRIFT, TIMEOUT, CRASHED, UNREACHABLE or BLOCKED. Chaos runs are left out on purpose, because their failures are injected and would teach the model the wrong thing.

The model and its fallback

SituationMethodOutput
At least 12 pooled samples, with at least 4 problem and 4 cleanLogistic regression, trained on sliding windows of all tests and browsers togetherScore of 0 to 100 from the model probability
Fewer samples than thatHeuristic: 50% EWMA of problems, 20% alternation, 20% flaky rate, 10% duration instabilityScore of 0 to 100
Fewer than 3 runs for a testNoneNo row; the count of such tests is reported

Scores of 60 or more are HIGH, 30 to 59 are MEDIUM, and the rest are LOW. The model block reports its accuracy against a majority-class baseline, measured by leave-one-out cross-validation (5-fold above 200 samples), not by accuracy on its own training data.

Flaky, broken or unclear

A test that fails every time is predictable: it is broken, and a retry will not fix it. So each row also carries a pattern. Alternating results mean flaky, a failure rate of 80% or more without alternation means broken, and anything else is unclear. The Studio ranks tests the same way, in three separate lists: non-deterministic, broken, and cost in seconds spent on failed runs. A pair needs at least 5 runs before its rate is treated as a claim.

Where it runs and where it shows up

It runs with nexus predict [TC] [--browser b] [--json] [--no-save] and writes evidence/mlops/flakiness-predictions.json in your workspace. The Studio shows the ranking in the Evidence Locker. It uses no outside service and no extra dependency: the training is plain JavaScript, so the history never leaves your machine.

Common questions

  • Is this a large language model?

    No. It is a small logistic regression trained on your run history, in plain JavaScript. It is deterministic: the same history gives the same scores.

  • Does it predict a bug in the application?

    It predicts the risk that the next run of a test has a problem, which includes flakiness and environment failures. It does not read your code or say what is wrong.

  • How much history does it need?

    At least 3 runs of a test to score it at all. To train the model it needs at least 12 pooled samples with at least 4 problems and 4 clean runs; before that the EWMA heuristic is used.

  • Why does a score show factors?

    Each score lists up to three human-readable reasons, such as a high problem rate in the last runs or an intermittent pass and fail pattern, so the ranking can be checked against the history.

See which tests will fail next.

Everything runs on your own machines. No card needed.