AI test risk prediction from your own history.
Nexus StudioQA scores every test and browser from 0 to 100 for the risk that its next run has a problem. A logistic regression written in plain JavaScript learns from the run history on your machine. When there is too little history, an exponentially weighted average takes over. Each score lists the factors behind it.
By the Nexus StudioQA team, Verdict System LLC.
Last updated: .
What it looks at
For each test and browser, the predictor reads the chronological runs in History and builds eight features from the last 10 runs.
| Feature | What it measures |
|---|---|
| fail_rate_5, fail_rate_10 | Share of problem runs in the last 5 and the last 10 |
| flaky_rate_10 | Share of runs that needed a retry or were flaky |
| alternation | How often the result flips between pass and fail |
| last_problem | Whether the last run was a problem |
| streak_signed | The current streak: positive for consecutive problems, negative for consecutive passes |
| duration_cv, duration_drift | How unstable the duration is, and how the last run compares with the median |
A problem run is BREACHED, BREACHED_FLAKY, TEST_DRIFT, TIMEOUT, CRASHED, UNREACHABLE or BLOCKED. Chaos runs are left out on purpose, because their failures are injected and would teach the model the wrong thing.
The model and its fallback
| Situation | Method | Output |
|---|---|---|
| At least 12 pooled samples, with at least 4 problem and 4 clean | Logistic regression, trained on sliding windows of all tests and browsers together | Score of 0 to 100 from the model probability |
| Fewer samples than that | Heuristic: 50% EWMA of problems, 20% alternation, 20% flaky rate, 10% duration instability | Score of 0 to 100 |
| Fewer than 3 runs for a test | None | No row; the count of such tests is reported |
Scores of 60 or more are HIGH, 30 to 59 are MEDIUM, and the rest are LOW. The model block reports its accuracy against a majority-class baseline, measured by leave-one-out cross-validation (5-fold above 200 samples), not by accuracy on its own training data.
Flaky, broken or unclear
A test that fails every time is predictable: it is broken, and a retry will not fix it. So each row also carries a pattern. Alternating results mean flaky, a failure rate of 80% or more without alternation means broken, and anything else is unclear. The Studio ranks tests the same way, in three separate lists: non-deterministic, broken, and cost in seconds spent on failed runs. A pair needs at least 5 runs before its rate is treated as a claim.
Where it runs and where it shows up
It runs with nexus predict [TC] [--browser b] [--json] [--no-save] and writes evidence/mlops/flakiness-predictions.json in your workspace. The Studio shows the ranking in the Evidence Locker. It uses no outside service and no extra dependency: the training is plain JavaScript, so the history never leaves your machine.
Common questions
Is this a large language model?
No. It is a small logistic regression trained on your run history, in plain JavaScript. It is deterministic: the same history gives the same scores.
Does it predict a bug in the application?
It predicts the risk that the next run of a test has a problem, which includes flakiness and environment failures. It does not read your code or say what is wrong.
How much history does it need?
At least 3 runs of a test to score it at all. To train the model it needs at least 12 pooled samples with at least 4 problems and 4 clean runs; before that the EWMA heuristic is used.
Why does a score show factors?
Each score lists up to three human-readable reasons, such as a high problem rate in the last runs or an intermittent pass and fail pattern, so the ranking can be checked against the history.
See which tests will fail next.
Everything runs on your own machines. No card needed.