Your tests stay Playwright, Selenium and pytest. We don't replace them. We make them enterprise-ready, and you own your code, your evidence and your history.

Visual regression testing in Nexus StudioQA.

Visual regression testing in Nexus StudioQA compares a screenshot from each run with a baseline image you approved. The first run with --visual plants the baseline. Later runs count the pixels that moved, ignore anti-aliased edges and keep a diff image when the change passes the tolerance. By default a regression is recorded but does not fail the test.

By the Nexus StudioQA team, Verdict System LLC.

Last updated: .

What each run compares

Two kinds of checkpoint, each with its own tolerance: the share of pixels allowed to differ before the change counts as a regression.

CheckpointWhat is photographedDefault tolerance
Whole pageThe end-of-test screenshot of the viewport, saved under the name baseline.1.0% of pixels (NEXUS_VISUAL_THRESHOLD)
ElementOne element, set inside the test with visualCheck(page, 'cart-total', page.locator('#cart-total')).0.1% of pixels (NEXUS_VISUAL_ELEMENT_THRESHOLD)

How a change is decided

  • Baselines are kept per browser and platform

    They live in visual-baselines/<browser>@<platform>/ in the project folder, so they can be versioned with your code. A Windows image is never compared with a Linux one, because fonts and anti-aliasing differ. A browser and platform with no baseline plants its own.

  • Anti-aliased edges are not counted

    A pixel that only moved because an edge was re-blended is painted amber in the diff and left out of the count. A pixel counts as changed when a color channel differs by 10 or more, and NEXUS_VISUAL_PIXEL_TOLERANCE changes that between 0 and 255.

  • Photos are retaken until they settle

    A photo is retaken until two in a row are byte-identical, three shots at most. A page that never settles is reported as unstable instead of becoming the baseline unnoticed.

  • A different size is not stretched

    The two images are compared at native size over the region they share, aligned top-left. Area that does not overlap counts as different.

  • A check that did not run fails

    If the comparison cannot run, for example an image is missing or the native image library is not installed, the test fails whatever the strict setting says. A check that never happened is not reported as a pass.

The settings that matter

Each one is on the Visual Regression screen of the Studio, with the default below.

SettingDefaultWhat it does
Compare pixels on every runOffTurns the comparison on for every run. --visual turns it on for one run.
A regression fails the testOffWith NEXUS_VISUAL_STRICT=true, moved pixels past the tolerance fail the test. Off, the regression is recorded and the test keeps its verdict.
Masked on every photoNoneCSS selectors painted over in every photo, such as a clock or an ad (NEXUS_VISUAL_MASK).
Regression folders kept10Newest folders kept per test and checkpoint (NEXUS_VISUAL_REGRESSION_RETAIN).

What you review and approve

Each regression is filed under visual-regression with its own files, and the Studio shows them in four views: Side by side, Swipe, Diff and Blink, next to the earlier runs of the same checkpoint.

FileWhat it is
baseline.pngThe approved image.
current.pngThis run’s image.
diff.pngRed for changed pixels, amber for anti-aliasing ignored, gray for unchanged.
visual-report.mdThe numbers in plain text.
result.jsonThe same result as data.

What it cannot tell you

Pixels cannot tell a planned redesign from a bug, so a person approves each change: Approve this change in the Studio replaces one checkpoint’s image, and nexus baseline --update <TC_ID> does the same from the command line. Anything that changes on its own, such as a clock, an ad or an avatar, has to be masked or it will keep turning the comparison red.

Common questions

  • Does a visual regression fail my test?

    Not by default. The regression is recorded and the test keeps its verdict. Turn on NEXUS_VISUAL_STRICT and a regression past the tolerance fails the test.

  • Why is there a baseline per operating system?

    The same page renders with slightly different fonts, anti-aliasing and scrollbars on Windows, Linux and macOS. Comparing across them reports a change that is only the renderer, so each browser and platform keeps its own baseline.

  • How do I stop a clock or an ad from failing the check?

    Mask it with a CSS selector in the visual settings, or use an element checkpoint on the part of the page you care about. An element checkpoint is held to a tenth of the whole-page tolerance, so it is strict without a moving ad turning it red.

  • Where are the baseline images stored?

    In visual-baselines in your project folder, not in the evidence folder, so they can be committed with the code. The regression diffs are run output and live in visual-regression under evidence.

Run it on your own machine.

Try every feature free, on your own machines. No card needed.