Glossary

Status: Stable

This glossary defines the terms a person or an agent meets when working on @qs/design-tests and on the design tests of its consumers. The file groups entries by subject and orders them alphabetically within each group. A definition that names another term refers to that term's entry.

Artefacts and directories

Actual: The image the current run captures. The runner writes it to the actual directory, design-tests/actual-images/ by default, under the same relative path as its Baseline.

Baseline: The reference image, committed to the repository. The default directory is design-tests/baseline-images/; a config that names no directory reads design-tests/, not tests/. The component Storybook and the CMS site described in these docs both set ./design-tests/... explicitly. A Baseline is valid only for the Capture platform that rendered it.

BASELINE.json: The record of the commit the Baseline came from and of the Capture platform (os, arch, Chromium version, Playwright version) that rendered it. design-tests baseline-update writes it to the baseline directory during Promotion. A run compares it with CURRENT.json and warns when the two platforms differ.

CURRENT.json: The same record as BASELINE.json for the run that has completed. design-tests run writes it to the Report directory.

Diff: The generated image that marks the pixels that differ between a Baseline and its Actual. The runner writes it to design-tests/diff-output/ by default.

Error screenshot: The capture the runner writes when a page fails to load. Its item belongs in the errors bucket of manifest.json. It never becomes a Baseline: Promotion skips the errors bucket.

manifest.json: The library's own result file, written to the Report directory. It holds the buckets passed, failed, new, errors and deleted, a summary with one count per bucket plus flaky and total, and the current and baseline metadata. Schema version 1.1.0 added summary.flaky and each item's retryCount; 1.2.0 added the errors bucket and summary.errors. manifest.json separates a failed comparison from a page that did not load; reg.json does not.

Promotion: Copying an Actual over its Baseline. design-tests baseline-update promotes the new and failed items of manifest.json and skips the errors items. It refuses a promotion across a Capture platform change unless the caller passes --allow-platform-change.

reg.json: The result file reg-cli reads to build the HTML report. It lists failedItems, newItems, deletedItems and passedItems. Jenkins reads it to set the build status, UNSTABLE or SUCCESS, and does not read the Exit codes.

Report: The directory design-tests/report/ by default. It holds the HTML report reg-cli generates, reg.json, manifest.json and CURRENT.json. The HTML report references images by relative path, so the archive step must include the Report, Baseline, Actual and Diff directories together.

Comparison

Flaky versus intentional: A changed capture is either a real design change or a nondeterministic capture. Neither a person nor a model can separate the two by looking at the images. Capturing the same item twice in one run separates them: two different images prove flakiness.

Per-pixel tolerance: The config value comparison.pixelmatch.threshold. It sets how different one pixel may be before pixelmatch counts it as changed. The default is 0.1, which is pixelmatch's own default. It is a separate setting from the Rate threshold: a config that sets comparison.threshold: 0 and expects a pixel-exact comparison still compares at 0.1 per pixel unless it also sets comparison.pixelmatch.threshold: 0.

pixelmatch: The in-memory image diff library the runner uses for the per-capture comparison. It reads the Per-pixel tolerance and produces the Diff.

Rate threshold: The config value comparison.threshold. It sets what fraction of a capture's pixels may differ before the capture fails. The component Storybook and the CMS site both set 0. It does not set the Per-pixel tolerance.

reg-cli: The tool that generates the HTML report from the Baseline, Actual and Diff directories and writes reg.json. It references images by relative path, so the archive step must include the Report, Baseline, Actual and Diff directories together. It compares with the same Per-pixel tolerance as the runner; see Two-tolerance defect.

Retry policy: resolveRetryPolicy() resolves a story's own retries parameter first, then falls back to retries.default, 2 by default. Both the sequential and the parallel path run captures through runWithRetries(). A pass is final. A completed comparison that differs is also final, unless retries.onDiff is true. Every other result (a capture failure, an HTTP error, a dimension mismatch) is retried, and the loop stops early when two attempts capture identical bytes. Diff retrying is off by default: commit b581cbc turned it off after build 74 of the CMS suite spent about 20 minutes re-capturing 49 deterministic diffs. retries.onDiff turns it back on for a project whose diffs are nondeterministic. The schema rejects playwright.retries and names retries.default as its replacement.

Two-tolerance defect: The state in which the runner and the HTML report read different Per-pixel tolerances. One run of the component Storybook reported 173 changed captures in manifest.json and 1142 in the HTML report from the same images, because executor.ts resolved an unset tolerance to 0.1 and html.ts resolved it to 0. resolvePixelmatchThreshold() in src/core/constants.ts is the only reader of the default, so both paths compare at the same tolerance.

Capture stability

Capture platform: The operating system, CPU architecture, Chromium version and Playwright version that rendered a capture. Font rendering and subpixel anti-aliasing differ between macOS, Windows and Linux, so a Baseline is valid only for the platform that rendered it. BASELINE.json and CURRENT.json record the platform.

Compositing layer promotion: The effect of translate3d on an element: the browser moves the element onto its own compositing layer. The browser evicts the layer's raster cache under memory pressure, so a downscaled photograph inside the layer rasterises differently between runs. A Style override that forces transform: none on the element removes the promotion. The tiny-slider and .slideshow__slider elements in the component Storybook show this effect.

Frozen counter: A use-count-up counter with isCounting={onScreen} stops at its current value when the element leaves the viewport, instead of completing. While isCounting is false, the hook reports its start value, so gating alone renders 0. The component must render the end value under prefers-reduced-motion.

image-rendering: pixelated: A CSS property the library does not set. Version 1.0.0-rc.6 set it against a Chromium scaler nondeterminism that does not exist at the Playwright version the consumers pin; the property changed 28.4% of the pixels in any capture that carries a scaled image, and 1.0.0-rc.7 removed it.

Mask: A Playwright screenshot option, configured through stability.maskSelectors, that covers the bounding box of each matched element before capture. A box that changes size between runs still produces a Diff, so a Style override or an application change has to stabilise the content inside a fixed box instead.

Pointer parking: The capture step that moves the mouse pointer to (-1, -1), outside the viewport, after navigation and before the waits. Playwright starts the pointer at (0, 0), and headless Chromium hovers nothing there until a mouse event arrives, so the step changes no capture of a headless run that sends none. With the pointer outside the viewport, a mouse event at its position hovers no element. stability.parkPointer: false turns it off.

prefers-reduced-motion: A media query the runner sets to reduce. It reaches CSS animations and transitions only. A requestAnimationFrame counter, an autoplaying video and a SMIL animation keep running, so the application has to read the preference itself and stop them.

Shadow root: The root of a shadow DOM subtree. page.locator pierces an open shadow root; document.querySelectorAll does not. A closed shadow root returns null from .shadowRoot, and no selector reaches it.

Style override: An entry in the stability.styleOverrides config: a selector and a set of CSS properties. The runner applies each property with el.style.setProperty(prop, value, "important") before capture. The element lookup walks open Shadow roots to depth 10. A Style override alters pixels, so it changes the Baseline of every capture it matches.

Timing versus pixels: The rule that classifies a library change by its effect on captures. A change that alters only when the runner takes a capture leaves every Baseline valid. A change that alters what the browser renders invalidates every consumer's Baseline and ships as a major version.

Story selection

ignoreList: A per-project list of story id prefixes. The runner removes every Story whose id starts with one of the prefixes. It applies after includeList.

includeList: A per-project list of story id prefixes. The runner keeps only the stories whose id starts with one of the prefixes, then ignoreList removes entries from what survives. ignoreList applies after includeList, so a Story a project has excluded stays excluded.

Story: One Storybook entry with type: "story", identified by an id such as slideshow--with-many-images. A docs entry carries type: "docs" and is not a Story. The runner checks type, not the id suffix, because an MDX file can produce a story id that ends in --documentation.

Viewport: A named width and height the runner captures each Story at. The component Storybook captures Desktop and Wide among other viewports.

Storage

Git LFS pointer: A 130-byte text file that git stores in the working tree in place of a binary. It begins with version https://git-lfs.github.com/spec/v1. A capture compared against a pointer reports as changed.

GIT_LFS_SKIP_SMUDGE: The documented environment variable that leaves Git LFS pointers in place at checkout, equivalent to the Smudge filter set to --skip. The Jenkins git-client plugin launches git with an environment that does not include the declarative pipeline environment block, so the variable does not reach the checkout.

Repository size: The size Bitbucket measures against its 4 GB cap. The cap counts git objects; LFS objects have a separate quota. git lfs migrate import on the component Storybook's baselines took the repository from 4.13 GiB to 1.8 MB, with 11020 baselines in LFS.

Smudge filter: The git filter that turns a Git LFS pointer into the real file on checkout. filter.lfs.smudge and filter.lfs.process set to --skip leave pointers in place. git lfs pull writes content through those filters, so a skipped filter also stops the pull from writing any file.

Pipeline

core.hooksPath=/dev/null: A git config value the Jenkins git plugin sets so a checked-out repository cannot run hooks during a build. git lfs install writes hooks and fails against it with mkdir /dev/null: not a directory. Setting filter.lfs.clean, filter.lfs.smudge, filter.lfs.process and filter.lfs.required directly configures LFS without the hooks.

Declarative: Checkout SCM: The checkout Jenkins runs to obtain the Jenkinsfile. It precedes everything the Jenkinsfile declares, including its environment block. skipDefaultCheckout() turns it off so a Checkout stage can run in its place.

Exit codes: design-tests run exits 0 by default once the run completes and writes the Report. With --fail-on-changes, it exits 1 when the run detects diffs and 2 on a capture failure; either mode exits 2 on an empty run. The last console line names the code and the mode. Jenkins reads reg.json for the build status, not the exit code.

git plugin timeout: A timeout the Jenkins git plugin states in minutes. A log line that reads # timeout=10 means ten minutes.

Registries: quatico-npm receives published releases, quatico-npm-snapshots receives Snapshot versions, and quatico-npm-proxy is the registry consumers resolve from.

Release candidate: A version that publishes under the next dist-tag. It carries no compatibility promise and consumes no changesets.

Snapshot version: A version of the form 1.0.0-SNAPSHOT.N. The pipeline reads the increment N from the npm registry and records it as a git tag. When the registry loses the package, the counter restarts at 1 while git keeps its tags, and the tag step fails; one such loss left git with 23 tags against a counter restarted at 1.