CLI Reference
Status: Stable
The design-tests binary runs the suite, promotes screenshots into the baseline, validates the configuration, and builds the report. Every command takes -c, --config <path>, which defaults to .designTests.js, then .designTests.ts, resolved against the working directory. Run it through your package manager so the local install is used:
pnpm exec design-tests --version
pnpm exec design-tests <command> --help
run
Capture every test of the selected projects, compare them against the baseline, and write the report.
pnpm exec design-tests run --config .designTests.js --all
| Option | Description |
|---|---|
-c, --config <path> | Path to the configuration file. Defaults to .designTests.js, then .designTests.ts. |
-p, --project <key> | Project key to test. The PROJECT or DESIGN_TESTS_PROJECT environment variable sets the same value. |
--all | Run every project in the configuration. |
--workers <number> | Number of parallel browser workers. Overrides playwright.workers (default 1). |
--set <key.path>=<value> | Override any configuration value. Repeatable. |
--debug | Headed browser and verbose logs. |
--fail-on-changes | Exit 1 on visual differences and 2 on capture failures. Without it the command exits 0 whenever the run completes. |
Select one project or all of them — a run with neither --project nor --all has nothing to do.
# One project, eight workers, headed
pnpm exec design-tests run --config .designTests.js --project portal --workers 8 --debug
# Project by environment variable
PROJECT=contracts pnpm exec design-tests run --config .designTests.js
# Override configuration values
pnpm exec design-tests run --config .designTests.js --all \
--set projects.cms.url=http://localhost:3000 \
--set playwright.workers=4
A run writes screenshots to actualDir, diff images to diffDir, and reg.json, index.html, manifest.json and CURRENT.json to reportDir. It never writes into baselineDir: a test without a baseline is reported as NEW, and its screenshot stays in actualDir until baseline-update promotes it.
The console shows one 📸 line per test (OK, DIFF, FAIL, NEW!), the capture platform (🖥️ Capture platform: linux-x64, Chromium 112.0.5615.29, Playwright 1.32.3, Node v24.0.0), a warning when BASELINE.json records a different platform, the Tests complete: summary, the 🔗 Non-Passed Test URLs list, and a final 🚪 Exit line naming the exit code and the mode.
Exit codes
| Code | Without --fail-on-changes | With --fail-on-changes |
|---|---|---|
0 | The run completed and the report was written, with or without differences. | Every test passed. |
1 | — | At least one visual difference, and no capture failures. |
2 | The run could not complete: a crash, a configuration error, or no tests. | A capture failure, or no tests. |
Use the default for local work, where the report is the output. A pipeline that reads reg.json or manifest.json for the build status runs without the flag. A pipeline that wants the exit code to carry the verdict passes it.
baseline-update
Promote the last run's new and changed screenshots into the baseline, and write BASELINE.json.
pnpm exec design-tests baseline-update --config .designTests.js --dry-run
pnpm exec design-tests baseline-update --config .designTests.js
pnpm exec design-tests baseline-update --config .designTests.js --project cms
pnpm exec design-tests baseline-update --config .designTests.js --prune
| Option | Description |
|---|---|
-c, --config <path> | Path to the configuration file. Defaults to .designTests.js, then .designTests.ts. |
-p, --project <key> | Promote only this project's items. |
--prune | Delete the baseline images the run reported as deleted. Use it only after an --all run, because a partial run reports every uncovered project as deleted. |
--allow-platform-change | Accept a run whose capture platform differs from the one BASELINE.json records. |
--dry-run | Print the plan and write nothing. |
The command reads reportDir/manifest.json and copies each new and failed item from actualDir to baselineDir. It never copies errors items: those are tests where no screenshot of the page could be taken, and their slot may hold an error screenshot. BASELINE.json receives the run's CURRENT.json content — deployment reference, os, arch, browserVersion and playwrightVersion — plus updatedAt.
The command exits 2 and writes nothing when no manifest exists, or when the run was captured on another platform or browser than the baseline and --allow-platform-change is absent. The refusal names each differing field.
baseline-update does not run git. A pipeline runs baseline-update, then commits and pushes baselineDir itself.
config
Validate the configuration and print what it resolves to, without launching a browser.
pnpm exec design-tests config --config .designTests.js
pnpm exec design-tests config --config .designTests.js --set playwright.workers=4
DESIGN_TESTS__PROJECTS__CMS__URL=http://localhost:3000 pnpm exec design-tests config --config .designTests.js --json | jq .projects.cms.url
| Option | Description |
|---|---|
-c, --config <path> | Path to the configuration file. Defaults to .designTests.js, then .designTests.ts. |
--set <key.path>=<value> | Apply an override, as run would. Repeatable. |
--json | Print only the effective configuration. |
The default output names the projects, viewports and worker count, and lists every DESIGN_TESTS__* variable and --set flag with the path and value it set. It warns about variables that start with DESIGN_TESTS_ but reach nothing, then prints the configuration as JSON with functions rendered as [Function name]. A validation error exits 2, with the overrides still listed.
report
Generate reg.json and the HTML diff report from the images already on disk. run writes the report itself; this command rebuilds it without capturing.
pnpm exec design-tests report --config .designTests.js
| Option | Description |
|---|---|
-c, --config <path> | Path to the configuration file. Defaults to .designTests.js, then .designTests.ts. |
-p, --project <key> | Project key, used to resolve directories. |
-o, --output <path> | Output directory. Defaults to reportDir from the configuration. |
--url-prefix <prefix> | Prefix prepended to image URLs in the report. Defaults to jenkins.baseUrl. Needed when the report is served from a path that differs from where it was written — see Jenkins Resource Root URL. |
debug
Registered but not implemented. It prints a notice and exits 0. Use run --debug instead, which opens a headed browser and turns on verbose logging.
Configuration overrides
Every configuration value is overridable from outside the file, in this precedence:
--setflagsDESIGN_TESTS__*environment variables- The configuration file
- Built-in defaults
Environment variables use __ to separate path segments and are case-insensitive. A single _ inside a key marks a camelCase boundary, so STABILITY__WAIT_BEFORE_SCREENSHOT sets stability.waitBeforeScreenshot:
DESIGN_TESTS__PLAYWRIGHT__WORKERS=8 pnpm exec design-tests run --config .designTests.js --all
DESIGN_TESTS__PROJECTS__CMS__URL="http://localhost:3000" pnpm exec design-tests run --config .designTests.js --project cms
Values are coerced to the type the schema expects and then validated, so a bad type or an invalid URL fails immediately with a message naming the key. design-tests config shows which overrides applied.
Set environment variables inline on the command rather than exporting them. An inline assignment keeps the command reproducible and scoped to that one run.
Environment variables
| Variable | Read by | Effect |
|---|---|---|
DESIGN_TESTS__<PATH>__<KEY> | run, report, config, baseline-update | Override a configuration value, as described above. |
PROJECT, DESIGN_TESTS_PROJECT | run | Project key when --project is absent. |
DESIGN_TESTS_FORCE_HEADLESS=true | run | Headless even with --debug, for containers without a display. |
DESIGN_TESTS_DIAGNOSTICS=true | run | Write a .diag.json sidecar beside each screenshot with the page's state at capture time; see Diagnostics sidecar. |
CURRENT_REFERENCE, CURRENT_REFERENCE_LINK | run | Deployment reference when /git-metadata.json is not served. |
ENABLE_NETWORK_CACHE=true, NETWORK_CACHE_DIR | run | Experimental network cache; see Network Cache. |
DEBUG=capture:fs,comparison:fs,report:fs | all | Trace file operations through the debug package. |
design-tests config lists the variables that are set and read, and names those that are set but read by nothing.
Diagnostics sidecar
With DESIGN_TESTS_DIAGNOSTICS=true, run writes <screenshot>.diag.json beside each screenshot in the actual directory. Each attempt reads the page after the stability waits, immediately before its screenshot. The probe only reads the page, so it changes no pixels, and a probe that fails does not fail the capture.
| Field | Content |
|---|---|
runStartedAt, testId, url, viewport | The run and the test the sidecar belongs to. |
attempt | The attempt that the top-level fields below describe: the last one that read the page. |
rootFontSize, htmlClass, innerWidth, clientWidth | The computed html font size, the html classes, and the window and layout widths. |
grids | For each match of diagnostics.gridSelectors, the laid-out column count. |
images | How many images the page holds, how many have no intrinsic size yet, and how many lack width or height attributes. |
mediaQueries | Each query of diagnostics.mediaQueries with its matchMedia result. |
classes | For each selector of diagnostics.classSelectors, the tag and the classes of its first 10 matches. The lookup includes open shadow roots. |
capture | method names the screenshot taken: root (#storybook-root), viewport (a root with no area), body or page. rootBoxAtLoad is the root's box that decided the method, measured after the first network-idle wait; rootBox is its box at capture time. |
attempts | One entry per attempt of the current run, ordered by attempt, each with the fields above. An attempt that threw during the capture has an error field. An attempt that captured and then differed from its baseline has the fields only. A retry follows only an attempt that did not pass, so every entry before the last belongs to such an attempt. |
Configure the probes in the global diagnostics block or in a project's; a project's list replaces the global one:
diagnostics: {
mediaQueries: ["(min-width: 900px) and (hover: hover) and (pointer: fine)"],
classSelectors: ["header.navbar", ".card-slider"],
},
A sidecar left in the actual directory by an earlier run is replaced, not extended: an attempt joins attempts only when the sidecar's runStartedAt is that of the current run. baseline-update promotes no sidecar.
Related
- Getting Started — install, first config, first run
- Adding Projects — multi-project configuration
- Baseline Management — reviewing and updating baseline images