Stabilization playbook

Status: Stable

Techniques that made a visual-regression suite reproducible, with the evidence behind each one. Every entry names the consumer it was proven in and the measurement that proved it. Read this before adding a mask, an ignore-list entry, or a wait.

The first question: does the fix change pixels?

Split every candidate stabiliser into two classes before considering it.

A timing stabiliser makes the capture wait longer for the page to reach the state it was always going to reach. Waiting for image decode, waiting for network idle, waiting for fonts. It moves the capture toward the settled render, but a longer wait also lets an animation the page never stops reach a later frame. A timing change therefore needs one run against the existing baseline before it ships.

A pixel-altering stabiliser makes the page render differently from the way a user sees it. image-rendering: pixelated, hiding an element, swapping an asset. It invalidates every existing baseline and puts a lie in the reference images, so it needs evidence that the instability is real and that no timing fix reaches it.

The library shipped one of each in consecutive candidates and conflated them. rc.6 set image-rendering: pixelated on every image to stop a scaler non-determinism that does not exist in Chromium 1055: six sequential captures of a 1600-to-1425 downscale, six of a 1220-to-740 downscale, six with the decode margin removed and eight concurrent full-page captures were all byte-identical. The property itself changed 367,580 pixels, 28.4% of a capture, at pixelmatch threshold 0. It was reverted in rc.7.

Measure before you attribute

A diff report names a file, not a cause. Three attributions in a component Storybook's investigation were wrong because they were read off filenames.

Localise the diff spatially first. Compute the bounding box of the differing pixels, then ask the DOM which element occupies that box. On pages-detail--report-page the box was x=194..1037 y=1034..1088, inside acme-highlight-box-grid — which identified the counter row in one step, after two wrong guesses from the story name.

Correlate across the whole run. A component Storybook's build 17 reported 3,658 failures in 12,288 captures. All 3,627 diff markers sat in image-bearing components (Detail page 430, Page intro 384, Home page 330, Image slideshow 265, Image 160); 2,080 captures of text-only components produced zero (Form components 0/1248, Table 0/160, CTA button 0/352, Accordion 0/224). A defect that partitions that cleanly has one cause, not eleven.

Check your instrument before believing a negative result. A throwaway shadow-DOM walk that stopped at depth 6 reported two distinct renderings where the shipped depth-10 walk found one. A local repro of the scaler claim used a browser that was not the CI browser, because Chromium 1055 does not launch on an arm64 Mac (spawn error -88) — run the experiment inside the CI image.

Compare against what the pipeline actually does. A determinism sweep that omits the config's maskSelectors measures raw rendering, not the suite. The same sweep reported one unstable story with masks omitted and nothing once they were applied.

Reduced motion reaches CSS, not JavaScript

The library sets the preference on every context and kills CSS animation unconditionally, in src/capture/screenshot.ts (preparePageForScreenshot) and at each context creation in src/core/browser-pool.ts and src/cli/executor.ts:

await page.addStyleTag({ content: `*, *::before, *::after {
  animation-duration: 0s !important; animation-delay: 0s !important;
  transition-duration: 0s !important; transition-delay: 0s !important; }` });
await page.emulateMedia({ reducedMotion: "reduce" });

That covers CSS animations and transitions and nothing else. A counter driven by requestAnimationFrame, an autoplaying video, a drag inertia loop and a SMIL animation inside a background-image SVG all ignore it, because none of them is a CSS animation.

The app has to read the preference for those. A CMS site does it with one helper and gates the JS at each site:

// frontend/source/assets/scripts/lib/prefers-reduced-motion.js
export const prefersReducedMotion = () => matchMedia('(prefers-reduced-motion)')?.matches || false;

// c-hero.js
this.playing = this.config.autoplay && !prefersReducedMotion();
startVideo() { if (prefersReducedMotion()) return; }

This is the preferred fix, because it is the accessibility behaviour the preference already asks for: the test runner gets determinism as a side effect of the app being correct, and no baseline records something a user never sees.

Two cases it does not reach. An animation isolated inside a background-image SVG cannot be gated from the host document. A CMS site swapped in a static per-theme SVG that shows a frozen frame, which keeps the affordance visible. And a component that freezes mid-animation rather than completing needs the end state forced, not the animation stopped: use-count-up with isCounting={onScreen} halts wherever it was when the element left the viewport, so three counters on one page of a component Storybook recorded 360/12/40 on one load and 65/2/7 on another — the same ~18% of each animation, frozen at a timing-dependent point. Waiting longer made it worse (4000ms settle gave 3 distinct captures of 4) because extra time only adds variance to when the freeze happened.

Masks

A Playwright mask paints over an element's bounding box. It stabilises content that changes inside a fixed box. It does not stabilise a box whose own size changes, because then the painted rectangle differs between runs and the masks become the diff.

Test the claim before recording it. A component Storybook's config carried a comment that said these masks did not work, over .report-number__counter; masking that selector gives one distinct capture in five, in every tenant theme tried. The evidence behind that comment concerned a different selector, .report-number-map__counter, whose box does change.

A CMS site made the same error in the other direction: masked regions came out magenta once the selectors reached the capture, so the option was never ignored — the selector list was being dropped upstream, which looked the same from outside.

When a mask appears not to work, confirm the selectors reach the capture before concluding anything about masking. Masked regions render magenta; if nothing is magenta, the list was dropped.

WebGL maps

A WebGL map keeps drawing after networkidle. In a Storybook suite, a map component came out half drawn in 3 of 20 captures, without labels or with a missing tile. The same map placed its markers only once it knew its size: the first marker appeared 100 ms after networkidle, at x = 60,050 px. The page looked finished to every wait the suite had.

The Storybook suite waits for the map itself. An init script counts WebGL draw calls. Before each screenshot the suite waits until every map has placed its markers and has not drawn for 750 ms, at most 10 s. With that wait, 20 of 20 captures of the map component matched the baselines in three runs in a row.

A CMS suite takes the other road. The component exposes its map instance, and the screenshot step waits for the map's idle event plus 750 ms.

@qs/design-tests has four waits: waitForNetworkIdle, waitForPageBusy (src/stability/page-busy.ts), waitForImageDecode and the fixed waitBeforeScreenshot (1000 ms by default). waitForPageBusy counts timeouts shorter than 7 s, requests in flight per URL and DOM mutations that clear on the next animation frame. None of the four looks at WebGL draws or at requestAnimationFrame. That is why a map can be half drawn after all of them. The library offers no map wait; a project that needs one adds it in its own harness.

Measurement. The script scripts/real-browser/webgl-wait.ts in the package repository (outside src/, not published) reproduces it. A fixture page on 127.0.0.1 draws one WebGL triangle on every requestAnimationFrame, and an init script wraps drawArrays and drawElements to record the time of the last draw. The colour fades in over a set time. Headless Chromium 112.0.5615.29 (Playwright 1.32.3, macOS arm64) created the WebGL context without flags. Each cell is five captures; "distinct" counts the different PNGs, so 1 means all five were byte-identical. Two runs of the script gave these counts, a range where they differ:

Variant	Library, waitForPageBusy on	Library, waitForPageBusy off	Bare Playwright, no reduced motion
Fade of 1.5 s	1 distinct	1 distinct	1 distinct
Fade of 6 s	1 to 2 distinct	1 to 2 distinct	2 distinct
Fade of 1.5 s that stops at once under prefers-reduced-motion	1 distinct	1 distinct	1 distinct
Loop that never stops	4 to 5 distinct	4 to 5 distinct	3 distinct

The library arms call captureScreenshot with waitBeforeScreenshot 1000 ms and the default pointer park; captureScreenshot sets reducedMotion: "reduce" itself, as in a real run. The bare arm is not a library run: it opens the page with reducedMotion: "no-preference", waits for networkidle and 1000 ms, and screenshots body. The count of distinct PNGs is a weak signal for the 6 s fade, because every run of one arm reaches the screenshot at nearly the same point of the fade. The time of the last draw is the stronger one.

• The page-busy check does not see the canvas. Polled every 500 ms while the 6 s fade drew, isPageBusy reported idle at each of 10 polls. waitForPageIdle returned idle after 1012 ms on the loop that never stops, while the canvas was still drawing.
• On the 6 s fade and the never-stopping loop, the last draw came 0 to 8 ms before captureScreenshot returned in every arm, so the screenshot landed on a moving canvas. The 1.5 s fade had stopped 222 to 1303 ms before the call returned (about 230 to 250 ms with waitForPageBusy off, about 1230 to 1300 ms with it on), because the scroll steps and waits that precede the screenshot already take longer than the fade. A fade shorter than that sequence needs no extra wait; a longer one does. The times include the screenshot itself.
• A loop that reads prefers-reduced-motion draws its final frame at once. The library arms gave 1 distinct capture for it, and the last draw came 1747 to 2771 ms before the call returned. Reduced motion helps a map only if the map library reads the preference. The page-busy check did not see the draws in any variant, with or without the preference, so a draw-call wait does not depend on it.

Conclusion: the existing waits settle a canvas only when it stops drawing before the capture sequence ends, and reduced motion stops it only when the map reads the preference. The measurement did not cover maplibre-gl, its reduced-motion option or tile loading from a server. One run of the Storybook suite's map component in its CI image, with and without its draw-call wait, would settle whether that map reads the preference.

Third-party iframes

A story that embeds a live survey from a third party changes without a change in the suite, so its captures flip. A Storybook suite met this with a third-party survey whose layout changed on the vendor's side.

Hide the iframe. Do not mask it. A Playwright mask paints a box over the element's bounds, so the capture holds a magenta box and the baseline stays tied to the size of that box. The screenshot option style (Playwright 1.41, release notes) with visibility: hidden hides the iframe and keeps its container, so the layout around it does not move. The Storybook suite hides the iframe this way.

@qs/design-tests does not pass the screenshot style option and runs Playwright 1.32.3, which lacks it. It offers two routes:

• stability.maskSelectors is a Playwright mask (see "Masks"). It paints the box.
• stability.styleOverrides sets inline styles with !important through el.style.setProperty and walks open shadow roots to depth 10 (applyStyleOverrides, src/capture/screenshot.ts). styleOverrides: [{ selector: 'iframe[src*="survey"]', styles: { visibility: "hidden" } }] gives the result of the style option.

styleOverrides alters pixels, so it belongs to the first class in "The first question" (pixel-altering): it needs evidence that the instability is real, and it invalidates the existing baselines of the story.

The cost: the story tests little afterwards, because the hidden iframe no longer proves that the survey renders. The Storybook suite accepts this.

Layout and viewport

Width selects behaviour, not only type scale. A CMS site captured at 601px, which sat 0.02px above a CSS bucket edge and produced up to 46% diff against 600px; and at 768px, where the search-results component's initial-view check matched mq-xsmall through an indexOf('small') test and forced list view. Widths moved to 390 / 1024 / 1440 / 1920, each well inside its bucket. A component Storybook's config documents the margin beside each width for the same reason.

Images without intrinsic size shift the page during scroll. A CMS site added imgWidth/imgHeight to 121 image objects across 11 stories and rescoped a selector that had left <img> at height: 0 until load; target drift during scroll went from 2,807px to 0.

Captures taller than the window

A component must not change its appearance during a capture. A capture taller than the window can show a component in a state that lasts one to two frames.

An element screenshot fires a resize event in the page when the element is taller than the window. Probe in Chromium 112.0.5615.29 (Playwright 1.32.3), window 1024×683:

Captured element	resize events
682 px element	0
685 px element	1
3000 px element	1
body of a 3000 px page	1

Transitions inside shadow roots

The stylesheet that stops transitions before a capture is added to the document, and a document stylesheet does not reach open shadow roots. In a second probe with the same versions, a child with transition: opacity 3s inside a shadow root was still at opacity 0.956 150 ms after the change, while the same transition in the light DOM stopped at once. document.getAnimations() does not list the shadow-root transition; el.getAnimations() on the element does.

A component Storybook's card slider story must hide its card actions. At the Tablet viewport the window is 683 px tall and #storybook-root is 684.45 px tall. The table lists how many of 8 captures showed the actions:

Window height	Captures with card actions visible
683 px	6 of 8
684 px	8 of 8
685 px	0 of 8
800 px	0 of 8

• A per-frame sampler during the root screenshot shows the same sequence in every capture: the actions at opacity 0, then all of them at opacity 1 about 90 to 150 ms into the capture, then back to 0 after 10 to 27 ms. innerHeight stays 683 px, and the media query that hides the actions stays true. A capture shows the actions only when the shot lands inside that flash.
• A long detail page of the same Storybook (4348 px at Desktop) flashes for 70 to 110 ms at a 683 px window and for 90 to 120 ms at a 960 px window. 7 of 8 and 8 of 8 captures show the actions.
• No library setting changed the result, and a plain Playwright script with a 683 px window reproduced it.
• The components fade the actions with transition: opacity inside their shadow roots. With transition: none forced on the actions, 32 of 32 captures were correct and the sampler showed no flash.
• What inside Chromium's capture beyond the viewport triggers the flash is not traced.

The runner stops transitions inside open shadow roots, to depth 10, before the waits and again right before the screenshot: it adds a stylesheet with transition-duration: 0s and transition-delay: 0s, both !important, to each root's adoptedStyleSheets. The step leaves animations untouched, because a duration of 0 can render the end state of an infinite animation instead of its start state. Closed shadow roots stay out of reach. Set stability.stopShadowRootTransitions: false to turn the step off.

The step also changes what a component that waits for transitionend shows. A transition inside a shadow root that starts between the first pass of the step and the shot does not run, so no transitionend event fires, and a component that sets its end state on that event stays in its intermediate state. In Chromium 112, a panel inside a shadow root that sets hidden on transitionend after a fade-out stayed in the layout at opacity 0 with the step on. With the step off, Playwright's animations: "disabled" finished the fade, transitionend fired, and the capture showed the panel hidden. Transitions in the light DOM behave this way since 1.0, through the document stylesheet; the step extends the behaviour to components with a shadow root. When such a component's capture differs from its baseline, set stability.stopShadowRootTransitions: false for the project, or force the end state with stability.styleOverrides.

The rule uses the selector *, whose specificity is 0, and comes last in each root. It stops a component's normal transition rules and an unlayered !important rule of the same specificity. It does not stop a component's other !important transition rules: a more specific selector such as .c { transition: opacity 1s !important } wins, and so does an !important rule inside @layer, because important declarations reverse the layer order. In Chromium 112 both kinds of transition still ran 150 ms after the change with the step on. For such a component, set transition: none on the element with stability.styleOverrides: the override is an inline !important style and stopped both kinds in the same probe.

The library repository holds a real-browser check, scripts/real-browser/shadow-root-transitions.mjs. Its fixture shows a child at once on resize and hides it with a 1000 ms fade inside a shadow root; the capture is a 1500 px root in a 683 px window. A sampler records the child's opacity and its running transitions in every animation frame, and the check asserts on those frames. In Chromium 112.0.5615.29 on macOS, with the step off, the first frame after the resize event showed the child at opacity 1 with one running transition in 24 of 24 captures, and 6 to 14 ms later the child was at opacity 0 again. With the step on, no frame of 24 captures showed the child above opacity 0 or a running transition. Whether the shot lands inside that short flash varies between runs, so the check reports the pixels as information only: with the step off, the child was in the pixels of 11 of 24 captures over two macOS runs and of 0 of 24 in a third.

A version without the option needs a style override on the affected selectors:

stability: {
  styleOverrides: [{ selector: ".card__actions", styles: { transition: "none" } }],
},

To diagnose a capture that shows a transient state, run it with DESIGN_TESTS_DIAGNOSTICS=true and compare capture.rootBox.height in the sidecar with viewport.height in its header. A root taller than the window fires a resize event in the page during the screenshot. A body or page capture records no rootBox; a page that scrolls is taller than the window. Then look for a component that reacts to resize or fades an element inside its shadow root.

Layout code reacts to the capture's resize

Layout code must not depend on the window size during a capture. A capture taller than the window resizes the window, and a layout that rebuilds itself on that resize can reach the capture in a transient state.

An element screenshot of an element taller than the window makes Chromium resize the window to 1×1 and back. In Chromium 112.0.5615.29 (Playwright 1.32.3), window 1024×683, root 1500 px:

• Without load, on macOS arm64 and in the Linux arm64 image of Playwright 1.32.3, the page receives one resize event about 130 ms into a shot of about 150 ms, and innerWidth reads 1024. A requestAnimationFrame sampler, a timer sampler, ResizeObserver, visualViewport and matchMedia never read a width below 1024. Media-query change events fire for every query, with the final value.
• Under load the page reads the narrow window. With CPU throttling through CDP and several pages that capture at the same time in one browser, the resize handler read innerWidth 1 in 0 to 1 of 8 shots per run (×8, 8 pages, shots of 0.1 to 0.6 s) and in 6 to 9 of 16 shots per run (×20, 16 pages, shots of 0.2 to 3.2 s). A second resize with width 1024 followed each time.
• A CMS site's card list computes its column count from the window width in JavaScript and rebuilds on resize, debounced by 250 ms, with a 500 ms width-probe fallback. In one verification run, 7 of 12 changed captures showed the list flipped between 2 or 3 columns and one narrow column.

Experimental: stable capture and its options stableCaptureMaxShots, stableCaptureInterval and stableCaptureFallback may change or be removed in a minor release; see Experimental options.

Stable capture (stability.stableCapture, off unless true) takes screenshots into memory until two consecutive ones are byte-identical, and writes the second one. It waits stableCaptureInterval (1000 ms) between the end of one shot and the start of the next, and stops after stableCaptureMaxShots (4) shots. 1000 ms covers the 250 ms debounce and the 500 ms width probe of the card list, with 250 ms for the rebuild itself. When no two consecutive shots match, the runner writes the last shot, reports the capture as successful and prints one line with the shot count. An error on shot 2 or later fails the attempt, as an error on a single shot does, and retries decide. With stableCaptureFallback: true (off unless set), the runner writes the last successful shot instead, reports the capture as successful, prints one line with the error, and the sidecar records matched: false and stableCapture.error. An error on the first shot always fails the attempt. Stable capture runs inside one attempt and never reads the baseline.

The library repository holds a real-browser check, scripts/real-browser/stable-capture.mjs. Its columns fixture computes columns = round(width / 300 px), at most 3. A resize to fewer columns rebuilds at once, and every resize restarts a rebuild debounced by 250 ms that applies its layout 500 ms later. A shot whose resize handler reads the 1×1 window therefore shows one column, and the columns return 750 ms after the restoring resize. On macOS arm64 at ×20 with 16 concurrent pages, 5 runs of 20 captures showed one column in 15 of 100 captures with the option off (4, 0, 6, 1 and 4 per run) and in 2 of 100 with it on. With the option on, 9 captures had a one-column shot: 7 ended after 3 or 4 shots with the columns restored, and 2 matched two consecutive one-column shots and wrote one column. At ×8 with 8 pages, no shot read the narrow window in 2 runs. The loop ends on any two identical consecutive shots, so two shots with the same transient end it too. Whether stable capture removes the card-list flip of a real suite is not measured.

To diagnose a capture that shows a transient layout, run it with DESIGN_TESTS_DIAGNOSTICS=true:

1. Compare capture.rootBox.height in the sidecar with viewport.height in its header. A root taller than the window fires a resize event in the page during the screenshot.
2. Read stableCapture.shots and stableCapture.matched in the attempt's entry. Two shots and matched: true mean the page did not change between the shots. More than two shots mean the page changed after the first shot and settled. matched: false means the page did not settle within stableCaptureMaxShots shots, and the file holds the last shot. matched: false with stableCapture.error means a later shot failed and stableCaptureFallback kept the last successful shot; stableCapture.shots counts the successful shots.
3. With DEBUG=capture:stable, the runner logs the size of each shot, whether it matched the previous one, and the result.

The options:

Option	Cost
Fix the page: compute the layout in CSS (grid auto-fill, media or container queries), or ignore a resize that reports a width below the smallest supported viewport	None in the run
Set stability.stableCapture: true with the defaults	At least one extra shot and one 1000 ms interval per capture; total run time not measured
Raise stableCaptureInterval or stableCaptureMaxShots for a page whose rebuild takes longer than 1000 ms	Each extra shot and interval per capture; not measured

Without stability.stableCapture: true the runner takes one screenshot. A retry repeats the shots of its own capture, so a suite with retries.onDiff pays the cost per retry.

A suite without stable capture needs a fix in the page. A waitBeforeScreenshot longer than the rebuild does not help: the wait ends before the screenshot, and the screenshot itself fires the resize.

Stable capture is off by default, for two measured reasons:

• In a Storybook suite, a second element screenshot re-rasterises backdrop-filter layers differently in headless Chromium 112. Shot 1 is correct; shots 2 and 3 are equally broken and match each other, so stable capture writes the broken render: glass buttons lose their text and a blurred hero veil smears. Of 12,408 captures, 658 changed bytes against a run with stable capture off, and 14 Mobile captures differed by 5 to 26 %.
• In a CMS suite with the card-list flake, stable capture added 7 % to the run time and showed no measurable reduction of the flake, with 3 runs per arm.

Turn it on for a page that rebuilds on resize and has no backdrop-filter, and compare a run with it on against a run with it off before you rebase baselines.

Stable capture is not a fix for a flash inside a shadow root. The check's shadow-root fixture fades a red block for 30 ms after each resize, and runs with stopShadowRootTransitions: false. On macOS arm64 at ×20 with 16 pages, 5 runs of 20 captures showed the block in 25 of 100 captures with the option off and in 21 of 100 with it on. 19 of the 21 were written after two consecutive shots that held the block at the same opacity and matched. At ×8 with 8 pages, 2 runs showed the block in 4 of 40 captures with the option off and in 16 of 40 with it on, all 16 after a match. A flash with the same pixels in two consecutive shots passes stable capture.

Per-run state

Clear web storage in an init script, before navigation, not after. A CMS site's search-results component persists grid-or-list under acme-navigation-history and reads it during its own init; with 32 parallel workers the story order varies per run, so the same story rendered grid in one run and list in the next — 52% diff, no code change.

Strip variable text out of error placeholders. Timestamps, retry counters, UUIDs and file positions in a failure image make every identical failure a fresh diff.

Time

A story that derives a date from today changes with the calendar, so its baseline expires. A Storybook suite and a CMS suite each fixed this, in two different ways.

Road 1: mock the browser clock. A Storybook suite pins the clock to 28.10.2025 12:00 Europe/Zurich. It calls page.clock.install({ time }) when it creates the context and setSystemTime with the same instant before each story. The clock then runs from the pinned instant. Do not use clock.setFixedTime: it freezes Date, and Highcharts then renders its columns at height 0. The suite reproduced this with Playwright 1.55.1: the consumption chart of one Storybook stayed empty in all five viewports under setFixedTime, and rendered like the real clock under install plus setSystemTime. A second Storybook stayed pixel-identical under both.

A baseline taken with the real clock can hide a mistake of its own. In the same suite, the baseline of the consumption chart at the xs viewport had caught the Highcharts animation before its end, so its columns were slightly short. Once two runs agreed with each other, the suite replaced the baseline with the run's image. If two runs agree with each other and disagree with an old baseline, test the old baseline before the runs.

Road 2: inject the time. A CMS suite gives each component that reads the time a timestamp prop and sets it in the story. The component is deterministic in every environment, also in unit tests, and nothing in the browser is faked. The price is one prop on every component that reads the time. A component that forgets the prop falls back to the real clock, and nothing reports it.

Choose by ownership. Pin a running clock for a suite, because it covers every component, third-party code included, without a change to product code. It also fakes Date, timers, performance.now and requestAnimationFrame for the whole page. Inject the time into components that the team owns, because the fix then holds outside the design tests too.

page.clock exists from Playwright 1.45 (release notes, "Clock"). @qs/design-tests pins Playwright 1.32.3 in its own development and accepts >=1.32.3 <2, and neither the CLI nor the runner offers a hook that hands the page to consumer code. The library neither pins nor fakes the clock. A suite that owns its Playwright harness can call the exported captureScreenshot and waitForPageIdle, which take a Playwright Page, and so can install the clock first. The clock road applies to such a suite on Playwright 1.45 or later.

Pointer

Playwright starts the pointer at (0, 0) on every new page, on the element in the top-left corner. Headless Chromium applies no :hover state until a mouse event arrives. In Chromium 112 (Playwright 1.32.3), on Linux and on macOS, the top-left element of a test page stayed unhovered after navigation, a scroll down and back up, a viewport change and an element screenshot, in three runs of each. Eight captures through the CLI showed no hover either. One page.mouse.move(0, 0) hovered the element every time, and a move to (-1, -1) after it removed the hover every time.

A mouse event at the start position therefore hovers the top-left element. The runner moves the pointer to (-1, -1) after navigation and before the waits, so no element is under it, and the waits absorb the repaint that the move causes. In a headless run that sends no other mouse event, the move changes no pixels. A headed run was not measured: there the real pointer of the screen can also reach the page. Set stability.parkPointer: false to leave the pointer at (0, 0).

A Storybook suite found the same cause in its own CI image. The pointer rests at (0, 0) on the top-left element of a story, and Chrome applied that element's :hover depending on timing. A card component there zooms its photo to 1.05 under :hover. The baseline of one story was hovered and the baseline of another was not, and each flipped in some runs. With the pointer at (10, 10) the first story differed from its baseline by 0 px, and with the pointer outside the card by 491,670 px. The second story differed by 484,654 px in one measurement and by 32 px in another. The suite parked the pointer at (-1, -1) before each story. That changed formerly hovered baselines once: 52, 60, 20 and 5 images in four Storybooks. Afterwards every job passed twice in a row. A second consumer reported the same pointer defect on a card component.

The Chromium 112 measurement above and the Storybook suite's observation differ: Chromium 112 hovered nothing without a mouse event, and the Storybook suite saw hover. That suite ran its own CI image, and this page does not know the Chromium version of that image. The park costs one mouse move and changes nothing where no hover occurs.

A hover state that flips between headless runs has a cause other than the pointer. One pointer hovers one chain of elements only, so a hover style on several sibling cards at once comes from somewhere else. Check these sources:

• A class that a scroll listener sets. The capture scrolls down and back up, and a header that turns opaque on scroll can stay opaque if its listener runs late (see stickyClassSelectors).
• A media query that gates the hover style, such as (hover: hover) and (pointer: fine). Log matchMedia(...).matches at capture time for each viewport.
• Focus inside the element, which :focus-within and :focus-visible styles show.

Lists that load from an absolute https URL

Symptom. A listing page shows its spinner in the baseline and in every run. The suite reaches the site over http://, and the page's own code calls an absolute https:// URL of the same host.

Diagnosis. Run with DEBUG=pw:api and look for a request to the https:// URL that fails with net::ERR_SSL_PROTOCOL_ERROR: the target serves no TLS, so the request never leaves the browser and the server logs nothing. Then set DESIGN_TESTS_DIAGNOSTICS=true and read requestRewrites in the .diag.json sidecar (see Request rewrites).

The rule. requestRewrites sends the page's requests for from to to:

requestRewrites: [
  {
    from: "https://www-test.example/",
    to: "http://www-test.example/",
    headers: { "X-Requested-With": "XMLHttpRequest" },
  },
],

Why the rule carries X-Requested-With. The rewritten request stays cross-origin for the browser, because the scheme differs from the page's. jQuery sets X-Requested-With only on a same-origin request, so the page's code omits it, and a CMS endpoint that checks the header answers 403 without it. The library adds no header the rule does not state. In a real-browser run on Playwright 1.32.3 (scripts/verify-request-rewrite.mjs), the same rule without headers produced the 403 outcome in 5 of 5 captures; with the header, 5 of 5 captures showed the rows.

Cache bypass. A rewritten request does not go through the network cache. The server sees one request per capture, and the page's other requests stay cached.

Limits. The rule applies to every request type whose URL starts with from; write a longer from to narrow it. A redirect response (3xx) reaches the page unchanged, so the page's redirect handling decides. If the fetch of the new URL fails, the original request continues and the page sees the failure it saw without the rule. Not verified: cookies on the rewritten request, playwright.proxy, service workers, and large or streamed responses.

The baseline changes on purpose. A page that showed a spinner now shows its rows, so the baseline of every affected page differs once. Check that the rows, not the spinner, are in the new capture before baseline-update.

Retry

A retry captures the page again and compares the new capture; it never compares the same two images twice, because pixelmatch is deterministic. Timeouts, HTTP errors, capture failures and size mismatches are retried. A completed pixel comparison that differs is retried only where the project sets retries.onDiff, because a CMS site measured the cost of retrying every difference at 49 deterministic diffs times two retries times about 12 seconds. A component Storybook's Pages/Home "Interactive Home" differs on a different set of tenants in every run while two Storybook builds produce byte-identical captures of it, so the capture is the variable there and only a second capture settles it. The loop stops as soon as two attempts write the same bytes, so a deterministic difference costs one extra capture. A test that passes after a retry is reported as flaky, with its URL, so the fix starts from the report.

Artifact footprint

Archiving is part of stability, because a full controller fails builds that captured perfectly. A component Storybook's build 18 completed all 12,288 captures and then failed with No space left on device during archiveArtifacts: a run archives about 9GB onto a 40GB volume shared with every other job on the instance. Bound retention per job, and prefer serving images from somewhere other than the controller.