Repo-history example

`setup-buildx-action`: one builder-argument change the repo's own checks missed.

In a measured replay of six real first-parent commits from docker/setup-buildx-action, the same selected checks passed in standard CI every time. DriftFence stayed quiet through three commits, then flagged one builder-argument change and stayed red on the later first-parent merges.

With approved builder behavior fixed in Git, DriftFence would have started flagging the moment unknown drivers stopped receiving the default buildkitd entitlement flags.

See the first flagged change Back to results summary

First divergence

Unknown drivers stopped receiving default `buildkitd` flags.

The first three commits in this window stayed conforming. On the fourth commit, the action stopped appending the default insecure-entitlement flags for unknown drivers, while the same selected checks still passed in standard CI.

Blocked case

Unknown drivers stopped receiving default entitlement flags.

In the approved baseline, the selected unknown-driver path still forwarded the default --allow-insecure-entitlement flags. After the feature landed, the action stopped forwarding those flags for unknown drivers while keeping the same docker-container and remote controls.

DriftFence report

Scenario

docker.setup-buildx-action.context.unknown-driver-buildkitd-flags

Expected

output.buildkitdFlagsForwarded = --allow-insecure-entitlement security.insecure

Observed

output.buildkitdFlagsForwarded = null

Classification

violating

The nearby docker-container and remote controls did not trigger DriftFence on the same six-commit slice.

Across the replay window

Three quiet commits, then one clear builder-argument onset.

This replay is stronger than a recurring-all-red window because the onset comes after three quiet commits on the same selected test surface and keeps two matched quiet controls.

Quiet commits first

Three toolkit bumps stayed conforming

Three consecutive @docker/actions-toolkit bumps stayed quiet.
None of them changed the protected unknown-driver flag behavior or the nearby control behavior.
The selected builder-argument surface stayed conforming until the feature merge landed.

Then the onset

One feature stopped forwarding the default flags

a56031a merged context: only append flags if we know the driver supports them.
The action stopped appending the default buildkitd flags for unknown drivers.
DriftFence flagged that change while the same checks stayed green.

Nearby controls

Two comparison cases stayed quiet

The standard docker-container driver kept forwarding the default flags across the whole slice.
The remote driver kept forwarding only its endpoint without the flags.
Both controls stayed quiet on the same selected test surface.

Method

How it was measured.

This replay fixes one builder-argument baseline, reruns the same selected tests on each commit, and counts the onset once because only one scenario diverged in the window.

Replay size: 6 commits
Fixed workflows: unknown-driver default flag forwarding, the standard docker-container path, and the remote driver endpoint control
The same selected checks reran on every commit in the window
DriftFence reran on every commit with the same approved baseline
The default follow-up window contained one onset unit and no later reconciliation

Source material

Method and source files.

The links below show the benchmark notes and pinned replay plan for this Docker builder example.

Measurement notes

The benchmark results log records the completed setup-buildx-action window, including the replay size, outcome mix, and first flagged change.

Benchmark results

Fixed test plan

The replay definition fixes the three selected builder workflows and the exact June to July 2023 replay window used here.

See the product, results, pricing, and trust pages alongside this Docker builder example.