Repo-history example

`metadata-action`: one annotations change the repo's own checks missed.

In a measured replay of six real first-parent commits from docker/metadata-action, the same selected checks passed in standard CI every time. DriftFence stayed quiet through five commits, then flagged one annotations change.

With approved metadata behavior fixed in Git, DriftFence would have started flagging when default OCI annotations began mirroring generated label values and custom annotation inputs started overriding the earlier null description.

See the first flagged change Back to results summary

First divergence

Default and custom annotation values changed on the same merged commit.

The first five commits in this window stayed conforming. On the sixth commit, default OCI annotations started mirroring generated label values and explicit annotation input started overriding the earlier null description, while the same selected checks still passed in standard CI.

Blocked case

Default annotation values stopped staying null.

In the approved baseline, the selected annotation path kept annotationVersion unset even while the generated label version stayed dev. After custom annotations support landed, the same annotation value started mirroring the generated label.

DriftFence report

Scenario

docker.metadata-action.meta.default-annotations

Expected

output.annotationVersion = null

Observed

output.annotationVersion = "dev"

Classification

violating

On the same merge, the custom-annotations path changed output.annotationDescription from null to "this is a \"bad\" example". The existing labels-control case did not trigger DriftFence on the same six-commit slice.

Across the replay window

Five quiet commits, then one clear metadata-annotation onset.

This replay is stronger than a recurring-all-red window because the onset comes after five quiet first-parent merges on the same selected test surface and keeps the label-forwarding control quiet.

Quiet commits first

Five earlier merges stayed conforming

Split bake files, an alternate annotations branch, a toolkit dependency bump, empty-image handling, and README documentation all stayed quiet.
Even the earlier annotations-alt merge stayed conforming on the selected default/custom annotation paths.
None of those merges changed the protected annotation outputs.

Then the onset

One merge changed two annotation outputs

e6428a5 merged custom annotation support.
Default annotations started mirroring generated labels, and custom descriptions started forwarding explicit input.
DriftFence flagged that change while the same checks stayed green.

Nearby control

Custom labels stayed quiet

The existing custom-label description path stayed conforming across the whole slice.
Longitudinal follow-up also kept four matched CI-pass control units.
The drift stayed confined to the annotation outputs.

Method

How it was measured.

This replay fixes one annotation baseline, reruns the same selected tests on each first-parent commit, and counts the onset once because only one merged commit in the window drifted.

Replay size: 6 first-parent commits
Fixed workflows: default-annotations, custom-annotations, and labels-control
The same selected checks reran on every commit in the window
DriftFence reran on every commit with the same approved baseline
Longitudinal follow-up found 2 onset units and 4 matched quiet controls

Source material

Method and source files.

The links below show the benchmark notes and pinned replay plan for this metadata-action example.

Measurement notes

The benchmark results log records the completed metadata-action window, including the replay size, outcome mix, and first flagged change.

Benchmark results

Fixed test plan

The replay definition fixes the annotation workflows and the exact late-November 2023 replay window used here.

See the product, results, pricing, and trust pages alongside this Docker metadata example.