See a real run

One real change to Spectre.Console (commit 01027ef), from coverage map to job summary. Every output on this page is copied from that run, not written by hand.

The change

A one-line refactor in the Rule widget. It changes no behaviour, but it is a code change, so a normal pipeline would run all 787 tests in 141 classes on every target framework.

// src/Spectre.Console/Widgets/Rule.cs, GetLineWithoutTitle
 var text = border.GetPart(BoxBorderPart.Top).Repeat(maxWidth);
+var style = Style ?? Spectre.Console.Style.Plain;

 return
 [
-    new Segment(text, Style ?? Spectre.Console.Style.Plain),
+    new Segment(text, style),
     Segment.LineBreak
 ];

The run

The map was recorded on main with td record. The pull request then runs:

$ td test --base main -- -f net10.0td: 1 changed file(s), 1 affected test project(s).Tests to run:  filtered src/Spectre.Console.Tests/Spectre.Console.Tests.csproj    (14 of 141 test class(es) run or reference the changed code, 0.6 s of 3.0 s (80% saved))Recorded test time: 0.6 s of 3.4 s (83% saved).Passed!  - Failed:     0, Passed:   156, Skipped:     0, Total:   156 - Spectre.Console.Tests.dll (net10.0)td summary:  passed  src/Spectre.Console.Tests/Spectre.Console.Tests.csproj  (14 class(es), 4.6 s)1 project(s) run, 0 failed, 0 skipped.Recorded test time: 0.6 s of 3.4 s (83% saved).

156 tests in 14 classes ran instead of 787 in 141. The Spectre.Console.Ansi.Tests project was not even built: nothing it depends on changed.

Were those the right 14?

To check, we made GetLineWithoutTitle throw and ran the whole suite. Exactly 14 classes failed, and they are the 14 TestDetta selected: AlignTests+Center, +Left, +Right, CanvasTests, ColumnsTests, ExceptionTests, LayoutTests, PadderTests, PanelTests, RecorderTests, RowsTests, RuleTests, TableTests and TextPathTests. Most of them never mention Rule: they reach it through layouts and panels, which is what the coverage map knows and a name search would not.

The job summary

With --summary-file "$GITHUB_STEP_SUMMARY" (the GitHub Action sets it for you) every pull request gets this summary: what ran, why each class ran, and what was skipped.

Class names are shortened here; the summary prints them in full (Spectre.Console.Tests.Unit.RuleTests).

Asking why

Every decision has a reason you can read. -v writes the trail to stderr:

[DBG] CodeChangeDetector: src/Spectre.Console/Widgets/Rule.cs: Modified Spectre.Console.Rule GetLineWithoutTitle(RenderOptions,int)
[DBG] CodeChangeDetector: src/Spectre.Console/Widgets/Rule.cs: 1 changed symbol(s), affects whole project: False
[DBG] TestSelector: Spectre.Console.Rule GetLineWithoutTitle(RenderOptions,int) is affected because it changed
[DBG] TestSelector: Spectre.Console.Rule Render(RenderOptions,int) is affected because GetLineWithoutTitle
[DBG] TestSelector: Spectre.Console.Rule GetLineWithoutTitle(RenderOptions,int): test classes taken from coverage

And --format json gives every class's decision, with its evidence and a time estimate, for scripts and dashboards:

{
  "project": "src/Spectre.Console.Tests/Spectre.Console.Tests.csproj",
  "mode": "Filtered",
  "reason": "14 of 141 test class(es) run or reference the changed code",
  "estimate": {
    "fullSeconds": 3,
    "selectedSeconds": 0.6,
    "savedSeconds": 2.4
  },
  "decisions": [
    {
      "class": "Spectre.Console.Tests.Unit.AlignTests+Center",
      "selected": true,
      "kind": "coverage",
      "evidence": "ran Spectre.Console.Rule.GetLineWithoutTitle(RenderOptions,int), which changed (map 01027efcd6c4)"
    },
    {
      "class": "Spectre.Console.Tests.PublicApiTests",
      "selected": false,
      "kind": "skipped",
      "evidence": "ran none of the changed code and names none of it (map 01027efcd6c4)"
    },
    "…139 more"
  ]
}

Running the tests your own way

td affected --format commands prints the dotnet test call TestDetta would make, so you can run it in your own script:

dotnet test src/Spectre.Console.Tests/Spectre.Console.Tests.csproj --filter 'FullyQualifiedName~Spectre.Console.Tests.Unit.AlignTests+Center.|FullyQualifiedName~Spectre.Console.Tests.Unit.AlignTests+Left.|FullyQualifiedName~Spectre.Console.Tests.Unit.AlignTests+Right.|FullyQualifiedName~Spectre.Console.Tests.Unit.CanvasTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.ColumnsTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.ExceptionTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.LayoutTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.PadderTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.PanelTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.RecorderTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.RowsTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.RuleTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.TableTests.|FullyQualifiedName~Spectre.Console.Tests.Unit.TextPathTests.'

Across 40 repositories

One change is one data point. In the evaluation corpus, members of 40 open-source libraries are changed one at a time; for a change to one member, TestDetta ran a median of 28% of the test classes, and a quarter or less in 18 of the 40 repositories. Libraries where nearly every test touches the core (Scriban, Markdig) gain little; layered code gains most. Every failing class was checked against the selection: 3 were missed out of 2,941 failing changes, and each was fixed.