See a real run
One real change to OpenTelemetry .NET (commit f9dd754, 30 test projects), from coverage map to job summary. Every output on this page is copied from that run, not written by hand; only long class names are shortened.
The run used a development build made after 0.1.3 (it reports itself as 0.1.3). Run again with 0.2.0 on 15 October 2026, the same change selected the same 8 classes and ran the same 229 tests.
The change
A rename in the declarative configuration package: the flag that says whether OTEL_CONFIG_FILE is overridden gets a name that says so. It changes no behaviour, but it is a code change, so a normal pipeline would build and run every test project in the repository.
-bool differs;+bool overridden; try {- differs = new FilePath(configFile) != filePath;+ overridden = new FilePath(configFile) != filePath; } catch (Exception ex) when (ex is ArgumentException or …) {- differs = true;+ overridden = true; } -if (differs)+if (overridden)
How the decision was made
TestDetta follows the change from the file to the tests that can observe it. The changed method is private; only UseDeclarativeConfiguration calls it. The coverage map, recorded on main, knows which test classes ran it: 8 classes in one test project. The other 266 ran none of it and name none of it.
WarnIfConfigFileEnvVarDiffersUseDeclarativeConfigurationmain, f9dd754The run
The map was recorded on main with td record. The pull request then runs:
$ td test --base main -- -f net10.0TestDetta 0.1.3 · 1 changed file · 2 affected test projectsTests to run┌────────────────────────────────────────────────────────┬──────────┬─────────┬───────┬────────────────────────────────┐│ Test project │ Run │ Classes │ Saved │ Why │├────────────────────────────────────────────────────────┼──────────┼─────────┼───────┼────────────────────────────────┤│ test/OpenTelemetry.Configuration.Declarative.FuzzTests │ skipped │ - │ 100% │ no test class runs or ││ │ │ │ │ references the changed code ││ test/OpenTelemetry.Configuration.Declarative.Tests │ filtered │ 8 / 28 │ 5% │ 8 of 28 test class(es) run or ││ │ │ │ │ reference the changed code │└────────────────────────────────────────────────────────┴──────────┴─────────┴───────┴────────────────────────────────┘ test/OpenTelemetry.Configuration.Declarative.Tests → 8 classes DeclarativeConfigurationDocumentAccessorTests DeclarativeConfigurationDocumentAcquisitionTests DeclarativeConfigurationEventSourceTests DeclarativeConfigurationExtensionTests DeclarativeConfigurationHostIntegrationTests DeclarativeConfigurationSdkIntegrationTests StrictModeDiagnosticsTests StrictModeMaskTestsRecorded test time 22.4 s instead of 5 m 59 s ███████████████████░ 93% saved; 7 test project(s) without timings are not countedPassed! - Failed: 0, Passed: 228, Skipped: 1, Total: 229, Duration: 21 s - OpenTelemetry.Configuration.Declarative.Tests.dll (net10.0)td summary┌────────────────────────────────────────────────────────┬──────────┬──────────────────┬────────┐│ Test project │ Result │ Ran │ Time │├────────────────────────────────────────────────────────┼──────────┼──────────────────┼────────┤│ test/OpenTelemetry.Configuration.Declarative.Tests │ ✓ passed │ 8 classes │ 28.0 s ││ test/OpenTelemetry.Configuration.Declarative.FuzzTests │ skipped │ nothing affected │ │└────────────────────────────────────────────────────────┴──────────┴──────────────────┴────────┘1 project run · 0 failed · 1 skippedRecorded test time 22.4 s instead of 5 m 59 s ███████████████████░ 93% saved; 7 test project(s) without timings are not counted
229 tests ran instead of 6,078, and only the projects on the path from the change to those tests were built. The selected classes are the slow integration tests of their project, so inside it little is saved (22.4 s of 23.6 s); the saving is the 18 other test projects that never had to run. The 7 projects without timings are mobile and stress-test apps that td record could not run on this machine; they were not affected either.
Class names are shortened here; the output prints them in full (OpenTelemetry.Configuration.Declarative.Tests.StrictModeMaskTests).
Were those the right 8?
To check, we made WarnIfConfigFileEnvVarDiffers throw and ran the whole solution: 6,078 tests in 19 test projects. The other 11 of the 30 are the 7 apps above and the Android, Apple, Blazor WebAssembly and MAUI test projects, which only the repository's extended solution (OpenTelemetry.Extended.slnx) builds. 70 tests failed, in exactly the 8 classes TestDetta selected: nothing missed, nothing extra. None of them names the method, which is private; they reach it through the configuration builder, which is what the coverage map knows.
| Test class | Selected by TestDetta | Fails with the method broken | Match |
|---|---|---|---|
DeclarativeConfigurationDocumentAccessorTests | yes | yes | ✓ |
DeclarativeConfigurationDocumentAcquisitionTests | yes | yes | ✓ |
DeclarativeConfigurationEventSourceTests | yes | yes | ✓ |
DeclarativeConfigurationExtensionTests | yes | yes | ✓ |
DeclarativeConfigurationHostIntegrationTests | yes | yes | ✓ |
DeclarativeConfigurationSdkIntegrationTests | yes | yes | ✓ |
StrictModeDiagnosticsTests | yes | yes | ✓ |
StrictModeMaskTests | yes | yes | ✓ |
| 266 other classes | no | no | ✓ |
Two Prometheus HTTP listener tests failed in both runs, with and without the change, because of the machine's network setup; they are left out of the comparison.
The job summary
With --summary-file "$GITHUB_STEP_SUMMARY" (the GitHub Action sets it for you) every pull request gets this summary: what ran, why each class ran, and what was skipped.
TestDetta test selection
1 changed file(s) compared with main.
- 1 of 2 affected test project(s) ran, 1 skipped.
- Recorded test time: 22.4 s of 5 m 59 s (93% saved); 7 test project(s) without timings are not counted.
- All selected tests passed.
test/OpenTelemetry.Configuration.Declarative.FuzzTests/OpenTelemetry.Configuration.Declarative.FuzzTests.csproj: skipped, none, recorded time 0.0 s of 0.4 s (100% saved)- why:
no test class runs or references the changed code
- why:
test/OpenTelemetry.Configuration.Declarative.Tests/OpenTelemetry.Configuration.Declarative.Tests.csproj: passed, 8 test class(es), recorded time 22.4 s of 23.6 s (5% saved)- why:
8 of 28 test class(es) run or reference the changed code classes (8)
DeclarativeConfigurationDocumentAccessorTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)DeclarativeConfigurationDocumentAcquisitionTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)DeclarativeConfigurationEventSourceTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)DeclarativeConfigurationExtensionTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)DeclarativeConfigurationHostIntegrationTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)DeclarativeConfigurationSdkIntegrationTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)StrictModeDiagnosticsTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)StrictModeMaskTests:ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)
- skipped: 20 class(es) that ran none of the changed code and name none of it
- why:
Asking why
Every decision has a reason you can read. -v writes the trail to stderr:
DBG CodeChangeDetector src/OpenTelemetry.Configuration.Declarative/Extensions/OpenTelemetryBuilderDeclarativeConfigurationExtensions.cs: Modified OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions WarnIfConfigFileEnvVarDiffersDBG CodeChangeDetector src/OpenTelemetry.Configuration.Declarative/Extensions/OpenTelemetryBuilderDeclarativeConfigurationExtensions.cs: 1 changed symbol(s), affects whole project: FalseDBG ImpactAnalyzer 4 affected project(s), 2 of them tests, 0 unmapped file(s)DBG TestSelector OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions WarnIfConfigFileEnvVarDiffers(FilePath) is affected because it changedDBG TestSelector OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions UseDeclarativeConfiguration(this IOpenTelemetryBuilder,string) is affected because WarnIfConfigFileEnvVarDiffersDBG TestSelector OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions WarnIfConfigFileEnvVarDiffers(FilePath): test classes taken from coverageDBG TestSelector test/OpenTelemetry.Configuration.Declarative.FuzzTests/OpenTelemetry.Configuration.Declarative.FuzzTests.csproj: Skipped (no test class runs or references the changed code)DBG TestSelector test/OpenTelemetry.Configuration.Declarative.Tests/OpenTelemetry.Configuration.Declarative.Tests.csproj: Filtered (8 of 28 test class(es) run or reference the changed code)
And --format json gives every class's decision, with its evidence and a time estimate, for scripts and dashboards:
{
"project": "test/OpenTelemetry.Configuration.Declarative.Tests/OpenTelemetry.Configuration.Declarative.Tests.csproj",
"mode": "Filtered",
"reason": "8 of 28 test class(es) run or reference the changed code",
"estimate": {
"fullSeconds": 23.6,
"selectedSeconds": 22.4,
"savedSeconds": 1.2
},
"decisions": [
{
"class": "OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationDocumentAccessorTests",
"selected": true,
"kind": "coverage",
"evidence": "ran OpenTelemetry.OpenTelemetryBuilderDeclarativeConfigurationExtensions.WarnIfConfigFileEnvVarDiffers(FilePath), which changed (map f9dd7546cc3d)"
},
{
"class": "OpenTelemetry.Configuration.Declarative.Tests.ConfigPropertiesTests",
"selected": false,
"kind": "skipped",
"evidence": "ran none of the changed code and names none of it (map f9dd7546cc3d)"
},
"…26 more"
]
}
Running the tests your own way
td affected --format commands prints the dotnet test call TestDetta would make, so you can run it in your own script:
dotnet test test/OpenTelemetry.Configuration.Declarative.Tests/OpenTelemetry.Configuration.Declarative.Tests.csproj --filter 'FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationDocumentAccessorTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationDocumentAcquisitionTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationEventSourceTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationExtensionTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationHostIntegrationTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.DeclarativeConfigurationSdkIntegrationTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.StrictModeDiagnosticsTests.|FullyQualifiedName~OpenTelemetry.Configuration.Declarative.Tests.StrictModeMaskTests.'
A smaller change: Spectre.Console
A one-line refactor in the Rule widget of Spectre.Console (commit 01027ef). The coverage map knew which 14 of its 141 test classes ran the changed method, most of them through layouts and panels that never mention Rule. Making the method throw failed exactly those 14 classes.
Across 53 repositories
One change is one data point. To know how TestDetta behaves on code it has never seen, we run it on 53 open-source .NET libraries and check every selection against the tests that really fail.
- RecordA coverage map at a pinned commit of the default branch.
- Change one memberUp to 100 sampled methods, properties and constructors, one at a time.
- Break it and run everythingThe member throws, and the whole suite runs to see which classes fail.
- Check the selectionEvery failing class must be one TestDetta selected. A miss is confirmed by running the class alone.
Every repository
Sorted by the share of test classes run for a change to one member; click a column to sort. Changes counts the changes that broke at least one test. Per failure is the median number of classes that ran for each class that failed: 1.0 means nothing extra ran.
The evaluation before the 0.2.0 release, 2026-10-13 to 2026-10-15, judged strictly: a test run outside the selection that crashes or hangs counts as a miss when it does so again with the change and not without it; a crash that does not come back leaves the change inconclusive. Of the 58 repositories in the corpus, 53 gave a verdict: Akka.NET, Lucene.NET, ABP and MSBuild were not run in this round, and Meziantou.Analyzer's test projects failed to restore on a new NuGet security advisory. The 5 misses are three known gaps, all on the Limitations page: NSec's algorithm self-tests, which run once behind a static flag (two changes); server code that handles a test's request on another thread, in ManagedCode.Communication (one); and Mediator's generated code, read as empty (two). Inconclusive changes: 1. Earlier real misses were either fixed, each with a test that fails without the fix, or are on that page as well (NLog's state left by an earlier test class).
The hard cases it covers
The corpus is chosen for the code that trips test selection up. Number of repositories that exercise each:
Where it gains most
Layered code, where each test reaches a small part of the system, gains most: Polly ran 3% of its test classes per change, Spectre.Console 6%, OpenTelemetry .NET 9%. Libraries where most tests run the same core gain little: Mapperly 97%, AutoMapper 86%, EF Core 68%. There, a change to the core really does reach most tests, and TestDetta runs them. LibGit2Sharp ran everything until 0.2.0, because a reference that depends on the target framework kept TestDetta conservative; 0.2.0 compares each framework's build on its own, and it now runs 26%.