docs: describe the extended test strategy

ADR 0033, ADR 0032 update, README test section, agent rules and
changelog entry.
This commit is contained in:
davide committed 2026-11-03 11:45:43 +01:00
1 parent d85c1e39ce
commit d431996ddc
7 files changed
+73 -5

No files matched your search

+2 -1
View File
@@ -15,6 +15,7 @@ The suite (unit, property-based, fuzz-corpus replay, in-process fake-server test
- **test**: restore, build (Release) and `dotnet test` of `tests/PalladiumWallet.Tests` on `ubuntu-latest` and `windows-latest`, uploading the TRX and coverage results as artifacts. Only the test project and its references are built, because the Android head needs a workload and SDK that CI does not provision.
- **fuzz-smoke**: one job per parser target, running `--random 20000` mutations; any escaping exception fails the job.
- **docs**: `docs/check-docs.sh`.
- A **coverage gate** on the Ubuntu test job fails the run if `Core` drops below the thresholds in the workflow (see 0033).
The pipeline is offline: `LiveServerTests` does nothing without `PALLADIUM_TEST_SERVERS`, so no third-party indexing server is contacted. Coverage-guided fuzzing (afl++) and release builds stay manual.
@@ -30,4 +31,4 @@ The pipeline is offline: `LiveServerTests` does nothing without `PALLADIUM_TEST_
- `.github/workflows/ci.yml`
- `docs/check-docs.sh`
- `tests/PalladiumWallet.Fuzz/Program.cs`
- Related: 0025, 0026, 0027
- Related: 0025, 0026, 0027, 0033
@@ -0,0 +1,34 @@
# 0033. Test the UI layer and CLI headlessly and gate coverage
- **Status:** Accepted
- **Area:** Testing
- **Recorded:** 2026-11-03
## Context
`Core` was covered at about 99% of lines, but `App` (ViewModels, localization) and `Cli` had no tests at all, although they hold logic that moves funds and handles secrets: the send flow, password and lock handling, seed and private-key reveal, the wizard. Nothing stopped coverage from silently dropping.
## Decision
- **ViewModels** are tested headlessly with `Avalonia.Headless` (no window, no GPU). The helper package `Avalonia.Headless.XUnit` is not used because it requires xunit v3 and the suite is on xunit 2; `UiSession` runs each test body on a shared headless session instead. Tests are sandboxed: the data root is a temp folder, the saved server points at the in-process `FakeElectrumServer`, and two internal seams keep them offline: `MainWindowViewModel.UpdateCheck` (stubs the GitHub check) and `MainWindowViewModel.ProfileOverride` (removes the real bootstrap servers).
- **CLI** is tested end to end by invoking the real entry point in-process with a captured console, against the fake server (create, restore, info, sync, send).
- **Localization** has consistency tests: every key has all seven languages, placeholders match across languages, and every key used in XAML or C# exists.
- **Tests that touch process-wide state** (`AppPaths`, `Loc`, `Console`, the seams above) share the xUnit collection `AppPaths` so they run serially.
- **Coverage** is measured with `tests/coverage.runsettings` (test tooling excluded). CI fails if `Core` drops below 95% of lines or 88% of branches (`tests/check-coverage.py`); `App` and `Cli` are reported but not yet gated.
## Consequences
- Regressions in the wizard, reveal gating, lock handling and send preparation are caught without a display.
- Writing the tests found a real defect in `ElectrumClient` (requests hung forever after a protocol error); it is fixed and covered.
- `MainView` code-behind (window-bound) and the Android head remain untested.
- Headless tests run background timers; the test seams must stay in place for the life of the test process so that leftover ViewModels cannot reach the network.
## References
- `tests/PalladiumWallet.Tests/App/HeadlessSetup.cs`
- `tests/PalladiumWallet.Tests/App/VmTestBase.cs`
- `tests/PalladiumWallet.Tests/App/CliTests.cs`
- `tests/PalladiumWallet.Tests/App/LocalizationTests.cs`
- `tests/coverage.runsettings`
- `tests/check-coverage.py`
- Related: 0025, 0026, 0032
+1
View File
@@ -36,6 +36,7 @@ One ADR per feature or significant design decision. Format: `template.md`. Workf
| 0030 | [Single-source versioning with a mandatory changelog entry](0030-single-source-versioning-and-changelog.md) | Distribution | Accepted |
| 0031 | [Explicitly defer multisig, Lightning, Tor and hardware wallets](0031-defer-multisig-lightning-and-hardware-wallets.md) | Scope | Accepted |
| 0032 | [Run tests, fuzz smoke and docs checks in GitHub Actions](0032-github-actions-ci.md) | Testing | Accepted |
| 0033 | [Test the UI layer and CLI headlessly, gate coverage](0033-test-strategy-beyond-core.md) | Testing | Accepted |
ADRs `0002` onwards were written retroactively on 2026-11-03 from the code, comments and changelog.
If one misstates the original reasoning, fix it in a pull request.