docs: describe the extended test strategy
ADR 0033, ADR 0032 update, README test section, agent rules and changelog entry.
This commit is contained in:
1 parent
d85c1e39ce
commit
d431996ddc
7 files changed
+73
-5
No files matched your search
@@ -15,6 +15,7 @@ The suite (unit, property-based, fuzz-corpus replay, in-process fake-server test
|
||||
- **test**: restore, build (Release) and `dotnet test` of `tests/PalladiumWallet.Tests` on `ubuntu-latest` and `windows-latest`, uploading the TRX and coverage results as artifacts. Only the test project and its references are built, because the Android head needs a workload and SDK that CI does not provision.
|
||||
- **fuzz-smoke**: one job per parser target, running `--random 20000` mutations; any escaping exception fails the job.
|
||||
- **docs**: `docs/check-docs.sh`.
|
||||
- A **coverage gate** on the Ubuntu test job fails the run if `Core` drops below the thresholds in the workflow (see 0033).
|
||||
|
||||
The pipeline is offline: `LiveServerTests` does nothing without `PALLADIUM_TEST_SERVERS`, so no third-party indexing server is contacted. Coverage-guided fuzzing (afl++) and release builds stay manual.
|
||||
|
||||
@@ -30,4 +31,4 @@ The pipeline is offline: `LiveServerTests` does nothing without `PALLADIUM_TEST_
|
||||
- `.github/workflows/ci.yml`
|
||||
- `docs/check-docs.sh`
|
||||
- `tests/PalladiumWallet.Fuzz/Program.cs`
|
||||
- Related: 0025, 0026, 0027
|
||||
- Related: 0025, 0026, 0027, 0033
|
||||
@@ -0,0 +1,34 @@
|
||||
# 0033. Test the UI layer and CLI headlessly and gate coverage
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Area:** Testing
|
||||
- **Recorded:** 2026-11-03
|
||||
|
||||
## Context
|
||||
|
||||
`Core` was covered at about 99% of lines, but `App` (ViewModels, localization) and `Cli` had no tests at all, although they hold logic that moves funds and handles secrets: the send flow, password and lock handling, seed and private-key reveal, the wizard. Nothing stopped coverage from silently dropping.
|
||||
|
||||
## Decision
|
||||
|
||||
- **ViewModels** are tested headlessly with `Avalonia.Headless` (no window, no GPU). The helper package `Avalonia.Headless.XUnit` is not used because it requires xunit v3 and the suite is on xunit 2; `UiSession` runs each test body on a shared headless session instead. Tests are sandboxed: the data root is a temp folder, the saved server points at the in-process `FakeElectrumServer`, and two internal seams keep them offline: `MainWindowViewModel.UpdateCheck` (stubs the GitHub check) and `MainWindowViewModel.ProfileOverride` (removes the real bootstrap servers).
|
||||
- **CLI** is tested end to end by invoking the real entry point in-process with a captured console, against the fake server (create, restore, info, sync, send).
|
||||
- **Localization** has consistency tests: every key has all seven languages, placeholders match across languages, and every key used in XAML or C# exists.
|
||||
- **Tests that touch process-wide state** (`AppPaths`, `Loc`, `Console`, the seams above) share the xUnit collection `AppPaths` so they run serially.
|
||||
- **Coverage** is measured with `tests/coverage.runsettings` (test tooling excluded). CI fails if `Core` drops below 95% of lines or 88% of branches (`tests/check-coverage.py`); `App` and `Cli` are reported but not yet gated.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Regressions in the wizard, reveal gating, lock handling and send preparation are caught without a display.
|
||||
- Writing the tests found a real defect in `ElectrumClient` (requests hung forever after a protocol error); it is fixed and covered.
|
||||
- `MainView` code-behind (window-bound) and the Android head remain untested.
|
||||
- Headless tests run background timers; the test seams must stay in place for the life of the test process so that leftover ViewModels cannot reach the network.
|
||||
|
||||
## References
|
||||
|
||||
- `tests/PalladiumWallet.Tests/App/HeadlessSetup.cs`
|
||||
- `tests/PalladiumWallet.Tests/App/VmTestBase.cs`
|
||||
- `tests/PalladiumWallet.Tests/App/CliTests.cs`
|
||||
- `tests/PalladiumWallet.Tests/App/LocalizationTests.cs`
|
||||
- `tests/coverage.runsettings`
|
||||
- `tests/check-coverage.py`
|
||||
- Related: 0025, 0026, 0032
|
||||
@@ -36,6 +36,7 @@ One ADR per feature or significant design decision. Format: `template.md`. Workf
|
||||
| 0030 | [Single-source versioning with a mandatory changelog entry](0030-single-source-versioning-and-changelog.md) | Distribution | Accepted |
|
||||
| 0031 | [Explicitly defer multisig, Lightning, Tor and hardware wallets](0031-defer-multisig-lightning-and-hardware-wallets.md) | Scope | Accepted |
|
||||
| 0032 | [Run tests, fuzz smoke and docs checks in GitHub Actions](0032-github-actions-ci.md) | Testing | Accepted |
|
||||
| 0033 | [Test the UI layer and CLI headlessly, gate coverage](0033-test-strategy-beyond-core.md) | Testing | Accepted |
|
||||
|
||||
ADRs `0002` onwards were written retroactively on 2026-11-03 from the code, comments and changelog.
|
||||
If one misstates the original reasoning, fix it in a pull request.
|
||||
Reference in new issue
Block a user