Conner Majic
ce137dd1c2
Update contributor, security, validation, and compute handoff documentation to reflect new runtime safeguards, CI gates, and expected regression checks. Made-with: Cursor
86 lines
3.3 KiB
Markdown
86 lines
3.3 KiB
Markdown
# Validation plan
|
||
|
||
Last Updated: 2026-04-16
|
||
|
||
## Determinism
|
||
|
||
- Same XML + same `solverModel` + `workflow` → identical `solver.card` arrays (bitwise for floats at printed precision in C stdout path).
|
||
- API responses include `runMetadata.generatedAt` — **stripped** before golden hash in tests.
|
||
|
||
## Golden file
|
||
|
||
- File: `data/golden/default.solve.sha256`
|
||
- Content: SHA-256 of **canonical JSON** string from `GET /solve/default?solverModel=fdm` with `generatedAt` removed and stable key ordering (`stableStringify` in tests).
|
||
|
||
## Unit coverage
|
||
|
||
- XML parser: numeric + array + unit conversion.
|
||
- `cardQa.js`: min samples, cycle closure, spike rejection.
|
||
- C: `test_solver` — determinism, bounds, static equilibrium helper, undamped wave CFL identity, FDM vs FEA peak tolerance.
|
||
|
||
## Cross-model (FDM vs FEA)
|
||
|
||
- C unit gate (`solver-c/tests/test_solver.c`): `|maxPolishedLoad_FDM - maxPolishedLoad_FEA| < 7e5 N` on an SI-normalized fixture (interim; tighten when BC parity improves).
|
||
|
||
## Integration
|
||
|
||
- `solver-api/tests/api.test.js` — solve routes, diagnostic path, golden hash, `validate-card`, `both` comparison keys.
|
||
- API error contract checks: assert `schemaVersion` + machine-readable `code` on expected 4xx responses.
|
||
- `comparison` schema v2 includes:
|
||
- `peakLoadDeltas` for Pmax/Pmin and Dmax/Dmin
|
||
- `pointwiseResiduals.series[]` (position + polished/downhole residuals)
|
||
- `residualSummary` RMS
|
||
- `fourier` optional baseline block (`null` unless `options.enableFourierBaseline=true`)
|
||
|
||
## Extended physics validation
|
||
|
||
- `options.enableProfiles=true` must return `solver.profiles` with `nodeCount > 0` and finite trajectory/side-load/friction arrays.
|
||
- `options.enableDiagnosticsDetail=true` must return `solver.diagnostics` with valve-state booleans and finite chamber/gas series.
|
||
- `options.enableFourierBaseline=true` must return non-null `comparison.fourier` and finite residual RMS metrics.
|
||
|
||
## Priority validation roadmap (1–5)
|
||
|
||
### 1) Correctness gates (mandatory)
|
||
|
||
- Field sensitivity checks: each mapped physics input gets ±1% perturbation and expected directional assertions.
|
||
- Invariant checks:
|
||
- finite loads/stresses/profiles,
|
||
- gas fraction within bounds,
|
||
- consistent valve-state transitions,
|
||
- no malformed profile/diagnostic arrays.
|
||
- Deterministic checks on multi-case golden set.
|
||
|
||
### 2) Fidelity checks
|
||
|
||
- Equation-backed vs heuristic term audit must be explicit in docs.
|
||
- Stability checks under high-deviation/high-friction cases.
|
||
|
||
### 3) Cross-model checks
|
||
|
||
- Case matrix compares FDM vs FEA on:
|
||
- peak polished load,
|
||
- peak downhole load,
|
||
- net stroke,
|
||
- residual RMS.
|
||
|
||
### 4) Contract checks
|
||
|
||
- Default `/solve` and `/solve/default` responses remain backward compatible.
|
||
- Option-gated payload behavior is tested for on/off combinations.
|
||
|
||
### 5) CI checks
|
||
|
||
- C sanitizers (ASan/UBSan) lane.
|
||
- Dependency audit lane (`npm audit --omit=dev --audit-level=high`) for API and GUI packages.
|
||
- Performance budget lane (fixed representative cases).
|
||
- Artifact lane emits comparison/drift summaries.
|
||
|
||
## Analytical targets (C tests)
|
||
|
||
- **Static rod:** uniform bar vertical hang — numerical mean tension trend vs \(\sum \rho g A \Delta x\) (approximate check).
|
||
- **Wave CFL:** \(a \Delta t / \Delta x \le 1\) for explicit scheme after clamp.
|
||
|
||
## GUI
|
||
|
||
- Smoke: import `base-case.xml`, run solver — manual / Playwright future.
|