rods/docs/engineering/validation.md

Validation plan

Last Updated: 2026-04-16

Determinism

• Same XML + same solverModel + workflow → identical solver.card arrays (bitwise for floats at printed precision in C stdout path).
• API responses include runMetadata.generatedAt — stripped before golden hash in tests.

Golden file

• File: data/golden/default.solve.sha256
• Content: SHA-256 of canonical JSON string from GET /solve/default?solverModel=fdm with generatedAt removed and stable key ordering (stableStringify in tests).

Unit coverage

• XML parser: numeric + array + unit conversion.
• cardQa.js: min samples, cycle closure, spike rejection.
• C: test_solver — determinism, bounds, static equilibrium helper, undamped wave CFL identity, FDM vs FEA peak tolerance.

Cross-model (FDM vs FEA)

• C unit gate (solver-c/tests/test_solver.c): |maxPolishedLoad_FDM - maxPolishedLoad_FEA| < 7e5 N on an SI-normalized fixture (interim; tighten when BC parity improves).

Integration

• solver-api/tests/api.test.js — solve routes, diagnostic path, golden hash, validate-card, both comparison keys.
• comparison schema v2 includes:
  • peakLoadDeltas for Pmax/Pmin and Dmax/Dmin
  • pointwiseResiduals.series[] (position + polished/downhole residuals)
  • residualSummary RMS
  • fourier optional baseline block (null unless options.enableFourierBaseline=true)

Extended physics validation

• options.enableProfiles=true must return solver.profiles with nodeCount > 0 and finite trajectory/side-load/friction arrays.
• options.enableDiagnosticsDetail=true must return solver.diagnostics with valve-state booleans and finite chamber/gas series.
• options.enableFourierBaseline=true must return non-null comparison.fourier and finite residual RMS metrics.

Priority validation roadmap (1–5)

1) Correctness gates (mandatory)

• Field sensitivity checks: each mapped physics input gets ±1% perturbation and expected directional assertions.
• Invariant checks:
  • finite loads/stresses/profiles,
  • gas fraction within bounds,
  • consistent valve-state transitions,
  • no malformed profile/diagnostic arrays.
• Deterministic checks on multi-case golden set.

2) Fidelity checks

• Equation-backed vs heuristic term audit must be explicit in docs.
• Stability checks under high-deviation/high-friction cases.

3) Cross-model checks

• Case matrix compares FDM vs FEA on:
  • peak polished load,
  • peak downhole load,
  • net stroke,
  • residual RMS.

4) Contract checks

• Default /solve and /solve/default responses remain backward compatible.
• Option-gated payload behavior is tested for on/off combinations.

5) CI checks

• C sanitizers (ASan/UBSan) lane.
• Performance budget lane (fixed representative cases).
• Artifact lane emits comparison/drift summaries.

Analytical targets (C tests)

• Static rod: uniform bar vertical hang — numerical mean tension trend vs (\sum \rho g A \Delta x) (approximate check).
• Wave CFL: (a \Delta t / \Delta x \le 1) for explicit scheme after clamp.

GUI

• Smoke: import base-case.xml, run solver — manual / Playwright future.