Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Tier 1: the utility routines

This page is generated. cargo xtask validate --tiers 1,2,3 --smoke wrote it from commit b6ef4ee-dirty. Every number on it was captured from that run's own output. To change what it says, change what the suite measures and run the command again.

TESUB1 (enthalpy), TESUB2 (temperature from enthalpy by Newton), TESUB3 (heat capacity) and TESUB4 (liquid density) are swept against the Fortran over a simplex grid, a Dirichlet sample and a boundary pool, at every temperature in the physical range, for each of the three ITY modes. PLAN.org sets the gate at a maximum relative error below 1e-13, with a ULP histogram reported rather than a verdict.

Reduced volume. This run passed --smoke, so the sweeps are the short ones the CI gate uses rather than the full ones PLAN.org specifies. The case counts in the tables below are what actually ran. Drop --smoke for the gate volume.

Produced with rustc 1.97.1 (8bab26f4f 2026-07-14), gfortran 15.2.0. The oracle's compiler flags are fixed in crates/tepsim-oracle/build.rs and asserted by a test; changing them invalidates every number on this page, which is why it is a logged re-baseline and not an edit.

What ran

2 test binaries: 7 test(s) passed, 0 failed, 0 ignored.

targetlibmpassedfailedignored
tier1_enthalpyvendored500
tier1_temperaturevendored200

Figures

GENERATED by `cargo xtask validate --tiers 1,2,3 --smoke` from commit `b6ef4ee-dirty`. Do not edit by hand: the next run overwrites it. Tier 1: every comparison against the 1e-13 gateA logarithmic strip plot. Each dot is one comparison against the Fortran, placed at its maximum relative error, in a lane named for the test target that produced it. 13 of 14 comparisons are exactly zero and sit in the separate lane at the left. The gate at 1e-13 is a dashed vertical line with the region beyond it shaded; 1 dots lie beyond it. Tier 1: every comparison, against the gate 14 comparisons over 2 target(s). 13 are bit-identical to the Fortran. 1 lies beyond the gate. 1e-14 1e-13 1e-12 1e-11 1e-10 1e-9 1e-8 1e-7 1e-6 1e-5 1e-4 1e-3 maximum relative error against the Fortran = 0 gate 1e-13 tier1_enthalpy tier1 TESUB1 ity=2, offset as f64: 1.597e-5 (vendored libm, reading_the_offset_as_double_precision_would_fail_the_gate_by_orders) tier1 TESUB1 ity=0: exactly 0 (vendored libm, tesub1_matches_the_fortran_over_the_full_sweep) tier1 TESUB1 ity=1: exactly 0 (vendored libm, tesub1_matches_the_fortran_over_the_full_sweep) tier1 TESUB1 ity=2: exactly 0 (vendored libm, tesub1_matches_the_fortran_over_the_full_sweep) tier1 TESUB3 ity=0: exactly 0 (vendored libm, tesub3_matches_the_fortran_over_the_full_sweep) tier1 TESUB3 ity=1: exactly 0 (vendored libm, tesub3_matches_the_fortran_over_the_full_sweep) tier1 TESUB3 ity=2: exactly 0 (vendored libm, tesub3_matches_the_fortran_over_the_full_sweep) tier1 TESUB4: exactly 0 (vendored libm, tesub4_matches_the_fortran_over_the_full_sweep) tier1_temperature tier1 TESUB2 ity=0 start=warm: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) tier1 TESUB2 ity=0 start=cold: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) tier1 TESUB2 ity=1 start=warm: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) tier1 TESUB2 ity=1 start=cold: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) tier1 TESUB2 ity=2 start=warm: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) tier1 TESUB2 ity=2 start=cold: exactly 0 (vendored libm, tesub2_matches_the_fortran_and_the_silent_failure_never_fires) vendored libm vendored libm beyond the gate beyond the gate hover a dot for the comparison it came from
Every comparison, against the gate. Every comparison this tier made, at its own maximum relative error, in a lane named for the target that ran it. Hovering a dot names the comparison. A dot is orange because its value is past the 1e-13 gate and for no other reason: no test is recognised by name, so a regression and a deliberate positive control are drawn identically. Beyond it in this run: tier1 TESUB1 ity=2, offset as f64, and nothing else. Any other dot crossing the line is a failure. The bit-equality half of the claim is falsified separately, by any dot leaving the = 0 lane under the platform libm, where the two sides call the same exp. Drawn by cargo xtask validate --tiers 1,2,3 --smoke at commit b6ef4ee-dirty; the measurement it repeats was first recorded in B-0009, B-0010 and B-0011 (LOG.org).
GENERATED by `cargo xtask validate --tiers 1,2,3 --smoke` from commit `b6ef4ee-dirty`. Do not edit by hand: the next run overwrites it. Tier 1: how many bits differA bar chart on a logarithmic count axis. Each bar is the number of comparisons whose result differed from the Fortran's by that many units in the last place, grouped by which libm the port was built against. 103,350 comparisons in total. Tier 1: how many bits actually differ 103,350 comparisons, by units in the last place. The count axis is logarithmic. One block beyond the gate is left out. vendored libm: 103,350 of 103,350 identical to the last bit 0 ULP 103,350 1 10 1e2 1e3 1e4 1e5 1e6 A bar exists only where the count is not zero, so a tier with one bar differed nowhere.
How many bits actually differ. The same comparisons counted by how many bits differ rather than by how much. A relative error is a ratio, and dividing by a large number makes any difference look small; a count of differing units in the last place cannot be flattered that way. This tier is straight-line arithmetic over constants already proved bit-identical, so the claim is that the only bar is the one at zero. A second bar appearing anywhere is a regression to chase rather than a tolerance to widen. Drawn by cargo xtask validate --tiers 1,2,3 --smoke at commit b6ef4ee-dirty; the measurement it repeats was first recorded in B-0009, B-0010 and B-0011 (LOG.org).

Measurements

15 block(s), lifted from the transcripts below. The columns are whatever fields the run printed, so a new field in the reporter becomes a new column here rather than data this page drops.

The test column matters as much as the numbers, because not every row is the port being measured. Some tests deliberately mis-type a constant, or solve from the wrong guess, to show what that would cost; a row from one of those is supposed to be enormous, and its test name says so. The what column carries a tier1 prefix in every tier, because it is the shared comparison reporter's own label rather than a claim about which tier printed it.

targetfrom testwhatcasesmax rel errmax ulpulp percentilesulp histogramnon-finiteabandonedround tripelapsedFortranport
tier1_enthalpyreading_the_offset_as_double_precision_would_fail_the_gate_by_orderstier1 TESUB1 ity=2, offset as f6479501.597e-5 at grid#704 T=21.875103098852352 at grid#704 T=21.875p50>=16 p90>=16 p99>=16 p100>=16>=16:79500 seen, 0 mismatched
tier1_enthalpytesub1_matches_the_fortran_over_the_full_sweeptier1 TESUB1 ity=079500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub1_matches_the_fortran_over_the_full_sweeptier1 TESUB1 ity=179500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub1_matches_the_fortran_over_the_full_sweeptier1 TESUB1 ity=279500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub3_matches_the_fortran_over_the_full_sweeptier1 TESUB3 ity=079500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub3_matches_the_fortran_over_the_full_sweeptier1 TESUB3 ity=179500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub3_matches_the_fortran_over_the_full_sweeptier1 TESUB3 ity=279500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_enthalpytesub4_matches_the_fortran_over_the_full_sweeptier1 TESUB479500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched [0.0 s]
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=0 start=warm79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 0e0 C0.0 s
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=0 start=cold79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 5.684341886080802e-14 C at grid#3637 T=131.250.0 s
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=1 start=warm79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 0e0 C0.0 s
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=1 start=cold79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 1.9895196601282805e-13 C at grid#4617 T=1700.0 s
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=2 start=warm79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 0e0 C0.0 s
tier1_temperaturetesub2_matches_the_fortran_and_the_silent_failure_never_firestier1 TESUB2 ity=2 start=cold79500.000e0 at grid#0 T=00 at grid#0 T=0p50=0 p90=0 p99=0 p100=00:79500 seen, 0 mismatched0 (delta D-001)max |solved - true| = 5.115907697472721e-13 C at face#96 T=144.143050562349230.0 s
tier1_temperaturethe_fortran_silently_returns_the_guess_where_the_port_reports_failureD-001 demonstrated:returned 120.4 silentlyNewton did not converge in 100 iterations from a guess of 120.4 C: reached -48598544002334720 C with a final step of 24299272000832196, which is not below 1e-12

Transcripts

Each block below is a test binary's own output, verbatim, with the command that produced it. The summary above is derived from these; they are not derived from it.

tier1_enthalpy, vendored libm

cargo test -p tepsim-oracle --features oracle --release --test tier1_enthalpy -- --nocapture --test-threads 1

running 5 tests
test both_routines_are_bit_identical_to_the_fortran ... sweep: SMOKE, 7950 cases. The 9987490-case gate is `cargo xtask validate --tiers 1`.
ok
test reading_the_offset_as_double_precision_would_fail_the_gate_by_orders ... tier1 TESUB1 ity=2, offset as f64
  cases          : 7950
  max rel err    : 1.597e-5 at grid#704 T=21.875
  max ulp        : 103098852352 at grid#704 T=21.875
  ulp percentiles: p50>=16 p90>=16 p99>=16 p100>=16
  ulp histogram  : >=16:7950
  non-finite     : 0 seen, 0 mismatched
ok
test tesub1_matches_the_fortran_over_the_full_sweep ... sweep: SMOKE, 7950 cases. The 9987490-case gate is `cargo xtask validate --tiers 1`.
tier1 TESUB1 ity=0
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
tier1 TESUB1 ity=1
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
tier1 TESUB1 ity=2
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
ok
test tesub3_matches_the_fortran_over_the_full_sweep ... sweep: SMOKE, 7950 cases. The 9987490-case gate is `cargo xtask validate --tiers 1`.
tier1 TESUB3 ity=0
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
tier1 TESUB3 ity=1
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
tier1 TESUB3 ity=2
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
ok
test tesub4_matches_the_fortran_over_the_full_sweep ... sweep: SMOKE, 7950 cases. The 9987490-case gate is `cargo xtask validate --tiers 1`.
tier1 TESUB4
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched  [0.0 s]
ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

tier1_temperature, vendored libm

cargo test -p tepsim-oracle --features oracle --release --test tier1_temperature -- --nocapture --test-threads 1

running 2 tests
test tesub2_matches_the_fortran_and_the_silent_failure_never_fires ... sweep: SMOKE, 7950 cases. The 9987490-case gate is `cargo xtask validate --tiers 1`.
tier1 TESUB2 ity=0 start=warm
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 0e0 C
  elapsed        : 0.0 s
tier1 TESUB2 ity=0 start=cold
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 5.684341886080802e-14 C at grid#3637 T=131.25
  elapsed        : 0.0 s
tier1 TESUB2 ity=1 start=warm
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 0e0 C
  elapsed        : 0.0 s
tier1 TESUB2 ity=1 start=cold
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 1.9895196601282805e-13 C at grid#4617 T=170
  elapsed        : 0.0 s
tier1 TESUB2 ity=2 start=warm
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 0e0 C
  elapsed        : 0.0 s
tier1 TESUB2 ity=2 start=cold
  cases          : 7950
  max rel err    : 0.000e0 at grid#0 T=0
  max ulp        : 0 at grid#0 T=0
  ulp percentiles: p50=0 p90=0 p99=0 p100=0
  ulp histogram  : 0:7950
  non-finite     : 0 seen, 0 mismatched
  abandoned      : 0 (delta D-001)
  round trip     : max |solved - true| = 5.115907697472721e-13 C at face#96 T=144.14305056234923
  elapsed        : 0.0 s
ok
test the_fortran_silently_returns_the_guess_where_the_port_reports_failure ... D-001 demonstrated:
  Fortran: returned 120.4 silently
  port:    Newton did not converge in 100 iterations from a guess of 120.4 C: reached -48598544002334720 C with a final step of 24299272000832196, which is not below 1e-12
ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s