Normally, run npm test. The eight upstream suites, including WG17 syntax,
reuse Git-ignored .cache/neumerkel/. A missing or incomplete cache is downloaded
automatically. npm run sync explicitly refreshes upstream, checks the suites
and updates the tracked report. Use npm test -- --offline to skip those suites
entirely, or node test/run-neumerkel.mjs --cached to run them without downloads.
For development, run a focused check directly from the repository root:
node test/run-regression.mjs # all regression sections
node test/run-regression.mjs docs # documentation checks only
node test/run-conformance-all.mjs # conformance layers; accepts --offline
node test/run-iso-strict.mjs # strict ISO core
node test/run-iso-part2-amendment.mjs
node test/run-neumerkel.mjs # reuse cache; download if missing or incomplete
# --live forces a refresh; --cached never fetches
# --verify-report fails on a stale tracked report
node test/run-neumerkel-tests.mjs # upstream-fetch harness
node test/run-examples.mjs
node test/run-playground.mjs
node test/run-architecture.mjs
node test/run-properties.mjs # seeded random-term invariant checks (see below)
node test/run-openrulebench.mjs
node test/run-http-json.mjs
node test/run-interop.mjs # requires the comparison engines
npm run neumerkel runs node test/run-neumerkel.mjs --live --verify-report:
it fetches upstream and makes a stale tracked report an error. Normal tests
warn about report drift while still checking every discovered case.
npm run sync uses --live --update-report to refresh and record that evidence.
The version lifecycle syncs once before the full test run; tests reuse the
same snapshot. Every other runner is invoked directly.
Focused checks do not replace the full release gate.
run-properties.mjs is different in kind from the rest of the suite: instead
of hand-picked inputs, it generates many random ground terms from a seeded
PRNG and checks invariants that must hold for any input. It covers term/list/
arithmetic/string library predicates (write/read round-tripping, sort/2
ordering and idempotence, append/3/length/2 agreement, compare/3
symmetry, keysort/2 stability, =.. round-tripping, copy_term/2 sharing,
reverse/2 involution, atom_codes/2/atom_chars/2/char_code/2/
number_codes/2 round-tripping, succ/2 and abs/1/sign/1 arithmetic
identities, nth0/3/nth1/3 agreement, last/2, min_list/2/max_list/2
bounds, sum_list/2 additivity, atom_concat/3/string_concat/3 regrouping,
atom_string/2/number_string/2 round-tripping, list_to_set/2 idempotence,
permutation/2, between/3 range generation, split/3/join/3 inversion)
and, more broadly, control constructs, exceptions, the database, and grammar
rules (catch/3 ball delivery, findall/3 order/duplicate/no-match
behavior, double negation, the exact existence_error indicator for an
undefined predicate, assertz/1+retract/1 leaving no trace, a DCG list
rule accepting exactly the list it was built from, bagof/3 agreeing with
findall/3, and numbervars/3 counting distinct variables) – around 35
properties, each reported as its own trial (currently ~530 individual test
lines) so a failure names exactly which trial of which property broke rather
than leaving a reader to dig through one aggregate error. The whole file
still runs in about two seconds. The seed (SEED in that file) is fixed so a
failure is exactly reproducible by rerunning it – change it only
deliberately, and say why, never to make a transient failure disappear.
There is no separate wall-clock benchmark harness: npm test’s own elapsed
time is the project’s performance indicator.
Example, proof-example, and proof-checking tests print results as they finish, so long-running proofs do not hide completed tests behind alphabetical order. Every test still compares its results with the same golden files.
See the conformance guide for report maintenance.