In Silico
Reproducibility
Commands that rebuild every published result from fixed inputs, with the SHA-256 of each evidence file so a replay can be compared byte for byte.
Run the commands from the project root with Python 3.10 or later; no external package is required. The runner and the audit of a harness use the same file permissions; a separate host, a container and the auditor’s own replay have not been run.
Common synthetic suite
Re-runs the fixed cases and compares the result with the stored document.
python3 -B engine/run_audit.py --verify evidence/audit.json| File | SHA-256 |
|---|---|
| audit.json | f495b4ab6b3f1d962cecfa96f6254d20486e81b3263d9afaf94dd042c0afb6c5 |
19-1 ranking on published cases
Replays the split and the aggregate from the fixed source files; the registry check compares the manifests.
sh gate1/rare19/replay.sh
python3 -B gate1/assemble.py --check| File | SHA-256 |
|---|---|
| rare19-published-results.json | 803e7c1a7cb4e5348d6c90d80f5ec46fefc8d1611dc39bf38e60627dc53fc9b8 |
| rare19-evaluation.md | ff676ab4a0d3b75a84ee8fa48ddf07e69e52a8aec0fd1951d04a11ab3ef631bd |
Public-record audit · acmg19
Figures come from ClinGen, ClinVar, gnomAD, HPO and published text, and from patients simulated out of those records. No patient data were used and no clinical performance was measured. No human auditor has signed, and this is not performance of the public call.
python3 -B -m acmg19.summarize --check
python3 -B -m acmg19.evaluate > /tmp/acmg19-recheck.json
cmp acmg19/result.json /tmp/acmg19-recheck.json
python3 -B -m unittest discover -s tests -p 'test_acmg19.py'
python3 -B -m acmg19.orchestrate --vcf acmg19/inputs/examples/PAH-02-P.vcf --case acmg19/inputs/examples/PAH-02-P.json --claims acmg19/inputs/examples/PAH-02-P.claims-from-memory.json| File | SHA-256 |
|---|---|
| acmg19-audit.json | dfc76ebcff4956f9755f081a8439ac1ee45af16029aace47cfcbd64954320477 |
| acmg19-run.json | 32924e4d864f887cb776cb6915711dd5cd95bc4b43f4ff0e32cb5bdd0b92a3a4 |
| acmg19-method.md | 0512527dadf7a3742b927e95b73d08994adfbf0e09818926d9858528d72173aa |
| acmg19-aja.md | 5f426b2cf1e0b0bd7633dfa3eb63aac9ca38220f75d53d2224d7d642fbd28097 |
Synthetic harness · cog16
These are virtual metrics on synthetic cohorts. No ADNI, OASIS-3 or DementiaBank data were read, no real cohort has been evaluated, clinical validity has not been assessed and no auditor has signed the result.
python3 -B -m cog16.orchestrate --check evidence/cog16-run.json
python3 -B -m cog16.audit --run evidence/cog16-run.json --output evidence/cog16-audit.json
python3 -B -m unittest discover -s tests -p 'test_cog16_*.py' -v| File | SHA-256 |
|---|---|
| cog16-audit.json | 20c0a233414396d83b066f44289820dd1e78008e0d97255351799793ce2dbbac |
| cog16-run.json | 23792e7c3a0fc60db93d0427857d9c2135d096af33ef56f9c66935daed67e1eb |
| 16-2-spec.md | 4f862f4657dfaa5ee429887dae4dc32460ca82fa64fe0287f8bbfc0d399bf0d9 |
| 16-3-spec.md | 71e054a1b3fda13213b8dd0d1a32570e6a9a173473a92649274e0c4df7691751 |
| cog16-method.md | 5490946ed733f43b864344f71e042280e37220b6fd76722371c69448b65bf4cf |
| cog16-review.md | 792dbe2ebbcb646ad673d4378667598634e4ad417696ed50e672b1eb22d1d52f |
Synthetic harness · dispense18
These figures come from a synthetic gallery, simulated vision outputs and orders written in abstract codes. No pill image was analysed, no real prescription or DUR rule table was used, clinical performance was not assessed and no auditor has signed the result.
python3 -B -m dispense18.orchestrate --output evidence/dispense18-run.json
python3 -B -m dispense18.audit --run evidence/dispense18-run.json --output evidence/dispense18-audit.json
python3 -B -m unittest discover -s tests -p 'test_dispense18.py' -v| File | SHA-256 |
|---|---|
| dispense18-audit.json | e7e300b4a449d268d9c2a84805f2c041bc202f2c03a85838c516a77ce6cfb105 |
| dispense18-run.json | cd411365a2280c168373343984a957087e395ed2a15272d23f42ebab97eaa1f6 |
| 18-2-spec.md | 141324a961a8e5da40be7429795dce09e647b9c9bd64a23760f0859ab2da2125 |
| dispense18-method.md | 0c5fd2aa6185feac6050deb404fb6dc613c682324de49ba0028832f0f4ddc2d5 |
Synthetic harness · prevent12
These are virtual metrics on synthetic inputs. No real cohort has been evaluated, clinical validity has not been assessed and no auditor has signed the result.
python3 -B -m prevent12.orchestrate --output evidence/prevent12-run.json
python3 -B -m prevent12.audit --run evidence/prevent12-run.json --output evidence/prevent12-audit.json
python3 -B -m unittest discover -s tests -p 'test_prevent12.py' -v| File | SHA-256 |
|---|---|
| prevent12-audit.json | 4372ccf135890e9f982bf7b627b1705ec271a7299db9eb39b80f0f46cd900160 |
| prevent12-run.json | 8a8b2e44a1e007a119d5cfdf73f30463f612fe62bb66ef02235c304db35ac924 |
| 12-1-spec.md | bbb79083dc45d1082eac2f9a112522c53e142f257ba1277cecdcff708c85fdbb |
| 12-2-spec.md | a9a70be4d1d76abde8a4b361e65b3473307bb054998e7aee51773ecc49af312d |
| 12-3-spec.md | 488f4ea42838d69c412c5f653abcd4867c76d92918c804cf842adf1a208f70a7 |
| 12-4-spec.md | 330db6c41a27a076eb5e494be32f6ea034712f7f67a7a4c21934e14a2f213de8 |
| prevent12-method.md | 31d9f911eb6fefd1052d9b242ce55be782a9ed5425c6cca4587b4373d6f64408 |
Synthetic replay · rare19
Scope: synthetic inputs and evidence-code review. Patient VCFs, clinical performance and auditor sign-off are not included.
python3 -B -m rare19.orchestrate --output /tmp/rare19-run.json
python3 -B -m rare19.audit --run /tmp/rare19-run.json --output /tmp/rare19-audit.json| File | SHA-256 |
|---|---|
| rare19-audit.json | 53bca4bb94935346a04a018e9e1d0fa1d14767967e823457710e893d7e33d28c |
| rare19-run.json | 5ae7109993ea35969d4cf16248428a39867d1c0ed5b8b9041986a7ee42e57273 |
| rare19-method.md | dcdfc674fc1fd5ae9a1bd4cfe34560c8c01cc943ca5ca34ca7e549a19cd51a39 |
Synthetic harness · synth8
These figures come from a synthetic multi-omics world generated by code. No real cohort (TCGA, GEO, MIMIC-IV) was evaluated, the main latent-diffusion generator was specified but not trained, and no auditor has signed the result.
python3 -B -m synth8.orchestrate --output evidence/synth8-run.json
python3 -B -m synth8.audit --run evidence/synth8-run.json --output evidence/synth8-audit.json
python3 -B -m unittest discover -s tests -p 'test_synth8.py' -v| File | SHA-256 |
|---|---|
| synth8-audit.json | 005a03124a066622193732c1756b9a8f23ce4dd64f9882a2f32b8663e494ad5c |
| synth8-run.json | 0ef973728bff9305e54d040b3bd904120c4e0046aca0a66709bef13cab7b82e8 |
| 8-1-spec.md | 0c6880260a1cf5780e85fa53d8f10b2959ce8e9c851e830a96bace093833ac28 |
| synth8-method.md | d4fd76b6f9530cf787d3e21aa8a364a654b99d1be44afba29359bb5d4f24be0c |
| synth8-calibration.json | c68c620e19c3e435aa15a10739e0b0864ff2ddf5b77bd015bd2e4baf26466056 |
| synth8-run-digest.json | 8c58f7503293903105cb080424e60c812e6e703118fb6bd7f1a51f0614ca874b |
| synth8-dispense18-review.md | e0945318ab1fb0baaa7b5e569b2e17ea08900548e7b2720ecf812e2fa8264bbc |
This site
The generator renders every screen and download in memory, checks each harness summary against its audit, and writes only when all of them pass.
python3 -B scripts/build_site.py --validate
python3 -B scripts/build_site.py
./scripts/check