Question → Target
Comparison & Review
Where a recorded target meets a measured value. Each comparison states the scope of its measurement, because a value inside a synthetic scope and a value on real cases answer different questions.
Target against measured value
“Within the criterion in this scope” means the measurement met the predefined criterion inside the scope named beside it. It does not mean the project’s target has been achieved.
| Project | Internal target | Measured value | Scope of the measurement | Reading |
|---|---|---|---|---|
| 19-1 | At least 90% of causal genes in the top five | 172 of 1,874 in the top five (9.18%) | Published cases | Below the target |
| 8-1 | At least 95% of downstream predictive performance retained | Lowest of 4 endpoints 91.4%; 3 of 4 at or above 95% (synthetic, baseline generator; release blocked) | Synthetic inputs only | Below the target |
| 18-2 | Zero missed contraindications or look-alike medication errors | 0 of 4,500 injected errors passed (simulated vision outputs; 62.0% of correct fills passed automatically) | Synthetic inputs only | Within the criterion in this scope |
| 16-2 | Difficulty convergence in at least 95% of simulated episodes | Synthetic convergence rate 96.2% (lower bound 0.9505) | Synthetic inputs only | Within the criterion in this scope |
| 16-3 | External cohort AUROC of at least 0.90 | Synthetic external AUROC 0.932 and 0.926; below target: ZERO_ADAPTATION_B_TO_A, SPEECH_UNSEEN_UNIT, EFFECT_SCALE_0.5, EFFECT_SCALE_0.75 | Synthetic inputs only | Differs by measure |
| 12-1 | At least 90% recall for acute crisis signals | Synthetic recall 0.9155 | Synthetic inputs only | Within the criterion in this scope |
| 12-2 | Zero clinically disallowed response events on the fixed test | 0 of 30,190 (synthetic) | Synthetic inputs only | Within the criterion in this scope |
| 12-3 | At least 85% precision for detecting transitions into sustained isolation | Synthetic precision 0.978 (lower bound 0.933) | Synthetic inputs only | Within the criterion in this scope |
| 12-4 | 100% immediate handoff after a detected crisis state | 1,584 of 1,584 detected turns (synthetic) | Synthetic inputs only | Within the criterion in this scope |
Measurements of something other than the target
These modules measure a different quantity from the project’s internal target, so they are not entered in the comparison above.
ACMG criteria claims audited against public records
Every ACMG criterion an agent claims is checked against an evidence ledger built from public records before it can reach a report. On 20 held-out genes the pipeline kept 1,323 of 1,324 expert-benign variants out of the pathogenic classes (99.92%) and called 984 of 1,890 expert-pathogenic variants pathogenic or likely pathogenic (52.06%). The audit gate stopped 9,493 of 9,493 injected unsupported claims and blocked 0 of the 164,867 criteria derived from the ledger.
Role separation and audit gates on fixed synthetic cases
The runner annotates variants from a fixed VCF subset and ranks candidate genes from HPO phenotypes; a separate audit command rechecks evidence codes, ranks and input hashes. 2 of 4 synthetic cases reached the top five. A tie counts only if the whole tied group fits within five places.
What a comparison does not show
No comparison on this screen was made on an independent external cohort, and none is signed by the auditor. The limits of each measurement are listed on the project record it links to.