Question → Target
Validation Roadmap
Follow each project from its current evidence to the next recorded check, and see what that check would unlock.
A completed computation is separate from a claim about the project’s target. The registry state says whether the data for the next check exists.
2Projects with evaluation data readyA public split is locked for the original endpoint
10Projects partly establishedA reference label or linkage is still missing
0Projects signed by the auditorSign-off follows isolated replay
Next check for each project
| Project | Evaluation data | Current evidence | Next check | Unlocks |
|---|---|---|---|---|
| 5-1 · AI drug discovery | Evaluation data partly established | Common synthetic checks | Select records with measured binding values, specify assay conditions and lock the split by scaffold, target family and date. | Binding free-energy error of 1 kcal/mol or less |
| 8-1 · Synthetic data | Evaluation data ready | Synthetic harness (synth8) · Common synthetic checks | Run the same harness on the fixed TCGA-BRCA institution split with the baseline generator, then train the main generator and require it to beat the baseline on every endpoint at the same privacy budget. | At least 95% of downstream predictive performance retained |
| 18-2 · Medication safety | Evaluation data partly established | Synthetic harness (dispense18) · Common synthetic checks | Train the vision modules on the fixed ePillID split, build per-category calibration sets from its validation split and fill the same table on its test split; categories below the minimum calibration size are reported as uncertified with no automatic pass. | Zero missed contraindications or look-alike medication errors |
| 19-1 · Rare disease diagnosis | Evaluation data ready | Published-case evaluation · Public-record audit (acmg19) · Synthetic replay (rare19) · Common synthetic checks | Read the literature for every expert-pathogenic missense variant instead of a sample, so that functional, segregation and case findings enter the ledger. Then re-measure sensitivity on genes outside the current 40, with the condition that specificity does not fall. | At least 90% of causal genes in the top five |
| 16-2 · Adaptive cognitive training | Evaluation data partly established | Synthetic harness (cog16) · Common synthetic checks | Import stratum-level summary statistics inside the data-use zone, re-freeze the fixture and re-run; then test a response generator whose functional form differs from the scheduler's model. | Difficulty convergence in at least 95% of simulated episodes |
| 16-3 · Dementia diagnosis | Evaluation data partly established | Synthetic harness (cog16) · Common synthetic checks | After approved data import, run the same harness with ADNI and OASIS-3 as the two external directions and evaluate speech separately; the synthetic figures are then replaced. | External cohort AUROC of at least 0.90 |
| 18-1 · Fall early warning | Evaluation data partly established | Common synthetic checks | Register participant and event splits, alert lead time and the time denominator for false alarms. | At least 92% early-warning sensitivity in the high-risk group |
| 18-3 · Surgical safety | Evaluation data partly established | Common synthetic checks | Lock visibility, occlusion and tracking ID criteria and define separate video and synthetic count metrics. | Zero instrument count omissions on the fixed test |
| 12-1 · Acute crisis signals | Evaluation data partly established | Synthetic harness (prevent12) · Common synthetic checks | Use structured event markers for synthetic evaluation. Define the requirements for longitudinal labels in external evaluation. | At least 90% recall for acute crisis signals |
| 12-2 · Safety-plan transitions | Evaluation data partly established | Synthetic harness (prevent12) · Common synthetic checks | Lock the synthetic transition table and denominator. Define the independent outcomes required for further follow-up. | Zero clinically disallowed response events on the fixed test |
| 12-3 · Social isolation | Evaluation data partly established | Synthetic harness (prevent12) · Common synthetic checks | Register the scale, time window and eligibility criteria for isolation. Test missing sensor readings and ordinary interruptions in simulation. | At least 85% precision for detecting transitions into sustained isolation |
| 12-4 · Student crisis handoffs | Evaluation data partly established | Synthetic harness (prevent12) · Common synthetic checks | Lock decision, success state and maximum delay for synthetic events. Register eligibility criteria for adolescent reference material. | 100% immediate handoff after a detected crisis state |
Order of work
A project moves in one direction: fix the data, labels and split; verify the decision rules on synthetic inputs with an independent audit; evaluate on public data; evaluate on an external cohort once access is approved; then replay in isolation for the auditor’s signature. A later step never substitutes for an earlier one.