Early Boost for Core Health R&D Technology

Research question and target

Align variables and clinical time series by schema, then assess predictive utility and privacy protection together.

Internal target

At least 95% of downstream predictive performance retained

Programme

Discovery and data

Evidence on this record

Synthetic harness (synth8)Common synthetic checks

Evaluation data

• TCGA-BRCA PanCancer Atlas RNA, copy number and progression data

  PanCancer Atlas 2018 · cBioPortal dda832891555904730395d476c5c888b2e1553e1

  Manifest locked

Evaluation registry · Labels and split definitions (KR)

Synthetic harness · synth8

Privacy-preserving synthetic data engine and TSTR validation harness

The decision apparatus classified 4 of 4 generators with known answers correctly. The differentially private baseline generator, measured at epsilon 2.0, kept 100.1% of C-index, 99.2% of macro-F1, 97.4% of AUROC and 91.4% of biomarker rank agreement relative to a model trained on the reference table of the same synthetic world. The last is below the 95% threshold, so the release rule blocked it. Its membership-inference AUROC was 0.508 (upper bound 0.538).

Checks recomputed by the audit

860

Model trained on the synthetic world's reference table (TRTR)

AUROC 0.908, C-index 0.721, macro-F1 0.733, rank agreement 0.958

Privacy budget of the baseline generator

epsilon 2.0, delta 1e-05; formal AUROC bound 0.881

Membership inference, two attacks

worst AUROC 0.508, upper bound 0.538, threshold 0.55

Distance to closest record

0 duplicates; gate PASS

Known answer: verbatim copy

573 of 600 rows flagged as duplicates; attack AUROC 0.920

Known answer: 15% of rows copied

81 duplicates; attack AUROC 0.557; true-positive rate 13.2% at 1% false positives

Lowest preservation across privacy budgets

epsilon 1: 34.1%; epsilon 4: 82.9%; epsilon 8: 92.5%; epsilon 16: 91.4%

DP-SGD noise multiplier, TCGA-sized plan

sigma 15.94 at epsilon 1; sigma 5.62 at epsilon 3; sigma 2.41 at epsilon 8

Epsilon that bounds every attack at the threshold

0.201 (sigma 78.0 in the TCGA-sized plan)

Members and non-members in the privacy audit

720 each; 701 needed for 90% power

Release verdict for the baseline generator

BLOCKED: UTILITY_FAIL

Limits

• The world is a toy: three subtypes, four shared factors, three batches and 26 encoded features. It exercises the gates on known answers and says nothing about TCGA, GEO or MIMIC-IV.
• The baseline generator is an analytic Gaussian model in a public low-rank basis. With negligible noise its rank agreement averaged 96.3% (lowest 92.8%) over 4 probe seeds: it sits at the threshold by its structure. The shortfall is a property of this baseline, not of the main generator, which was not trained.
• The membership-inference result covers two specified attacks. The formal guarantee at this budget is the AUROC bound in the table, far weaker than the threshold; the release rule therefore requires both the ledger and the empirical audit.
• The audit recomputes metrics, bounds and verdicts from the score and distance arrays of the run. Refitting the generator and the learners is covered by byte-identical re-execution, not by the audit.
• A raw preservation ratio can pass with both models at chance level, so the rule also requires skill above chance and an informative reference model. Ratios above 100% are sampling variation, not improvement.

Synthetic inputs only (synthetic_only). These figures come from a synthetic multi-omics world generated by code. No real cohort (TCGA, GEO, MIMIC-IV) was evaluated, the main latent-diffusion generator was specified but not trained, and no auditor has signed the result. A separate audit program passed 4 of 4 gates.

Audit record (JSON) · Run output (JSON) · 8-1 design specification: privacy-preserving synthetic data and TSTR validation (KR) · 8-1 harness modules and replay record (KR) · 8-1 null calibration of the distance gate (KR) · 8-1 run digest without score arrays (KR) · 8-1 and 18-2 cross-model review record (KR)

python3 -B -m synth8.orchestrate --output evidence/synth8-run.json
python3 -B -m synth8.audit --run evidence/synth8-run.json --output evidence/synth8-audit.json
python3 -B -m unittest discover -s tests -p 'test_synth8.py' -v

Sources and eligibility

• NCI GDC / TCGA

  Open and controlled files

  Files are divided into open and controlled tiers. For controlled access, dbGaP approval and eRA Commons authentication are required.

• NCBI GEO

  Open

  Platforms and variable definitions vary by study. Verify the endpoint and subject-level linkage for each series.

• PhysioNet MIMIC-IV

  Credentialed access

  Deidentified records still require credentials, training and a data use agreement. Sharing conditions apply to derived models and data.

Boundary cases

• Original rows or repeated records for the same person appearing across splits
• Differential privacy budget and proximity of synthetic rows to source records
• An internal target of zero biologically impossible variable combinations

Common synthetic checks

• Synthetic record schema and variable consistency · 5/5 matched

Expected and observed values for these cases

What the official call asks

Open to existing or ongoing national health R&D projects at TRL 3 or above. Stage one supports commercialisation strategy and consulting; stage two supports competitive follow-on development.

Proposal requirements, printed pages 31–36 · All twelve requirements

Interpretation and next evaluation

TCGA, GEO and MIMIC-IV are separate sources. Validation follows each source's endpoint, access terms and verified linkage.

Next check

Run the same harness on the fixed TCGA-BRCA institution split with the baseline generator, then train the main generator and require it to beat the baseline on every endpoint at the same privacy budget.