16-3 · Cognitive and neurological health · 2026-11-14

Multimodal Dementia Diagnostic Research

Cohort and simulationSynthetic checks matchedEvaluation data partly established

Study whether imaging and speech features retain their predictive value across hospitals and equipment.

Research question and target

Evaluate MRI and PET features along with pauses, vocabulary and sentence features from speech. Compare fixed models on independent cohorts.

Internal target: External cohort AUROC of at least 0.90
Programme: Cognitive and neurological health
Evidence on this record: Synthetic harness (cog16)Common synthetic checks

Evaluation data

OASIS-2 MRI visit metadata
Official OASIS-2 demographic workbook and MRI archives
Manifest locked
DementiaBank ADReSS Challenge 2020
2020 official split and test labels
Split rule locked
ADNI / OASIS-3 / DementiaBank
Provider data definitions and access conditions
Blocked

Evaluation registry · Labels and split definitions (KR)

Synthetic harness · cog16

Multi-site MRI and speech fusion: cross-cohort validation on synthetic cohorts

On synthetic multi-site cohorts, the external AUROC for separating controls from MCI or dementia was 0.932 (DeLong 95% lower bound 0.909) on 540 held-out cases in one direction and 0.926 (0.905) on 560 in the other, with harmonisation referenced to control cases. The internal target is 0.90.

Seed replications at or above the target: 9 of 10; lower bound above target in 7; minimum 0.881
Synthetic paired fusion: AUROC 0.955
Conditions below the target: ZERO_ADAPTATION_B_TO_A, SPEECH_UNSEEN_UNIT, EFFECT_SCALE_0.5, EFFECT_SCALE_0.75
AUROC figures recomputed by the auditor: 21

Limits

All cohorts are synthetic and the AUROC follows the assumed disease effect size: at 0.5, 0.75 times the assumed effect it falls below the target. No ADNI, OASIS-3 or DementiaBank data were read.
The public datasets do not link MRI and speech from the same participant. Paired fusion was evaluated only on a synthetic paired cohort and is reported separately from the primary result.
Harmonising a scanner that was absent from training needs labelled control cases from that site, kept apart from the test split. Without them the design holds the score instead of reporting it.
FreeSurfer and acoustic feature extraction, the PyTorch modules and domain-adversarial training are specified but were not run. The harness trains a standard-library reference network on already extracted synthetic features.
The auditor recomputes each AUROC and interval from stored per-case scores. It does not refit the models that produced the scores.
The output is a research risk score for clinician review. It does not issue a diagnosis.

Synthetic inputs only (synthetic_only). These are virtual metrics on synthetic cohorts. No ADNI, OASIS-3 or DementiaBank data were read, no real cohort has been evaluated, clinical validity has not been assessed and no auditor has signed the result. A separate audit program passed 4 of 4 gates.

G1 · Frozen inputs and submission structureG2 · Verdict definition, control completeness and structure rulesG3 · Independent recalculation of rates, intervals and verdictsG4 · Virtual metric verdict and result hash

Audit record (JSON) · Run output (JSON) · 16-3 design specification: multi-site MRI and speech fusion (KR) · 16-2 and 16-3 replay record (KR) · 16-2 and 16-3 cross-model review log (KR) (KR)

python3 -B -m cog16.orchestrate --check evidence/cog16-run.json
python3 -B -m cog16.audit --run evidence/cog16-run.json --output evidence/cog16-audit.json
python3 -B -m unittest discover -s tests -p 'test_cog16_*.py' -v

Sources and eligibility

OASIS-3
Account and data agreement
The longitudinal imaging, clinical and cognition cohort requires a NITRC account. Linkage to DementiaBank participants remains unverified.
ADNI
Application required
Includes imaging, diagnosis and cognitive assessments. Confirm coverage by study phase and visit-level variables.
DementiaBank
Membership required for most data
Audio, transcripts and diagnostic coverage vary by corpus. Matching the same people to scans remains unverified.

Boundary cases

Whether MRI and speech can be matched by participant and visit
Performance changes across scanners, sites and audio settings
Lock model weights and diagnostic definitions before external evaluation

Common synthetic checks

Disjoint sites, scanners and participants, with subject-level modality pairing · 5/5 matched

Expected and observed values for these cases

What the official call asks

Prediction and diagnosis areas require a completed exploratory clinical performance study or an approved confirmatory study plan. The imaging area requires validation at two or more independent tertiary hospitals. Both areas set paper targets.

Proposal requirements, printed pages 82–84 · All twelve requirements

Interpretation and next evaluation

Single-modality evaluation is available pending imaging and speech matched to the same participant and diagnostic visit.

Next check: After approved data import, run the same harness with ADNI and OASIS-3 as the two external directions and evaluate speech separately; the synthetic figures are then replaced.

← All twelve projects