PCI RR — early data collection and software revisions
Origin. raw/_4_for_WS_PCI_RR_Early_Data_Collection_and_Software_Revisions.docx
One-line gist. A practical assessment of whether starting data collection before Stage
1 in-principle acceptance is compatible with a PCI Registered Report, which problems stay
repairable during a 12-week study, and which are not.
Bottom line, as stated in the source
The PCI RR procedure is not necessarily destroyed. Software development, testing, and the final evaluation may be corrected during data collection, provided that outcome-relevant study data remain unseen and the revised pipeline is approved and frozen before evaluation. If the acquisition itself is invalid, however, later software repair may not rescue the already collected observations.
Key claims
- The decisive distinction is not timing but access. Accumulating study data must not be used to decide how the software, features, thresholds, exclusions, statistical model, or evaluation procedure should work. Development and validation should use synthetic data, external data, or a clearly separated pilot dataset.
- Bias-control level at in-principle acceptance depends on data access, not merely on whether collection has begun. → PCI Registered Reports
- What cannot safely be postponed: the prospective definition of the measurement, its validity criteria, and the safeguards preventing accumulating data from influencing development decisions.
Repairable versus fatal, per the source’s own table
| Day-30 criticism | Status | Response |
|---|---|---|
| Analysis or evaluation plan insufficient | Usually repairable | Lock accumulated data, revise plan before inspecting outcomes |
| Evaluation code not yet developed | Usually repairable | Implement and test on synthetic, external, or pilot data |
| Software testing or validation inadequate | Usually repairable | Add tests, validation datasets, prospective acceptance criteria |
| Feature definitions or algorithm need clarification | Often repairable | Freeze revised specification prospectively, document the change |
| Acquisition software produced unreliable recordings | Potentially fatal for early data | First 30 days may be unusable; missing information cannot be reconstructed |
| Review requires changing what or how data are recorded | Major difficulty | Pre- and post-change observations may not be comparable |
| The therapy procedure itself must change | Major difficulty | Study may no longer implement one consistent protocol |
| Outcome-related variables already inspected | Serious threat | Prospective bias protection falls substantially; safeguards or redesign required |
Notable details
- Six-step recommended safe timeline, from Day-1 specification submission through to running the frozen pipeline after the 12-week collection window.
- Three defensible fallback options if the acquisition method turns out to have been incapable of valid measurement: exclude pre-correction participants prospectively, treat early observations as pilot data and restart the confirmatory cohort, or redesign and resubmit Stage 1.
- The handling recommender should be informed immediately about any inflexible start date and the data-access safeguards in place.
- Source of the taxonomy: PCI Registered Reports, “Guide for Authors”.
Pages touched by this source
PCI Registered Reports (created) · Speech Gap · Semmelweis / VIKOTE
Open questions raised by this source
- The guidance is general. Applying it to the specific Speech Gap / BPD timeline requires the recruitment start date and the current data-access arrangements, which this source does not fix.