Skip to content

Validation tiers: definitions block, expectations warn, comparisons review

TCKDB validates uploaded science in four distinct places, and which place a check lives in is a scientific decision, not an implementation detail. Pydantic schema validation refuses a payload outright at upload. UploadWarning records a machine-readable (field, code, message) alongside a payload that is accepted anyway. HardFailReason in the trust evaluator labels a stored record at read time without refusing anything. machine_review runs asynchronously under versioned rubrics and raises curator tasks. Those are four different consequences — refusal, annotation, labelling, and human referral — and until now checks were assigned to them case by case.

The rule is that a check may block only when it asserts a definition or a contract, never when it asserts an expectation. "A transition state has exactly one imaginary frequency" is the definition of a first-order saddle point; zero means the geometry is a minimum and two means it is a higher-order saddle, so a record declaring otherwise is internally contradictory and no correct calculation can produce it. "The imaginary frequency should exceed 100 cm⁻¹ in magnitude" is an expectation; flat and variational transition states are real, and blocking them would reject correct science. The test to apply is simply whether the check could plausibly fire on a correct novel result. If it could, it must not block, because a database that refuses correct results is worse than one that flags them — the same reasoning that led 0005 to record execution environments rather than grade them, and 0002 to keep trust, review, and reproducibility as independent judgements that may legitimately disagree.

That rule sorts the existing checks cleanly, and doing so exposes two problems. The first is that contradiction and absence are currently mixed in one tier. Of the twenty upload warnings, ten are W_MISSING_*_PROVENANCE-style absences — missing level-of-theory provenance, missing statmech source calculations, missing tunneling application — and those are correctly non-blocking, because an incomplete record is still a true record and the trust and reproducibility layers exist precisely to grade that incompleteness. But W_N_IMAG_CONTRADICTS_MINIMUM, W_N_IMAG_HIGHER_ORDER_SADDLE, W_CHARGE_MISMATCH, and W_MULTIPLICITY_MISMATCH are not absences. Each is a direct contradiction between what the uploader declared and what the parsed evidence shows, and each is definitional. They belong at the blocking tier and are currently advisory.

The second problem is duplication across tiers. W_N_IMAG_HIGHER_ORDER_SADDLE (upload_reconciliation.py:33) and HardFailReason.frequency_source_has_multiple_imaginary_modes_for_validated_ts (trust/models.py:127) encode the same physical fact, evaluated twice, at different times, with different consequences: the first annotates an accepted upload, the second labels the record at read time. A reader cannot tell from either one what the system's actual position is. Where the same fact is checked in more than one tier, the blocking tier owns it and the others cite it rather than re-deriving it — otherwise the tiers can disagree about the same record, and the more permissive one silently wins at upload time.

Cross-checks against external reference data — comparing an uploaded enthalpy against RMG-database group additivity, or a rate against an evaluated kinetics library — belong in machine_review and nowhere earlier. Such a comparison can be wrong in both directions: the reference can be stale or inapplicable, and a genuine disagreement may be the interesting result rather than the error. It is therefore an expectation by construction and can never block. Putting it in the upload path would additionally make ingestion depend on a multi-gigabyte dataset, pin validation to one reference commit that will drift, and couple upload availability to a comparison that is not needed for the record to be correct. machine_review already versions its rubrics, so a finding recorded there stays interpretable later: it says which reference version produced the judgement, which is what makes a disagreement actionable rather than merely alarming.

Consequently, adding a new check requires stating which tier it belongs to and why, in the terms above. The audit implied by this decision — promoting the four contradiction warnings to blocking, and collapsing the duplicated imaginary-mode check onto a single owner — is deliberately not performed here, because promoting a warning to a blocker will reject payloads that are accepted today and that is a migration with its own evidence requirements, not a refactor.