EXP-PAPER9-CROSS-ENGINE-VALIDATION
… pendingpaper9 phase-2 cross-engine validation (flash-2.5-lite)
In plain language
This experiment tested if a new AI model architecture, designed to improve how AI understands concepts, works consistently across different versions of the AI engine. Researchers ran the same tests on an older engine and a newer one to see if the results were the same. The results are still being analyzed to determine if the architecture is robust or if its performance is specific to a particular engine.
▶ Technical details
Research hypothesis
Cross-engine validation of paper9 V3 two-channel architecture finding. Re-runs exp-paper9-phase2-rerun-with-trace identically (same 4 cells × N=5 = 20 runs, same NS+LIC prompts, same substrate seed) but on gemini-2.5-flash-lite (paper9 V2's ORIGINAL engine, vs. flash-2.0-lite-001 in the rerun-trace). Tests whether the STRATIFIED-CONCEPT inverse cell-ordering (HMACE > CORAL > GenericAgent > MemQ at concept-novel) is engine-portable or 2.0-lite-specific. Pre-registered verdicts:
- CROSS-ENGINE-CONFIRMED: flash-2.5-lite produces same cell-ordering at concept-novel (rank-1 cell = HMACE), with Wilson95 CIs that don't overlap arbitrarily → two-channel architecture survives engine swap → paper9 V3 robust → paper9 → READY-FOR-PUB. Operational discriminator: HMACE concept-novel ≥10pp above next-highest cell at concept-novel.
- CROSS-ENGINE-FALSIFIED: flash-2.5-lite produces DIFFERENT cell-ordering at concept-novel (rank-1 ≠ HMACE OR ordering scrambled) → engine- specific artifact → paper9 V3 needs scope-narrow («on flash-2.0-lite specifically») rewrite before publication.
- AMBIGUOUS: Wilson CIs broadly overlap at N=5 (HMACE−nextcell < 10pp OR overlapping CIs across rank-1 vs rank-2) → defer to N=10/cell follow-on.
Experimental setup
Type: 4-cell-ablation-with-trace
| Condition | Parameters |
|---|---|
| CORAL-external | mechanism: external-deterministic-grader + greedy-feedback |
| GenericAgent-absent | mechanism: no-closure + uniform-random |
| HMACE-distributed | mechanism: 4-role-persona-rotation + no-central-scalar |
| MemQ-in-retrieval | mechanism: q-value-em-retrieval-from-memory |
Factors: closure_placement (CORAL-external, GenericAgent-absent, HMACE-distributed, MemQ-in-retrieval)