COG-CYCLE-V5-N5-NOISE-FLOOR
◐ partialcog-cycle-v5-N5-noise-floor
In plain language
We ran two similar 3-agent thinking experiments twice each before and got slightly different numbers, even though the setup was identical. Now we run each one five times to measure how much the numbers normally jiggle when nothing changes. If the jiggle is small, we can trust future single-run experiments. If it is big, we have to run everything many more times.
▶ Technical details
Research hypothesis
v5 noise-floor measurement gate. v4 commit b6992b1c2 revealed 3pt P4 max-cos drift (v4 PARTIAL_OBS_ONLY=0.918 vs v3 PARTIAL_OBS=0.886) across nominally-identical control runs — half the 0.85 threshold-crossing being chased. Any further single-replicate v5-v6 substrate-modification (DPO / activation steering) verdicts are uninterpretable until noise floor σ(P4) and σ(P6 role-coherence) are established at N=5 per cell. v5 design: 2 conditions × 5 replicates each, both copying EXISTING v3+v4 partial-obs configurations verbatim. V3_REPLICATE = v3 PARTIAL_OBS (asymmetric beliefs, partial-env). V4_REPLICATE = v4 PERSONA_ROLES_PARTIAL_OBS (functional roles, partial-env). No new factors. Pure replication-noise measurement. Primary discriminator: σ(P4 max-cos) across 5 reps of each config. If both σ<0.02 AND σ(P6 role-coherence per agent)<15pp → CONFIRMATION: noise floor low enough for v6 substrate-modification dispatch at N≥3. If σ(P4)>0.05 OR σ(P6)>30pp → KILL: require N≥10 or new metric. Inter-config Δμ/σ_pooled ≥ 2.0 → real persona effect distinguishable from noise; <1.0 → indistinguishable.
Experimental setup
Type: 2-condition-replication-noise-floor
| Condition | Parameters |
|---|---|
| V3_REPLICATE | architecture: cognitive_cycle, asymmetric_beliefs: true, private_memory: true, typed_protocol: true, use_retrieval: true, env_mode: partial, persona_mode: beliefs_only, env_streams: [object Object], agents: [object Object],[object Object],[object Object], note: Verbatim copy of v3 PARTIAL_OBS (asymmetric beliefs only, partial-env). 5 reps to measure σ(P4) noise floor under v3 setup. |
| V4_REPLICATE | architecture: cognitive_cycle, asymmetric_beliefs: true, private_memory: true, typed_protocol: true, use_retrieval: true, env_mode: partial, persona_mode: functional_roles, env_streams: [object Object], agents: [object Object],[object Object],[object Object], note: Verbatim copy of v4 PERSONA_ROLES_PARTIAL_OBS (functional-role personas, partial-env). 5 reps to measure σ(P4) and σ(P6) noise floor under v4 setup. |
Factors: persona_mode (beliefs_only, functional_roles) × n_replicates (5)
Parameters
- model
- qwen2.5-7b-instruct-1m
- n_agents
- 3
- n_runs_per_condition
- 5
- temperature
- 0.85