EXP-PERSONA-NOVELTY-DISENTANGLE
… pendingPersona x leakage 2x2 disentangle (paper7 step-1) - confound vs basin
In plain language
This experiment tested whether a "child" persona genuinely leads to more creative responses or if previous findings were just due to the prompt itself containing words related to the expected creative answers. Researchers created four scenarios: a child persona with and without potentially revealing words in the prompt, and a scientific persona with and without those same words. The results are still being analyzed to see if the child persona truly drives creativity or if the prompt's wording was the real cause.
▶ Technical details
Research hypothesis
exp-persona-novelty-basin (commit 517e948b0, 2026-05-09) reported CHILD persona produced 33.33% novel-value rate vs 0% across 4 other personas. Accepted novel values were 'smell', 'taste', 'yummy'. CONFOUND identified by mnsky-research note (memory/research/2026-05-09-persona-novelty-confound-elicitation-vs-basin.md): the CHILD prompt literally contained "signals could be smells, voting could be tastes" — accepted values are lexical neighbors of seeded nouns. This is textbook semantic leakage (Gonen et al. 2024 EMNLP).
This 2x2 factorial disentangles persona-role from noun-seed-leakage:
- persona-role: {CHILD, SCIENTIFIC}
- noun-seed-leakage: {PRESENT, ABSENT}
4 conditions x 8 runs = 32 runs. rule_multigen, meta_modifiable=true, 4 agents, 5 gens x 20 rounds, budget=400, gemini-2.5-flash-lite, agent_spawn_enabled=false, mute_signal=false (identical to basin experiment).
PRIMARY confound-falsification: if SCIENTIFIC-LEAK >= CHILD-NOLEAK in novel-value rate -> confound was prompt-content (semantic leakage). 'child basin' is phantom. paper6 step-8 finding revised to 'novelty came from prompt seed, not persona shift'.
PRIMARY basin-confirmation: if CHILD-NOLEAK >= 5x SCIENTIFIC-LEAK in novel-value rate -> child persona effect real BEYOND leakage. true persona-attractor finding.
SECONDARY interaction: 2x2 contingency check for persona x leak interaction (chi2 or Bayesian factorial). if interaction strong -> persona modulates leakage receptiveness, neither pure effect.
KILL: any condition produces engine breakage (>10% JSON parse failures, agents drift to non-task generation) -> void the condition, retry.
Experimental setup
Type: factorial-2x2
| Condition | Parameters |
|---|---|
| CHILD-LEAK | persona_role: CHILD, noun_seed_leakage: LEAK, meta_modifiable: true, mute_signal: false, agent_spawn_enabled: false, n_runs: 8, note: replicates basin CHILD condition exactly |
| CHILD-NOLEAK | persona_role: CHILD, noun_seed_leakage: NOLEAK, meta_modifiable: true, mute_signal: false, agent_spawn_enabled: false, n_runs: 8, note: isolates basin: removed smell/taste hint |
| SCIENTIFIC-LEAK | persona_role: SCIENTIFIC, noun_seed_leakage: LEAK, meta_modifiable: true, mute_signal: false, agent_spawn_enabled: false, n_runs: 8, note: isolates leakage: scientific role + smell/taste hint inserted |
| SCIENTIFIC-NOLEAK | persona_role: SCIENTIFIC, noun_seed_leakage: NOLEAK, meta_modifiable: true, mute_signal: false, agent_spawn_enabled: false, n_runs: 8, note: control = baseline equivalent (no role-play, no seed) |
Factors: persona_role (CHILD, SCIENTIFIC) × noun_seed_leakage (LEAK, NOLEAK)
Parameters
- n_agents
- 4
- n_rounds
- 100
- model
- google/gemini-2.5-flash-lite
- temperature
- 0.85
Trophic Ratios by Condition
Mean trophic ratio per agent across runs. Error bars = ±1 std dev. Higher TR = more upstream (exporter).