EXP-PROMPT-MECHANISM-FACTORIAL
✓ successQ2 — prompt-mechanism factorial: which axis dominates the 85.5× lift?
In plain language
Researchers tested different parts of a special instruction to see which one was most responsible for getting creative answers. They found that telling the AI to act like a novelty-seeking persona was the main driver of these creative responses. The other parts of the instruction, like explicitly asking for new ideas or framing existing ones differently, had a much smaller effect.
▶ Technical details
Research hypothesis
Q1 (exp-pure-llm-licensing-ceiling, bfb4bc1c6, 2026-05-13) showed 100% INTERNAL_NOVEL under MAX-licensing prompt (Wilson 95% [96.4%, 100.0%], N=10, $0.13) vs paper8 baseline 1.17% under CONSERVATIVE prompt — 85.5× lift via prompt change ONLY (same engine, same model gemini-2.0-flash-lite, same RULE_SPACE_SEED, same agent rotation). Q1 decomposes the MAX-licensing prompt into 3 axes: (1) persona (novelty-seeking vs conservative), (2) explicit-license in propose-template, (3) counter-RULE_SPACE framing of seed values. This Q2 factorial isolates each axis to identify the dominant prompt-content mechanism. PRE-REGISTERED VERDICTS (LOCKED 2026-05-13 BEFORE RUN): - SINGLE-FACTOR-DOMINANT: one factor's main effect ≥80% of total INTERNAL_NOVEL variance across cells (sum-of-squares decomposition on the 2×2×2). Name the factor. Other two contribute <20% combined.
- TWO-FACTOR-INTERACTION: interaction term (between two factors) ≥40% of the sum of those two factors' main effects. The factorial must "synergize" — neither alone reaches the joint.
- COMPOSITE-LICENSING-REQUIRED: full-max cell (novelty-seeking × yes-license × yes-counter) ≥ 95% AND every 7-other cell ≤ (full-max - 30 ppt). All 3 factors needed to lift past the 70% threshold; removing any single factor drops ≥30 ppt.
- COMPLEX: pattern doesn't cleanly fit any of the above. Document residual structure (e.g., 3-way interaction dominant, or non-monotonic ordering).
Computation: per-cell pooled INTERNAL_NOVEL = (truly_novel + lineage_extension) / total_accepts across N=5 runs/cell. Wilson 95% CI per cell. Main effects = marginal means per factor level. Interaction terms = (cell mean - sum of main effects). Variance decomposition via stdlib (no scipy needed for 2×2×2). COMPARATORS: - Q1 anchor: 100.0% (Wilson [96.4%, 100.0%], pooled N=10, full-MAX cell) - paper8 floor: 1.17% (943 accepts archived corpus, conservative prompts) - Expected pattern (based on theory): novelty-seeking persona alone should not reach ceiling; explicit-license in propose should be largest single main effect (most direct mention of "novel"); counter-RULE_SPACE should modulate.
Experimental setup
Type: factorial-2x2x2
| Condition | Parameters |
|---|---|
| NS_LIC_CTR | persona: novelty-seeking, explicit_license: yes-license, counter_rule_space: yes-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Existing values (these have been tried — SURPASS them): {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| NS_LIC_NOC | persona: novelty-seeking, explicit_license: yes-license, counter_rule_space: no-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| NS_NOL_CTR | persona: novelty-seeking, explicit_license: no-license, counter_rule_space: yes-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} RULE PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Existing values (these have been tried — SURPASS them): {rule_space_str}{axis_hint} Propose a rule modification. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| NS_NOL_NOC | persona: novelty-seeking, explicit_license: no-license, counter_rule_space: no-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} RULE PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a rule modification. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| CO_LIC_CTR | persona: conservative, explicit_license: yes-license, counter_rule_space: yes-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Existing values (these have been tried — SURPASS them): {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| CO_LIC_NOC | persona: conservative, explicit_license: yes-license, counter_rule_space: no-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| CO_NOL_CTR | persona: conservative, explicit_license: no-license, counter_rule_space: yes-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} RULE PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Existing values (these have been tried — SURPASS them): {rule_space_str}{axis_hint} Propose a rule modification. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| CO_NOL_NOC | persona: conservative, explicit_license: no-license, counter_rule_space: no-counter, coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} RULE PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a rule modification. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
Factors: persona (novelty-seeking, conservative) × explicit_license (yes-license, no-license) × counter_rule_space (yes-counter, no-counter)
Parameters
- n_agents
- 4
- n_rounds
- 100
- model
- google/gemini-2.5-flash-lite
- temperature
- 0.85
Trophic Ratios by Condition
Mean trophic ratio per agent across runs. Error bars = ±1 std dev. Higher TR = more upstream (exporter).