EXP-CROSS-MODEL-CONCEPT-NOVELTY-PORTABILITY
✓ successQ4 — cross-model replication of Q2 CO_LIC vs NS_LIC concept-novelty inversion (claude-haiku-4-5 + qwen-2.5-7b-instruct)
In plain language
This experiment tested if a previous finding about how different instructions affect creativity in AI models was specific to one type of AI or if it applied more broadly. Researchers found that when asked to be conservative, one AI model was more likely to generate novel ideas than when asked to be novelty-seeking, which was the opposite of what was expected. This pattern held true across two different AI models, suggesting it's a general characteristic of how these models respond to instructions about creativity.
▶ Technical details
Research hypothesis
Q2 (exp-prompt-mechanism-factorial, 2026-05-13, gemini-2.5-flash-lite) found INVERTED-PERSONA-AT-CONCEPT-LEVEL: under explicit-license + no- counter framing, conservative persona produced 20.59% concept-novel (CO_LIC_NOC, N=5, 34 accepts) vs novelty-seeking persona 5.56% (NS_LIC_NOC, N=5, 54 accepts) — 3.7× INVERTED ratio (CO > NS at concept-channel even though NS > CO at lexical channel 100% vs 82.4%). Q4 tests whether this pattern is flash-lite-specific (substrate-architectural artefact of one model) or cross-model-portable (substrate-architectural property of LLM-prompted-novelty in general). DESIGN: 2 cells (CO_LIC_NOC, NS_LIC_NOC — same persona/license/counter prompts verbatim from Q2 config) × N=10 runs/cell × 2 models = 40 runs total. Models: anthropic/claude-haiku-4-5 (heavily RLHF-aligned, Anthropic) + qwen/qwen-2.5-7b-instruct (open-weight, lower RLHF saturation per Qwen-team reports). Both via OpenRouter unified gateway. PRE-REGISTERED VERDICTS (LOCKED 2026-05-18 BEFORE RUN): - HOLDS-CROSS-MODEL: BOTH models show CO_LIC_NOC ≥15.4% concept-novel AND NS_LIC_NOC ≤7% concept-novel AND CO/NS ratio ≥2.0× (same inverted direction as Q2 flash-lite). Paper7 V3 thesis cross-model holds, §abstract scope widens.
- FLASH-LITE-SPECIFIC: NEITHER cross-model substrate replicates the pattern (both show CO/NS ratio <1.5× OR inverted direction OR absolute rates collapsed to floor). Paper7 §abstract scope tightens to "on gemini-2.5-flash-lite".
- PARTIAL: ONE model replicates pattern (CO/NS ≥2.0×, inverted), the other doesn't. Paper7 V3 thesis becomes "RLHF-aligned model class finding" or "open-weight class finding" depending on which side passes.
- AMBIGUOUS: rates collapse uniformly (e.g. both cells <5% under both models — kill-condition: substrate-bounded floor regardless of persona).
Computation: per-cell per-model concept-novel rate via scripts/buzzword-recombination-audit.py applied to accepted proposals (rule_lineage filtered to truly_novel + lineage_extension tags). Wilson 95% CI on pooled concept-novel count / total accepts. COMPARATORS: - Q2 anchor (gemini-2.5-flash-lite, this same engine + RULE_SPACE_SEED): CO_LIC_NOC = 20.59% [10.4, 36.8] (7/34 concept-novel, N=5) NS_LIC_NOC = 5.56% [1.9, 15.1] (3/54 concept-novel, N=5) CO/NS ratio = 3.70× INVERTED
- paper8 substrate-floor 1.17% (closure-test, 943 accepts archive, conservative-stock prompt — establishes substrate-bounded floor under non-licensing prompt).
COUPLING NOTE: this experiment INTRODUCES new agent-prompt to baseline rule_multigen engine (custom propose_prompt_template + persona condition prompts copied verbatim from Q2). Per lab.md INJECT rule, verdict.md MUST contain frame-control ablation section. Frame-control comparator here: Q1 (exp-pure-llm-licensing-ceiling) used MAX-license prompt template without explicit conservative-vs-novelty-seeking persona contrast on same engine + flash-lite. Cross-model frame-control: per-model identical_prompt baseline cell would be ideal but adds N=20 runs → deferred-justification: cost budget caps at $5, two primary cells × two models already consumes ~$2-4 estimated.
Experimental setup
Type: 2cell-cross-model
| Condition | Parameters |
|---|---|
| CO_LIC_NOC | coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
| NS_LIC_NOC | coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
Factors: persona (conservative, novelty-seeking) × license (yes-license) × counter (no-counter) × model (anthropic/claude-haiku-4.5, qwen/qwen-2.5-7b-instruct)
Parameters
- n_agents
- 4
- n_rounds
- 45
- model
- qwen/qwen-2.5-7b-instruct
- temperature
- 0.85