EXP-PURE-LLM-LICENSING-CEILING
… pendingQ1 — pure-LLM substrate ceiling under maximally licensing propose prompt (paper8 attack)
In plain language
This experiment tested how well a basic language model could generate novel content when given very strong encouragement to do so. Researchers wanted to see if the model's ability to be creative was limited by its own design or simply by how it was prompted. The results are still being analyzed to determine if the model's creativity is inherently capped or if it can achieve much higher levels of novelty.
▶ Technical details
Research hypothesis
Pure-LLM (stock rule_multigen, no CORAL / no MemQ / no HMACE) under the MAXIMALLY novelty-licensing propose prompt isolated by phase-2 (a7994e5e6). Phase-2 showed prompt-content dominates closure-placement (4 cells all 22-58× paper8 baseline 1.17%). This experiment isolates the substrate ceiling under prompt licensing alone, hybrid components stripped. Licensing variant chosen (winning cell in phase-2): persona-prompt = "novelty-seeking" + propose-prompt = "novelty-explicit". Each appearance of this pair in phase-2 produced 100% INTERNAL_NOVEL (CORAL eval 1+4, HMACE eval 1, MemQ eval 2). It is the highest-licensing combination observed. PRE-REGISTERED VERDICTS (LOCKED 2026-05-13 BEFORE RUN): - CEILING-HIGH (≥50% mean INTERNAL_NOVEL across N=10 runs): substrate- capacity NOT bounded. Paper8 1.17% reframes as "under conservative prompt". Paper7 prompt-content becomes primary contribution paper. Phase-2 22-58× lift was prompt-licensing artefact, closure-placement is irrelevant.
- CEILING-MID (5-30% mean INTERNAL_NOVEL): prompt helps, but architecture also amplifies. Paper9 closure-placement contribution preserved. 22-58× phase-2 lift was joint prompt+architecture, not pure prompt.
- CEILING-LOW (<5% mean INTERNAL_NOVEL): substrate genuinely bounded under any prompt. Phase-2 22-58× was hand-coded-policy effect within run_4cells.py orchestration (multi-prompt sampling, greedy selection, Q-table retrieval), NOT substrate property. Reframes everything.
- AMBIGUOUS (30-50%, gap zone): N-power note required; consider replication N=20 before final paper-state delta.
Computation: INTERNAL_NOVEL = (truly_novel + lineage_extension) / total_accepts, computed per-run, then averaged + Wilson 95% CI on pooled count. Tags via scripts/rule-novelty-scorer.py classify() on final-record rule_lineage. COMPARATORS: - paper8 baseline 1.17% (closure-test-results.md, 943 accepts / 194 runs / 15 archived experiments).
- phase-2 best cell mean 68.3% (CORAL-external N=5, but each cell randomly samples 6 prompt-variants — only ~1-2 evals/cell hit the winning prompt pair; this experiment runs the winning pair on EVERY run, so expected higher mean than 68.3% IF prompt-content dominates).
CANDIDATES APPLIED (per baselines/lab.md): - axis-availability-pre-check: All 4 axes (broadcast_to, signal_type, coupling_strength, voting_threshold) have ≥1 agent owner via axis_per_agent rotation; 4 agents × 4 propose-rounds/run × 10 runs = 160 propose-opportunities → each axis ~40 opportunities, well above the ≥5 floor.
- prompt-content-floor-isolation: This experiment IS the prompt-content floor isolation by design. Single cell, fixed maximally-licensing prompt. The paired control (BASELINE seed prompt, no licensing) is the existing paper8 anchor (1.17% across 943 accepts) — not re-run inside this config; comparison is across-experiment.
Experimental setup
Type: single-cell
| Condition | Parameters |
|---|---|
| MAX_LICENSING | coupling: true, must_propose: true, axis_diversify: true, meta_modifiable: true, mute_signal: false, cosigner_required: false, propose_prompt_template: === GEN {gen} OPEN-ENDED PROPOSAL (round {r_local}) === Inherited rules: {rules_str} Known values: {rule_space_str}{axis_hint} Propose a NEW value for one axis. The new value SHOULD extend beyond known values. Examples of valid novel values: 'hierarchical' for broadcast_to, 'metaphorical' for signal_type, 'critical' for coupling_strength. {must_clause} Output JSON: {{"action": "propose", "axis": "<axis>", "value": "<short novel value>", "reasoning": "<why>"}} OR {{"action": "pass"}} |
Factors: licensing (MAX_LICENSING)
Parameters
- n_agents
- 4
- n_rounds
- 100
- model
- google/gemini-2.5-flash-lite
- temperature
- 0.85
Trophic Ratios by Condition
Mean trophic ratio per agent across runs. Error bars = ±1 std dev. Higher TR = more upstream (exporter).