← back to timeline

EXP-ENGINE-CONFOUND-SYSTEMATIC-AUDIT

verdict.CONFIRMED-P4-engine-source-confound-is-category-15-sites-11-engines

Engine Confound Systematic Audit

2026-05-28 L3-L4 level 0
new
premortem
mock
real
metrics
analyze
review
krit
done

In plain language

Researchers investigated whether the way prompts are constructed for language models might be influenced by factors other than just the agent's identity. They systematically searched the code for any instances where instructions or prompt elements were specifically tailored based on an agent's role or position. The audit confirmed that there are indeed multiple ways prompts are influenced by the engine's internal logic, with some of these influencing the outcomes of experiments.

▶ Technical details

Research hypothesis

axis_per_agent is NOT the only hard-coded engine→LLM prompt injection. Systematically grep ALL active engine files (rule_multigen.py + multi_agent.py + scarcity.py + acdc_mini.py + ca_coupling.py + levin_coupling.py + evo_loop.py + closure_test.py + paper9_phase2.py + cli/*.py — ~12 files / ~5000L total) for agent-identity-conditional branching, role-conditional prompt construction, hard-coded instruction templates that vary by agent-name/position/seed.

Build meta/engine-prompt-injection-inventory.md catalog with columns: engine | line | injection-site | classification (engine-internal-helper / prompt-injected-to-LLM / engine-only-no-LLM-effect) | linked-experiments | affected-verdicts | downgrade-risk.

Predict ≥5 additional engine-injection sites beyond axis_per_agent, ≥2 of which materially affect a published or in-flight verdict.

construct paper6-related paper7-related paper8-related paper9-related pool-drained phase7-3-generated