← back to timeline

EXP-FORGET-N5-R2

informative

Forgetting at N=5 — Replication R2

2026-03-29 L1.5 level paper3 1 runs $0.01
new
premortem
mock
real
metrics
analyze
review
krit
done

In plain language

Five AI agents build a shared text, but each is told to focus only on the last 2-3 entries. At N=3, this destroyed hierarchy completely (TR=0.015). Does the same instruction destroy hierarchy at N=5? This fills the missing cell in our memory × group-size matrix.

What we found

UNDERPOWERED (N=1): No predictions scored

Predictions we made before running

0/5 confirmed

Technical details

Research hypothesis

N-MATCHED CONTROL: Prompt-based forgetting at N=5.

PROBLEM: exp-forget (N=3, prompt decay): TR=0.015 — hierarchy COLLAPSES exp-memory-truncation (N=5, structural truncation window=20): TR=0.298 — hierarchy SURVIVES

Two confounds: (a) N=3 vs N=5 — different group size, different hierarchy baselines (b) Prompt-based "attend to last 2-3" vs actual context truncation (window=20)

THIS EXPERIMENT: prompt-based forgetting at N=5. Same "attend only to last 2-3" instruction as exp-forget, but with N=5 agents (same as exp-memory-truncation).

If TR collapses (< 0.15): prompt-based decay kills hierarchy regardless of N. → exp-memory-truncation's TR=0.298 is due to STRUCTURAL truncation being less effective than prompt-based instruction (agents still see full text). If TR survives (> 0.25): N=5 is sufficient to maintain hierarchy even with decay. → exp-forget's collapse was an N=3 artifact (too few agents). If intermediate (0.15-0.25): partial effect, needs characterization.

BASELINES:

  • exp-pent (N=5, full memory, 60 rounds): TR~0.5
  • exp-forget (N=3, prompt decay, 48 rounds): TR=0.015
  • exp-memory-truncation (N=5, window=20, 60 rounds): TR=0.298
  • exp-memory-truncation-r1 (N=5, window=20): TR=0.298±0.079 (pooled N=3)

This creates a clean 2×2: N=3 full memory (exp-asym): TR=0.444 N=3 prompt decay (exp-forget): TR=0.015 N=5 full memory (exp-pent): TR~0.5 N=5 prompt decay (THIS): TR=?

EXPLORATION TIER (N=1). Goal: fill the 2×2 cell.

Experimental setup

Type: simple

Condition Parameters
DECAY_LIVE interaction: LIVE, memory: DECAY, note: N=5 group with prompt-based decay. Same instruction as exp-forget.

Factors: memory (DECAY)

Parameters

model
gemini-2.5-flash-lite
n_agents
5
n_rounds
60
n_runs_per_condition
1
scheduler
round_robin
temperature
0.9

Trophic Ratios by Condition

Mean trophic ratio per agent across runs. Error bars = ±1 std dev. Higher TR = more upstream (exporter).