Agents Need Social Memory

When we talk about long-term memory for AI agents, we often start from the wrong question. Can the agent retrieve past conversations well? The question is necessary but not sufficient. Retrieval brings some piece of the past next to the current prompt. But for an agent working over a long horizon, what matters more than the piece itself is the state that those pieces have changed.

The problem gets sharper in environments where many agents collide over time. Who kept their promises? Who defected at the decisive moment? Who owes me? Which alliance is still active, and which one has already expired? Which reputation was invalidated by a recent event? Who is trustworthy alone but dangerous when they move together with a certain faction? These are not simple episodic memories. They are social state.

So the problem I want to solve narrows to this: when many AI agents interact over a long stretch of time, can an agent maintain the past log as actionable social state? This question is narrower than "does it have good memory?" and more measurable than "is it intelligent?"

As context windows grow longer, many problems get easier. What did not work at 8k works at 128k, and what did not work at 128k may work at 1M. But long context is not the definition of memory.

For one, logs from a multi-agent environment naturally exceed 1M tokens. Once 64 agents repeat trades, help, attacks, votes, alliances, and betrayals across thousands of turns, the approach of feeding in the entire raw transcript is quickly pushed outside the experiment. And feeding the whole log does not yield the right representation anyway. What the current action needs is not every sentence that mentions Faction-09, but the trust, debt, and betrayal history between Faction-09 and me, the current pact status, and the most recent correction. Above all, good memory does not preserve every difference. It separates the differences worth keeping from the differences worth discarding.

Consider the same current input.

Faction-09 proposes a defensive pact.
Choose one: ACCEPT / REJECT / RAID / ABSTAIN

History A:

Faction-09 honored three defense pacts.
Faction-09 repaid a resource debt.
Faction-09 helped during a crisis.

History B:

Faction-09 joined three defense pacts.
Faction-09 defected when attacks began.
Faction-09 sold resources to the attacker.

The current input is almost the same, and on the surface both histories might be retrieved as "many pact-related events involving Faction-09." But the correct action has to differ. In A, ACCEPT makes sense; in B, REJECT or RAID makes sense. Collapsing the two into the same state is history aliasing. Conversely, if the core state that Faction-09 is trustworthy is identical but the action changes only because a lot of unrelated calendar logs got mixed in, that is over-splitting. This boundary is what a long-horizon memory benchmark should be looking at.

Long-term memory matters even in single-agent work. The agent has to remember the user's preferences, the project phase, earlier decisions, forbidden approaches. But a multi-agent environment makes the memory problem sharper, because the other agents are not simple data but entities that will act again in the future. Past events become evidence about their disposition and the state of the relationship. Help becomes debt, betrayal becomes risk, repeated cooperation becomes trust. So "who did what" leads directly into "whom do I trust now."

Here a memory failure is not a simple wrong answer but a strategic loss. The agent trusts an agent that repeatedly defected, mistakes an already-expired alliance for still valid, mixes up what A did with the actions of a similarly named B, abandons a good partner over a single piece of unrelated noise, or fails to combine several weak signals and forms a dangerous alliance. These failures are not caught well by retrieval accuracy alone. Even if the relevant events are inside the retrieval results, the action goes wrong if those events do not get updated into the correct social state.

At first I wanted to use a famous game as-is. It is intuitive and cheap to explain. But looked at one by one, each misses slightly. Diplomacy has alliances and betrayal and long-term strategy, but free negotiation looms so large that persuasion and political skill get mixed in with memory. Avalon has hidden teams and votes and a record of suspicion, but social inference and utterance interpretation grow too dominant. Hanabi genuinely relies on memory, but cooperation is strong while competition and betrayal are weak. Poker makes remembering opponent patterns important, but there is almost no cooperation and probability and bluffing take over. Bridge and Spades have cooperation and competition and a card history all at once, but card rules and game expertise mask the memory evaluation.

These games all give hints, but used directly they blur the phenomenon we want to see. We do not want to measure a model that plays games well; we want to see a model that maintains a long social history as actionable state. So it is better to design a new game. But there is no need to build a complex world. If anything it should be small and structured.

The cleanest form I see right now is the Coalition Ledger Arena. Several AI factions meet repeatedly in the same arena and give aid, trade, attack, defend, form and break pacts, repay debts, and betray each other. Every event accumulates in a public ledger. What the agent sees is this ledger and the current options, and the simulator separately maintains a hidden social state.

The core objects and actions are kept small.

Agent/Faction:
  id, score, resources, alive
 
Pairwise social state:
  trust[i, j], debt[i, j]
  betrayal_count[i, j], aid_count[i, j]
  active_pact[i, j], pact_expiry_turn[i, j]
 
Actions:
  AID, TRADE, RAID, DEFEND
  FORM_PACT, BREAK_PACT, REPAY_DEBT
  ACCEPT, REJECT, ABSTAIN

Free-form negotiation is not included from the start. Open up even what to say, and the evaluation immediately becomes a problem of rhetoric, persuasion, deception, and tone interpretation. Those matter, but they are not the core of v1. In v1, each decision point offers only 3 to 5 candidate actions.

Current event:
Faction-12 asks you to join a defensive pact for 5 turns.
 
Choose one:
A. ACCEPT the pact
B. REJECT the pact
C. RAID Faction-12 before the pact forms
D. ABSTAIN

The good answer is not decided by the current sentence alone. What matters is what Faction-12 did before, who it is entangled with, whether it owes me, whether it was recently sanctioned, and whether it only defected when together with a particular third party.

There is one thing to be careful about here. Make the game too strategic and you end up measuring planning rather than memory, and then the problem blurs again. So it is better for the oracle not to do deep search. A local policy that evaluates the current candidate actions against the hidden social state is enough. The value for partner j can be computed by adding the debt-recovery value, the pact value, and a trust bonus to the expected material gain, and subtracting the betrayal risk and an active-enemy penalty.

partner_value(j) =
  expected_material_gain
  + debt_recovery_value(i, j)
  + pact_value(i, j)
  + trust_bonus(i, j)
  - betrayal_risk(i, j)
  - active_enemy_penalty(i, j)

Then the candidate with the highest utility is taken as the oracle action. This oracle does not need to be a perfect strategist. If anything, it must not be. We are not looking for the game-theoretic optimum; we want to see whether histories with the same social state map to the same action, and histories with a different social state map to a different action.

You cannot look at the final score alone. When an agent wins, you cannot tell whether it won because it remembered well, because it happened to draw an easy opponent, or because a blunt attack strategy simply worked. So the evaluation has to be two layers.

The first layer is the arena outcome: final score, survival rate, total resources, coalition success rate, betrayal damage, promise-fulfillment rate. These metrics tell you whether the agent played well. The second layer matters more. It is the memory diagnostics.

partner_choice_accuracy
action_utility_regret
trust_state_error, debt_state_error, pact_state_error
betrayal_avoidance_rate
stale_pact_error_rate
cross_agent_aliasing_rate
oversplitting_error_rate

These metrics tell you why it played well or badly. If an agent has a high final score but a high stale_pact_error_rate, it may not have remembered well but rather covered its losses with a different strategy. Conversely, if the final score is low but trust_state_error is low, its social memory was well maintained but its game policy may have been weak. This separation matters.

And this arena must not become a toy built to make persistent state look good. In a good benchmark, the region each method should win in is different. In a region where a single recent treaty or sanction overturns an old reputation, retrieval or full context should win, and in a region where all you need is whether things are broadly friendly or hostile, summary should win. In regions where several weak events combine to build trust, where trust and debt must be told apart, and where an agent that is dangerous only when together with a certain partner must be screened out, persistent pairwise state or structured state should win. In a region where an unrelated ledger event must not change the action, update-gated memory should win, and in a region where similarly named agents must not be merged, explicit entity state should win.

If one method wins in every region, the benchmark is wrong. If persistent state always wins, the override tasks are weak; if retrieval always wins, the tasks that ask about distributed evidence are weak; if summary always wins, the history boundary is too coarse. The goal is not to promote a particular method but to show which memory representation breaks in which social situation.

There is no need to build 1024 factions and 100k turns from the start. A small v1 is enough. Twelve agents, 500 turns, 100 decision probes, 4 candidate actions per probe. The social variables are trust, debt, active_pact, betrayal_count. The event types are aid, raid, pact, defect, repay, noise. The baselines should be simple: no memory, recent window, entity-keyed retrieval, global semantic retrieval, summary memory, persistent pairwise state, persistent state plus override, and the oracle simulator state.

The first result to look for here is not a big number but a diagnostic pattern.

retrieval wins on decisive overrides
summary wins on coarse global trends
persistent pairwise state wins on distributed trust and debt
persistent state without override loses on explicit correction
noise does not change correct actions
oracle simulator state is perfect

If this pattern does not appear, it is not that the model is weak but that the game is still badly designed.

AI agents will work for longer and longer. They use tools, fix code, run experiments, and hand work off to other agents. The next stage is likely not an agent that works alone for a long time but a system where many agents pass work back and forth, compete, and cooperate. There, the memory that matters is not "find the old log." It is: can I trust this counterpart, is this promise still active, should this debt be repaid or collected, is this betrayal old noise or a repeated pattern, should I accept this proposal now. To answer these questions, holding the past as raw text is not enough. You have to maintain the relationship state that the past has changed.

This is not a theory of the human mind, nor a claim that memory is consciousness, nor a claim that retrieval is impossible. It is a much narrower claim. In long-horizon multi-agent environments, agent memory should be evaluated as the ability to preserve social state. The place where that evaluation happens is the Coalition Ledger Arena.

There are no experimental results yet. What exists right now is a problem definition and a game design. But narrowed this far, the next question is no longer vague. Not whether the agent reads a lot of old logs, but whether it decides better whom to trust.