Samarpit Santoki

, 3 min read

What should an agent remember?

I’m building Context Engine at Decklar, a memory layer for our agents and products so they can carry what they learned in one conversation into the next. I explored and adopted existing memory tools first. Storage turned out to be the easy part. Deciding what’s worth keeping, and what to bring back for a given task, is where the work is.

A bigger context window isn’t memory

The obvious move is to put more history in the prompt. Context windows keep growing, but every extra token makes each call slower and more expensive. Models also use a short, relevant context better than a long, noisy one.

Things go out of date at different speeds

“Prefers email over phone calls” might stay true for a year. “Was annoyed about a dropped call on Tuesday” matters for a few days. “Uses tool X” is true until the day they switch, and then the old answer is worse than no answer.

So a memory needs more than its content. It needs to know where it came from, when it was learned, and how long it’s likely to stay true. When something new contradicts an old memory, the old one should be replaced or marked as outdated, not left sitting next to it. Otherwise the store gets noisier every week.

Similar isn’t the same as relevant

Vector search finds records that look like the current conversation. But an outdated memory can look very similar to the question.

Scope is a filter, not a score. Another customer’s memory should never be a candidate at all. Inside that scope, similarity is a good starting signal, with freshness, source, and the task itself counting too. The test is whether the agent does the task better with what was retrieved.

Sharing between agents

If several agents serve the same customer, they should share what they learn. The open parts are which agent can read what, who can update a memory, and what wins when two agents write conflicting facts. That’s what I’m working on now.