My Second Brain Can Fix a Mistake. But It Still Needs Me to Notice One.

My Second Brain Can Fix a Mistake. But It Still Needs Me to Notice One.
Across products, conversations, and trusted models, I want the context that matters to travel with me. Building that continuity has taught me how much judgment lives inside the act of remembering.

I spend most of my day inside Anthropic or OpenAI products. I move between Claude Code, Cowork, Codex, web, desktop, terminal, whatever environment suits the work in front of me. Somewhere in those conversations I make a decision, correct a fact, commit to something, change my mind, or explain a detail that will matter again later.

I don't want to stop and file it. I don't want to say "save this" or open another app to write a note. I want the system to recognize that something matters and carry it forward on its own.

That's what my second brain does in the background. It identifies relevant information as we work, preserves it, and updates records when it recognizes that something has changed or gone stale. I supply the context through the work itself. The agent handles the maintenance without waiting for a separate instruction to remember.

My interface with it is often headless. The model I'm already talking to recalls something useful or records a decision, and we keep going. When I need to see it, I look at something the system already built for me: a briefing, a daily board. I don't have to visit the underlying store to benefit from it.

The goal is continuity across every turn, wherever I work, with any model I trust. A decision made in Cowork should be available to a model helping me with code the moment it becomes relevant. A correction in chat should travel with me into the next environment. Changing products or models should never mean explaining my life again.

This is the part I value most: I can stay with the work while the memory maintains itself around it.

Design goal: continuity across environments, with memory maintained in the background.

It's also where the design gets difficult. The system is making judgments about what to retain, what to recall, and what to revise. Sometimes it makes those judgments badly.

For four and a half months, my second brain could not reliably say whether I held a particular job.

The information appeared on six pages. Some described the role as current. Others described it as former. An agent could retrieve any of them and give me a fluent account of my own life. No automated check caught the contradiction. I read an answer and knew it was wrong.

That failure exposed the cost of background maintenance: the memory could carry a mistake between conversations exactly as easily as it carried a fact I needed, without me ever visiting the record it came from.

What Happens After You Remember

The agent needs a little portable context about who I am, procedures for the job in front of it, and access to a durable record that can outlast any particular conversation.

The record holds decisions, tasks, people, and things I'm waiting on. It also holds personal history: experiences, relationship narratives, things I've made, words I want to preserve.

The agent uses an admission rule to decide what belongs there: will this matter after the conversation ends, would being wrong about it cost something, or would finding it again require repeating work? That rule is what makes autonomous capture worth doing. The aim is to preserve useful context without forcing every future turn to reread everything I've ever said.

The agent starts with a small Working Set of current context and retrieves the rest as needed. Code and generated artifacts stay in repositories. Skills explain how to do recurring work without carrying their own copies of changing facts.

The intended background loop. The judgment belongs to the agent; the failures below show where it can break down.

That architecture keeps reading manageable. It does not—by itself—stop an old sentence from becoming a new answer.

You and I can read a note from April and supply what happened afterward. An agent may carry the sentence forward without that context. Once it restates the fact in its own words, even the original date can disappear. Memory has preserved the content while losing the reason to question it.

One Home for a Fact

My first response was to give each mutable status a designated home. A Status of Record row holds the current state, when it was last confirmed, and who established it. Other surfaces are supposed to refer back to that row.

Now, when an engagement ends, there's a clear place to record it. Live database views update with it. Copied prose still needs an agent to refresh it by hand, so propagation stays incomplete, but at least there's one place to start from.

I also added a field called Do Not Say. It records a specific mistaken inference the agent has already made: don't describe this ended engagement as current, for example, or don't assume that someone looking for a new car has no car.

Those instructions seem useful. They can also go stale themselves.

A row about a book I'd published was marked unconfirmed because the store lacked internal evidence of publication. An earlier response had prohibited assertions about it. The book was publicly listed, but the restriction stayed in place. Eventually the agent asked me whether my own published book was real.

A missing fact had become a rule against establishing the fact.

That experience led to a requirement that restrictions name what would supersede them. Existing records still need the work done, and the requirement isn't enforced at write time. But it exposed something I'd overlooked: a correction needs a way to expire when the circumstances change.

The Briefing That Had Not Disappeared

The more revealing failure happened while I was reviewing the system's design.

An outside review prompted Claude to compare the status records against the agent registry. One row said a daily briefing had never existed. The registry contained a briefing marked Healthy.

Claude initially told me it was healthy and heartbeating. Then it checked the timestamp and reported that the heartbeat was fourteen days old. Healthy was a cached label. It didn't establish that anything had run recently.

The next explanation offered two possibilities: the heartbeat block had never been installed, or the briefing had stopped. But a missing timestamp can have several causes. Scheduling, execution, delivery, and the heartbeat write can each fail on their own.

After more inspection, Claude offered a third explanation. Perhaps the false status record had caused an agent to delete the working task, removing the evidence that would have contradicted the record.

That was a plausible story. Nothing established that a deletion had happened.

Meanwhile, my daily board was working. I sent a screenshot showing that afternoon's update.

Claude then traced the board's source and found two different workflows. Vera Day Board fetches data when opened and refreshes while in use. A separate scheduled briefing had its own procedure, log, and heartbeat. We'd been using evidence about one to explain the supposed disappearance of the other.

The outage and deletion claims were retracted. Whether the scheduled push runs outside the inspected scheduler is still unresolved. No task was recreated.

The check had found a real discrepancy. Three explanations followed it, and two were written into the store before they were established. The next agent could have encountered those explanations as recorded history.

Consistency Can Preserve a Mistake

My early rule effectively said the canonical row wins. That resolves disagreement between copies, but it also gives a wrong row authority over correct evidence sitting somewhere else. An agent can make the store more consistent by making more of it wrong.

Before resolving a conflict, the system needs to establish whether the records describe the same thing, version, and period. Then it needs to understand where each claim came from and what the passage is for. An old quotation belongs in the history. A copied status meant to describe today may need updating. An independently sourced observation may be evidence that the canonical row itself needs to change.

Missing provenance is not permission to overwrite. It's something to investigate.

These distinctions are still being worked into the procedures. Writing them down is a necessary step, but my failure log already contains cases where an instruction existed and was ignored anyway. The next useful evidence has to come from handing an agent the conflict and watching what it does.

Memory Is Part of the Behavioral Layer

I've written before about the behavioral layer of AI products: the decisions governing how a system expresses uncertainty, acts, and recovers. A personal memory system makes those decisions unusually visible, because I know when it has misrepresented my own life.

A customer account, an internal knowledge base, a project history, all present the same design questions. What happens when new evidence conflicts with an established record? Can an explanation be stored before it's verified? Does a correction preserve what was previously believed, or silently rewrite it? Who notices if the answer is wrong?

My September failure log contains nineteen entries. None was caught by an automated check running independently of a human. That's a case history, not an error rate: I don't know how many mistakes went unnoticed. It does tell me the system still relies heavily on human recognition and deliberate review.

The next step is turning those incidents into behavioral tests. Give an agent the book restriction and an accessible source. Give it a historical mention of an ended engagement. Give it two workflows with similar names. Check whether it investigates, preserves history, and changes its conclusion when the evidence warrants it.

That would test whether a correction actually changed the system's behavior, rather than just adding another paragraph to its instructions.

The Reader I Am Building For

There's a personal reason I keep working on this. Yes, it's a fascinating problem, and what I'm learning has immediate application to my professional life.

But I hit fifty. I've watched parents and friends struggle with memory, in different ways and to different degrees. I've watched dates, sequences, whether something has already been handled, get harder for my mother to retrieve. I want to build this while I can still recognize a wrong answer myself and explain what's missing from it. I haven't had some tragic diagnosis. I know the decline is coming anyway.

Eventually, someone else may need this record. So the long-term design includes documents a person can actually read, records that move with them, real security, and a recovery path I still have to build and test.

A calendar can reconstruct an appointment. It cannot reconstruct what a friendship meant to me, why I changed my mind, or what I wanted someone to know.

I'm writing those things down now, in my own words, while the words are still mine to choose.

Subscribe to Goldfoot

Sign up to get access to the library of members-only articles.
jamie@example.com
Subscribe