Our AI Memory System: How We Built a Second Brain for the Practice

Here's a scenario that happens in every small business, every week: someone asks "how did we handle that insurance situation last time?" and the one person who knows the answer is on leave. Or worse — they left six months ago and took the answer with them.

Institutional knowledge lives in people's heads. It's built up over years — little decisions, supplier recommendations, lessons learned from mistakes, policies that were agreed on verbally but never written down. It's the most valuable asset in your business and it has zero redundancy. If the person who holds it walks out the door, it's gone.

We got tired of that. So we built a system that captures everything important, understands what it means, and makes it instantly searchable. We call it TwoEyeSee — a play on "2IC," as in second-in-command. Because that's exactly what it is: an always-available deputy that remembers everything the team has ever decided, discussed, or learned.

The problem we were actually solving

It's not a filing problem. We had filing. We had shared drives, Notion pages, Google Docs, and a wiki that nobody updated. The information existed — scattered across twelve different places, buried in chat histories, saved in someone's personal notes, or stored exclusively in Sarah's memory because she was the one who dealt with that supplier two years ago.

The problem is retrieval. When you need an answer, you need it now — not after 20 minutes of searching through folders, scrolling through old messages, and eventually giving up and calling Sarah on her day off.

We tracked it for a month. Our team was spending an average of 35 minutes a day looking for information that someone in the practice already knew. That's nearly three hours a week per person, wasted on searching for things that had already been figured out.

And the really painful version: when nobody can find the answer, you just figure it out again from scratch. You re-research a supplier. You re-negotiate a policy. You make a decision that contradicts one made six months ago because nobody remembers the first one. The wheel gets reinvented constantly.

How it works

The concept is simple, even if the technology underneath is clever. There are four steps:

1. Capture. Someone has a thought, makes a decision, or learns something useful. They capture it — either by typing a quick note or recording a voice memo. It doesn't need to be formal. "Just spoke to the lab — they're increasing prices 8% from March. We agreed to stay with them but review in June" is a perfectly good entry. So is "veteran referral process: call DVA first, get approval number, then book appointment. Don't book without the number or we won't get paid."

2. Classify. The AI reads the entry and figures out what it's about. Is it a business decision? A supplier update? A clinical procedure note? A patient interaction? A policy? It tags the entry with the right categories and identifies the key details — people, companies, dates, amounts — without anyone having to fill in forms or pick from dropdown menus.

3. Create a semantic fingerprint. This is the part that makes it powerful. The AI converts the text into a mathematical representation of its meaning — not its exact words, but what it's actually about. Think of it like creating a unique signature for the concept behind the words. Two entries that say completely different things but are about the same topic will have similar fingerprints.

4. Store and index. The entry, its classification, and its semantic fingerprint all go into a database. From that moment on, it's searchable — not by keywords, but by meaning.

That last distinction matters. Traditional search (like searching your email or shared drive) matches words. If you search for "cancellation policy," it finds documents containing those exact words. If the policy was described as "what we do when patients don't show up" — no match. You'd have to guess the exact words someone used when they wrote it down.

Semantic search matches meaning. You can ask "what's our policy on same-day cancellations?" and it will find the entry even if nobody ever used the word "cancellation" in it. It understands that "didn't show up," "no-showed," "cancelled last minute," and "same-day cancellation" are all talking about the same thing.

What people actually ask it

The best way to explain the system is through the questions it answers every week:

• "What's our policy on same-day cancellations?" — finds the decision from three months ago where we agreed on a $50 fee for less than 24 hours notice, with exceptions for emergencies.
• "Who did we get the waiting room chairs from?" — finds the conversation where someone noted the supplier name, the model, and the fact that they offered a 10% discount for bulk orders.
• "What happened with that patient complaint in February?" — finds the full context: what the complaint was about, who handled it, what was agreed, and how it was resolved.
• "What are the steps for processing a veteran's referral?" — finds the procedure notes, including the bit about getting the DVA approval number before booking.
• "Why did we stop using that lab in Cairns?" — finds the entry from eight months ago explaining the quality issues, the conversations that were had, and the decision to switch.
• "What did we agree about the staff Christmas party budget?" — finds it, even though nobody filed it anywhere formal.

Every one of those questions used to require interrupting someone, calling someone on their day off, or just guessing. Now they take about five seconds.

Why this is different from a shared drive or a wiki

We've tried shared drives. We've tried wikis. We've tried "just put everything in Notion." They all fail for the same reason: they require discipline. Someone has to write things up properly, file them in the right folder, use consistent naming conventions, and keep everything updated.

In theory, that works. In practice, people are busy. They make a mental note to document something later, and later never comes. Or they document it, but in their own way, in their own spot, using their own words — and nobody else can find it.

TwoEyeSee works because the barrier to entry is almost zero. You don't need to file anything. You don't need to categorise it. You don't need to put it in the right folder or use the right template. You just capture the thought — a few sentences, a voice note — and the AI handles the rest. It classifies, tags, indexes, and stores it. The messier the input, the more the AI earns its keep.

And because it searches by meaning rather than by filename or folder structure, it doesn't matter how disorganised the inputs are. The retrieval is always clean.

The knowledge continuity problem

Every business owner has felt this: a key staff member gives notice, and your stomach drops — not because they're irreplaceable as a person, but because of everything they know. The supplier relationships, the workarounds, the "we tried that in 2024 and here's why it didn't work" context that exists only in their head.

Handover documents help, but they're always incomplete. People don't know what they know — they can't write down every micro-decision and contextual detail they've accumulated over three years. The important stuff surfaces only when someone asks the right question, and by then, the person who knew the answer might be gone.

With TwoEyeSee, the knowledge is captured as it happens, not during a frantic two-week handover. Every decision, observation, and lesson learned goes into the system in real time. When someone leaves, their knowledge stays. The new person can ask the system the same questions they'd have asked their predecessor.

It doesn't replace human mentoring. But it means the new hire isn't starting from zero, and the team isn't re-learning things that were already figured out.

Privacy and data control

Because we handle patient information in our practice, we built TwoEyeSee to run on our own infrastructure. The AI processing happens on our own servers. Nothing gets sent to OpenAI, Google, or any third-party cloud AI service. The data stays where it should — under our control, on hardware we own.

For businesses that don't handle sensitive data, a cloud-hosted version would work fine and would be simpler to set up. But for healthcare, legal, financial, or any practice where confidentiality matters, local processing is a significant advantage.

What this looks like in practice

We've been running TwoEyeSee for several months now. The system has hundreds of entries — decisions, supplier notes, procedure documentation, policy discussions, patient interaction summaries, and random observations that turned out to be useful later.

The time spent searching for information has dropped dramatically. More importantly, decisions are more consistent. When someone asks "have we dealt with this before?", the answer is almost always yes — and the system can tell you exactly what was decided and why.

It's one of those tools that doesn't feel revolutionary on any single day. You capture a note, you search for something, you get an answer. But over months, the compound effect is enormous. The practice develops a genuine institutional memory that doesn't depend on any one person being present. It's the kind of advantage that's hard to quantify but impossible to give up once you have it.

If you're curious about what else AI can do for a small business beyond memory systems, we wrote a broader overview in our guide to practical AI for small business.

Would this work for your business?

If your team ever wastes time searching for information that someone already figured out — or if you've ever lost critical knowledge when a staff member left — this kind of system would make a real difference.

It doesn't have to be complex. Even a basic version that captures decisions and makes them searchable is a massive step up from "I think Sarah mentioned something about that last year."

Get in touch and we'll walk through how your team currently handles knowledge sharing, where the gaps are, and whether a memory system like this would be worth building. No jargon, no pressure — just a practical conversation about keeping your business's knowledge where it belongs: accessible to everyone who needs it.