A pattern for building personal knowledge bases using LLMs.

[…]

Most people’s experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There’s no accumulation.

[…]

The idea here is different. Instead of just retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources.

As a human reader, I get immense value out of online wikis (like Wikipedia) and will often not even need the actual source material for the vast majority of my searches. When reading fiction, I will frequently use book-specific wikis as supplemental reading when I’m trying to remember certain details (e.g. where else have I seen this character?). My personal knowledge base (and this site, to some degree) is an effort to build a wiki-like system, that I control, to augment my own memory. Wikis save me a lot of time.

Applying this same principle to LLM systems is a fascinating idea. Wikis (theoretically) can save a lot of tokens.

The knowledge is compiled once and then kept current, not re-derived on every query.
[…]
You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it. You’re in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work

Architecture

There are three layers:

Raw sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.

The wiki — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. You read it; the LLM writes it.

The schema — a document (e.g. CLAUDE.md for Claude Code or AGENTS.md for Codex) that tells the LLM how the wiki is structured, what the conventions are, and what workflows to follow when ingesting sources, answering questions, or maintaining the wiki. This is the key configuration file — it’s what makes the LLM a disciplined wiki maintainer rather than a generic chatbot. You and the LLM co-evolve this over time as you figure out what works for your domain.

Operations

Ingest. You drop a new source into the raw collection and tell the LLM to process it. […]

Query. You ask questions against the wiki. The LLM searches for relevant pages, reads them, and synthesizes an answer with citations. […] The important insight: good answers can be filed back into the wiki as new pages. […] This way your explorations compound in the knowledge base just like ingested sources do.

Lint. Periodically, ask the LLM to health-check the wiki. Look for: contradictions between pages, stale claims that newer sources have superseded, orphan pages with no inbound links, important concepts mentioned but lacking their own page, missing cross-references, data gaps that could be filled with a web search. […]

The tedious part of maintaining a knowledge base is not the reading or the thinking — it’s the bookkeeping.

[…]

The right way to use this is to share it with your LLM agent and work together to instantiate a version that fits your needs.

I’ve applied the concepts for this idea to one of my projects. My first attempt was a failure. It generated a wiki, but the instructions written to CLAUDE.md contained a fatal flaw: Claude would greedily consume the wiki on every query. This was filling my context window quickly. I’ve since updated those instructions and seen efficiency improvements. Plenty of room to improve, though. I plan to explore implementing a few of the operations as commands and an agent for coordination.

Another interesting idea I’ve seen around this concept: for software projects, using compiled artifacts like AST to provide the basis for the wiki. See Codesight for more details.

Everything I do runs through Claude Code. Not just coding. My morning routine, my CRM, my content pipeline, my lead sourcing, my follow-ups. All of it.

I built a system that runs my entire business from the terminal. One command in the morning, and my whole day is laid out. I copy, paste, check boxes, move on.

At some point I stopped thinking of Claude as something I chat with and started treating it as infrastructure. That changed everything.

Don’t get me wrong, I still chat with it, but only on [Claude Code].

Anyone else gone full Claude Code for non-coding work?

I’ll have a lot more to write about this in the future, but I have been using Claude Code to help launch my business . I have a lot of experience building software but I lack experience in the other skills that are required to run your own business: managing infrastructure, product strategy & research, marketing, design, and the day-to-day details of administration. As a solo indie developer, it’s empowering to have a (semi-)reliable personal assistant that can do a little bit of everything.

I even use Claude Code to semi-automate the running of this blog. Claude has helped me:

• Write a custom Hugo theme that I’m more happy with and which allows me to drop an external dependency.
• Automate the boilerplate and grunt work required to manage a static site.
• Audit the site to catch mistakes and inconsistencies.
• Add a new linked list feature to the blog.
• Proofread my writing to catch mistakes before publishing.

Now I get to focus almost entirely on the writing.

I went to the New York Times to glimpse at four headlines and was greeted with 422 network requests and 49 megabytes of data. It took two minutes before the page settled. And then you wonder why every sane tech person has an adblocker installed on systems of all their loved ones.

I’ve been a long-time user of 1Blocker , Hush , and Vinegar to help wrangle the madness of today’s internet.

Simplified versions like text.npr.org, lite.cnn.com and www.cbc.ca/lite still exist out there. And RSS feeds do too.

For the record, a typical post on this site (no images) weighs in as follows:

At just 23 KB, that’s roughly 2,000× lighter than the home page of the New York Times. This site is built to be fast and lean.²

Some supplementary (and seething) commentary from Daring Fireball :

Even websites from publishers who care about quality are doing things on the web that they would never do with their print editions.

[…]

The web is the only medium the world has ever seen where its highest-profile decision makers are people who despise the medium and are trying to drive people away from it.

This post from Daring Fireball weighs in at a trim 157 KB, over half of which comes from the screenshot (an extremely rare inclusion on DF).

Back to Shubham Bose to close it out:

They built a system that treats your attention as an extractable resource. The most radical thing you can do is refuse to be extracted.

Daniel_Langworthy (April 13, 2025):

@codinghorror do you know what happened to WASDKeyboards? I am so sad, I love my keyboards I got from them but they seem to just be gone

JimPas (April 13, 2025):

Hi Daniel. I checked this out and the last post on their support site was back in Nov 2024. Scrolling down to the bottom of the page shows this, so it doesn’t look good:

It shows a screenshot of a notice that reads: “Service Temporarily Unavailable”.

Glenwing (April 15, 2025):

I stopped by their office address in early January. Their name was still on the sign, but they’re gone. Replaced with some Solar startup.

This is sad news indeed!

After I started my first software engineering job in 2014, one of the first things I purchased was a mechanical keyboard from WASD: a CODE V1 87-Key Mechanical Keyboard with Cherry MX Clear keys.

I felt strongly (and still do) that writing software is a craft and any good craftsperson should invest in quality tools, learn how to use them effectively, and care for them. That keyboard went with me to every new job and moved across the country from Tennessee to California to Colorado. I’ve never had a single issue with it.

In 2022 when I needed both an office and home setup for Apple’s hybrid schedule, I added the CODE V3 (87-Key Mechanical Keyboard - Cherry MX Clear) to my setup. The V3 was for work; the V1 for home.

Now that I work independently , the V3 is my daily driver but the V1 is still alive and well, boxed up in my office awaiting a day when it’s needed again.

Yellow highlight | Location: 564
My awesome quote!

Yellow highlight | Location: 566
Another cool quote...

Using Sublime Text , you can use easily clean this up by:

1. Select Find > Replace from the menu bar
2. Ensure regular expressions is selected
3. Past the following regex into the find field (note the trailing period):

^.*Yellow highlight.*\n

4. Replace all with an empty space!

This leaves you with a relatively clean copy of your highlights.

1. Striving for Smallness
2. Documentation
3. Organization
4. Access Control
5. Early Exits
6. Closure-Based CollectionType APIs
7. Property Configuration
8. Lazy Sequences
9. Debugging Slow Build Times

I’d love to hear your thoughts and feedback. Feel free to reach out to me on Twitter @blakemerryman .

• Powered by plain text.
• Preview/rendering options.
• Multi-media support.
• Hand writing / drawing support.
• Customizable organization (via tags).
• Search.
• Mobile friendly. There is apps for iPhone, iPad, & Mac.
• Affordable and viable. Subscription pricing: $1.49/monthly or $14.99/annually.

Still missing from my wishlist:

• Varied storage options. For now, iCloud is the only supported storage option. This is fine for now as I live in an Apple dominated world.
• Text transcription. I can sketch and write on notes but can’t easily search this hand-crafted content.
• Web service. There is no way to access this content from the web.
• Customizable templates. There is not any template support.
• Extensible. There is no way to extend bear with your own functionality.

The silver lining in all of this is that I ostensibly control the content and it is being stored in an easily transferable format. This app is also well maintained and will hopefully add many of the features that I’m looking for.

And yet, I’ve never loved Evernote. I feel like we have always been at odds. Things never work quite how I want them to and a lot of minor details leave me frustrated. I’ll save that rant for another post but the bottom line is this: I’ve been looking for a new note taking system for a long time and I’ve always been forced come back to Evernote because it is the lesser of many evils.

With all of the discussion about the “open web” lately, I’ve been toying with the idea of building my own system, using existing technologies and supplementing with my own open-sourced utilities. This post won’t go into the nitty-gritty details and is more of an outline of what features I think this system should have in order to meet my needs. So let’s begin.

Wish List

This list is not all encompassing, nor are all items required. These are just things I want (and use in my current system).

1. Powered by plain text files. Whether you prefer to take notes in Markdown , HTML, plain text, or LaTex , this system should take any and all text files without regard for the contents.
2. Preview/rendering options. This system should easily allow the user to preview their marked-up plain-text content. Preferably with a live preview option. Also, presentation mode!
3. Varied storage options. A user should not have their thoughts and ideas locked into a third party’s cloud. They should be able to pick and choose and easily relocate their data at anytime. Ideally, the user can even host their own data.
4. Supports multi-media. As much as the developer in me wishes that everything existed in text, even I have to add images & videos to my notes every now and again. This should be easy & intuitive.
5. Hand writing support. Some people just prefer to take notes the old fashioned way or have sketches interspersed throughout.
6. Customizable organization. If the user wants to file things away in folders by month or year, great. If the user wants to file everything away in one large, hot-mess of a folder, also great.
7. Search. It should be easy to find what you’re looking for.
8. Text transcription. This is one feature that Evernote really nailed and probably the hardest to replicate. Ideally, this would tie in with search as well.
9. Mobile friendly (first?). We live in a progressively mobile first world whether we like it or not. This system should function equally well on phones, tablets, computers, and the web.
10. Web Service. This system should be able to live on the web, including self-hosted.
11. Affordable but viable. Fairly priced for consumers and the developer(s).
12. Customizable templates. Whether you’re bookmarking sites or journalling, a good template can save you time.
13. Extensible. Feature missing? Easily add it in yourself.

Basically what I want is a fast, simple, powerful note taking platform that is also useful for bookmarking, planning, todo-list managing, and cross-platforming utility. Not much… just everything.

This will probably (read: definitely) be on long, drawn out process as I piecemeal this all together (or give up and move on to something else, as I’m want to do on projects like these).

If you have suggestions or thoughts, feel free to contact me on Twitter .

• Lack of “Status Updates”

  In the past, Apple has kicked off every event with a stacatto of numbers about how great Apple has been doing. The past few events, however, Tim Cook has basically said “we’ve already crammed so much into this time slot that we don’t have time for updates; things are great; let’s do this”… which, in my opinion, is a welcome change that shows Cook’s leadership style comfortably in place at Apple.

• Apple Watch & watchOS 2

  I’m not über thrilled about the Watch-related announcements but there were a few gems. The updated watchOS will bring some much needed stability & performance improves to the fledgling platform (including a few nice new features like third party complications). Additionally, I think it’s great that not only is Apple continuing to produce new finishes and bands, but they are beginning to partner with serious companies to branch out with new collections of bands.

• iPad Pro, Keyboard, & Pencil

  A huge (and yet light) screen powered by iOS 9. Multi-tasking. Touch ID. Retina Display. Stereo speakers. All day battery life. Coupled with a ridiculously engineered Pencil (I didn’t say stylus!) and an interesting keyboard case, this could be the general purpose computing device that I’ve been waiting for. Slim, beautiful, powerful, versatile. I’m am stoked about this product combo.

• AppleTV & tvOS

  This will be in my living room as soon as it comes out. The AppleTV finally has an app platform. Apps are universal. Unified, refine-able search across content sources. A multi-touch remote that also acts as a microphone for voice commands. Powerful Siri voice search. A powerful, 64-bit system on a chip capable of Apple’s new Metal technology (high performance graphics). Support for bluetooth controllers.

• iPhone 6S & 6S Plus

  The new iPhones are pretty amazing. Tougher materials. 3D Touch technology (the “peak” and “pop” paradigm is pretty ingenious). 12MP camera capable of 4K video. Moving photos from the Wizarding world. Rose gold isn’t for me but it is beautiful.

  If only Apple would stop shipping 16GB phones. An uncompressed 4K video a few minutes in length can be close to 10GB in size (depending on format). Seriously Apple, come on. The fact that Apple continues to ship 16GB iPhones seriously rained on my parade.

• iPhone Upgrade Program

  This may be the announcement that I’m most excited about. Apple’s new upgrade program will finally allow one to affordably decouple our iPhone purchases from our service carrier bills. Here are the key points:

  • New iPhone every year
  • Unlocked
  • Low monthly payments (2 year commitment required through the Apple Store)
  • AppleCare+ (protection plan)

  I think decoupling hardware purchases from cellular service carriers is a good thing that will benefit the consumer and further drive carriers to focus on their service reliability and affordability rather than the handsets.

• Naming Conventions

  With iOS, watchOS, & now tvOS, I don’t think it will be long before we see Mac OS X become macOS. And I really like this naming convention.

• Power Connectors

  I had thought that with Apple’s adoption of USB Type C on the new MacBook that we might see the iPhones transition to the new connector in a year or so. However, with several new products (the TV Remote & Pencil) adopting the Lightning Connector for charging, I’m not so sure.

• Incremental builds for increased build performance
• Native Set collection class that bridges to NSSet, joining Array/NSArray and Dictionary/NSDictionary
• static methods and properties support for classes
• Improved type inference for single-expression closures
• Swift enums can now be exported to Objective-C
• let constants no longer need immediate initialization
• General performance enhancements
• General improvements to Swift-Objective-C interoperability

And my personal favorite:

• Nested & enhanced optional binding

Now, this:

if let x = self.x {
    if let y = self.y {
        if let z = self.z {
            if z == true {
                // Do something at the top of the Pyramid of Doom!
            }
        }
    }
}

becomes this:

if let x = self.x, y = self.y, z = self.z where z == true {
    // Do Something without climbing the Pyramid!
} 

Beautiful!

Apple has even included a handy Migration Tool to get your code from Swift version 1.1 to version 1.2.

There are a lot of other great changes as well (including enhancements to Objective-C) so head over to the Apple Developer Center to download the beta and read the full release notes.

First, Some Stats & Facts:

• In January 2012 , Google had over 4 billion views per day. It’s definitely more now, but this is the most up-to-date stat I could find.
• Every time you load a Youtube video, you are downloading that view counter (along with the video and everything else on the page).
• 32-bit integers take up 4 Bytes; 64-bit integers take up 8 Bytes.

Now For Some Math:

oldCounterBandwidth = ( 4 Bytes / view ) * ( 4 billion views / day ) = 16 GB / day
newCounterBandwidth = ( 8 Bytes / view ) * ( 4 billion views / day ) = 32 GB / day

Note: Final values were converted from Bytes to GigaBytes for clarity.

By converting to 64-bit integers to store the view counters, they are doubling their bandwidth from 16 GB per day to 32 GB per day…. just to load view counters. Keep in mind that the daily view stat I found is from January 2012.