I like to have it clean. Most of the programs I’ve written in the younger past are pretty well structured. Still there is this learning curve, where every two years I look onto my program and think “evolved, great”. Stagnation is dead. But the code is well structured, follows the “right” rules, whatever that means. The most important thing is, it follows a common thread.

That means even switching projects is not a problem. I am a freelancer, so even switching between customers where I wrote IIOT applications, web application, message based backends, infrastructure automation, pipelining, all follow the same basic idea. So I am mostly in the luxury position to be able to forget what I did and understand is pretty fastly watching the lines and the folder structure.

But what I very rarely do, is document the why. All the design patterns and strategies, DRY and KISS, Tell don’t ask, SOLID, the libs and the procedures. They help to make things recognizable.

I never document the why.

Religious wars

You may remember the religious discussions about “what to document”. Use fancy features in your IDE to auto generate documentation of classes and/ or methods that actually have no value at all. Having something like this:

/// <summary>
/// Gets a blue collar worker 
/// </summary>
/// <param name="logger"></param>
/// <param name="blueCollarWorkerAdapter"></param>
public class GetBlueCollarWorkerRequestHandler(
    ILogger<GetBlueCollarWorkerRequestHandler> logger,
    IBlueCollarWorkerAdapter blueCollarWorkerAdapter) : IRequestHandler<GetBlueCollarWorkerRequest, GetBlueCollarWorkerResponse>
{
    public async Task<GetBlueCollarWorkerResponse> Handle(
        GetBlueCollarWorkerRequest request,
        CancellationToken cancellationToken)

Okay, when the method is called “Handle” and the class is called ” GetBlueCollarWorkerRequestHandler I really don’t need this “Gets a Blue Collar Worker” documentation. So mainly I personally document two kind of things:

• official interfaces: swagger/ rest api, nuget/ npm/ whatever packages, libs
• problematic areas: when I needed to write something that is not understandable, I need to document my decisions, mostly with links to sources

Sometimes that tells the story, but this does not help to get an overview about the decisions of a program. Documentation of the “why” takes time. I use documentation software like Arc42 that leverages Architecture Decision Records. But this is outside of the code in an own repository. And even this only declares architectural thoughts and directions. It may not necessarily mean to define the decisions taken when implementing certain features.

Usual problem

As a result, six months after having shipped that feature, nobody remembers what decisions had been taken why. Sure, let’s call this guy with the last commit. If he’s still in the company. And if he is, let’s see if he remembers.

In a well structured program, usually this is not going to break my neck. But sometimes it is more than annoying to remember, and when there is nobody remembering, then hopefully the automatic tests will hold their promise to catch my mistakes.

What if the why was just part of the output?

Developer’s work changes a lot nowadays. With the rise of Claude and all his buddies, there are some more possibilities to write code and let it be documented. Anyway all of these AIs need to have some kind of plan to write the actual code. When it’s not just “vibe”- coding but more constructive.

When the AI came up, I needed something to get my hands dirty. In my case it was Agent Smith. When working with Claude in ZED, VsCode and Rider, I noticed a few really embarassing things:

• when the tab with Claude does not work anymore or for any other reason Claude stops to work, the context is completely unclear. So I waste a lot of token of my subscriptions to get back. And Claude is kind of bulky when restarting
• Having long threads with Claude to straighten out what needs to be done creates a long chat of conversation. When this is gone, it is really cumbersome to explain all the stuff again.
• Explaining all the past’s features even extending my frustration

Plans And Runs as the Rescue

So I created two different kind of modes to implement. One is the “vibe” coding kind of thing. Me in the IDE planning features, bug fixes, changes, tests. whatever. Claude implementing this. This is “phases”.

Agent Smith is meant to execute implementations via tickets in Azure DevOps, Jira, GitHub or Gitlab. When doing this, there are “runs”.

Phases have state being active done or planned. Each phase is a md file.

flowchart TD
    classDef box fill:#1e1e2e,stroke:#7c3aed,color:#e2e8f0,rx:6
    classDef container fill:#0f0f1a,stroke:#7c3aed,color:#a78bfa
    
subgraph Phases["Phases"]
        direction TB
        P1[Create phase doc]:::box
        P1 --> P2[Move to active]:::box
        P2 --> P3[Implement]:::box
        P3 --> P4[Move to done]:::box
        P4 --> P5[Enhance context.yaml]:::box
        P5 --> P6[Commit]:::box
    end

Runs are executed by Agent Smith and come with plan.md and result.md. They are part of a pipeline that either can fix bugs or create tickets with multi agency.

flowchart LR
    subgraph Runs["Runs"]
        direction TB
        R1[Receive ticket] 
        R1 --> R2[Generate plan.md]
        R2 --> R3[Execute]
        R3 --> R4[Write result.md]
        R4 --> R5[Enhance context.yaml]
        R5 --> R6[Create PR]
    end

    

Phases are how Agent Smith itself gets built. Before any code, there’s a phase document. I discuss with claude in the web browser my next ideas. Claude will create a small document describing the problem, approach, definition of done and sometimes code, abstractions. This is what I add in the IDE. The “IDE Claude Buddy” will ask questions when there is something unclear. I read it briefly but mostly I can just move these questions back to the web Claude. He will answer. Give it back. And then the predefined process starts which is described in claude.md, see first few lines:

# Agent Smith — Claude Code Instructions

## Context Files (read in this order)

1. `.agentsmith/context.yaml` — architecture, stack, integrations, phase status
2. `.agentsmith/coding-principles.md` — code quality rules (ALWAYS follow)
3. `.agentsmith/phases/active/p{NN}-*.md` — prompt for the phase being implemented
4. `.agentsmith/runs/r{NN}-*.md` — prompt for the runs already implemented

## Phase Directory Structure

```
.agentsmith/phases/
├── done/       # completed phases (historical reference)
├── active/     # phase currently being worked on (max 1)
└── planned/    # upcoming phases with requirements
```

## Runs Directory Structure

```
.agentsmith/runs/ # all runs already executed by agent smith
```

Saving Tokens and make Claude knowing

Keeping it this way, Claude always knows what the current state is without wasting tokens. Have a look onto the context.yaml compressed information about the phases:

With that information in place, claude nows exactly what happened in this programs, it knows where to look when it needs more information as the phase and run documents contain all “what” and “Why”s.

Beside that context.yaml contains information about how to implement the program. Architecture, stack, meta and a lot more.

# yaml-language-server: $schema=context.schema.json
meta:
  project: agent-smith
  version: 1.0.0
  type: [agent, pipeline]
  purpose: "Self-hosted AI orchestration framework: code, legal, security, workflows."

stack:
  runtime: .NET 8
  lang: C#
  infra: [Docker, K8s, Redis]
  testing: [xUnit, Moq, FluentAssertions]
  sdks: [Anthropic, OpenAI, Google-Gemini, Octokit, LibGit2Sharp, YamlDotNet]

arch:
  style: [CleanArch]
  patterns: [Command/Handler, Pipeline, Factory, Strategy, Adapter]
  layers:
    - Domain        # entities, value objects — no deps
    - Contracts   # interfaces, DTOs, config models
    - Application   # handlers, pipeline executor, use cases
    - Infrastructure # AI providers, git, tickets, Redis bus
    - Host          # CLI entry point, DI wiring
    - Dispatcher    # Slack gateway, job spawning, intent routing

What’s in that documents?

Let’s take a simple sample for the generated documents of Agent Smith’s runs.

A fixed a NullReferenceException in a todo API. What comes out are:

Plan.md: Planning the task at hand

# Plan: GET /todos returns 500 when database is empty

**Ticket:** #60 — GET /todos returns 500 when database is empty

## Summary
Fix NullReferenceException in GET /todos endpoint when TodoStore is empty by ensuring the internal collection is properly initialized

## Steps
1. [Modify] Initialize the _items collection in TodoStore constructor to prevent NullReferenceException when accessing an empty store → Services/TodoStore.cs

Result.md: what actually happened

---
ticket: "#60 — GET /todos returns 500 when database is empty"
date: 2026-02-24
result: success
duration_seconds: 51
cost:
  total_usd: 0.0720
  phases:
    scout:
      model: claude-haiku-4-5-20251001
      turns: 4
      usd: 0.0091
    primary:
      model: claude-sonnet-4-20250514
      turns: 7
      usd: 0.0628
---

That’s a small fix. The overall pipeline took 51 seconds, and it cost 7 cents. The result is part of a Pull Request in Azure Devops or one of the other supported tools. Here you got the why. And everything else a developer would never add, especially when he’s in a hurry.

Three layers of knowledge that accumulate

flowchart TD
    A["Phases
what & why
.agentsmith/phases/done/"] 
    B["Runs
what happened with cost calc 
.agentsmith/runs/r01.../result.md"]
    C["Decisions
the why-not
.agentsmith/decisions.md"]
    D["Wiki
compiled knowledge
.agentsmith/wiki/"]

    A --> D
    B --> D
    C --> D

Phases capture the reasoning before implementation. It contains comprehensive information about the strategy, decisions, thoughts and methodic.

Runs capture the execution. Has plan and cost outcome. Tells the full story.

Decisions capture the why-not. This tells why alternatives do not have taken into account due to weighting the pros and cons. This also never gets written down.

And all of it compiles into a wiki that the agent maintains automatically.

Andrej Karpathy noticed the same gap

Karpathy wrote recently about using LLMs to build personal knowledge bases. When I recognized it I need to compare his thought against the implementation approach inside of Agent Smith.

For Andrej, it is collecting external material into a raw/ directory, letting the LLM compile it into a linked markdown wiki, then running Q&A against it. Obsidian is used as the frontend. Markdown files are used in a directory the LLM writes and humans read. I do really like this pattern.

The structural parallel here is obvious. But there’s a key difference.

Karpathy collects external knowledge. Papers, articles, datasets. These things that exist in the world and get pulled in manually.

The proof is in the timing and the content

You may want to have a look at the documentation of Agent Smith. This documentation has been fully generated by Claude. This is not that dramatic, actually. Guess lots of people are already done it before.

How long did it take? It was just something about 15 minutes. Of course it was another phase following this paradigm.

docs.agent-smith.org/
├── Getting Started
│   ├── Installation (binary, Docker, source)
│   ├── Quick Start — your first bug fix in 5 minutes
│   └── Quick Start — your first API scan
├── Pipelines
│   ├── fix-bug / add-feature
│   ├── security-scan
│   ├── api-scan
│   ├── legal-analysis
│   └── mad-discussion
├── Concepts
│   ├── Pipeline System (commands, handlers, presets)
│   ├── Phases & Runs (context.yaml, result.md, the workflow)
│   ├── Multi-Skill Architecture (roles, triage, convergence)
│   ├── Decision Logging (decisions.md, why not what)
│   └── Cost Tracking (token usage, pricing, result.md frontmatter)
├── Configuration
│   ├── agentsmith.yml Reference
│   ├── Skills YAML Reference
│   ├── Tool Configuration (nuclei.yaml, spectral.yaml)
│   └── Model Registry (providers, routing, pricing)
├── AI Providers
│   ├── Claude (Anthropic)
│   ├── OpenAI / GPT-4
│   ├── Gemini
│   ├── Ollama (local models)
│   └── OpenAI-Compatible (Groq, Together, vLLM, LiteLLM)
├── CI/CD Integration
│   ├── Azure DevOps
│   ├── GitHub Actions
│   ├── GitLab CI
│   └── Generic (binary in any pipeline)
├── Deployment
│   ├── Single Binary
│   ├── Docker / Docker Compose
│   ├── Kubernetes
│   └── Chat Gateway (Slack, Teams)
├── Architecture
│   ├── Clean Architecture Layers
│   ├── Project Structure
│   └── Extending Agent Smith (custom pipelines, providers, skills)
└── Contributing
    ├── Development Setup
    ├── Coding Principles
    └── Phase Workflow (how we plan and track work)

code.agent-smith.org  CNAME  holgerleichsenring.github.io

As all of the features, bugfixes, ideas and decisions are documented in the code anyway, it is not surprising that it can create the documentation rapidly with a pretty precise content.

The same if true for the website.

That’s the thing that’s hard to explain until you’ve seen it. Documentation usually takes time because you have to remember things. Visit the code, reconstruct the reasoning after the fact. When the reasoning is captured while it’s happening, there’s nothing to reconstruct.

When the code breaks six months from now, you’ll know exactly what the agent was thinking when it wrote it.

That’s not a documentation strategy. That’s just how the work is structured.

Have a look at the github repo.