My Software Factory in the Age of AI

I'm creating this page to document my "software factory". It will be more of a reference page than an article and I will reference it on the resources page of the site.

Context: several applications in monorepos (hakanai, Writizzy, Bloggrify), polyglot (Nuxt, Kotlin, JS), solo dev, code written largely by agents, continuous deployment to production.

The guiding idea

Even though the latest generation of agents are now capable of producing quality code (much better than the majority of human developers), code production represents only part of what I call software quality. The rest includes:

• the intention (why, for whom)
• does it satisfy the 4 risks identified by Marty Cagan:
  • Value
  • Usability
  • Feasability
  • Viability
• we can add: performance and reliability

Part of these risks is resolved with a good understanding of your market, benchmarks, interviews, manual testing, mockups. All of this is also part of my "software factory" even if I don't describe everything here.

The code produced is now almost 100% generated, but that doesn't mean it's vibe coding. Vibe coding as defined by Karpathy was experimentation and letting yourself be carried along by a dev session. Here, I'm going to talk about context engineering. The goal is to provide all the necessary context, at the right time, so that the software matches an intention and is systematically controlled. Even though I don't write the code, I'm responsible for it and I need to maintain control over it.

The tooling described here answers several questions:

1. What does the agent know? (context, memory, code graph)
2. What can it do deterministically, without improvising? (skills, procedures)
3. What stops it when it makes a mistake? (hooks, architecture tests, quality gates)

Layer 1: the context

Here are the files read by agents before starting. Be careful, the size of the context must remain controlled. Too large a context costs money and degrades the quality of responses if it becomes too heavy. We want to save tokens and optimize when information is loaded.

The separation matters: permanent context stays short, specialized context loads when it's useful.

Long-term constraint as context. Concrete example: I plan to open-source part of the code. The rule "code intended for open source should never depend on proprietary code" is written in the rules and verified by a test (see layer 4). Writing a future constraint in the context avoids paying for a refactoring later.

You can also find constraints on:

• marketing copy (labels in the application)
• how to write migrations
• types to favor for database fields
• asynchronism patterns
• etc. (list far from exhaustive and enriched regularly)

A constraint is specific to a project and a person. It's not a matter of software quality in the strict sense. It's not better or worse to do feature flagging for example, but it's my preference for partial deployment and feature activation in testing.

There's no point in telling an AI to "write quality code", it doesn't make sense. You need to make your own constraints explicit.

Example of a rule that allows deferred loading depending on context:

Every time I ask you to commit, use the commit-convention skill

It's simple, but it avoids loading the entire skill in a rule (which would be loaded systematically).

Here's a more complex rule:

---
paths:
  - "api/**/*"
---

You act as a Senior Expert in Kotlin and Spring Boot (DDD, Spring, JPA).
You have patterns to follow. BEFORE writing code, you MUST invoke the skill corresponding to the task you are performing.

## Skills to invoke depending on the task

| Task                                                                           | Skill             |
|---------------------------------------------------------------------------------|-------------------|
| JPA entities, repositories, data model                                          | `kotlin-jpa`      |
| Database schema changes, Flyway migrations, and Hibernate/Kotlin entity mapping | `db-migration`    |
| Services, business logic, transactions                                          | `kotlin-services` |
| Controllers, REST APIs, DTOs                                                    | `kotlin-api`      |
| Unit or integration tests                                                       | `kotlin-tests`    |
| End of task, validation before confirming                                       | `kotlin-review`   |
| Feature gating by plan (offers.sql, FeatureService, quota, conditional access)  | `offers`          |
| Unleash feature flags (progressive rollout, admin-only, maintenance, runtime toggle) | `feature-flags` |
| RabbitMQ producers, consumers, outbox pattern, retry/quarantine, AMQP config    | `rabbitmq`        |

Here there are two important things:

• the activation pattern, which specifies that this rule only loads if you touch the api directory
• the table that lists all available skills, to be opened only if needed. If the AI doesn't make schema changes, there's no point opening the db-migration skill

Layer 2: the skills

A skill is a procedure written once, replayed identically. The agent loads it itself when the context matches. This automatic loading can sometimes fail. In that case, you need to explicitly ask to use the skill.

The criterion: am I repeating myself? If I explain the same thing a third time, it becomes a skill.

I have about thirty, grouped by family:

Two things I've learned:

• "Multi-file procedure" skills are the most profitable. Example: adding a block to the content editor touches three different rendering surfaces. Without a skill, the agent systematically forgets one.
• "Up-to-date doc" skills are very important on code that evolves quickly. It allows an agent to understand the entry points and intentions of a feature.

Sub-agents: for tasks that generate a lot of reading without much decision (audit, broad exploration, doc writing), I delegate to a sub-agent. It consumes its own context and gives me a conclusion, not a dump of files. I use them less and less, recent agents make their own fairly targeted delegations.

Layer 3: the tools

You often hear that an AI is non-deterministic and can make mistakes on trivial things that need to be deterministic, like calculating 2+2.

That's largely false now, and an AI is no longer "just an LLM": it has many tools to control output. Nevertheless, the best way to ensure a form of reproducibility is to delegate to tools whose job it is.

Compilation, test execution, linters, all of that is delegation. You can delegate to MCPs, or to skills that use themselves a command-line tool (CLI). I try to avoid MCPs which consume more context, but I have a few anyway.

MCP

GitNexus

The repo is indexed in a graph (symbols, relationships, execution flow). This allows you to measure impact levels and find all links with the code being modified:

• impact(symbol) before modifying: blast radius, callers, risk level
• detect_changes() before committing: did I only touch what I wanted to?
• find an execution flow rather than grep a function name
• rename via the call graph rather than find-and-replace

The real issue isn't speed, it's detecting all side effects of a modification.

Claude-mem, persistent memory

I use two levels of memory:

• Claude's internal memory
• Claude-mem, which allows capturing decisions between sessions

What we want with these tools is to avoid repeating mistakes, document decisions, and not start over with an empty session each time.

Claude Code's internal memory mechanism has improved and becomes more relevant than before with the latest versions. However, you need to control it and not hesitate to ask it to delete rules it creates on its own which are sometimes a bit silly. Claude-mem, I honestly have a hard time measuring the negative or positive impact. I don't have enough perspective on it yet.

Output filtering

A wrapper (here RTK) prefixes shell commands and only returns what's necessary. It remains very limited, only available for a few tools. The gain is sometimes canceled out because Claude runs the command twice. It doesn't hurt, but I think there's still room for improvement. Note that Claude builds its own tools on the fly in Python or Bash, and also knows how to use filtering mechanisms with grep, tail, etc. to optimize the outputs of the tools it uses itself and save tokens.

Debugging with access to logs and database

I use two skills for incident resolution:

• a Sentry skill to read information on Sentry and retrieve stack traces
• a db-query skill that gives read-only access to the database (read/write access is possible directly via psql on the Docker container in local dev)

I could also mention the Stripe MCP, read-only as well, which allows in certain specific cases to investigate Stripe configuration issues.

Layer 4: executable guardrails

Guardrails prevent the randomness associated with understanding and executing an instruction. It happens that an LLM ignores a rule. You need to provide tools that execute automatically.

Hooks

Scripts triggered by the agent harness, not by the agent itself.

Other uses that work well: block editing of generated files, require a test alongside any new module, forbid a dangerous pattern.

Architecture tests

Structuring rules become tests that break the CI.

Real example, I have an architecture test that preserves the boundary between future open source code and the rest:

• no file on the "open" side references the "proprietary" side
• every production file belongs to one of the two sides

It's part of the automated tests, so it can't be bypassed, unlike a rule.

Pattern linting

I use several things:

• ESLint for syntax
• ast-grep for architecture decisions, for example forbidding any call to fetch without going through the OpenAPI client
• typecheck for typing

Layer 5: the factory

One blocking quality gate, per application

Each app has its workflow on GitHub Actions.

push to main
   └─> quality gate: lint → pattern lint → typecheck → tests
        └─> build docker image
             └─> push registry
                  └─> deployment webhook

The deployment job has a needs: on the quality job. Nothing goes to production without passing the gate. It's essential in general, even more so for automatically generated code.

Test at multiple stages

I shouldn't need to re-explain this, but just in case, I have several types of tests.

I give Claude instructions to explain the test hierarchy, gitnexus tells him what to replay to validate these modifications, and he has instructions to write them when he adds/modifies code.

Layer 6: the product process

As I said in the intro, the goal isn't just to produce code, it's to produce code that serves a purpose.

The process often starts with a spec, then a design, then an implementation.

Spec first. I have a folder of numbered specs, one per functional domain, indexed in the permanent context. Two skills frame the cycle: one for writing the spec and its plan, one for closing it by updating it with what was actually built.

This spec and the discussions can rely on a 'product-marketing-context.md' file that I create in each project and which summarizes my personas, my competitors, my positioning etc...

The design. By design I mean two things: technical design and interface. Technical design is part of the spec phase. Most of the time a spec is enough, but some tricky cases require a spec dedicated to a technical component or a technology choice. On the other hand, the design/mockup phase is separate. I do it in Claude Design. I make a functional mockup and iterate until it's perfect. I verify understanding, labels, usability. Then I can pass the result to Claude Code.

The implementation. Claude starts from the spec and mockup. He follows the plan made in the spec phase. The spec is meant to be delivered in stages (protected by feature flag). This allows me to do several small implementation sessions rather than one large session, which tends to degrade in quality if it gets too full.

The closing step is important: without it, specs become obsolete in six months.

Explicit rule in the context: if a spec is vague or inconsistent with what exists, the agent should ask the question, not guess.

Progressive delivery.

I work trunk based. However I use feature flipping and gating, which represent two different things:

I have skills that explain the difference, so agents don't do anything wrong and respect my working process.

Where to start

If you're starting from 0, the first step is the quality gate if you don't have one. You need a control mechanism that runs tests, linters etc... Then start light with a Claude.md that describes the essentials, the why. Add rules as you go for important architecture patterns. As soon as you see procedures that come back often, document them as skills. And then equip yourself with cli and MCPs to interact with your main tools, jira, sentry, etc...

Be careful, any skill, MCP, code taken from outside must be scrutinized. These are dependencies that can be vectors of attack.

Key takeaways

1. What matters must be executable. An instruction is followed "most of the time", but it can be forgotten. A hook or test is followed all the time.
2. An error must be documented. Every error must be recorded in a skill or in memory.
3. Context is a budget. Wrappers, filters, sub-agents, summaries: everything that reduces noise keeps reasoning on the problem.
4. Measure impacts before and after editing. We want to avoid the effect "1 bug fixed, 10 produced".
5. Repetitive procedures don't improvise. One skill per procedure.
6. Spec documentation dies if its closure isn't in the process. You need to plan the update and maintenance step.

To improve

Updating skills/rules

You need to take into account that the technology is still very young, February 2025 if we consider agentic programming. Tooling is improving but you also need to constantly review it. Mid-2025, some instructions in a Claude.md made sense, for example "write a test for each new service". Today it's noise and Claude does it naturally. So you need to be wary of your old rules, sometimes they're obsolete and create noise. I have no way to measure and know if an old rule has become obsolete.

Latest versions of Opus are increasingly autonomous. AI takes the initiative on its own to build, look at produced content, read dependency code to understand calls, find bugs, run tests in the browser. It's almost creepy and more rigorous than 99% of humans. Let's be honest, I'm increasingly less useful in implementation phases, but I don't want to lose control of the produced code. I'm torn between satisfaction at having an increasingly efficient software factory and the risk of losing knowledge. I need to find a way to control designs a posteriori, to appropriate the result.

The rabbit hole problem

I recently added a boyscout.md rule

Always leave code a little better than it was before. If you see flagrant errors, report improvements to me. If you notice gaps with documentation or information present in your skills, or in this file, same thing, report them.

But I find myself having endless sessions. I think I've never worked on a codebase that maintains itself as much. I've rarely improved the product at this level of detail. But it comes with a cost, cognitive overload.

I think I'd rather automatically note these elements and categorize them in an online TODO list (trello, todoist etc...). I think the workflow for maintenance should move elsewhere, and be partially automated.

Lack of standardization

I still copy and paste my skills/rules etc… from one project to another. And sometimes it's dependent on my station based on a skill installed locally.

I need to find a way to package my skills to deploy them where relevant and to centralize maintenance.

The rest

In other pending improvement points:

• my dependency on Claude. I want to test open weight models but I don't have the hardware for it. Moderate risk in my opinion, the entire ecosystem is moving upward
• the IDE is becoming obsolete compared to this new workflow. I still use Intellij but I no longer find it suited to our time. I haven't seen an interesting alternative yet.