Practical Blog

Knowledge Platform

Practical Blog

Knowledge Platform

Your code can't always teach your agent the rules

During the first round of feedback on the AI software factory we built, a developer flagged a problem in code an agent wrote.
A call into the platform was made without providing a proper user ID.
In this system almost every operation takes one.
The system uses it to decide whose permissions are checked, and who the action is recorded against.

Fair enough. So which user should the code pass?
I asked an agent to find the rule in the code.

ONE QUESTION, FOUR ANSWERS

The agent scanned the system's code, some of the product's own documentation comments, and one production-like test environment.
It found four answers:
- most calls pass no user at all, which skips the permission check,
- some pass a preconfigured ID, under a different name in different projects,
- some pass the user who triggered the action,
- and in one case the code does exactly what the developer said was wrong.

All four in production. All four working.

The agent did the research well, quickly and thoroughly.
It even suggested a guideline: take the user who triggered the action, else the user in context, else a configured default user, else none.
Kind of a combination of everything it found.
What it could not tell was which of the four is the desired behavior today.
Nothing in the code says so.

It did not stay neutral either. Twice it turned thin evidence into a general rule.
The developer's comment about one call became, in the agent's write-up, a rule for every call.
To get to the actual answer, we needed a discussion with the people involved.

THE CODE HOLDS A HISTORY OF RULES

In a young codebase the code is a fair teacher. It mostly reflects one set of decisions, made by a small group, recently.

A system that has been in the field for years reflects all the decisions made along the way.
Approaches change. Teams change. A better way is introduced, and remnants of the old way stay, because rewriting what works is not common practice, and even when it happens, people miss things.

None of this is an inherent defect. A system is a living thing that evolves over time.
The result is that the code no longer holds a single rule.
It holds a history of rules.

THE AGENT VOTES WITH THE MAJORITY

An agent investigating the code learns by default from the most common practice.
In a long-lived system the most common practice is often just the oldest one, simply because there is more old code than new, or because replacing it can take weeks or months.
When that happens, the agent reproduces what the team already moved away from, and it does so confidently, because the evidence is overwhelming.

The obvious fix is to tell the agent to prefer newer code. We considered it, and it fails the other way.
Newer is not the same as approved.
New approaches are always being tried. Some work out and become the new standard. Some fail, and are mostly taken out.
But while they are being tried, they are the newest code in the system.
So an agent that follows "newest is best" would bring them straight back in.
And all of this assumes the simple case, where there is a single rule to begin with.

WHERE THE EFFORT ACTUALLY GOES

This is why an agent, harness or tool out of the box is not enough on its own, however capable the model.
What is missing is not intelligence. The answer is not in the code for any model to find.

Teaching the agent means turning unwritten knowledge into something it uses on every task:
- which approach is current, and which is legacy that new code should not copy,
- the exceptions, stated explicitly, because otherwise the agent generalizes from them,
- where each kind of logic belongs,
- and where the answer depends on the customer, so the agent asks instead of guessing.

Each of these needs a source, and someone who ruled on it.
The agent can propose a rule from what it finds. It cannot decide which of four working practices is the standard.
That takes someone who understands both the system and the business and its history, and who will stand behind the choice.

Teaching the agent the unwritten knowledge is where most of the work in a real adoption sits, much more than in choosing the tool.

A NEW HIRE HAS THE SAME PROBLEM

The problem isn't new. Even the most experienced developer needs ramp-up time on a new system.
In a big system, it can take weeks, even months.
They spend that time learning which of the four ways is current, mostly by asking.
An agent doesn't ask unless you build that in, and it forgets everything when the session ends.
So effort has to go into gathering the tribal knowledge and organizing it so that the agent draws on it in every session and every task.

WHERE TO START

One way to approach it is to list the questions a new developer asks when they start working at your company:
which user an operation runs as, how errors are handled, where configuration is read from.
Then have a senior developer answer them one by one, and pick a reference example for each answer.
In other words, define your standards.
Then feed this into your agent, and tell it that the given examples are the rules and the rest of the code is history.
Slowly, you build your own knowledge base that helps the agent write the code.

Remember, your codebase records what was done. Only your people know what should be.

This post expands one finding from "What building an AI development lifecycle taught us".

* * *

Written by Lior Friedman.
Lior is a Senior Software Architect and R&D Leader with 25+ years of experience building mission-critical systems and leading large-scale engineering transformations. Today, he focuses on helping R&D organizations adopt AI-enabled engineering practices-turning AI capabilities into practical, repeatable, and scalable workflows across coding, testing, and documentation.

* * *

Practical AI helps organizations move from individual AI experimentation to a shared, scalable way of working. We combine deep software engineering expertise with organizational change to build the standards, workflows, and practices that make AI work across teams — not just for individuals.
Let’s talk about building your AI capability at scale.