For the past few months I have been building an AI development lifecycle with a delivery team.
One system, many years in the field, and a team that handles customer-specific requests on top of it.

We started like any other change project, with a two-week assessment. In it we mapped how work actually moves today, from a customer’s requirements document to a package installed at the customer.
Then came seven weeks of building an AI-based software factory, and we had it focus on back-end work first.
The result was a proof of concept, and we gave it to developers to try on real requirements.
While we collect their feedback and act on it, we keep extending the system to the parts it doesn’t cover yet.

Today a requirements document goes in, and code comes out that builds and passes its own, human approved, acceptance and unit tests.
Along the way the AI writes the user stories, turns them into executable acceptance scenarios, writes the design, implements test-first and reviews the result.
A person answers what the document left open, signs what “done” means, signs the design and rules on every review finding.
Front-end work, deployment and proving the result in a running system stayed as they were, for now.

That is the background. The interesting part is what we learned, and a lot of it was unexpected.

THE PLAN WAS ONE AUTONOMOUS AGENT. IT BECAME A SKILL PER STAGE, USED AND GATED BY PEOPLE.

What the team asked for at the start was an autonomous agent: hand it a feature, get back working code, with nobody in between. A developer would then review it, maybe test it, and deploy it.
What they got is a multi-step software lifecycle in which each stage has a skill (or a set of skills), and a person helps clear up vagueness, fill in gaps and approve at every major gate.
You might call this a compromise. For us, it’s what worked.

An agent working end to end makes a guess early and builds on it for hours. By the time anyone looks, the guess is under a design, code and tests that all agree with each other.
Splitting the work into stages, each producing something a person can read and sign, puts the decisions where they are cheap to change.
Stages also let people take one piece, use it and test it in isolation.
A skill they run at a stage they already know fits into their day.
A full cycle inside a black box that just returns a finished change does not.

EXISTING CODE IS A BAD TEACHER

This one was a reality check for me.

The natural instinct is to point the agent at the codebase and tell it to learn the product, the system and how things are done from the code.
In a young codebase that may work.
This product, however, has been in the field for years. Its code holds a history of rules: duplicated techniques that evolved over time, decisions that were later changed, and sometimes direct conflicts.
In real life, approaches change, old code stays, and things that work don’t always get cleaned up.
Two working examples from different years may contradict each other, and both are still active side by side.

A typical agent learns the most common practice, and in a long-lived system the most common practice is usually not the most modern one. In many cases it’s a practice the team has already moved away from.
It copies it confidently, because the evidence is overwhelming.
How often a pattern appears says nothing about whether it is still the rule.
We thought about giving priority to newer techniques, but that opens the door for experiments that were ruled against to sneak back in.
For us, what worked is having a person rule on these development standards (see below).

DOMAIN KNOWLEDGE SETS THE QUALITY

When we started testing the chain on real requirements, dozens of issues came up. We fixed most of them, and a few were deferred or ruled out.

The interesting bit is that almost none of them were coding mistakes.
Nearly every miss traced back to product knowledge the agent didn’t have: how a particular flow really works, which mechanism already exists for a job, what a term means in this product.
The rest were inputs it never asked for, like an image in the requirements it didn’t read, and decisions it made on its own when it should have stopped and asked.

Almost all of these were fixed by adding knowledge, and with stricter rules that make the agent ask when it doesn’t know.
At this point it doesn’t feel like we need a better AI engine.

REQUIREMENTS ARE NEVER COMPLETE

In one run, the produced code met every written requirement and still missed standard behavior that the code implements everywhere else.
Nobody wrote it down as a requirement, because everyone knows it.

This touches an old debate.
R&D always complained that requirements were never good enough, and for the most part the industry’s answer was to invest more in writing better requirements.
I saw this fail more than once. What usually did help was turning requirements gathering into a conversation.
But the honest answer was always that finding what is missing is part of the developer’s job, and as an industry we never quite got there.
In practice, a human developer filled the holes quietly, from what they knew about the product, and that was usually good enough.

An agent fills them quietly too, but with guesses.
Every requirements document I tested had gaps (even a few that were already fully developed and signed).
And gaps are only one kind of problem: there are also mistakes, conflicts and details that are simply wrong.
With AI helping to write requirements, I expect people to go back to expecting that a requirement reaching development is complete.
I don’t believe AI makes that any more realistic.
To me the development stage still has to assume that requirements are not perfect, and be built for it:
The agent reads the existing product as part of the specification, and every hole becomes a question for a person.

JUDGING WAS THE MISSING HALF

The team already had some AI skills before we started.
Almost none of them judged anything. Mostly they were focused on creation.
They produced code, and nothing checked that code against a standard.

Review became the core of what we had to build.
Stories and designs are reviewed by a second agent that never saw them being written, because a self-review re-derives the same assumptions and confirms them.
Acceptance scenarios are signed by a person before any code exists, so “done” is something the machine can check.
Code review reports its findings and a person rules on each one.
Generating is just one half. Most of the effort went into the half that says whether the result is good enough.
And we are still improving those parts.

STANDARDS LIVE IN PEOPLE’S HEADS

To judge code against a standard, the standard has to be written somewhere.
Mostly it wasn’t. It lived with the senior developers, who answer the question when someone asks them.

Gathering and writing those standards down as checks the agents run is a piece of work of its own.
It is not a side effect of building the tooling, and it cannot be done from the outside.
It needs an owner on the client’s side, someone with the authority to say which of the ways the code does something should be used.

Over the coming weeks I will take each of these findings apart in its own post.

* * *

Written by Lior Friedman.
Lior is a Senior Software Architect and R&D Leader with 25+ years of experience building mission-critical systems and leading large-scale engineering transformations. Today, he focuses on helping R&D organizations adopt AI-enabled engineering practices-turning AI capabilities into practical, repeatable, and scalable workflows across coding, testing, and documentation.

* * *

Practical AI helps organizations move from individual AI experimentation to a shared, scalable way of working. We combine deep software engineering expertise with organizational change to build the standards, workflows, and practices that make AI work across teams — not just for individuals.
Let’s talk about building your AI capability at scale.