Chapter 6

Chapter 6

Building the Engineering Knowledge AI Needs

Building the Engineering Knowledge AI Needs

When AI is operating as an assistant beside an experienced developer, many of these gaps can be covered by the developer. The developer recognizes when the AI is heading in the wrong direction, adds a missing detail, explains an exception, points the AI toward the correct component, or rejects a recommendation because it conflicts with something the developer knows about the system. In that environment, the human is doing more than reviewing AI output

The human is continuously supplying organizational knowledge and judgment that the AI does not have.

The question, therefore, is not simply “How do we give AI more context?” The more useful question is “What does AI need to know to make this engineering decision correctly, where does that knowledge come from, and how do we know it can be trusted?” That distinction matters because more context is not necessarily better context. An organization can give an AI system thousands of pages of documentation and still leave it uncertain about which information is current, which source is authoritative, which architecture decision still applies, or which business rule contains an exception. The objective is not to give AI everything the organization knows. The objective is to make the right engineering knowledge available at the point where that knowledge changes a decision.

The Knowledge That Exists Outside the Code

Software organizations accumulate this type of knowledge continuously. Architecture decisions, business rules and exceptions, integration constraints, security requirements, approved and prohibited dependencies, deployment assumptions, customer-specific behavior, historical failures, regulatory requirements, data-handling rules, coding and testing conventions, and the reasons apparently simpler solutions were rejected all become part of the real engineering environment.

The difficulty is that this information usually exists in different places and with different levels of reliability. Some is in architecture documentation, some is in tickets, some is in source-code comments, some is in a wiki, some is buried in an old email or meeting note, and some exists only because an experienced engineer remembers it.

Start With What Experienced Engineers Keep Repeating

• “Before you touch that, you need to understand…”
• “I know this looks strange, but we do it this way because…”
• “We tried that several years ago and it caused…”
• “That customer works differently…”
• “Don’t use that library here because…”
• “That service looks unused, but…”

Those statements are clues. If experienced engineers repeatedly need to provide the same explanation to developers, reviewers, new employees, or AI systems, the organization may be depending on knowledge that should be made more durable. AI interactions can make these patterns particularly easy to identify. An engineer may repeatedly tell AI that functionality already exists somewhere else, that business logic belongs in another layer, that an application cannot use a particular dependency, that a business exception must be considered, or that the pattern dominating an older repository should no longer be used for new development.

A single correction may simply be part of normal engineering work, but the same correction appearing repeatedly is different. It is evidence. The organization should ask why the AI keeps requiring the correction. Perhaps the information is missing. Perhaps it exists but cannot be found. Perhaps two sources conflict. Perhaps the source is outdated. Perhaps the standard is unclear. Perhaps the workflow is receiving too much irrelevant information. Or perhaps the decision genuinely requires human judgment and should remain with the engineer. Each explanation points toward a different improvement.

This is why the Human Correction Record introduced earlier in the framework becomes particularly useful here. It is not simply a record of mistakes AI made. It can become a map showing where the engineering environment itself needs to improve.

Capture Knowledge That Changes a Decision

A useful knowledge entry might explain what the engineer or AI needs to know, why the rule or constraint exists, what could reasonably be misunderstood without it, where the rule applies, what exceptions are known, what must be verified before changing it, where the authoritative supporting information resides, and who owns the information. The reasoning behind the rule is particularly important. Consider the difference between “Do not modify this integration” and “Do not modify this integration without reviewing the customer synchronization requirements. The external system depends on a legacy identifier that appears redundant in the current application but remains contractually required.” The second instruction gives both a human and an AI system something much more useful because it explains the engineering constraint rather than merely stating a prohibition. If conditions change later, the engineer can understand what must be reconsidered rather than blindly preserving a rule whose purpose has been lost. That is the kind of knowledge worth capturing.

Give Important Knowledge an Authoritative Home

For consequential engineering knowledge, the organization needs to identify an authoritative source. That does not mean every piece of documentation must be centralized into one enormous system. Different types of information naturally belong in different places. Architecture decisions may live with architecture documentation, application-specific instructions may belong near the repository, business rules may belong with domain documentation, and security requirements may be maintained by security. What matters is that the engineering system can answer a very simple question: Which source should win when two sources disagree?

Important knowledge should also have an owner. Without ownership, AI context can quietly become another form of technical debt. The information was correct when it was created, the application changed, and the AI continues receiving outdated instructions because nobody was responsible for changing or retiring them. For important engineering knowledge, identify the authoritative source, the owner, when the information was last validated, what should trigger an update, and what should cause it to be retired. AI does not eliminate documentation lifecycle problems; it makes solving them more important.

Current Knowledge Must Be Distinguishable From Historical Knowledge

This distinction is particularly important in mature repositories. AI often learns from what is common, and if most of the repository reflects an older architecture, the model may repeatedly reproduce that older pattern even though the engineering organization has decided that all new development should use a newer approach. Without clear guidance, the AI may be statistically reasonable and organizationally wrong. That is another reason authoritative engineering context matters. It tells AI not only what exists, but what the organization intends to continue doing.

Give AI the Smallest Useful Context

An effective context package might include relevant architecture boundaries, applicable engineering standards, business rules, security requirements, approved patterns, known exceptions, a small number of useful examples, escalation conditions, and retrieval paths to deeper authoritative information. The contents should depend on the decision being made. An agent updating documentation does not need the same context as an agent modifying application code. An agent reviewing dependencies does not need the same business rules as an agent implementing pricing logic. A testing agent may need acceptance criteria and business behavior that a repository-analysis agent does not.

The question is not “How much information can we give the model?” It is “What information does this workflow need in order to make its delegated decisions correctly?” This distinction will become important again in Chapter 7 because unnecessary context is not only a quality issue. It can also become an economic one.

Do Not Create a Separate Version of Organizational Truth for AI

Where possible, AI-ready context should be derived from or connected to authoritative organizational information rather than independently duplicating it. The goal should not be to maintain human documentation and AI documentation as separate sources. The goal is to maintain reliable engineering knowledge that can be presented appropriately to both.

Preserve Knowledge Before It Walks Out the Door

Ask experienced engineers:

• What would someone need to know before safely changing this system?
• What mistakes do people repeatedly make when they first work on it?
• What looks unnecessary but isn’t?
• What do you know that isn’t obvious from the repository?
• Which business rules cause the most surprises?
• Which architecture decisions should not be reconsidered without understanding the history?

Those conversations can produce valuable engineering knowledge for developers and AI alike. This is one of the larger opportunities hidden inside the AI transition. The organization is not simply making its applications easier for AI to understand. It is improving its own engineering memory.

From Engineering Knowledge to Agent Responsibility

That means knowledge and authority cannot be treated as separate AI engineering problems. Before an organization increases an agent’s independence, it should ask whether the knowledge required to exercise that authority has also become dependable. Does the agent know the architecture boundaries an experienced engineer would recognize? Can it find the applicable business rules? Does it know which source is authoritative when information conflicts? Does it understand the engineering standards that apply to the task? Can it recognize an exception rather than forcing a normal pattern onto an unusual situation? Most importantly, can the agent recognize when the information available to it is not sufficient to make the next decision safely? That question becomes increasingly important as the human moves farther away from the individual steps of the workflow.

Human Approval Should Protect a Real Decision

Capture the missing knowledge. Make it authoritative. Make it available to the workflow. Strengthen independent validation where possible. Then reevaluate whether the same human intervention is still necessary. If the human is making a genuine judgment—evaluating an unusual business consequence, interpreting an ambiguous requirement, approving an architecture change, or accepting a risk—keep the human involved. If the human is simply supplying information the organization already knows or manually verifying a deterministic condition, improve the engineering system instead.

This is a useful distinction because organizations can easily build agent workflows filled with approval points that provide the appearance of control without actually identifying what the reviewer is supposed to decide. The objective is not to remove human approval. It is to understand what the human is approving and why.

Better Knowledge Does Not Automatically Justify Greater Autonomy

A useful progression is to identify the knowledge a workflow requires, make that knowledge authoritative and accessible, determine which decisions can be made reliably from that knowledge, define what the agent is permitted to do, retain human approval where human judgment still matters, and establish the conditions under which the agent must stop and escalate. This creates an important feedback loop between engineering knowledge and agent orchestration. When an agent stops because it lacks information, that escalation may reveal knowledge the organization should capture. When a reviewer repeatedly supplies the same correction, that correction may belong in the agent’s context. When an agent cannot reliably distinguish two situations, the organization may need a clearer business rule, stronger validation, or an explicit human decision point.

The objective is not to remove the human from every workflow. It is to understand why the human is needed and make that requirement deliberate.

An Agent Also Needs to Know When to Ask for Help

An agent should be expected to stop when required context is missing, authoritative sources conflict, the task moves outside its approved scope, a new architecture decision is required, a business rule is ambiguous, validation repeatedly fails, a prohibited action becomes necessary, or human judgment is explicitly required. This is not a failure of automation. It is evidence that the organization has defined the limits of the automation correctly. An agent that knows when to ask for help can be considerably more useful than one that always attempts to finish.

The escalation itself should also be useful. A human should not receive nothing more than “Unable to complete task.” A useful escalation should explain what the agent was trying to accomplish, what it completed, what remains, what information it used, what failed, what it already attempted, why it stopped, and what decision or information it needs from the human. That preserves the work already performed and prevents the engineer from reconstructing the entire problem from the beginning.

It also creates another source of organizational learning. If the same escalation occurs repeatedly, leadership can ask whether additional authoritative context is needed, a standard is unclear, validation is insufficient, the authority boundary should change, or the decision should intentionally remain human. Each outcome teaches the organization something.

Agent Orchestration Is an Engineering Design Problem

This becomes particularly important when several agents or AI-enabled services participate in the same workflow. One agent may analyze a repository, another may produce a proposed change, deterministic tools may validate it, another system may review security conditions, and a human may approve the final action. The orchestration layer needs to make clear which component is responsible for which decision, what information each component receives, what evidence allows the workflow to continue, and where responsibility ultimately resides. Otherwise, an organization can create an automated process in which every individual component appears reasonable but nobody can clearly explain who was responsible for the final outcome. That is not mature agentic engineering. It is distributed ambiguity.

Knowledge, Authority, Validation, and Approval Belong Together

It also explains why the progression from assistants to agents is more than a technology decision.

The Chapter 6 Working Checklist

Begin by identifying the engineering knowledge that repeatedly affects decisions but may not be obvious from the code or current task. Pay particular attention to the information experienced engineers routinely supply from memory, as well as the corrections they repeatedly make when AI or less-experienced developers misunderstand the system.

• What do experienced engineers repeatedly have to explain?
• What important information cannot be reliably discovered from the code?
• Which business rules and exceptions materially change engineering decisions?
• Which architecture decisions require historical context?
• Where does AI repeatedly require the same human correction?

The goal is not to capture everything experienced engineers know. Focus first on the knowledge that changes what someone should do, prevents a predictable mistake, or explains why an apparently reasonable approach should not be taken.

2. Establish Authority

Once important knowledge has been identified, determine where the authoritative version of that information should live and who is responsible for keeping it accurate. AI cannot reliably use organizational knowledge if multiple sources disagree or if nobody knows which version represents the organization’s current position.

• Where does each important piece of knowledge currently live?
• Which source is authoritative?
• Can current guidance be distinguished from historical or deprecated information?
• Who owns the information?
• What should trigger an update or retirement?

The objective is not necessarily to centralize all engineering knowledge in one system. It is to make sure that when two sources disagree, developers and AI-assisted workflows can determine which source should be trusted.

3. Deliver the Right Context

After establishing authoritative knowledge, determine how the relevant portions of that knowledge will reach an AI workflow when they are actually needed. More context is not automatically better context, and providing large amounts of unrelated information can introduce ambiguity, increase cost, and make it more difficult for AI to identify what matters.

• What does each significant AI workflow actually need to know?
• Are you supplying relevant context or simply large amounts of information?
• Can the AI reliably retrieve the information when the decision occurs?
• Are you unnecessarily duplicating organizational truth in AI-specific instructions?

The goal is to provide the knowledge that changes the decision at the point where the decision occurs, while keeping authoritative organizational information connected to its source rather than creating a separate and potentially conflicting version specifically for AI.

4. Connect Knowledge to Agent Responsibility

As AI moves from assistance toward agentic work, determine whether the knowledge available to the agent is sufficient for the responsibility it is being given. Access to better information can support greater authority, but knowledge alone does not justify autonomy. Authority, validation, consequence, reversibility, and human judgment must still be considered.

• Which decisions can the agent make reliably from the knowledge available to it?
• Which decisions still require human judgment?
• What authority is the agent actually being given?
• What independent validation protects the result?
• What should cause the agent to stop?
• Where should it escalate?
• Does the escalation preserve enough information for a human to continue efficiently?

The objective is to make the relationship between knowledge and authority explicit. An agent should receive responsibility only for decisions it has the information, permission, and validation necessary to make reliably, while decisions requiring judgment or information outside that boundary should return to a human.

5. Build the Learning Loop

Finally, use AI-assisted work itself to identify weaknesses in the engineering system. Repeated corrections, recurring escalations, missing information, and unnecessary human approvals are not simply problems to resolve one at a time; they are evidence showing where the workflow can improve.

• Are repeated human corrections being captured?
• Are recurring escalations being reviewed?
• Can missing knowledge be added to the authoritative engineering system?
• Can repeated manual validation become an automated control?
• Is human approval protecting genuine judgment or compensating for a weakness elsewhere in the workflow?

The objective is to create a learning loop in which each correction or escalation has the potential to improve the next AI-assisted interaction. The organization does not need to document everything it knows. It needs to make the knowledge that changes engineering decisions reliable and available at the moment it is needed, and then use what it learns to improve the knowledge, validation, authority, and human involvement surrounding the workflow.

What to Do Tomorrow Morning

• Introduction → AI engineering framework
• Chapter 1 → AI in software development
• Chapter 2 → AI developer productivity
• Chapter 3 → AI engineering standards
• Chapter 4 → AI agents in software development
• Chapter 5 → AI technical debt
• Chapter 6 → AI engineering context
• Chapter 7 → AI cost governance
• Chapter 8 → AI engineering operating model

If you have questions, please let us know.

Take a few minutes to complete the assessment and gain a clear, practical view of your organization’s AI readiness—and what to do next.