Chapter 1

Chapter 1

Everyone Is Using AI Differently. Now What?

Understanding What Is Actually Happening Before You Try to Manage It

This makes the first stage of an AI engineering transition somewhat counterintuitive. Leadership’s first instinct may be to establish rules, choose approved tools, or create a standard process. Those things will eventually matter, but beginning there can cause the organization to overlook something valuable: its developers may already have discovered some of the best AI use cases available to the company. Before deciding how AI should be used, leadership needs to understand how it is being used now.

That does not mean conducting a traditional software inventory and recording which employees have licenses for which products.

Start With Discovery, Not Enforcement

A more useful set of questions would be: What engineering work are developers using AI to perform? Which uses have become part of their normal workflow? Where do developers believe AI is saving significant effort? Where have they stopped using it because the results were unreliable? Which practices are spreading from one engineer to another? What tasks still require substantial correction after AI is involved? Has anyone built an agent or automated workflow that goes beyond an individual developer asking an assistant for help? These questions reveal behavior rather than access.

The purpose of discovery is not to catch developers doing something wrong. If engineers believe that leadership is conducting an AI audit primarily to restrict usage, they have a strong incentive to describe only the safest and least interesting things they are doing. Leadership may then receive a reassuring report showing that developers use AI for documentation, code explanation, and unit tests while missing the engineer who has developed an innovative workflow that saves hours each week—or the engineer who has quietly given an agent far more authority than anyone realized.

A useful discovery process needs to communicate a different message: we are trying to understand what is working, what is not working, and what the organization can learn from the people already using these tools.

If leadership does not know what is happening, it cannot determine which practices deserve investment, which need better controls, which should become standards, or which should stop.

Inventory the Use Case, Not Just the Product

The same is true of a developer who says AI helps with debugging. That description is too broad to become useful organizational knowledge. Does the developer paste an error message into an assistant and ask for possible causes? Does AI inspect several source files and trace the likely path of the error? Does an agent reproduce the problem, change code, execute tests, and continue until the tests pass? Those are very different engineering uses with very different implications for value, authority, context, and risk.

• Engineering task: What is the developer trying to accomplish?
• AI role: What part of the task is AI performing?
• Tool or model: What technology is currently being used?
• Frequency: Is this occasional experimentation or part of the normal workflow?
• Human involvement: What does the engineer review, decide, approve, or correct?
• Perceived benefit: Why does the developer continue using AI for this task?
• Known problems: Where does the approach fail, require rework, or produce inconsistent results?
• Authority: Is AI recommending work, generating work for review, or taking actions through tools or systems?
• Potential value: Is this practice important enough to evaluate more formally?

Notice that we are not yet asking developers to prove every productivity claim. That comes in Chapter 2. At this stage, perceived benefit is a discovery signal, not proof of value. If ten developers independently tell us that AI dramatically reduces the effort required to understand unfamiliar legacy code, that is worth knowing. It does not yet establish how much engineering time has been saved, but it gives us a strong candidate for investigation.

This separation between discovery and proof is important. Trying to establish rigorous ROI during the initial inventory can make the process unnecessarily heavy and discourage participation. Conversely, treating developer enthusiasm as proof can cause leadership to scale practices before understanding their downstream effects. The first chapter finds the candidates. The second chapter will test the value.

Look for Patterns, Not Just Individual Success Stories

Suppose multiple developers report using AI to understand unfamiliar sections of a legacy application. At first, that may look like a straightforward productivity use case: AI helps developers comprehend old code faster. But if the same pattern appears across several teams, it may reveal something else. Perhaps the organization has inadequate architectural documentation, poorly captured business rules, or too much critical system knowledge residing in the memories of a few experienced employees. AI may be helping developers work around the problem while simultaneously exposing an engineering weakness that deserves attention of its own.

This is one of the reasons AI discovery can become more valuable than a simple adoption survey. AI usage often reveals where engineers are already experiencing friction.

The inventory therefore gives leadership two views at the same time. One view shows where AI may create leverage. The other shows where the engineering organization may already contain friction worth addressing. Both matter.

Find the Outliers

One developer may have discovered that a carefully structured context package allows AI to work effectively with a complex internal framework. Another may have created a repeatable process for generating tests that substantially reduces tedious work while preserving review quality. Someone may be using AI to compare implementation changes against architectural standards. Another developer may have experimented with an agent, discovered that it repeatedly makes the same type of mistake, and abandoned the approach. That failed experiment may be every bit as useful to the organization as the successful one.
Failure is information.
Organizations frequently document successful technology experiments and quietly forget unsuccessful ones. With AI, that can cause multiple teams to repeat the same mistakes. If one group has already learned that a particular type of task requires too much undocumented business context for AI to perform reliably, another team should not have to rediscover the same limitation six months later.

The discovery process should therefore capture not only what developers are using, but also what they tried and stopped using, and why. This becomes the beginning of an organizational learning system.

Separate Experimentation From Established Practice

A simple classification is enough at this stage:

• Exploring: The practice is still being tried and results are uncertain.
• Recurring: The developer or team uses the practice regularly because it appears useful.
• Candidate for evaluation: The practice appears valuable enough to measure more formally.
• Needs attention:The practice raises a question about quality, authority, security, cost, or another organizational concern.

These are not permanent labels, and they are not standards. They simply help leadership understand what deserves attention next. A recurring practice may remain an individual choice for years because there is no organizational reason to standardize it. Another may quickly become a candidate for broader evaluation because dozens of developers perform the same task. A third may need immediate attention because AI has been given authority that leadership did not realize existed. The purpose of classification is to help the organization direct its attention intelligently rather than responding to every AI use with the same level of concern.

This also helps prevent a common mistake: trying to govern every experiment before the organization has learned anything from it. Some experimentation should remain inexpensive and local. If a developer is using an approved tool to explore a low-risk way of improving a routine task, requiring a formal review before every variation would slow learning without providing meaningful protection. Govern the consequence, not the novelty.

As the potential consequence of AI activity increases, the organization needs stronger controls. As the practice becomes more common and more consequential, it deserves greater visibility. The chapters ahead will develop those controls in detail. At this stage, we simply need enough classification to recognize where they may eventually be required.

Build an AI Engineering Learning Loop

The loop begins with experimentation. An engineer discovers a useful approach or a limitation. That experience becomes visible to the team. If the practice appears promising, it is evaluated against representative engineering work. If evidence supports it, the organization decides whether the practice should remain optional, become recommended, or eventually become part of an engineering standard. If the practice fails, the reason is captured so the organization learns from the failure rather than repeating it.

The loop then begins again because the technology, engineering environment, and available evidence continue changing. In its simplest form, the learning loop is:

• Discover: What are engineers trying?
• Share: What appears useful or problematic?
• Evaluate: Does it improve the engineering outcome?
• Decide: Should we keep experimenting, recommend it, standardize it, restrict it, or stop?
• Learn: What should other teams know?
• Revisit: Has new evidence or technology changed the answer?

This loop is intentionally broader than an AI governance process. Governance is part of the system, but governance alone tends to focus on what is permitted. Engineering learning must also ask what is improving.

That distinction will become important throughout this book. The objective is not to build an organization that is exceptionally good at controlling AI while its competitors become exceptionally good at using it. Nor is the objective to encourage AI use so aggressively that engineering discipline becomes secondary. A successful transition has to do both: create room for useful experimentation and create a reliable path for turning evidence into better engineering practice.

The inventory gives us visibility. The learning loop keeps that visibility alive. What we do with what we discover is the next step.

Turning Discovery Into a Working AI Engineering Baseline

The word baseline matters. We are not trying to create a permanent record of AI usage, because such a record would begin becoming outdated almost immediately. We are establishing a starting point against which the organization can make decisions and later recognize change. Six months from now, leadership should be able to look back and determine whether AI use became more consistent, whether promising experiments became proven practices, whether previously unidentified agents appeared, whether particular engineering problems became easier to solve, and whether the organization learned enough to change its standards or operating model.

A useful baseline also prevents leadership from relying too heavily on anecdotes. Without one, the organization’s perception of AI can easily be shaped by whichever experience receives the most attention.

This does not require building an elaborate reporting system. In fact, the more difficult the baseline is to maintain, the less likely it is to remain useful. The objective is to capture enough information to make decisions without turning AI experimentation into another administrative process developers learn to avoid.

What the AI Engineering Usage Baseline Should Capture

For each meaningful use, capture:

• Engineering activity: What work is being performed?
• Current AI use: How is AI participating in the work?
• Users or teams: Where is the practice occurring?
• Tool/model: What AI technology is currently involved?
• Maturity: Is the practice exploring, recurring, a candidate for evaluation, or in need of attention?
• Human role: What decisions, review, correction, or approval remain with the engineer?
• Perceived benefit: What improvement do the users believe they are receiving?
• Known limitations: Where does the approach perform poorly or create additional work?
• Authority level: Is AI advising, generating work for approval, or acting through tools and systems?
• Next action: Ignore for now, continue experimenting, share, evaluate, or investigate.

The last field is particularly important because an inventory without a next action can quickly become a catalog. Leadership does not need a catalog of everything developers have tried. It needs a way to identify where attention is warranted.

Consider an engineering team that regularly uses AI to generate unit tests. The practice appears to save time, several developers use it, and generated tests are reviewed before being committed. The next action might be evaluate. We want to determine whether the apparent time savings remain after review and correction and whether the resulting tests provide comparable coverage and usefulness.

Another team may occasionally use AI to generate SQL queries. Developers report that it is convenient, but the practice is infrequent and the queries are reviewed carefully. There may be no reason to invest additional organizational attention in it. The next action can simply be continue experimenting.

A third group may have created an agent capable of modifying several repositories and opening pull requests. Even if only one developer is using it, the next action may be investigate because the authority and potential consequences are materially different.

The baseline therefore helps leadership direct scarce attention toward the AI uses that matter.

Do Not Confuse Popularity With Value

Developers naturally notice immediate savings. If AI generates a method in thirty seconds that would have taken twenty minutes to write, the benefit is visible. Costs that occur later are harder to associate with the original AI interaction. Another engineer may spend additional time reviewing the implementation. A subtle inconsistency may not become apparent until the next feature change. Generated tests may pass while failing to cover the business condition that actually matters. A dependency introduced for convenience may create an upgrade problem months later. None of these possibilities means the original practice is bad. They mean that the organization needs to distinguish perceived immediate benefit from demonstrated lifecycle benefit.

We will make that distinction rigorously in Chapter 2. At this point, the baseline should simply preserve both pieces of information: developers believe the practice is useful, and the organization has not yet established the full engineering value.

This keeps leadership from making two opposite mistakes. The first is dismissing developer experience because it is anecdotal. Developers closest to the work are often the first people to discover meaningful AI leverage, and their experience should be taken seriously. The second is turning enthusiastic reports into organizational conclusions before the results have been measured. Discovery identifies where to look. Measurement tells us what to do with what we find.

Identify AI “Leverage” Candidates

A good leverage candidate usually combines several characteristics. The work occurs often enough for improvement to matter, developers already perceive friction or wasted effort, the desired engineering outcome can be defined, and the organization has some way to determine whether the AI-assisted result is acceptable. The candidate is also sufficiently representative that learning from it will be useful beyond one unusual task.

Useful candidates might include repetitive implementation work, test creation, legacy-code comprehension, code review assistance, documentation tied to engineering changes, migration work, defect investigation, or another recurring activity identified by the organization’s own developers. The specific use case matters less than the selection discipline. We are looking for places where AI has a reasonable opportunity to improve the engineering system and where evidence can tell us whether it did.

For each candidate, leadership should be able to answer:

• What engineering problem are we trying to improve?
• How is the work performed today?
• Where does AI appear to create leverage?
• Who is already using the approach?
• What would a better outcome look like?
• Can we observe the downstream consequences?
• What would make us decide the approach is not worth expanding?
• What do we need to learn before making it a broader practice?

Notice that “Which model should we use?” is not the first question. Model selection may matter during implementation, but selecting a model before defining the engineering problem reverses the process. We want technology choices to follow the engineering objective.

Identify AI “Attention” Candidates

An attention candidate may involve an agent with unexpectedly broad access, AI-generated code entering a sensitive application without sufficient review, proprietary information being supplied to an unapproved service, repeated quality problems, significant uncontrolled consumption, or a workflow that has quietly become business-critical while remaining dependent on one developer’s personal configuration. Some concerns will belong to security, legal, privacy, compliance, architecture, or other existing organizational functions rather than to a new AI-specific process. The important thing is that discovery makes the practice visible so it can be routed to the right owner.

This is another reason the inventory should not be framed as an enforcement exercise. Leadership needs engineers to surface the interesting edge cases. An engineer who says, “I built something that works extremely well, but I’m not sure we should be giving it this much access,” is providing exactly the kind of information the organization needs. Punishing the disclosure teaches everyone else to keep similar experimentation invisible.

A mature engineering culture should be able to distinguish between discovering a control gap and deliberately ignoring one. The first is part of learning. The second requires correction.

Find the Engineering Weaknesses AI Is Exposing

Suppose an agent cannot reliably change a legacy application because there are few automated tests. The immediate conclusion might be that the application is not ready for agentic development. That is true, but incomplete. The deeper finding is that the organization has an application whose behavior cannot be validated easily. AI did not create that weakness. It exposed it.

The same can happen with architecture. If AI-generated changes repeatedly violate architectural expectations because those expectations are known informally but poorly documented, the immediate problem appears to be AI quality. The underlying problem is that important architectural knowledge is not available at the point where engineering decisions are being made. Human developers may have been compensating for that weakness through experience and conversation for years.
Business rules provide another example. AI may repeatedly produce technically sensible changes that are wrong because the business logic depends on conditions that are not obvious in the source code. An experienced employee knows those conditions because someone explained them years ago. A new employee may struggle with them as well. Again, AI has exposed a knowledge-management problem that already existed.

As you review the baseline, ask a question that will become increasingly important throughout this book: What does our difficulty using AI reveal about the engineering system AI is entering? Some possible findings may include:

• Inadequate automated testing.
• Poorly documented architecture.
• Business rules that exist primarily as institutional knowledge.
• Inconsistent engineering standards.
• Weak or outdated documentation.
• Excessive application complexity.
• Unclear ownership.
• Fragile deployment processes.
• Limited observability.
• Repetitive work that should already have been automated.
• Engineering practices that vary substantially between teams.

These findings should not automatically become prerequisites for AI adoption. Waiting until every engineering weakness has been corrected would ensure that AI transformation never begins. Instead, they become part of the organization’s readiness picture. Some weaknesses can be accepted for a particular use case. Others require additional human review. Some may prevent an agent from receiving greater authority. A few may be important enough that fixing the engineering foundation creates more value than the proposed AI initiative itself.

This is a recurring theme in AI engineering transformation: AI does not only automate the engineering system. It reveals it. That can be uncomfortable, but it is enormously useful.

Create a Lightweight Way to Share What Is Being Learned

A team might maintain a shared AI engineering channel, hold a short recurring engineering exchange, add useful patterns to internal documentation, maintain examples in a repository, or include AI findings in an existing architecture or engineering community of practice. The mechanism matters less than whether useful knowledge can travel.

What should be shared is not simply a collection of clever prompts. Prompts can be useful, but the more durable knowledge is the relationship between the AI technique and the engineering outcome. A useful contribution sounds less like, “Here is a great prompt,” and more like, “We used this approach to create tests for this type of service. It reduced the initial creation effort, but we found that developers still had to add these two categories of cases manually. Here is where it appears useful, and here is where it does not.” That is organizational learning.

It also prevents AI knowledge from becoming centered around personalities. Most organizations will quickly develop a few employees who are known as the people who “really understand AI.” Those individuals can be tremendously valuable, but the objective should not be to make the rest of the organization dependent on them. Their discoveries should help improve the engineering system.

Establish Ownership Without Creating an AI Bureaucracy

Someone should know who maintains visibility into meaningful AI engineering practices, who receives leverage candidates, who routes attention candidates to the appropriate organizational owner, and how decisions and lessons are communicated back to engineering teams. Without that ownership, the discovery process may produce an excellent initial report and then slowly disappear while AI usage continues changing underneath it.

Central ownership should also remain appropriately limited. The people closest to an application usually understand its engineering realities better than a centralized AI function does. The purpose of central coordination is not to make every implementation decision. It is to maintain the shared framework, surface cross-organizational learning, coordinate decisions that genuinely need consistency, and ensure that successful or problematic patterns do not remain invisible.

This distinction between shared organizational responsibility and local engineering judgment will become increasingly important as we build the operating model in later chapters.

The Chapter 1 Working Checklist

Before collecting information, define what part of the engineering organization is included. A smaller organization may reasonably begin with the entire development group. A large enterprise may start with several representative teams and expand after refining the process.

• Define the engineering teams included in the initial baseline.
• Identify the person responsible for coordinating discovery.
• Confirm existing organizational requirements for approved AI tools, data handling, security, privacy, and intellectual property.
• Explain to participating engineers that the objective is to understand useful practices, limitations, and emerging risks—not simply to measure tool adoption.
• Establish a reasonable completion date so discovery does not become an endless exercise.

2. Build the AI Engineering Usage Inventory

Capture meaningful recurring uses and significant experiments rather than every individual interaction. For each use, record:

• Engineering task or activity.
• How AI participates in the task.
• Tool or model currently used.
• Team or role using it.
• Frequency of use.
• Human review, approval, or correction involved.
• Perceived benefit.
• Known limitations or recurring problems.
• Level of AI authority.
• Current maturity: exploring, recurring, candidate for evaluation, or needs attention.
• Recommended next action.

3. Look Across the Inventory for Patterns

Once individual uses are visible, analyze the collection rather than treating each entry separately.

• Identify practices appearing across multiple developers or teams.
• Identify several different AI approaches being used for the same engineering task.
• Find high-frequency uses that may justify formal evaluation.
• Find unusual practices producing unexpectedly strong results.
• Capture experiments that failed and the reason they were abandoned.
• Identify AI uses that reveal recurring engineering friction.

4. Select AI Leverage Candidates

Choose a small number of promising uses for deeper evaluation rather than attempting to measure everything at once.

• Define the engineering problem each candidate is intended to improve.
• Record how the work is performed today.
• Identify where AI appears to remove meaningful effort or improve the outcome.
• Confirm that representative work is available for evaluation.
• Determine whether downstream consequences can be observed.
• Identify what must be learned before the practice should expand.

5. Identify AI Attention Candidates

Surface practices that deserve review because of their potential consequences, even if they are not candidates for broader adoption.

• Agent or tool authority that may exceed current expectations.
• Sensitive or proprietary information handling.
• AI-generated work entering important systems with inadequate validation.
• Recurring quality or maintenance concerns.
• Unexpected or poorly understood AI consumption.
• Critical workflows dependent on an individual’s undocumented AI configuration.
• Practices that may conflict with existing security, legal, privacy, contractual, architectural, or engineering requirements.

6. Record Engineering Readiness Findings

Use the discovery process to identify weaknesses in the engineering environment that may limit future AI leverage.

• Automated testing gaps.
• Missing or unclear engineering standards.
• Undocumented architectural decisions.
• Business knowledge dependent on individual employees.
• Inadequate or outdated documentation.
• Weak observability or validation.
• Repetitive processes that may be better addressed through conventional automation.
• Application complexity or ownership problems that make AI-assisted work difficult to evaluate.

7. Establish the AI Engineering Learning Loop

Define how useful discoveries move from individual experimentation into organizational knowledge.

• Where will useful practices and lessons be shared?
• Who determines whether a recurring practice deserves formal evaluation?
• Where will unsuccessful experiments and their lessons be captured?
• How will teams learn when guidance or standards change?
• How often will the organization revisit its AI usage baseline?
• Who owns the learning loop?

And finally…

What You Should Have When Chapter 1 Is Complete

You should know where AI is already participating in software engineering, which practices appear to be producing meaningful value, which deserve closer examination, where AI use may require attention, and which weaknesses in the existing engineering environment could limit what becomes possible next. You should also have a lightweight mechanism for keeping that knowledge alive as developers continue experimenting.

The principal artifacts from Chapter 1 are:

• AI Engineering Usage Baseline
• AI Engineering Usage Inventory
• AI Leverage Candidate List
• AI Attention Candidate List
• Engineering Readiness Findings
• AI Engineering Learning Loop

Do not rush to standardize everything on those lists. Do not assume the most popular practice is the most valuable. Do not assume the most advanced AI use is the one most deserving of investment. Chapter 1 has deliberately focused on making the work visible before making those decisions.

The next question is where the transition becomes much more demanding. Developers will tell us AI is saving them time. In many cases, they will be right. But how much time is the engineering organization actually saving? And perhaps more importantly, where did the work go?

• Introduction → AI engineering framework
• Chapter 1 → AI in software development
• Chapter 2 → AI developer productivity
• Chapter 3 → AI engineering standards
• Chapter 4 → AI agents in software development
• Chapter 5 → AI technical debt
• Chapter 6 → AI engineering context
• Chapter 7 → AI cost governance
• Chapter 8 → AI engineering operating model

If you have questions, please let us know.

Take a few minutes to complete the assessment and gain a clear, practical view of your organization’s AI readiness—and what to do next.