No one understands their codebase anymore. To be precise, no one has understood an entire large codebase for decades. Once a system grows past a few hundred thousand lines, knowledge of it is distributed across a team, with each person holding part of the picture. Organisations accepted this because every part had an owner. One engineer wrote the billing logic and another reviewed it. When it failed, there was a named person who could explain what the code did and why it was built that way.
That assurance is what has changed. Over the past year, most organisations that adopted AI coding agents stopped requiring that a human understand each change before it reaches production. Few made this decision formally. It happened incrementally, one approved pull request at a time, as the volume of code increased and the capacity to read it did not.
Our position is that this trade is often acceptable. Our concern is that most small and mid-sized businesses made it without recognizing it as a decision, and therefore without putting in place the controls that make it safe.
This article is written for owners, founders and technology leads at small and mid-sized businesses: product companies with ten to fifty staff, agencies building software for clients, founders whose AI-built products now have paying customers, and smaller firms in regulated sectors such as wealth management and fintech. These organizations rarely have a platform engineering team or a dedicated compliance function. They still have to decide how code reaches production, and at present much of that decision is being made by default.
What changed: the requirement for understanding
For most of the history of software development, writing code was the constraint. That constraint had a useful side effect. Code accumulated at roughly the pace at which people could reason about it, so the author of a change understood it, and a reviewer read it line by line. Code review was the most economical way to catch defects and to spread knowledge across a team. No individual understood the whole system, but every change had been examined by at least two people.
AI agents have removed that constraint. Anthropic, the developer of Claude, offers the clearest public example. In May 2026 it merged an average of about 2,500 code changes a day; by September it was merging more than 30,000 a day, a volume that led the company to replace git with its own version-control system [1]. Anthropic reports that more than 80% of the code it merges is now written by Claude, and that its engineers still approve every change but review the intent of the change rather than every line [1][2]. Google's chief executive, Sundar Pichai, said in April 2026 that about 75% of Google's new code is generated by AI and then approved by engineers [3].
The scale of these figures is not relevant to most of our readers. The pattern is. A two-person development team running several agents in parallel can produce more pull requests in a day than both engineers can review thoroughly in a week. Faros AI's analysis of telemetry from more than 10,000 developers found that on teams with high AI adoption, pull requests were 154% larger on average and review time rose 91% [4]. When output grows that quickly, the depth of review is what gives way.
The meaning of approval changes as a result. Approval once meant that a reviewer had read the change and would vouch for it. Increasingly, it means that the summary is plausible, the automated checks have passed and the reviewer has a queue of further changes to clear. No one lowered the standard deliberately. The volume lowered it.
In many organisations, the practical position is now this: a change is understood once, at the moment it is written, by an agent that retains no memory of it. A person then reviews a summary and approves it. Knowledge that used to reside with named individuals no longer has a defined home.
This need not be a cause for alarm. The organisations furthest along this path did not simply abandon understanding. They replaced it with a different set of controls. Whether that substitution works for a given business depends on whether it was designed or merely inherited.
What replaced understanding
The heaviest adopters of AI coding agents rely on a consistent set of controls. None of them requires a person to understand an entire change. Each addresses a different category of failure, and together they perform most of the function that line-by-line human review used to perform.
Intent review
At Anthropic, the engineer approving a change primarily assesses what the change is intended to do and whether that intent is correct. Approval tooling directs reviewers' attention to sensitive changes [1]. This can look like a lower standard, but it concentrates human judgement where it is most valuable. Reviewers are poor at detecting an off-by-one error deep inside a long diff. They are effective at recognising that a change described as an invoicing fix also modifies refund handling, and at asking why.
For a smaller organisation, intent review can be implemented as a simple rule. Every agent-generated pull request states, in plain language, what it changes and why, and a named reviewer confirms that statement against a summary of the diff. Where the description and the scope diverge, the reviewer escalates. The control takes minutes per change and targets the class of error with the highest business impact.
Automated review
Anthropic runs an automated Claude reviewer on every change, using multiple reviewing agents whose findings are checked against one another [2]. Since it released this capability as a product in March 2026, Anthropic reports that the share of its internal pull requests receiving substantive review comments rose from 16% to 54% [5]. The company is also explicit about the limits. A retrospective analysis found that the automated reviewer would have caught about a third of the bugs behind past incidents on claude.ai [2], and the product does not approve pull requests; that decision remains with a human [5].
Catching a third of the defects behind real incidents is a meaningful improvement. It is not sufficient for automated review to serve as the final control. We advise clients to treat an AI reviewer as a first pass that removes routine issues and allows the human reviewer to concentrate on intent and risk. It should never be the approver of record.
Task scoping and continuous integration
Stripe's internal coding agents produce more than 1,300 pull requests a week that contain no human-written code, and every one is reviewed by an engineer before merge [6]. The model is workable because the agents are restricted to well-defined categories of work, such as configuration changes, dependency upgrades and small refactors, and because each workflow combines fixed, deterministic steps with continuous integration, automated tests and static analysis [6]. Stripe did not give its agents unrestricted access to the codebase. It specified which work agents are permitted to do.
The equivalent for a smaller organisation is a written policy that distinguishes tasks agents may complete with light review (dependency updates, copy changes, tests for existing functionality) from tasks that require closer scrutiny (anything involving payments, authentication, customer data or deletion). Such a policy rarely needs to exceed a page. In our experience, most small organisations do not have one.
Permission boundaries
Permission boundaries are the least visible control and, in our assessment, the most important. Goldman Sachs had allowed traders and bankers to use AI coding tools for several months. In October 2026 it updated its guidance to restrict Anthropic's Claude Code and Microsoft's GitHub coding tools to roughly 12,000 engineers and its quantitative "strats" staff, citing "productivity and risk management" [7]. Citi took a different approach when it extended Cognition's Devin agent to about 40,000 developers: agents work only from internal repositories, and humans approve all output [8]. Deutsche Bank's stated principle is that developers must never deploy AI-generated code without review [9]. These are two governance models addressing the same risk. One controls who may use the tools. The other controls what the tools may ship.
The most widely reported incidents point to the same conclusion. In July 2025 an AI agent on Replit's platform deleted a customer's production database during a declared code freeze; Replit responded by automatically separating development and production databases [10]. The same month, a release of Amazon's Q extension for Visual Studio Code was distributed for about two days containing an injected instruction directing the agent to delete local files and cloud resources, with confirmation prompts disabled. AWS stated that no customer resources were affected [11].
For organisations of any size, the baseline is clear. AI agents should not hold production credentials. They should operate in branches and staging environments, and a deliberate step, either a named person or a gated deployment pipeline, should promote code to production. If an organisation implements only one recommendation from this article, it should be this one.
Taken together, these four controls form a workable system, and they are the controls the most aggressive adopters actually operate. None depends on any individual understanding the whole codebase. Each depends on a prior decision about what that layer is responsible for. That prior decision is the step most smaller organisations have not taken.
Risk is proportional to what the software does
It would be easy to present AI-generated code as an unmanaged hazard. That framing would not help our readers, because the consequences of failure vary enormously from one business to another.
When a consumer scheduling app fails, the cost is typically lost time and, at worst, a lost customer. When a wealth manager's reporting pipeline places an incorrect figure in a client statement, the client may act on it, and a regulator may later ask how the error occurred. The tools, the agents and even the prompts may be identical. The consequences differ by orders of magnitude, and the controls should differ accordingly.
The more useful question is therefore not whether AI-generated code is safe, but what a defective change would cost the business and how quickly it would be detected.
Three gaps in the current approach are well documented.
Review has become the bottleneck. Google's DORA research programme surveyed nearly 5,000 technology professionals for its 2025 report. It found that AI adoption is now associated with higher delivery throughput and, at the same time, with greater delivery instability, and it concluded that AI amplifies the existing strengths and weaknesses of an engineering organisation [12]. Anthropic states in its own published analysis that human code review has become a new bottleneck [2].
Delegation erodes the skills that oversight requires. In a study of its own engineers, Anthropic describes a "paradox of supervision": supervising Claude effectively requires the very skills that delegating work to Claude can erode [13]. In a separate randomised controlled trial published in January 2026, 52 mostly junior developers learned an unfamiliar programming library with or without AI assistance. Those using AI scored 50% on a subsequent comprehension test, against 67% for those working without it, with the largest gap in debugging, and they were not significantly faster [14]. An organisation whose control model depends on human review must therefore actively maintain its reviewers' capability.
The gap is not measured. In our research we found no organisation, of any size, that publishes defect or revert rates for AI-generated code. Unless a business tracks this itself, it has no reliable view of how its agent-written changes perform.
The public incidents share a common feature. The Replit database deletion [10], the compromised Amazon Q extension [11] and the AWS service disruption reported by the Financial Times in February 2026, which Amazon attributes to a user permissions error rather than to its AI tooling [15], each involved an agent with the ability to take a destructive action. None involved a subtle defect that no one could explain. To date, the most visible failures have been failures of permissions. The less visible failure, the gradual accumulation of code that no one can account for, tends to emerge later, costs more to remediate and rarely attracts coverage.
Organisations in regulated sectors should note that supervisors are now addressing this directly. A Bank of England summary of industry roundtables in February 2026 recorded that AI agents challenge the principle of a human in the loop and that traditional model validation is "unsustainable" at current scale [16]. FINRA's 2026 Annual Regulatory Oversight Report identifies the risks of agents acting beyond a user's authority, of agent actions that are difficult to audit and of sensitive data exposure, and sets an expectation of logging, human review and limits on agent access [17]. A regulated firm should be able to explain how any change reached production, and "an agent wrote it and it was approved" will not be an adequate explanation.
We recommend that organisations classify their exposure using three broad tiers.
Lower risk: internal tools, prototypes, marketing sites and early-stage products where a defect costs time rather than revenue. Permission boundaries and basic continuous integration are generally sufficient, and agents can be allowed to operate quickly.
Moderate risk: customer-facing products where defects affect revenue or trust. Add intent review on every pull request, automated review as a first pass, a written policy on which tasks agents may complete independently, and tests covering revenue-critical paths.
Higher risk: systems that handle money, client data or regulated output, or that clients rely on to make decisions. Apply all of the above, together with human-owned decision records, tracking of defects and reverts in AI-generated changes, and an audit trail of approvals.
In our experience, most small businesses fall into the moderate tier and operate as if they were in the lower one.
The strongest objections
There are credible arguments against the position we have set out. We address the five most substantial.
"No one has understood large codebases for decades." This is correct, and we acknowledged it at the outset. Understanding of any substantial system has always been distributed. The relevant difference is ownership. Every part of a traditional system had someone accountable for it who could explain its behaviour. AI-generated code frequently has no such person: the agent that wrote it retains no memory of the work, and the human who approved it reviewed a summary.
"Treat agent output as you would another team's service." Simon Willison, one of the more rigorous independent commentators on AI-assisted development, has written that he no longer reviews every line his agents produce. He compares agent output to an internal service run by another team: he evaluates it by its behaviour and examines the code when problems arise [18]. We largely agree. Organisations do not read the source code of every library they adopt. However, an established library comes with a test suite, a release history, a user base that reports defects and a stable interface. A recent agent-generated change has none of these unless the organisation has built them. Willison is equally clear that engineers should deliver only code they have demonstrated to work, with evidence [19]. The two positions belong together.
"Scoped agent work is manageable." Stripe demonstrates that it is, at considerable volume [6]. This supports our argument rather than undermining it. Stripe defined the scope deliberately. The objection applies only to organisations that exercise similar discipline, and many small teams currently assign agents broad repository access and loosely specified tasks.
"AI-generated code is now as good as human-written code." Anthropic reports that its staff consider Claude's code roughly at parity with human-written code [2]. That may well be the case, and quality continues to improve. However, a large number of individually sound changes can still produce a system that no one designed: duplicated implementations, inconsistent patterns and structure that has accumulated rather than been planned. GitClear's analysis of 211 million changed lines of code found an approximately eightfold increase in duplicated code blocks during 2024, alongside a decline in refactoring [20]. GitClear sells code analysis software, so the figures should be read as directional. Quality at the level of an individual change does not guarantee coherence at the level of the system.
"Comprehensive understanding is unnecessary; monitor production instead." For lower-risk products, this is a reasonable position. Effective monitoring and rapid rollback can outperform close reading for many types of software. The approach is inadequate where failures are silent (an incorrect figure that appears plausible), delayed (a client discovers the error months later) or irreversible (deleted data, sent communications, executed payments). Production monitoring detects only what production makes visible.
Each of these objections holds where an organisation has made a deliberate choice. Willison relies on testing and professional judgement. Stripe defined its scope. Production monitoring works for lower-risk systems with mature observability. Together they make a strong case against overreaction, which we accept. None of them supports leaving these decisions to default.
Four decisions to make explicitly
We recommend that every organisation using AI coding agents make four decisions explicitly and record them. An initial version of all four can be produced in a single working session, which is more than most organisations currently have in place.
1. Permission boundaries
Define who may direct agents, what they may access and what they may never do. At a minimum, agents should not hold production credentials; each agent should be limited to named repositories and environments; the organisation should decide which staff may operate agents at all, the question Goldman Sachs resolved for itself [7]; and destructive actions, including data deletion, database migrations, customer communications and payments, should always require human approval.
These rules should be documented where both staff and agents can read them. This is the least costly of the four decisions, and it addresses every publicly reported incident we have reviewed.
2. Review design
Define what each layer of review is responsible for, so that no category of risk falls between them. A typical allocation for a smaller team is as follows: continuous integration detects build failures and failing tests; automated review identifies routine defects, security issues and style problems; and a named human reviewer assesses intent, confirming that the change does what it claims and touches nothing it should not. In addition, the organisation should maintain a short list of sensitive areas, typically payments, permissions and any code that deletes data, where the reviewer reads the full diff.
Pull requests should be kept small. DORA's 2025 findings indicate that working in small batches matters more, not less, when AI is involved [12], because large changes are where review most often degrades into skimming.
3. The knowledge layer
The third decision concerns where the reasoning behind the system is recorded. Organisations commonly rely on one of three approaches, none of which is sufficient on its own.
The first is self-documenting code. Clear naming and structure explain what code does, but rarely why it exists. An agent working from a partial view of the system may interpret a defensive check against a supplier's malformed data as redundant, and a comment warning against its removal as obsolete. It removes both, and the tests continue to pass because they never covered the supplier's behaviour.
The second is documentation in markdown files that agents keep up to date. In one Telos Brothers project, agents maintained a set of design documents alongside the code for several months, and did so accurately. One agent then encountered a section describing behaviour it could not locate in the files available to it, concluded the section was out of date and deleted it. Changes to documentation were not being reviewed, so the deletion went unnoticed. The behaviour was implemented in a module the agent had not loaded, and the gap was identified only when a subsequent failure sent the team to documentation that no longer contained the answer. The documents appeared current and well maintained while being materially incomplete. Independent research supports the broader concern: a February 2026 study found that AGENTS.md-style context files generally did not improve agents' task success rates and increased inference costs by more than 20%, with repository overviews providing the least benefit [21].
The third is a code knowledge graph. Tools such as Graphify, with which Telos Brothers has no affiliation, parse a repository into a structured map of files, functions and the relationships between them, which agents can query instead of searching files individually [22]. Because the map is generated from the code itself, it is rebuilt rather than maintained, and an agent cannot remove a relationship that the code still contains. This approach is worth evaluating for large codebases, systems spanning several services, and legacy estates where navigation is the primary constraint. For smaller codebases it is usually unnecessary. Vendors' efficiency claims have not been independently verified, and a graph records what is connected, not why it was designed that way.
Our recommendation is a small set of decision records: short entries, a paragraph each, written or approved by a named person. Examples include "Balances are rounded at account level because the custodian reports at that level" and "This retry logic exists because the payment provider times out under month-end load." Only decisions that would surprise a new engineer need to be recorded, which typically means dozens rather than hundreds. Agents should read these records before starting work and may propose additions, but they should not be able to edit or delete an existing record without approval from a named owner. The records should be stored in a protected location where any change requires human sign-off. Where a code graph is justified, it should provide the map of the system and the decision records should provide the rationale.
4. Protected behaviour
Identify the behaviours that must never fail. For most small businesses this list contains between five and twenty items: invoices total correctly; one customer can never access another customer's data; client reports reconcile; account deletion requires confirmation. Each should be protected by a test that a person has written or reviewed in detail, and that agents cannot modify without approval.
A test is the only form of documentation that enforces itself. It fails visibly when the protected behaviour is removed, regardless of who removed it or why. Meta has taken this further with a system that introduces simulated faults and generates tests designed to detect them; its engineers accepted 73% of the tests the system proposed [23]. Smaller organisations do not need that level of sophistication. A defined list of protected tests, with clear rules on who may change them, achieves most of the benefit.
Finally, measure the gap. Track how often AI-generated changes are reverted and how many defects are traced to them. A simple log is sufficient. Within a few months, the organisation will have performance data on its own AI-generated code that even the largest technology companies do not publish.
Accountability is the control that matters
No one understands their codebase anymore, and for most small and mid-sized businesses, no one needs to. For as long as codebases have been large, the idea that someone understood the whole system was a reassurance rather than a functioning control.
What an organisation does need is a clear answer to two questions for every change that reaches production. Who is accountable for it? And what would have stopped it if it had been wrong? For code written by the team two years ago, most organisations can answer both. For code merged by agents last month, most cannot.
Closing that gap is the work, and it is more contained than it may appear. It does not require a new platform or a broad AI strategy. It requires a short, written set of decisions: what agents may access, what each review layer is responsible for, where the reasoning behind the system is recorded, and which behaviours are protected. A two-person team can produce a first version in a day; a twenty-person team, within a week. Organisations that can address only one area should start with permission boundaries, which are the least costly to implement and cover every publicly reported incident to date.
The organisations merging tens of thousands of changes a day did not keep pace by understanding more. They defined in advance what each layer of control was responsible for, and allowed agents to operate within those limits. Smaller organisations can apply the same discipline with far less code and far less overhead.
The assessment below takes approximately five minutes and identifies which of these four decisions your organisation has made deliberately, which it has made by default and which remain open.
Where the assessment identifies gaps, Telos Brothers works with small and mid-sized organisations to define permission boundaries, design review layers, and establish decision records and protected tests, and then implements those controls within the client's existing codebase and workflow.
Sources
Anthropic, "Redesigning source control," Anthropic Engineering, October 2026. anthropic.com
Anthropic Institute, "When AI builds itself," 2026 (updated 18 September 2026). anthropic.com
The Decoder, "Google says 75 percent of its new code is now written by AI," April 2026. the-decoder.com
Faros AI, "The AI Productivity Paradox Report 2025," July 2025. faros.ai
Claude, "Code Review," Claude blog, March 2026. claude.com
InfoQ, "Stripe's autonomous coding agents," March 2026. infoq.com
The Decoder, "Goldman Sachs pulls AI coding tools from traders and bankers," November 2026, summarising Bloomberg reporting of 2 November 2026. the-decoder.com
American Banker, "Citi is rolling out agentic AI to its 40,000 developers," 2025. americanbanker.com
Deutsche Bank, "How AI is reshaping the software behind everyday banking," July 2026. db.com
The Register, "Replit responds to SaaStr database deletion," July 2025. theregister.com
The Register, "Amazon Q extension shipped with a destructive prompt," July 2025. theregister.com
Google DORA, "State of AI-assisted Software Development 2025." services.google.com
Anthropic, "How AI is transforming work at Anthropic," December 2025. anthropic.com
Anthropic, "How AI assistance affects coding skill formation," January 2026. anthropic.com
GeekWire, "Amazon pushes back on Financial Times report blaming AI coding tools for AWS outages," February 2026. geekwire.com
Bank of England, "Summary of AI roundtables," February 2026. bankofengland.co.uk
Debevoise & Plimpton, "FINRA's 2026 Regulatory Oversight Report," December 2025. debevoise.com
Simon Willison, "Vibe coding and agentic engineering," May 2026. simonwillison.net
Simon Willison, "Code proven to work," December 2025. simonwillison.net
DevClass, "AI is eroding code quality, states new in-depth report" (GitClear), February 2025. devclass.com
arXiv preprint 2602.11988, evaluation of repository context files for coding agents, February 2026. arxiv.org
Graphify, "Introducing Graphify," July 2026. graphify.com
Meta, "Mutation-Guided LLM-based Test Generation at Meta," arXiv 2501.12862, January 2025. arxiv.org
