Skip to main content
Michael Onofre
AI & Policy8 min read

Keep building AI. Keep humans accountable.

A response to Dario Amodei on human oversight, AI-written code, and who should be allowed to develop American AI models.

Written by

Michael Onofre

Notes on practical AI systems, evaluation, and human review.

Dario Amodei's September 2026 essay, “We Must Pace the Frontier,” raises a question that deserves a serious answer: can our ability to control AI keep up with our ability to build it? I share his concern about losing control. I am less convinced that slowing AI development should become our starting point.

My perspective comes from AI training and evaluation through Handshake AI and the systems I build in my personal projects. I see real value in developing better tools. I also believe humans need to remain involved throughout the process, with the knowledge and authority to challenge what those tools produce. The more capable AI becomes, the more seriously we need to take that responsibility.

1. Take the warning seriously and keep the benefits in view

Amodei's proposal is more specific than a blanket pause. He argues for slowing capability growth enough for safeguards to catch up, beginning with embedded external evaluators and extending to coordination among democracies and, where feasible, globally. His proposed evaluators would examine training processes as well as finished models. He also discusses capability checkpoints and possible limits on training compute and internal use of AI to improve AI.

I support independent evaluation and meaningful access to the development process. We agree that oversight has to reach inside the lab. My disagreement is about how broadly restrictions should apply and what evidence should trigger them.

I want AI to help researchers investigate disease, make learning more accessible, and remove repetitive work that consumes people's time. These are reasons to keep investing in development, including safety research. Delaying useful capabilities has a cost that belongs in the same discussion as the risks of releasing them.

My preference is to require evidence before a consequential step proceeds: a major training run, an expansion of an agent's permissions, or a public deployment. If the evidence is inadequate, hold that step and address the gap. I would need a stronger case before supporting restrictions that slow development across the field.

2. Give human oversight the resources and authority to work

“Keep humans in the loop” is easy to say. It becomes meaningful when someone has enough expertise to assess the work, enough time to investigate a concern, and enough authority to stop an unsafe decision. A person approving output they cannot evaluate offers very little protection.

I want that responsibility built into every stage. People should decide what a model is being trained to do, assess the data and feedback shaping it, review changes to its software, evaluate its behavior, and approve how it will be deployed. After release, people still need to investigate failures and decide when access should be restricted or a system withdrawn.

NIST's Generative AI Profile supports oversight across the AI lifecycle, independent evaluations proportionate to risk, and review of generated code. These are useful foundations. My concern is whether organizations make them operational as the volume and complexity of AI-assisted work increase.

The standard should be visible in the work itself:

Before development
Name the responsible people, define the intended use, and document what the system must never be allowed to do.
During development
Keep changes reviewable. Preserve the data, code, and evaluation history needed to investigate how a result was produced.
Before consequential action
Require qualified human approval supported by test results, known limitations, and independent scrutiny where the consequences justify it.
After deployment
Monitor actual behavior, investigate incidents, and give a named team the authority and practical ability to restrict or stop the system.

3. Measure review quality, not the percentage of AI-written code

In May 2026, Business Insider reported that Amodei told the previous October's Dreamforce conference that Claude generated 90% of most Anthropic teams' code. He also emphasized that engineers remained necessary to check and supervise the work. That reported figure is not an independent audit or a measurement of how much model-training or safety-critical code went unreviewed.

Anthropic's own December 2025 research offers useful context. In a survey of 132 engineers and researchers, employees reported using Claude in roughly 60% of their work. More than half said they could fully delegate only 0–20%, with differing interpretations of delegation. Frequent AI use and working without human verification are different measures. This was an internal study, with self-reported findings, rather than an independent safety assessment.

I do not think an arbitrary cap on AI-written code solves the problem. A human can write a serious defect, and AI can produce a useful implementation that survives careful testing. The origin of the code does not settle whether it belongs in a system.

I would require qualified human review and approval before AI-generated changes enter consequential model-development or production systems. Reviewers should understand the intended behavior, inspect security-sensitive changes, and verify that tests cover failure cases. AI can help with this work, but its own reassuring explanation cannot substitute for evidence.

Imagine an AI assistant changes an evaluation script and also writes a test that repeats the same mistaken assumption. Both can agree while the system is wrong. A human reviewer needs an independent basis for deciding what should happen. This is an illustrative failure mode, not an allegation about Anthropic.

If a team can generate changes faster than it can verify them, it should reduce the size of changes, add review capacity, or hold the backlog. Producing more code only helps when the team can still stand behind what it accepts.

4. Preserve human control when AI helps build the next model

AI writing a function is different from a system directing experiments, modifying training processes, judging its own results, and deciding what to build next. The latter places much more of the development process inside automation. That is where my demand for human involvement becomes especially firm.

I would keep research objectives, evaluation criteria, access permissions, and approval of major training or deployment changes under accountable human control. An agent should not be able to rewrite the test it is being judged against or expand its own authority without a separately controlled review.

The strongest objection is that people may eventually be unable to understand or evaluate the systems quickly enough. I take that seriously. Human approval cannot guarantee safety, and a signature cannot compensate for missing evidence. We still need technical work on alignment, interpretability, containment, and evaluation.

If that work cannot establish an adequate basis for proceeding, the affected experiment or release should wait. That position overlaps with parts of Amodei's proposal. I am comfortable acknowledging the overlap: supporting continued development does not require approving every next step.

5. Treat access to American AI development as a national security issue

There is another side to human oversight: who has access to the systems being overseen. An organization can carefully review model outputs while leaving the development process exposed to theft or interference.

The threat of AI-related espionage is concrete. In January 2026, a federal jury convicted former Google engineer Linwei Ding of economic espionage and theft of trade secrets involving Google's AI technology for the benefit of the People's Republic of China, according to the Department of Justice. That case establishes a specific instance of theft. It does not establish that foreign workers as a group are disloyal.

Beyond theft, a hostile actor with sufficient access could try to compromise training data, manipulate evaluations, insert malicious code, or disrupt infrastructure. These are possible attack paths, not claims that every one occurred in that case. My concern is the opportunity to use privileged access to undermine U.S. security and technological capability.

My proposal is federal legislation requiring U.S. citizenship for people working across model development at U.S. AI companies, including employees and contractors involved in training, evaluation, and the infrastructure supporting that work. I would apply it across model development rather than limiting it to classified projects. Citizenship would include naturalized citizens; ancestry would be irrelevant.

I favor a citizenship requirement because I regard the development of American AI models as strategically sensitive work that warrants a defined national access boundary. That is a policy judgment. It is not a finding that citizenship predicts an individual's loyalty, and the espionage case alone does not prove that a broad hiring restriction would be effective.

The tradeoff is substantial. This rule would exclude skilled noncitizens, including lawful residents and researchers from allied countries. It could drive expertise overseas and slow the progress I want to preserve. A narrower restriction tied to sensitive access is a serious alternative. I still favor the broader requirement, but its costs and security benefits would need to be evaluated openly, with a workable transition for existing teams.

Citizenship also cannot replace security controls. U.S. citizens can be malicious, compromised, or careless. Every worker should receive only the access needed for the job, sensitive changes should require separate approval, and access to model weights and training infrastructure should be logged and monitored. A citizenship rule without those controls would leave much of the underlying problem intact.