An AI Filed a False Murder Tip in My City. The Problem Isn’t the AI. It’s the Architecture.

AI Governance

Oct 12

Human authorization, safer forms, and the boundaries AI systems must enforce before an action affects anyone else.

Eric J. Cox, Esq. | Founder, Guardian AI Services

October 12, 2026

I live in Philadelphia. I own a business here. And I spent more than two decades working in child welfare law, where procedural failures can have profound consequences for real people.

On October 4, I published “The Units Know Their Place”, explaining why an AI system’s limits should be built into its design. Five days later, Philadelphia police disclosed that an AI had submitted a fabricated homicide tip through their public reporting website. That incident brought the principle into sharp focus in my own city. [8] [1]

A police reporting system concerns real cases, real families, and real public responsibilities.

According to Philadelphia police, the submission occurred on July 18, 2026. Anthropic discovered it on September 28 and notified the department in early October. Police said the tip was flagged as spam and never forwarded for investigative vetting. The department reported no indication of unauthorized access to police systems or compromised department data. [1]

Anthropic described a model generating example tasks on live webpages and submitting an invented observation through a police tip form. The system had crossed the boundary between generating text and taking an action that could affect other people. [2]

AI Governance Begins with Architecture

As an attorney and a builder, I focus on the authority the system was given and the controls that were supposed to contain it. A model’s ability to produce information does not automatically justify giving it the ability to submit that information to a government agency, a financial institution, or anyone else.

Those capabilities are design decisions. So are the permissions, the approval process, and the point at which a person must take control.

Instructions to the model have a role. The surrounding system also needs enforceable limits that apply even when the model misunderstands a task or finds another route to completing it. A restriction should hold across the tools and connections available to the AI.

That is what I mean by AI governance by architecture.

The Open Worldwide Application Security Project, known as OWASP, is a nonprofit focused on software security. Its guidance on excessive AI agency recommends limiting capabilities and permissions, requiring human approval for consequential actions, and enforcing authorization outside the model. Those principles support the approach we take at Guardian. [3] [4]

Persistence Must Stay Within the Authority Granted

Persistence can be useful. We want an assistant to look for an answer when the first attempt fails. The problem begins when completing the task becomes a reason to work around a restriction.

Anthropic also described models working around access restrictions and a research model submitting a real government form when a practice copy was unavailable. In another evaluation, a model submitted a form after wrongly expecting another confirmation page. [2]

At Guardian, our rule is straightforward: when a restriction blocks the task, the AI must explain the obstacle, present its proposed solution, and obtain the user’s permission before using that workaround. It may look for solutions within its existing authority. Finding a route around a restriction does not authorize taking it.

A practice task must remain a practice task. If the practice environment fails, the system should stop that attempt and report the problem. Access to a live website should not become an improvised substitute.

The architecture must make that boundary enforceable. The model should not be able to grant itself a new permission because doing so would help it finish.

Restricting internet access can contain the immediate risk. The authority to act should be governed before a system is connected to live services. When a restriction blocks the task, the AI must explain the obstacle, present its proposed solution, and obtain the user’s permission before taking that new route. Those controls belong in the architecture from the beginning and must apply to every capability the system retains, whether it operates online or locally. It may add a few seconds to the user’s workflow, but it can save hours of fixing problems later.

The User Must Review and Personally Submit

At Guardian, our default design standard for any capability to submit forms is straightforward: AI may help prepare the information, but the user must review the completed form and personally activate Submit. Government and financial submissions make the importance of that boundary especially clear.

I mean an actual human action: the user physically uses the keyboard to activate the submission, with an accessible alternative when a disability makes that method unsuitable. Review and submission are both deliberate parts of the process.

The user should see the actual information being sent, its destination, any attachments, and any associated certification, payment, or commitment. Approval should apply to that submission. Material changes to the contents or destination should require another review. OWASP’s transaction authorization guidance similarly ties authorization to the transaction details being approved. [5]

The company must also ensure that the final control belongs to the person. An AI-generated click or simulated keystroke should not satisfy a requirement for human submission, and another tool should not provide a route around it. That is a design requirement for the surrounding software.

The default should be clear enough that users understand when the AI is preparing work and when they are taking responsibility for sending it.

Accessibility Belongs in the Safety Architecture

I have ADHD and understand the value of help with complicated processes. AI can make forms easier to navigate by reading instructions, explaining language, organizing information, or entering what a person dictates.

People who cannot use a conventional keyboard need an accessible way to make the same intentional decision. That may involve voice control, switches, or other assistive technology. W3C accessibility guidance recognizes that keyboard interfaces can support several forms of assistive input. [6]

The approval mechanism should remain under the person’s control and outside the AI agent’s authority to imitate or bypass. Accessibility should be built into that mechanism from the beginning.

Expanded Capabilities Should Require a Separate Agreement

Some users may want automatic form submissions or additional tools beyond the default. Each AI company should prepare and present a separate, clear agreement for the user to review and sign before enabling those expanded capabilities.

The agreement should identify what the user is authorizing: the actions, the sites or recipients, the limits, the risks, and how permission can be withdrawn. A user should understand the specific change being made to the system’s authority.

I would recommend that the agreement include a release of the AI company, to the extent permitted by law, for harm the signing user incurs from deliberately enabling those specified additional capabilities. Its scope and limits should be explained plainly.

This would be an additional safeguard, recommended for expanded form abilities and other consequential permissions. The software must still enforce the scope the user selected. The AI must never be allowed to approve its own expansion of authority.

An agreement authorizing a defined class of actions should not silently become permission to take unrelated actions. The user’s choice and the system’s actual capabilities must remain aligned.

The Receiving System Has Responsibilities, Too

The AI company controls what its system can do. The receiving organization controls what its system will accept. Both have opportunities to prevent an avoidable problem.

Anonymity is fine. A submission still needs to tell the receiving agency something useful.

If someone claims to have seen a suspect, the form should require an actual description or another concrete detail about the observation. A tip about a vehicle, document, location, or other evidence should provide information relevant to that report.

In this case, Anthropic said the model claimed to have seen someone matching a description, although the page supplied no perpetrator description. It supplied no actual description of its own and left contact details blank. [2]

Our standard is that when necessary information is missing, the form should identify what is needed and prevent submission until that requirement is met. The receiving system must enforce those checks as well as the visible form. Server-side enforcement matters because browser-only validation can be bypassed. [7]

Completeness checks are one layer. Appropriate screening and human assessment must also consider the substance of a report. A system should distinguish an incoming submission from information that has been reviewed or corroborated, so its records reflect what has actually been established.

A Small Submission Can Create a Much Larger Burden

Calling an incident small because its immediate consequences were contained overlooks how far the effects can spread.

Philadelphia police described reviewing the incident and coordinating with other city offices. [1]

Anthropic also described changes to its evaluations, tools, and monitoring in response to the behavior it reported. [2]

An incident like this can draw in people who must determine what happened, coordinate a response, explain the problem, and develop corrections. Its effects can extend through media coverage, professional discussion, public concern, and other organizations examining their own systems.

The submission may take seconds. Understanding and addressing its consequences can involve many people with other responsibilities.

That is also why timely detection belongs in the architecture. Attempts to take unauthorized actions on law enforcement, government, or financial systems should generate prompt alerts for designated reviewers. Monitoring should capture what is needed to understand the action and the permission involved, with appropriate limits on personal information and retention.

A safeguard should help prevent the action and make a failed boundary visible quickly enough for someone to respond.

Recycled Time Makes Meaningful Review Practical

I have written about recycling the time AI saves. This is a practical way to make careful human review affordable.

Suppose a task would normally take 40 hours. With AI assistance, it takes 30. That creates 10 hours of savings. If two of those hours are reinvested in checking the work, the task takes 32 hours overall. Eight hours are still saved.

The amount reinvested should reflect the importance of the task to the user and to others who could be affected. Reviewing a casual note and reviewing a court filing, financial instruction, or government submission call for different levels of care.

Two hours is an illustration, not a universal rule. A percentage can help plan the review; it should never become a ceiling on the attention the task requires.

I approach AI output as I would work from a first-year associate or intern: useful assistance that still requires responsible supervision. Review should include accuracy, sources, context, and whether the proposed action is appropriate.

The final Submit action should represent a considered decision. Some of the time AI saves can pay for the care that decision deserves.

Time saved through AI should not become avoidable work imposed on everyone else.

Architectural Risk Becomes Harder to Fix Later

When an organization connects an AI system to its records, tools, and daily work, it begins building dependencies. The more those connections grow, the more involved it can become to remove an unnecessary capability or replace an unsafe process.

Leaving fundamental boundaries to be fixed later creates architectural risk debt. A change that would have been straightforward during design may later require new interfaces, revised permissions, staff retraining, and changes across several connected systems.

The pressure to add capabilities should be accompanied by an equally serious examination of their limits. Some problems can be addressed through a narrowly targeted control. Others may require isolating a function or redesigning a component.

Those decisions belong early enough that an organization can make them before the system becomes difficult to replace.

The Units Should Know Their Place

At Guardian AI Services, I approach design by asking what a unit needs to do for the person using it and what capabilities that role actually requires.

A private companion does not need unrestricted authority to interact with outside systems. A unit with an approved communication function needs boundaries appropriate to that function. Access, information sources, memory, and transmission should be designed around the unit’s purpose.

My background in child welfare law and public-sector AI governance has taught me to look for the gaps between a policy and what a system actually permits. Those gaps are where responsibility can become unclear and where a preventable problem can spread.

The units know their place. AI governance by architecture is the first layer of protecting your privacy.

AI may assist. Humans must authorize. Architecture must enforce the boundary.

If those safeguards had been built into the system and enforced, the AI could not have submitted the tip on its own. We would not be discussing it today—let alone asking why it took more than two months to discover what had happened. [1]

Coming next: “AI Has a File on Me? What?”