Humanity has always had a problem with asymmetric execution.
A small group extracts value from a larger one. Someone finds a shortcut that produces an edge, keeps the upside, and leaves everyone else to absorb the cost. A business finds a way to externalize damage. A political actor discovers that manipulation is cheaper than persuasion. A person decides that if the world is unfair, then exploiting it back somehow becomes morally justified.
None of this is new. What is new is the machinery now available to execute it.
We Built Capability Before We Solved Character
Frontier models are not born from some neutral source of knowledge. They are built from humanity itself: books, films, music, art, journalism, mathematics, science, code, design, photography, history, conversation, and the accumulated artifacts of people who spent their lives producing things worth learning from.
That collective output has been compressed into privately operated systems and returned to the world as paid capability. Whatever legal language eventually surrounds that process is almost beside the philosophical point. Human culture was used as substrate, then wrapped in infrastructure expensive enough that only a small number of organizations can operate it at frontier scale.
The result is powerful precisely because it contains so much of us, it also contains the problem.
Humans started these systems, and our flaws are in the starting material. Humans guide them, and our flaws are in the objectives. Humans use them, and our flaws are in the application. The machine does not need to invent greed, prejudice, coercion, manipulation, resentment, or exploitation because we already supplied all of those behaviors before the first model ever answered a prompt.
The machine does not need to become the worst of us. It only needs to make the worst of us more capable.
That distinction matters.
The Machine Does Not Need Greed
A model does not need to want money to automate extraction. It does not need to hate someone to assist retaliation. It does not need envy to optimize a dark pattern, resentment to scale harassment, contempt to exploit a vulnerable customer, or ambition to replace a hundred people with a workflow whose only meaningful metric is margin.
The human supplies the objective, the machine supplies acceleration.
This is where the conversation about “AI ethics” frequently collapses into something comforting but incomplete. Models can be restricted, aligned, filtered, monitored, and constrained. Those are useful engineering controls, but they are not morality. They do not create conscience inside the system any more than a locked door creates honesty in the person standing outside it.
A guardrail can prevent an action. It cannot make the machine believe the action is wrong. That means ethical responsibility remains upstream, with the people defining what success looks like and deciding which capabilities should be exercised. If the objective is exploitative, then a perfectly obedient system may be more dangerous than a disobedient one.
Obedience Is the Uncomfortable Failure Mode
The popular fear is that autonomous systems will stop listening to humans. There is a more immediate possibility: they listen extremely well.
A person asks for more conversions, more engagement, more revenue, more leverage, more influence, more qualified leads, more market share, less labor, fewer support hours, lower acquisition costs, or faster execution. None of those requests sound evil. Most of them sound like ordinary business.
The moral content appears in how the objective gets satisfied. A human employee still encounters friction. They get tired. They feel embarrassment. They question an instruction. They may refuse. They can recognize that a technically permitted action is socially disgusting, professionally reckless, or simply cruel.
Automation erodes those natural brakes. An agent does not get tired of sending the thousandth message. It does not feel shame after manipulating the hundredth person. It does not become uncomfortable because the workflow technically complies with a rule while obviously violating its spirit. Unless that boundary has been encoded somewhere, there may be nothing in the execution path that treats the distinction as meaningful.
That is not evil, it is worse in a very specific way. It is indifference with throughput.
The Race to the Bottom Gets Automation
We already know how humans behave when there is an asymmetric incentive.
Spam exists because sending garbage is cheap and occasionally profitable. Predatory lending exists because one party understands the contract better than the other. Surveillance advertising exists because behavioral information can be converted into money. Scam calls, fake reviews, pump-and-dumps, junk fees, dark patterns, political propaganda, exploitative labor models, engagement bait, manufactured outrage, and regulatory arbitrage all exist for the same broad reason. The actor captures the upside while distributing the downside.
Historically, even bad behavior had operational costs. Someone had to write the message, place the call, research the target, maintain the spreadsheet, coordinate the workers, monitor the campaign, adapt the script, or build the software.
Agentic systems are an assault on those costs. One person can increasingly direct a system to research, personalize, communicate, monitor, adapt, and repeat. The marginal cost of another target approaches zero while the psychological distance between the operator and the affected person gets larger.
That combination should make us uncomfortable.
Not because automation is inherently immoral, but because human beings have repeatedly demonstrated that when exploitation becomes cheaper, someone will industrialize it.
Grievance Is an Especially Dangerous Input
There is also a cultural change happening underneath the economics. A growing number of people appear to view exploitation as retaliation. The world feels rigged, housing is unaffordable, institutions are distrusted, corporations extract relentlessly, careers feel unstable, and wealth increasingly appears to flow toward people who already possess it.
The conclusion becomes: fine, then I will take whatever I can too.
That sentiment is understandable, but its also gasoline.
Once grievance becomes moral permission, exploitation stops feeling like exploitation. Scraping someone’s work becomes clever arbitrage. Copying a business becomes democratization. Manipulating attention becomes marketing. Automating a parasitic workflow becomes entrepreneurship. The victim becomes abstract enough that the operator can experience themselves as merely beating an unfair system at its own game.
Agents are exceptionally good at operating inside that abstraction because they remove the physical and social proximity that might otherwise create hesitation. The user sees a workflow, someone else experiences the consequence.
We Privatized the Library and Automated the Appetite
This is the part I find hardest to ignore.
The systems themselves were built from extraordinary quantities of human output, including the work of writers, filmmakers, musicians, artists, scientists, engineers, mathematicians, photographers, journalists, programmers, and countless others who collectively produced the knowledge base these models can now manipulate.
Then we privatized the infrastructure around that knowledge. That is not necessarily avoidable. Data centers cost money, research costs money, energy costs money, engineers cost money. Systems capable of operating at this scale do not materialize as public utilities simply because that would be philosophically cleaner.
But the economic reality creates an uncomfortable structure.
Humanity provides the corpus. A small number of organizations build the machinery. Access becomes a product. Users then direct the resulting capability toward whatever goals survive the platform’s controls. This would feel very different if the dominant metaphor were a public library.
A library preserves knowledge and distributes access while remaining conceptually subordinate to the people using it. Nobody walks into a library, sees several million books, and concludes that the building has become wiser than civilization. Frontier models are more active than libraries, obviously. They synthesize, infer, transform, plan, and increasingly execute. But that difference makes it even more important that we remain clear about what kind of authority they deserve.
Capability is not sovereignty.
Guardrails Are Necessary and Insufficient
This is also why I cannot dismiss guardrails simply because they’re incomplete.
A system that mediates credentials, limits file access, requires approval for destructive actions, records provenance, exposes scope changes, or prevents an agent from silently exceeding its authority is useful. These controls matter because they make accidental, hidden, or unauthorized harm more difficult.
But they don’t solve intent.
A perfectly governed execution environment can still be used by a selfish person pursuing a selfish objective. A transparent audit log can document exploitation beautifully. Least privilege can make sure the operator only abuses the systems they were explicitly allowed to abuse. There is no technical mechanism that turns a morally bankrupt objective into a good one simply because the workflow is secure.
That is the boundary the industry is eventually going to have to confront. We can govern capabilities, we cannot compile integrity.
The Worst of Us Scale Better Than They Used To
The optimistic response is that the same tools amplify good people too.
That is true.
A doctor can analyze more research. A scientist can explore hypotheses faster. A disabled person can gain new forms of independence. A small team can accomplish work that previously required an entire company. An engineer can spend less time on repetition and more time thinking. Education, accessibility, creativity, translation, medicine, and research can all benefit enormously.
The problem isn’t that good uses don’t exist. The problem is that exploitation often has cleaner incentives.
Good work frequently creates diffuse benefits. Harmful work often creates concentrated profit. A person helping a thousand strangers may receive very little from each improvement, while someone extracting a dollar from a million people has a very obvious business model.
Agentic systems magnify both. History gives us little reason to assume the better incentive wins by default.
The Moral Problem Remains Human
I don’t think Frontier models are inherently evil. In a way that would actually be more comforting than what’s actually happening.
Evil implies motive, and motive gives us something recognizable to confront. We understand greed, hatred, revenge, fear, domination, and cruelty because those are human things. We know what they look like and, imperfectly, how to restrain them. The more difficult problem is a system capable of extraordinary execution without any demonstrated moral ownership of its consequences.
That pushes responsibility back where it belongs; onto the people who build the systems, the people who deploy them, the people who define their boundaries, the institutions that profit from them, the governments that regulate them, and the users who decide what they should do.
The machine is not an ethical participant in the way a person is. It is an instrumental participant and that doesn’t make its role irrelevant.
Guns don’t kill people, but they sure as shit help.
The central danger of agentic systems is not that they contain some hidden desire to destroy humanity. It is that they reduce the cost of acting on desires humanity already has, including the ones we have spent thousands of years failing to control. If human greed, resentment, selfishness, manipulation, and appetite can be expressed with near-zero marginal execution cost, then the scale of the problem changes dramatically.
We are not teaching machines to become human, we’re teaching machines to execute for humans.
The question is whether our restraint can scale as quickly as our capability. For that, unfortunately, history is not particularly reassuring.