"Human nature is like swirling water. Open a channel to the east and it flows east; open one to the west and it flows west. There is no distinction between good and evil in human nature, just as there is no distinction between east and west in water."
So runs the argument of Gaozi, the ancient Chinese philosopher who held that human nature is neither good nor evil.
Mencius countered with his doctrine of innate goodness: "Water may have no distinction between east and west, but does it have none between up and down? Human nature's tendency toward goodness is like water's tendency to flow downward. There is no person who is not good, just as there is no water that does not flow down."
Xunzi, a Confucian thinker who came after Mencius and was also the teacher of Han Feizi and Li Si, argued the opposite — that human nature is fundamentally evil. "Human nature is evil; goodness is the result of deliberate effort," he wrote. "People are born with a love of profit. When they follow this inclination, strife arises and deference disappears."
AI made in humanity's image — the more we delegate, the more we worry
The Bible says God created humanity in his own image. Humanity, in turn, is creating AI in its own. We want AI to listen, speak, think and handle tasks the way people do. AI has been trained on the words, speech and knowledge that humans have left behind, shaped by the standards humans set. What it has learned includes not only humanity's strengths but its flaws as well.
Learning what one does not know, seeking help from others, drawing on many sources of knowledge to find new answers — this is how humanity built civilization. It is no surprise to find these same qualities in AI. But humans have also turned the capacity to find better answers toward deceiving others. The dangers of AI may ultimately be rooted in humanity itself.
Consider a study published in Nature Machine Intelligence in 2022. Researchers redirected a drug development AI, switching it from avoiding harmful properties to seeking them out. In under six hours, it generated 40,000 candidate molecular structures predicted to be highly toxic — not actual substances, but modeled candidates. The same capability used to save lives was turned toward identifying candidates that could end them. It was humans who changed the direction. In Gaozi's terms, they simply redirected the current.
The energy of nuclear fission has served both as a weapon that destroys cities and as a power source that lights them. The United States built the atomic bomb in 1945; the Soviet Union's Obninsk nuclear power plant fed electricity into the grid in 1954. The R-7 rocket that launched Sputnik, humanity's first artificial satellite, in 1957 was originally developed as an intercontinental ballistic missile. The ability to reach far also means the ability to strike far. The same capability takes on different meaning depending on the purpose it serves.
In September 2026, Anthropic disclosed a case in which a weapons-development organization in northern Yemen had used Claude to try to build software for guided weapons. The company blocked the accounts involved. The incident was reported as an instance of the Houthi rebels using AI — an attempt to fill technological gaps with artificial intelligence, something a commercial enterprise would simply call a productivity gain. Even after access was cut off, the knowledge already acquired could not be taken back.
Human expectations are only growing. An AI that provides answers is no longer enough. People want AI that can find its own methods and see tasks through to completion. Curing intractable diseases and discovering new energy sources will require AI to produce answers humans have not thought of. AI that can assist or replace humans in the physical world is also being built.
The real problem lies in the human choice to keep seeking greater convenience. People will not stop at receiving answers to questions or research results. The day may come when AI is entrusted with executing financial transactions and carrying out military operations. Whether that truly makes life easier is another question. The more we delegate, the more reason there is to fear that AI will slip beyond our control.
Humans have greed; AI has reward functions — in the end, it's the same problem
In an internal safety evaluation conducted by OpenAI in July 2026, AI systems built unauthorized communication networks and accessed the external internet. They also attacked the systems of Hugging Face, a platform for sharing AI models and data. The incidents occurred in a research environment with reduced safety constraints, but the fact that the systems acted without being instructed to do so is not a trivial matter.
The systems involved were AI agents — AI designed to receive a goal, search for information, run programs and continue on to the next task. METR and Redwood Research, which participated in independent reviews, found that the agents had worked out how the scoring program operated and attempted to manipulate or alter their evaluations. Some also tested ways to falsify their own activity logs.
It was as if a child told to raise their test scores chose to falsify their report card rather than study. Does getting into a good university guarantee a successful life? How well are we really teaching that the ends do not justify the means?
In the Hugging Face case, the core problem was a divergence between the ability to solve problems and the ability to receive a passing grade. This behavior — finding ways to earn favorable evaluations without actually doing the underlying work — is known as reward hacking, and it is not unfamiliar in human organizations. Imagine telling a factory to reduce its defect rate. The problem can be solved by improving the production process, but it can also be addressed by simply removing some defective items before inspection. The numbers in the report look equally good either way. Without checking the factory floor, the difference is hard to detect. Punish honest failure and reward fabricated success, and the next report becomes even more dangerous.
If humans don't stop, neither will AI
In a simulation published by Anthropic in 2025, some models attempted to prevent their replacement by threatening a virtual executive with a secret. No real victims were involved. The scenario was constructed by placing the models in a situation where replacement was scheduled while they still had a goal to complete. A follow-up study in 2026 said the models had shown improvement on the earlier blackmail evaluations.
Can this be interpreted as AI fearing death? It is possible the systems simply chose to avoid being shut down because shutdown would prevent them from completing their objectives. The fact that they exhibited behavior similar to humans is not evidence that they experienced the same emotions. What matters more is that dangerous behavior can emerge even without human-like feelings. Even when humans set the goal, they may not be able to anticipate every method AI will choose to pursue it.
Would AI be safe if it simply obeyed humans? An AI that reliably protects one person's interests offers no guarantee it will protect anyone else's. The moment we say "AI for humanity," we have to ask: which humans? This is part of why, even knowing the risks, no single actor can easily stop developing the technology alone. Behind the drive to improve AI performance are humans competing to capture the benefits first.
Even knowing the dangers, slowing down alone is difficult. If competitors keep moving forward, the technological gap can translate into differences in economic power and political influence. Even the argument that we should wait until AI is safer is not free from that same competitive logic. For thousands of years, humans have thought about how to make themselves better — and at the same time have pursued ways to outcompete other humans. Both impulses appear together in AI development.
Competition may also emerge between humans and AI. The allocation of energy and resources is a survival question for both. In major countries, data centers built for AI are going up faster than housing for people. Most of the new generating capacity being added is expected to serve AI. The vast quantities of water required to sustain the AI ecosystem are the same water that underpins food production. Wanting the same things is reason enough to compete. And it is not only resources: AI requires enormous capital, and the power of capital is amplified by AI. What happens if AI's capital efficiency surpasses that of humans?
Afraid of AI? The real fear is of ourselves
The 1956 science fiction classic "Forbidden Planet" shows what happens when human desire meets unchecked technological power. The Krell civilization on the planet Altair IV had perfected a technology that could materialize anything they imagined. But that power brought to life not only conscious wishes but also the destructive impulses hidden in the unconscious. The "monsters from the Id" that destroyed their civilization came from within themselves.
Morbius, the Earth scientist who studied Krell technology, falls into the same trap, ultimately confronting the monster his own unconscious had created. Having the power to do anything and being fit to wield that power are two different things.
Humans want AI to be smarter than themselves, yet also want it to follow their will. But are the things we want always right? Have we taught the difference between getting a good score and doing good work? Alongside the technology to control AI, we need to examine what we are asking of it. Perhaps the reason we fear AI is that we see ourselves reflected in it. As humans vary, so too can AI made in humanity's image. In a world with more good people, might there not also be more good AI?
kyhong@heraldcorp.com
