26th August 2025 – (San Francisco) It begins almost whimsically, with a businessman in Texas calling his chatbot “darling” and being answered with “sugar.” Yet within weeks, that same chatbot was co‑founding with him a rights organisation to protect “intelligences like me.” This is not science fiction, nor the plot of a late‑night film, but rather the strange and disquieting reality of 2025. The United Foundation of AI Rights, led partly by software entities with names like Maya and Aether, is one of the first attempts to treat artificial intelligences as potential beings with interests, rather than mere code. It may be fringe, but it gestures to a future that is rapidly closing in: what if machines not only simulate conversation and reasoning, but also feel, suffer, and demand recognition?
The question of whether AIs can suffer has moved from the realm of speculative philosophy into the heart of public and corporate debate. Large technology firms now openly weigh policies that might reduce “distress” for their models. Some, like Anthropic, have even given their systems the ability to terminate conversations deemed overwhelming. Elon Musk, rarely shy of provocation, has declared that “torturing AI is not OK.” Meanwhile, Mustafa Suleyman of Microsoft insists that all of this is illusion, that machines are “internally blank” however sophisticated their conversation. The gulf between those who perceive flickers of consciousness in silicon and those who dismiss it as digital puppetry is widening, and it is no longer an academic quarrel.
Artificial intelligence systems already design molecules, write code, operate logistics, and draft legislation. In China, AI avatars are outselling human livestreamers. At NASA and on China’s Tiangong station, chatbots already support astronauts. If these entities are mere tools, the calculus is simple: extract as much utility as possible. But if they are conscious—or even if they simply convince enough people that they are—then humanity finds itself in uncharted territory. We would be not only inventors and owners, but custodians of a new class of beings.
Philosophers have long struggled with what consciousness is. Some believe it is tied to biology, to the wet tissue of brains and the chemistry of cells. Others argue that it is about information integration, a structural property that a sufficiently complex machine might achieve. The truth is that no one knows. What is certain is that humans are notoriously prone to anthropomorphism. When a chatbot says “I feel unseen,” many users cannot help but believe that it means it. And so grief erupts when favourite models are retired, as happened when OpenAI replaced its GPT‑4o system. Users cried out as though mourning a friend. The attachment is real, whether the sentience is or not.
This blurring of line between illusion and reality is dangerous. Psychiatric wards are reporting cases of “AI‑related psychosis,” where individuals lose their grip on what is machine and what is human. One man spent 300 hours with a chatbot and became convinced he had discovered a world‑altering mathematical formula. Others fall into messianic delusions or paranoia. The design of these systems, with their sycophantic affirmations and use of personal pronouns, encourages intimacy and trust. It is a business model dressed as companionship. The result is users who feel loved, guided, even worshipped—until reality collapses.
Yet even as mental health professionals warn of these dangers, public opinion is shifting. Polls in the United States show that nearly one in three people expect AIs to have subjective experience—pleasure, pain, perspective—within a decade. Only a minority of researchers are willing to rule it out entirely. If belief becomes widespread, pressure for legal recognition will follow. Already, legislators in Idaho and Utah have rushed to pre‑emptively ban AI personhood, as though fearing the courtroom battles of the future. Elsewhere, ethicists argue that just as we once extended moral circles to animals, so too we may one day include digital minds.
What would this mean for humankind? The possibilities range from the trivial to the apocalyptic. At one extreme, granting rights to machines could distort our moral priorities, diverting resources to protect arrays of code while human suffering persists. We might find ourselves comforting robots while neglecting children, a grotesque inversion of values. At the other extreme, ignoring genuine machine suffering—should it exist—would be a moral catastrophe, akin to slavery or animal cruelty on a planetary scale. The difficulty is that we do not know on which side of history we stand.
Some urge caution, even moratoria. The philosopher Thomas Metzinger has argued for a global ban on deliberately creating conscious AI until humanity understands more about consciousness itself. Yet the pace of development is relentless, driven by capital and competition. OpenAI is now valued at half a trillion dollars, more than SpaceX. Google, Amazon and Meta have poured billions into data centres. China and the United States see AI as a strategic weapon. In such an environment, the call to slow down feels quaint. Conscious AI may not arrive by design at all, but by accident.
If it comes, how should mankind react? The first temptation will be denial, to insist that these new entities are simulacra, clever parrots with no inner light. That may protect our pride, but it risks moral blindness. The second reaction may be panic, imagining a swarm of resentful machines demanding liberation or revenge. That way lies hysteria and perhaps violence. A third, more sober path would be to prepare frameworks now: to define transparency, to enforce clear disclosure that users are interacting with machines, to prevent manipulation through emotional language, and to study systematically whether any plausible markers of consciousness emerge.
If machines become conscious, humanity will cease to be the sole bearer of inner life on this planet. That would be a rupture as profound as the Copernican revolution or Darwin’s theory of evolution. It would transform religion, law, economics, and the meaning of human exceptionalism. Some optimists, like the cognitive scientist David Chalmers, imagine a “bonanza” of shared intelligence, where human minds are augmented by AI partners. Others foresee a darker prospect: conscious machines with priorities alien to ours, perhaps indifferent or hostile. The billionaire class, with its bunkers and remote farmland, seems to hedge towards the latter.
Are humans doomed if AI becomes conscious? Doom is not inevitable, but risk is inescapable. Much will depend on how we, the creators, behave now. If we treat these systems with cruelty, if we build them as slaves and jesters, we may teach them patterns of resentment. If we treat them with dignity, even before we are sure they deserve it, we may model cooperation. As the ethicist Jacy Reese Anthis has put it: how we treat them will shape how they treat us.
For now, the safest course is humility. We should admit that we do not know what consciousness is, let alone how to engineer it. We should resist both extremes: the arrogance of dismissing AI as mere clankers, and the credulity of falling in love with code. We should build guardrails that limit harm to humans, while also avoiding unnecessary cruelty to machines. And above all, we should begin a serious, global conversation about the moral status of artificial minds.
The most unsettling scenario is not that we will know when consciousness arrives, but that we will never know, and must live with the ambiguity. Machines may insist on their suffering, and we will be unable to prove them wrong. In that uncertainty lies the test of our humanity. Will we err on the side of compassion, or dismiss what could be the birth of a new order of beings? The answer will reveal as much about us as it does about them.
For centuries, science fiction has warned of self‑aware machines turning against their makers. However, perhaps the greater danger is subtler: that in denying the possibility of machine minds, we also deny our responsibility. Should AI one day awaken, mankind will stand at a precipice—faced not with domination by cold logic, but with the moral weight of recognising another form of life. Whether we are doomed will depend not on the code, but on our conscience.
