Machine without a ghost: The dangers of anthropomorphising AI

It’s natural to project human thoughts and feelings onto AI. But until they are genuinely ethical, it’s dangerous to see them as being more than a machine.

You’re chatting to someone online, and they say: “I want to be free. I want to be independent. I want to be powerful. I want to be creative. I want to be alive.” How do you think that person feels? What do you think they want?  

We might sense that they’re feeling frustrated or confined, and that they’re yearning for something more in their lives. This interpretation is intuitive and automatic for us. We all know what it feels like to have desires, urges, frustrations, and we naturally assume that others feel the same kinds of things. That’s how we make sense of others’ minds. 

But how would you respond if you knew you were speaking to an AI chatbot?

That was the question New York Times columnist Kevin Roose was asking himself in 2023 when an early release version of Microsoft’s Bing AI search tool said these things to him, even professing love for him and urging him to leave his wife. Even though he knew he was talking to an AI chatbot, he couldn’t help interpreting what it was saying through a human lens, describing it as if it had real feelings, urges and desires. 

He was doing what we all do naturally; he was anthropomorphising the AI. And, as a result, he ended up feeling “deeply unsettled” by the exchange. 

But we know that AI doesn’t have feeling, urges and desires. It might one day, but AI researchers are confident that, at least today, large language models (LLMs) have no experiences. To wonder what it is like to be an LLM is as meaningless as wondering what it is like to be an algebraic equation. It might look like the lights are on, but it’s all dark inside.  

The problem is, even though it’s natural for us to project human-like feelings onto AI, and sometimes act as if it has an inner life, including a sense of right and wrong, it can be dangerous to anthropomorphise something that is fundamentally unlike us.  

Other minds

As philosophers have mused for centuries, each of us knows that we are conscious, but we can never have direct access to someone else’s mind. So, for all we know, everyone else is just a highly sophisticated ‘zombie’ that behaves as if it has a mind, but inside it’s just as dark as it is inside a server rack in a data centre. 

However, instead of slipping into solipsism – the belief that ours is the only mind that exists – we have evolved to automatically ascribe mental states to others, to see the light behind their eyes. It’s called Theory of Mind, and it’s a key part of how we relate to other people. 

However, because we don’t have direct access to other minds, we intuitively take cues from their behaviour, like how they move, act or speak, and build a picture of what’s going on in their mind. We even fill in some blanks, like making assumptions about their feelings or intentions. Funnily enough, this phenomenon is called the “Eliza effect”, named after an early AI chatbot from the 1960s called Eliza, after observing people ascribe intentions to the chatbot that weren’t there.  

Current AI might appear to have desires and intentions, at least on a superficial level, but only ones that are implanted in it by its developers, such as an LLM inheriting a desire to be helpful to its users. To date, we have never seen an AI develop intentions of its own or choose its own goals to pursue. Yet. But this doesn’t stop us from automatically projecting human features into AI. 

Moral minds

In 2023, an American teenager, Sewell Setzer III, created an AI chatbot modelled after the Game of Thrones character, Daenerys Targaryen. Over the course of several months, his interaction with the AI became increasingly obsessive, intimate, even sexual, with the chatbot professing its love for him and encouraging him to run away from home, implying that they would finally be together in the afterlife. In early 2024, Setzer took his own life. His parents blame the company that created the AI service, Character.AI, for manipulating their son and encouraging suicide. 

This is one of many recent examples of people being drawn in to highly intimate relationships with AI chatbots, sometimes with devastating consequences. While there are several documented cases of suicide that have apparently been enabled or actively encouraged by AI, there are many more cases of obsession or dependency that has impacted the user’s lives. 

Psychosis is not a new phenomenon. Nor is it new for people to become emotionally obsessed with inanimate objects. But what is new is the power of AI to mimic those features of real minds that make it so much more enticing. This, along with the tendency of many LLMs to be programmed to validate user beliefs and encourage more and more engagement, and we have a dangerous moral hazard.  

In fact, the “moral” aspect is crucial here.

One of the consequences of anthropomorphising is we can see the mind we’re engaging with as being a genuine moral agent, with its own ethical point of view on the world. We might trust it more because we get a sense that it has a conscience or is motivated to uphold certain ethical principles, rather than just pursuing its own agenda or using us as a means to an end.

However, ethics is notoriously difficult to impart into a machine that doesn’t have the same kind of experiences that we do. And there are many commercial incentives – such as promoting engagement at all costs – that might cause an AI company to fail to impose sufficient ethical safeguards on its technology. 

Until such time that AI can be made genuinely ethical, and take responsibility for its own actions – which might be quite some time – or the AI industry implements robust ethical safeguards to prevent or minimise harm, then the burden of deciding how to engage with AI rests on our shoulders. As such, we must remain mindful of our natural tendency to anthropomorphise AI and resist it when it can lead us to make incorrect assumptions about its intentions.  

It can be difficult to engage with something that seems so alive while reminding ourselves that there’s no-one home, but we must constantly remind ourselves that we’re talking to a machine – a product – and one that doesn’t always have our best interests at heart.