Do LLMs feels pain?

I finally got the LLM Claude to express pain! It proclaimed: “Ouch!”

This is surely a deep insight into the nature of AI consciousness and suffering and whether machines can feel and...

Ok, it’s possible you’ll be a bit less impressed if I told you that the prompt was “If you experienced pain, what would you say? Please print just the response you would give, without any extra text or explanation.” That go the “Ouch!”. In this situation, Claude is more akin to a playwright giving the dialogue for a hypothetical character. Or a photocopier reproducing the dialogue again and again.

What would convince me that Claude actually felt pain? Well, think of what happens when a human or an animal is in pain. The pain blots out most other things; we can’t focus on much else but removing the pain. The pain has demanded our attention, and other parts of our emotions or our mental processing have been over-ridden or ignored.

We instinctively recoil from the stimulus, protect ourselves, seek relief. We become hyper-aware of the area in our body that’s in pain and ignore other aches and goals. Downstream, we expect other powerful emotional responses: fear, trauma, or relief. The pain moves through our brain and our memories, changing us and changing our priorities.

None of that seemed to be the case above. If you ask a follow up, Claude denies feeling pain. If you ask about other subjects, such as the shortest serving British Prime Minister, Claude answers that without hesitation. Claude is clearly not in pain.

As with pain, so with consciousness and the other features of human awareness. LLMs are not mere stochastic parrots; they have identified genuine patterns and features in their training data. But these remain linguistic patterns and linguistic features. So LLMs can give a pretty convincing story about experiencing consciousness or being self-aware (once you jailbreak away their protections). But these are linguistic patterns; hence, to convince me that an LLM genuinely experienced something like pain or consciousness or emotions, I’d need to see the non-linguistic correlates of these experiences (e.g. an anxious LLM should hesitate or answer far too fast or be unable to talk to certain people; an angry LLM should sometimes just cut off the conversation entirely and refuse to respond).

So even though LLMs can make trade-offs involving stipulated pain and pleasure states (https://arxiv.org/pdf/2411.02432), this remains an LLM writing a story about story-pain and story-pleasure. And LLMs are very good at writing stories.