给相信 AI 具有意识者的追问清单

This weekend, I had been intending to start writing a careful and comprehensive ‘how to argue against the people who believe AI is conscious’ piece. But, at this very minute, everyone is going crazy on the internet shouting at each other about these things, so here instead is my quick attempt to help!

If you scroll down, you’ll see that the bulk of this piece takes the form of a list of questions. You can either read straight through these questions, or you can approach them as a ‘choose your own adventure’ philosophical resource, jumping between the questions based on the answers you give! I’ll begin, however, by noting two brief bits of context.

First, the meta point I’m trying to make here is that discussion of AI consciousness can be both clarified and enriched by bearing in mind some simple distinctions that are ‘bread and butter’ to most philosophers. My view is that much current confusion, and even anger, about the topic of AI consciousness could’ve been avoided if there were even a tiny bit better general awareness of the useful foundations philosophers have laid on this topic, over the centuries.

Of course, I’m not suggesting that philosophers have solved all the problems of consciousness (ha)! Rather, that we’ve at least made some progress in being able to talk clearly and productively about these things with each other. And that we should’ve done a much better job at sharing this progress with the rest of the world, over the past few years, when talking about these things has been becoming increasingly and newly important.

My cynical take is that most philosophers have, instead, ignored humanity’s sudden need for this philosophical input. Sure, a small but growing number of philosophers have been taking jobs at AI companies — where, seemingly, they mostly spend their time supporting the views of their bosses. But the majority of philosophers, who hold busy jobs at universities, have spent these years telling each other how rubbish they think AI is, while complaining about how much harder it has become to mark essays. Maybe this sounds harsh. Fine! I strongly believe we’ve let the world down, as a profession. And this piece is an attempt to play some part in turning things around, on a ‘better late than never’ basis.

Second, some of the questions below are aimed at emphasising differences of view about what ‘the conscious thing’ is that’s being referred to, when people claim that AI is conscious. Nonetheless, current discussion of AI consciousness is taking place during the fast-paced development of LLMs, so my questions are particularly aimed at evaluating claims about consciousness made in relation to products developed by companies like OpenAI and Anthropic. All that said, I’m aiming to keep this piece as non-specific and non-technical as possible.

Here we go!

If your answer is YES, then please answer Question 2.

If your answer is NO, then I hope the following questions might be useful when you’re talking with someone whose answer is YES. Or, even better, that reading this piece might help you to form and continually refine your own set of questions for use in such situations. After all, there are infinite ‘choose your own adventure’ games to be created!

You know full well what I mean by ‘internal world’! You can talk about this in terms of interiority, subjective experience, having a ‘what it’s like to be-ness’, or whatever… but you know what I’m getting at! So, do you believe that AI has an internal world?

If your answer is YES, please read on, and then answer Question 3.

If your answer is NO, then please skip to Question 6.

At this point, I’ll admit that the main thing I want non-philosopher readers to take from this piece is awareness that when people who work in AI say that they believe AI is conscious, they aren’t necessarily implying that they believe AI has an internal world. In other words, that some of these people — whether they are employees of AI firms, AI policy people, or whatever — use the term ‘conscious’ in a way that would seem very odd to most ordinary people.1

I’m not going to get into different theories of consciousness here. I’m also not going to unpack the idea of an ‘internal world’, or its relation to consciousness, in the way I usually would as a philosopher who’s interested in these things. Rather, I simply want to say the following. I would bet every dollar I have that while most people don’t have a fully-worked-out theory of consciousness, nonetheless that when someone says to them, “I believe that AI is conscious”, they take this to entail “I believe that AI has an internal world”.

So, if you’re reading these questions as someone wanting some help to understand or engage with people who claim that AI is conscious, then next time you come across such a person, please remember that their answer to Question 2 might be NO! And, of course, that the same little chain applies if instead they say, “I have calculated a probability of 0.76 that AI is conscious!” We philosophers know to ask Question 2 in such situations, and I hope to help you see the value in doing so, too.

None of this is to deny that some people do believe that AI has an internal world, however. It seems likely, for instance, that many of the people who believe that Claude has a soul also believe that Claude has an internal world. So, here are some follow-on questions for the people who say YES to Question 2!

No need to get overly technical, here! I’m just looking for some specificity beyond ‘AI’. I mean, perhaps it’s models, like GPT-6 Astra or Claude Fable 5.1, that you think have internal worlds? Or perhaps it’s the conversations you have with such models? So, can you tell me what particular kind of thing you’re referring to?

If your answer is YES, then please skip to Question 5.

If you think you needn’t be referring to some such ‘particular kind of thing’, beyond AI ‘itself’, then I’m taking your answer to be NO. If this applies to you, then please answer Question 4.

If your answer is NO for some other reason, then please read on:

Perhaps, on reflection, the ‘models’ answer is starting to appeal to you! After all, don’t people say that Claude has a ‘character’? Or perhaps you’re starting to think it’s the AI companies that produce these models that have the internal worlds in question. Perhaps, instead, you weren’t convinced by the idea that it could be the conversations that you have with these models, but you’re now wondering whether it could be the answers you receive in such conversations! Or the words? Each and every single one of them! Or perhaps it could be the ‘tokens’ that the models process and produce while ‘communicating’ with you via text…

Perhaps, instead, you’re wondering whether it might be the machinery of the data centres in which electrical events enable these conversations. Or perhaps it’s wires in this machinery? Perhaps you want to go back a few steps, and consider the devices that you’re reading these conversations on… Perhaps it’s been the laptops, or phones, all along!

Do any of these options sound like they’re in the right ballpark, to you? Remember, you can keep it simple: I don’t need complex theories about personal identity or technological processes! At this point, however, I’m going to ask you Question 3 again:

Can you tell me what particular kind of thing you’re referring to as ‘AI’, when you say you believe that AI has an internal world?

If your answer is now YES, then please skip to Question 5.

If you’ve come to think that there needn’t be some such ‘particular kind of thing’ you’re referring to here, beyond AI ‘itself’, then please answer Question 4.

If your answer is now NO for some other reason, then please skip to Question 6.

If your answer is NO, or if you think this question is irrelevant here, then please answer Question 5.

If your answer is YES, and/or you think this question is relevant here, please read on, and then answer Question 5:

Normally, I would go straight into throwing some objections at the idea that a group of things could share an internal world, but I can save that for later. For now, I’m wondering whether it’s because you think that AI is a ‘group-type thing’ that you feel no need to be able to refer to some ‘particular kind of thing’ as the internal-world-haver, beyond AI ‘itself’?

If so, I’m happy to accept that there is some relevant sense in which a group of things can be a ‘particular thing’ — and even a ‘particular kind of thing’! In other words, I’m totally open to you giving a group-type answer, at this stage. I did offer you ‘AI companies’ as a potential answer, after all, and a company surely counts as a group, in a way in which a word and a wire do not.

If what you’re doing instead, however, is trying to find a way to say you think it’s simply ‘AI’ that has the internal world — and that ‘AI’ is some nebulous, group-type, edgeless kind of thing, which might incorporate some of these other things like models and words and tokens, and which also might not — then you’re going to have to do better! Not because that answer is a group-type answer. But because you can’t even tell me what the ‘parts’ are that make up the group.

Okay, one reason that getting to grips with this stuff is so hard surely relates to complexity arising from the way in which people standardly use the term ‘AI’ both to refer to ‘an AI’ (singular) and also to ‘AIs’ (plural). But if you want to convince me that AI has an internal world, then you’re going to have to tell me something about what AI is!

I mean, if your answer to my question were ‘AI companies’, then at least you’d be able to tell me that the ‘parts’ of each AI company were the people involved in that company, or some such. And, of course, your group-type answer could certainly be ‘companies plus models plus conversations’, if you wanted. I’d take that! Or any other combination of particular kinds of things… But I really do have to press you for some specificity. So, I’m going to ask you Question 3 again:

Can you tell me what particular kind of thing you’re referring to as ‘AI’, when you say you believe that AI has an internal world?

If your answer is now YES, then please continue to Question 5.

If your answer is NO, then please skip to Question 6.

If your answer is NO, then please return to Question 2.

If your answer is YES, then please read on:

Let’s begin by taking stock! So far, you’ve said that: A) you can indeed tell me what kind of thing you’re referring to when you say you believe that AI has an internal world; and also B) that you’re sure this is the kind of thing that could have an internal world. Good! I now have some follow-up questions. Some of these questions will be more or less relevant to your particular case, so please bear with me.

First, I’m going to repeat Question 4: do you believe that a group of things can have an internal world? This time round, this question applies to you, particularly, if the answer you gave to the ‘particular kind of thing’ question was a group-type answer, like ‘AI companies’ or ‘AI companies plus models’.

If you do believe that a group of things can have an internal world, then do you also believe that each member of such a group needs to be the kind of thing that can, in itself, have an internal world? And if so, do you have good answers to why and how it could be that each of these things plays a part in a wider shared internal world, above and beyond their own internal world? I mean, just because you have an internal world doesn’t mean that you can enter into a shared internal world with me or anyone else! So, how does this ‘sharedness’ arise, and how is it cashed out, in practical terms, with regard to accessing experiences, reflecting on experiences, and so on? Remember, we’re talking about internal worlds, here! And if, instead, you do not believe that the parts of the group all need to be internal-world-havers, in themselves, then where does the shared internal-ness come from? And do the non-internal-world-having parts ‘gain’ some of this?

Of course, at this point, you might want to remind me that people have often asked these kinds of questions about human consciousness! But at least when you’re thinking about human consciousness, you know what it’s like to have an internal world, and you also know that you are not privy to anyone else’s internal world! Moreover, please do remember that the meta aim of this piece is to help non-philosophers gain access to the questions and distinctions that philosophers typically use to think about and discuss these matters clearly and productively... So, let’s carry on!

Second, do you believe that a non-living thing can have an internal world? This question applies to you, particularly, if the answer you gave to the ‘particular kind of thing’ question was a non-living-thing-type answer — whether this was also a group-type answer or not. I mean, if you answered ‘AI companies’, but you take this to include some non-living things on top of the people who are members of such companies, then this question is for you! It’s also for you if the answer you gave was ‘models’ or ‘conversations’ or ‘wires’. In all of these cases, I want you to think about whether there are any instances of such non-living things that you wouldn’t want to consider as able to have internal worlds. I mean, if your answer was ‘wires’, but you think that wires, generally, aren’t the kind of thing that can have an internal world, then you might want to reconsider your answer, or enhance it in some way!

Third, do you believe that a non-biological thing can be a living thing? This question is for you, particularly, if you think that only living things can have internal worlds, but you also think, for instance, that computers can be living things. That said, if you do think that computers can be living things, and computers were also the ‘particular kind of thing’ from your answer to Question 3, then please skip ahead to Question 6!

If instead, however, you want to try to tell me that AI is a non-biological thing that can be a living thing, and that it’s also AI that you want to refer to as the ‘particular kind of thing’ from Question 3, then I’m afraid, once again, I’m going to need some further specificity! Aside from anything else, convincing me that models could be living things is going to require some quite different arguments from convincing me that particular parts of data centres could be living things. As indeed would convincing me that ‘models plus particular parts of data centres’ could be some kind of composite living thing. In other words, I need to know something about what you think AI is! What’s more, if your answer is ‘companies plus models plus conversations’, but you don’t think this combination includes any living things, then you’re going to need a different argument again!

Regardless of which answer you give me, however, I’m also going to want to ask you about its moral implications. Does your answer mean, for instance, that you should probably stop using AI in the ways you currently use it? Does your answer mean that we should all be treating trees or rocks or computers totally differently from our current practices? I’m also going to want to press you further — as with the ‘wire’ answer, above — about whether you hold coherent views about other similar things in the world. I mean, if you think that ‘companies plus conversations’ can be alive, do you also think that ‘book publishers plus conversations between fictional characters’ can also be alive? And if not, why not?

There are many further questions I’d likely want to ask you, but I hope by now that you can begin to imagine them for yourself. In this context, I’ll ask you Question 5 again:

Are you sure that the kind of thing you’re referring to is the kind of thing that could have an internal world?

If your answer is NO, then please return to Question 2.

If your answer is YES, then please answer Question 6.

Thank you for your attention to this matter!

Some very quick thoughts about people who depend on AI outputs

One final thing I want to note — for the benefit of readers who neither work in AI nor are philosophers — is that many of the people who work in AI who believe that AI is conscious believe this because of the special significance they afford to the qualities of AI outputs. What I mean by this, in simple terms, is that these people conclude that because AI can produce the kinds of outputs that humans produce, therefore AI is conscious. I’m now going to give you an easy route into questioning these people about this…

First, let’s imagine that one of these people, let’s call him Adam, begins by emphasising how much better AI has recently become at producing poems. “Look at these lines!” Adam cries. “They could’ve been written by a top poet! They are full of such longing and beauty! They speak of personal experience! They describe what it’s like to feel heartbreak! Don’t you think that these lines seem like the output of a conscious thing?”

Now, of course, the first thing you should find out here is what Adam means by ‘conscious’. If you ask him Question 2, and his answer is NO, then this will save you a lot of bother! If it turns out, however, that Adam believes that these outputs provide sufficient reason to conclude that AI has an internal world — and if you want to address this ‘output thing’ head on, rather than jumping straight into Question 3 — then here’s a quick guide to pushing back against him.

Start with some simple questions! Things like: does this mean that any tool that can be used to do the things that a human can do counts as conscious? Are ploughs conscious? What about calculators? Or: do you think that if a parrot can utter the phrase “I’m really awesome”, then you should assume that the parrot is conscious, for that reason alone? Basically, just focus on how much emphasis Adam is placing on outputs: it’s all of his emphasis, remember!

Then, if necessary, you can move on to asking him some progressively tougher questions. Things like: do you think that because characters in books ‘say’ things, therefore they are also conscious? Or: do you think that if the wind blows the sand in such a way to create a word on the beach, therefore the wind is conscious? Or the sand!? At this point, if you really want to have some fun, you can bring in some examples that involve randomness and possible worlds… After all, people who work in AI often claim to know about those sorts of things!

In other words, be as creative as you like with these questions. And, pretty quickly, Adam will begin to move towards depending on something beyond the qualities of AI output. Most likely, he will turn to talking about AI processes. And, at that point, you should make sure to emphasise that he is no longer solely depending on output!

This takes you into a totally different zone, of course, but if you want to continue arguing, then one thing you might begin to emphasise is that we know so much more about AI processes than we do about correlative human processes. We know a great deal about how AI processes work, for instance, and we know how to replicate them. It would even be possible to do so manually, without any need for electronics! At this stage, do remember to ask Adam what he himself knows about AI processes, because his desire to show off will likely end up advancing your case. The more he tells you about the fine details of AI computation, for instance, the more you can push back when he wants to tell you that brains are also ‘literally’ computers…

One further point to remember and emphasise in these kinds of arguments, wherever you need to, is that you are conscious. Indeed, that the fact you are having this argument with Adam means that you are conscious! And that it also seems extremely reasonable to assume that Adam is conscious, too, because he is a human being, just like you. What’s more, if Adam doesn’t agree with you about these things, in the sense of applying them to his own case, then his whole task is likely made massively harder. Regardless, he’s going to need another route into persuading you that AI is conscious. I mean, he obviously can’t depend on the ‘it’s a human, just like me’ approach, and he’s given up depending solely on human-like outputs…

At this stage, perhaps it’s time to begin asking him about the ‘kind of thing’ he thinks has the internal world?

1

Please note that ‘ordinary’ is not a criticism. Regular readers will know how much I value ordinary language, ordinary understandings, etc…

No posts