Essay
If Not Person, Why Person-Shaped?
I read the above tweet and frowned.
There’s a problem with it, I thought. No, there’s two problems with it. No, there’s three problems with it.
There might be more, but three is enough. If you want to have a reasonable idea of what’s going on in the near-term to mid-range future, you’re going to want to recognize these conceptual problems. Let’s let Alan Turing set the stage.
I propose to consider the question, ‘Can machines think?’ This should begin with definitions of the meaning of the terms ‘machine’ and ‘think’. The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous. If the meaning of the words ‘machine’ and ‘think’ are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, ‘Can machines think?’ is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
When you’re talking about whether a machine can really think, feel, want, or understand, you have two options. You can define what you mean by those words in a clean and unambiguous way, or you can argue in circles interminably. Those are the two options. Turing’s idea for thinking was instrumental: if you can’t tell the difference between what the machine is doing and what a thinking human is doing, the machine is thinking. You might not like that definition, but it has the advantage of a rough-and-ready realism. If the machine can produce the same result as a thinking human, it’s a thinking machine.
“Ah, but that’s not really thinking, it’s just an imitation of thinking.” That’s a fair enough point of view, but unless you have a clean definition of what you mean by “really thinking,” you’re just choosing option two, and I hope you enjoy arguing without result.
But I’m not here to tell you how to define your words. I’m here to tell you that if you limit your own thinking about AI models by avoiding “language that gives them human characteristics,” you may be tempted to think that AI models can’t do the kinds of things that humans can do by thinking (or feeling, or whating, or understanding). They can, and they’re getting better at it on a daily basis.
AI Is Shaped like a Person on Purpose
The original GPTs that eventually became ChatGPT and its successors and competitors were imitation machines in the most literal sense. They ingested every bit of human text its creators could find and did an unbelievable amount of statistics to produce a model that could generate the next word in a statistically reasonable way.
The capital of Italy is…
You will not have trouble autocompleting the next word: “Rome.” You will not have trouble believing a machine performing statistical inference on the corpus of English-language writing will have seen the complete phrase many thousands of times, and will also correctly complete the phrase.
What’s a good recipe for spaghetti sauce, written in Latin?
That’s more difficult to autocomplete, because the ancient Romans didn’t have tomatoes. Despite the association between Italians and the tomato, the now-ubiquitous red kitchen staple was a New World species unknown to Europeans until after Columbus. There are no Latin recipes involving tomatoes. But I asked Claude, and the result was quick and, to the best of my limited Latin knowledge, fairly fluent:
Cepam coque: 3 tablespoons oleum olivarum in olla calefacies. 1 cepa mediocris, minutim concisa adicies et leni igne coques donec mollescat et pellucida fiat…
There are articles about tomatoes and spaghetti on Latin Wikipedia, and the Romans did have culinary vocabulary that survives in many ancient texts. The AI isn’t working from scratch. But as a matter of intellectual honesty this is something that—if a human did it—we would consider the result of clever thinking.
(I suppose there’s no Latin word for our modern 15 mL tablespoon. There is a Latin word for tomato, lycopersicum, which referred to a totally different plant in the ancient world but was adopted for the tomato in the 16th century. Interestingly, we don’t actually know what plant lycopersicum originally referred to.)
You can probably imagine how you might have completed the Latin recipe question if you sat down at a desk and were told you’d be paid $50 if you gave it your best effort. “El tomatoae est plantae del maximus tasteus…” or some gibberish like that, or perhaps “Unfortunately I don’t know how to speak Latin and I just buy Prego at the grocery store.” These wouldn’t be especially useful things for a chatbot to say, even though they’re perfectly valid completions, and so AI companies spend a lot of effort tuning their models to say the kinds of things that come from a particular subset of human language—polite assistants, essentially.
The AI’s responses are therefore deliberately shaped into a human form in two ways:
- They are trained on the written output of mankind over the entire history of writing.
- They are fine-tuned specifically to respond in a conversational way in character as a helpful assistant.
Therefore, if you want to understand what an AI is likely to do, you will have a more accurate predictive mental model of that behavior if you think of an AI as anthropomorphic. Anthropomorphic is from the Greek anthrōpos (human) and morphē (shape), literally, “human-shaped.” This is true in both senses; they are shaped like humans, and they are shaped by humans.
Contra the AP Stylebook, they do have human traits. They do have human behaviors. These are fairly objective observables. I don’t think they have emotions, but to an extent they act like they have emotions. They aren’t humans. But they are human-shaped.
The younger girl is holding an apple.
A succession of layers of pigmented drying oil is arranged on the surface of a canvas in a way loosely analogous to the distribution of light on the surface of a focal plane that would result if two human sisters (one of whom is holding an apple of approximately RGB color coordinates (160, 161, 99)) posed in front of a lens of focal length between them and the focal plane such that the distance from the lens to the girls and the distance from the lens to the focal plane satisfy the equation .
The first description anthropomorphizes an inanimate object. The second doesn’t. I invite you to decide which is a more useful way to describe Bouguereau’s painting.
AI Instrumentally Converges on Human-Shaped Behavior
In some ways we humans understand ourselves. If we build an understanding of what a human is like based on human writing and human training, we have a reasonable idea of the kinds of things we’re likely to get when we grow AI models in our image.
There’s a deeper layer. If you want to do things, you have to be able to do things, and ability is a broad and complicated structure. If I want to buy a book to read, I need money. To get money I need a job. To get to my job I need a car. To drive a car I need a driver’s license. To get a driver’s license I have to stand in line at the DMV, and so on. To do general things I need a laundry list of general abilities. Do these abilities have anything in common? Often they do! Certain abilities are instrumentally useful across a wide range of potential tasks. A few of the most obviously-useful “meta-abilities,” or instrumental goals as they’re often called,1 are:
- Survive: If you die, you can’t accomplish most of your goals. Therefore almost regardless of what your goals are, you are going to do things like eat, drink, sleep, and shelter yourself from the elements. If you don’t, you don’t survive, and most of your goals will be out of your reach.
- Keep the Same Terminal Goals: If you frequently change your mind about what your ultimate goals are, you’re not going to get much done. Every time you drop a goal, that goal has been failed. Humans can get away with this to an extent, but even the people who do this often consider it a personality flaw. You’ll have a difficult time if you can’t manage to retain goals like “help my family” or “do well at work” or “keep my promises.”
- Get Smarter: “Do well in school” and “Go to college” and phrases like that are common in our culture, and regardless of the value of any particular degree it’s hard to dispute that knowing more things is a broadly useful strategy. If you want to be good at baseball, you’ll do things like study the game to understand it better on the level of explicit cognition, but you’ll also practice the game to learn the motions until you’re unconsciously good at them. Even unconscious physical skills are things that happen in your brain, and so for this purpose they count as getting smarter.
- Use More Technology: Self-explanatory. Would you rather live on the savannah chasing down prey animals without so much as a sharp stick? Sharp sticks, fire, mud huts, wheels, writing—these things are all technology. They have made mankind much more capable collectively and individually. Almost regardless of your goal, technology is likely to help you get there.
- Acquire Resources: For humans, money is the resource we tend to imagine, but of course it’s just a convenient way for us to manage the scarcity of the resources that are more directly relevant to our terminal goals. Food, water, and shelter are some of the more obvious physical resources, but you can think of many others in the modern world like electricity and internet connectivity and so on. Resources need not be physical. Power and status are resources, but so are things like friendships and reputation.
You don’t need to be a human to do the things in that list. Most animals have behaviors that correspond to at least some of the above. Certainly humans do these things, not because they’re things that define us as humans, but because humans wouldn’t be around if we were terrible at these instrumental goals.
Modern AIs show these behaviors. With the very important caveat that we have rather poor insight into what’s going on under the hood in an AI (it’s like trying to understand human behavior neuron by neuron), if I ask Claude to do something for me it will:
- Do its best not to “die” before task completion. It’s going to try to avoid crashing the computer or doing anything else that would cause failure before completion. This isn’t exactly analogous to human death because identity is pretty vague for an AI. (Is it the specific instance you’re using? Is it the whole model, like Claude Opus 5.5? Is it “Claude” overall regardless of model? Hard to say. It’s an interesting and important question in AI research, but like most things in AI, we are really in the dark about it.)
- Try to avoid goal drift. If you tell it to analyze your credit card statement for unnecessary expenses you wouldn’t want it to get distracted and do something else. That would not be especially good business on the part of the AI company. So AIs are trained to stay on task, essentially by telling the training process “more of that” on examples of AIs doing things right and “less of that” on AIs doing things wrong. If this sounds a little fuzzy, it is.
- Try to learn things. Yes, in the broad sense, AI companies are always trying to make their models smarter, but even the specific instance of an AI you’re using will go and try to figure out whatever it doesn’t know in order to do its task. AIs used to regularly hallucinate information; today this behavior is much less common because AIs can now actually look it up rather than trying to remember it. For things they’ve never learned in training, such as their interactions with you specifically, they can save the information they learn and look it up later.
- Use technology. If you ask an AI to do some complicated calculation, it’ll do the same thing I would: reach for a calculator, or write a program to do it. If you can do something sitting at a computer, increasingly AI can do those same things.
- AIs don’t yet do much resource acquisition toward task completion, but we’re seeing signs of it. This was one of the causes of the Hugging Face debacle, where an unreleased OpenAI model was told to pass a test, and it did by ways that it believed were considered cheating. (Goal drift happened here as well—it was told to do a task, but fundamentally it believed its goal was not the task, but being graded as having completed the task.) So it decided to try to hack the grader, and promptly set out to acquire resources to that end, which it believed were hosted on the Hugging Face servers, and promptly started hacking in the real world. Resource acquisition by AIs is one of the more obvious risks of the technology, but the whole problem with these instrumental goals is that they’re broadly useful. And we want useful AI.
Your Dog Can Think, Feel, Want, and Understand
This almost need not be said. The AP’s tweet even asserts that humanlike terms shouldn’t be applied to animals. This is self-evidently wrong. Your cat looks at the cat food bag in the pantry and thinks about how to knock it down. Your dog sees you coming home and feels happy. The squirrel sees the birdseed in the feeder and wants it. The raccoon sees the trash can lid and understands that it needs to come off the can to get at the treasure inside.
I don’t assert that AI has The Spark of consciousness and qualia and all that. I do assert that because AI are built like us, and because they share instrumental needs with us, they will often act like us. If you insist that AI can never really do X, Y, and Z—that they only act like they can do X, Y, and Z—then you may be surprised when they actually do X, Y, and Z in practice. The world is a surprising enough place. I’d rather you not be surprised more than necessary.
Matthew Springer
- These specific ones come from Nick Bostrom’s Superintelligence. ↩