AI vs. Humans, in Chess and Mathematics

Life at a Low Rating

In 2020, Netflix released The Queen’s Gambit to a public isolated by COVID and stuck online. Like everyone else, I watched it and thought “I should play some chess.” I already knew the rules from games played as a kid, but functionally I was a total beginner.

It was fun. I wasn’t any good at it, but it was fun. Over a few months playing online I improved from a total piece-hanging dumpster fire to a partial piece-hanging dumpster fire. But that’s ok! Online matchmaking will happily pair you with equally tragic dilettantes and in principle try to keep you at about a 50/50 chance of winning.

For a while I played on Lichess. In the roughly seventeen thousand moves in the database of my games, there are some good moves and some not-so-good moves. Here’s one of my most egregious blunders:

In the position above I’m playing as black. It’s a won position—mate in two. All I had to do was use my queen to take the rook on c1, checking the king. The opponent would have to move their other rook to f1 to block the check. Capture that rook with the queen and it’s checkmate.

Regrettably that’s not what I did. My evaluation of the possible moves was terrible. Of the 38 legal moves in this position, Qxc1 is the only good move. Every other move is garbage. The garbage move I picked was to use my bishop to capture the rook on f2:

What does this do? Well it flips the position from being able to win in two moves to the opponent being able to mate in three, but against me. White queen captures the hanging rook on a8 with check, I move the king out of check to g7 which manages to worsen my position to mate in one, and…

My opponent didn’t see that all they had to do was move Qf8 to win the game. The opponent instead moved Rf1, letting me escape the immediate threat, and after a comedy of errors on both sides I won the game seven moves later. All told, these were truly dreadful performances by both of us.

Chess is a game. If you compete honorably and enjoy yourself, the mere fact that you’re bad makes no moral difference.

If you want a competitive game though, you need to find an opponent as bad as you. For this, it helps to have a number to measure your skill.

Searching for Arpad Elo

In chess, games end in one of three ways. You can win, you can lose, and you can draw. In a tournament a loss earns you 0 points, a draw earns 0.5 points, and a win earns 1 point. Players who are exactly equally matched can expect to earn about half a point per game on average.

The rating system used in chess was created by Hungarian-American physicist Arpad Elo toward the middle of the last century. What matters for our purposes is that chess has numerical ratings. The ratings are called Elo ratings. Mine, at the time of that game, was 919¹. The rating scale is relative. The absolute number doesn’t matter; what matters is the difference in rating between you and your opponent. In principle there’s no top or bottom of the scale. It’s theoretically possible to have a negative rating or a stupendously huge rating, but in practice for the way the system is set up for competitive chess, absolute beginners tend to be in the low hundreds and the best grandmasters are in the 2700ish range.

If you want to know the likely outcome of a game or series of games between two rated players, just subtract their ratings and plug that number into the graph below. (I’ve marked a few specific rating differences for ease of use.)

A 400-point rating difference means an expected score of about 0.91—the better player very likely wins, but an upset stays live at roughly 9:1 odds. That makes 400 points a natural size for a tier:

• 400: Beginners just learning the rules
• 800: Beginners who know the basic ideas like development of the center
• 1200: Regular people who play chess a lot, maybe an average Joe at a chess club
• 1600: Among the better players at a chess club
• 2000: Seriously good, winning club tournaments, a lot of talented streamers like Alexandra Botez
• 2400: People with “Master” in front of their names
• 2800: Former FIDE World Champion Magnus Carlsen

How is a rating determined? Pretty easy. Remember that a rating difference determines your expected score. Each time you play, if you do better than expected, your rating goes up. If you do worse than expected, your rating goes down. So if I lose to streamer gothamchess (FIDE rating ~2400), our ratings are basically unchanged because I was expected to lose. If I win by some miracle, my rating goes up a fair bit and his drops a fair bit.

So how good are the best computers?

The Rules of Chess

Computer chess has a long and fascinating history, but for our purposes the most relevant fact is that IBM’s computer Deep Blue first defeated world champion grandmaster Garry Kasparov in 1997. Humans were able to keep things broadly competitive for a few years after that—Vladimir Kramnik in particular—but eventually it became pointless. The best computers now have nominal ratings a thousand points higher than the top human players, although there’s no human pool of competitors to keep the computer chess Elo pool statistically anchored. The computer ratings are mostly useful for comparisons between computers.

How do they do it?

In principle it’s easy. All you need is a way to evaluate how good a position is. One classic way is to say that a queen is worth nine points, a rook is worth five, a bishop is worth three, and so on. Then you look at every legal move on the board and pick the one that makes your numerical advantage as high as possible. Of course your opponent will respond and try to do the same thing, so you look at a variety of moves and countermoves and pick the move that comes out best against the opponent’s best attempt to do the same thing to you. A computer can do this millions of times per second, and even this very simple strategy will work pretty well if you evaluate it through enough moves and countermoves. More advanced programs might take into account more factors like passed pawns or bishop pairs, but the basic idea is still to maximize your points versus an opponent trying to do the same thing.

Modern computer chess engines use much more sophisticated algorithms, and correspondingly don’t plow through as many raw positions. They can focus on the positions that are more likely to result in a win, and therefore the machine’s thinking is more efficient.

The reason computers can be taught to do this is because chess is a formal concrete system of rules. When you have a finite set of things to check, a computer can just check them. Maybe not completely or perfectly, but well enough to beat me or to beat Magnus Carlsen.

If “formal concrete system of rules” sounds like mathematics in general, you might suspect that computers would be good at math. Until very recently, this was half-true.

The Rules of Mathematics

What’s 456 + 789? Well add the 6 and the 9 to get 15, carry the one… the point is it’s a concrete formal procedure. You can execute that procedure as a person fairly quickly, and a computer can execute it in silicon in a fraction of a microsecond.

What’s 239382190941616118566557150711365358018981181692566042773 × 948800338938251791383943269135443899210667819070908022997? You could theoretically do it. You know the procedure. That would probably take a 40 hour work week, but you could do it. (Assuming you don’t get to the end and realize you forgot to carry a 1 on Tuesday afternoon…) A computer just does the same thing you do, in a fraction of a microsecond. It can do in seconds calculations that a human couldn’t do in a million lifetimes. Its ability to do huge numbers of very simple operations at blinding speed is unmatched.

Computers have been superhuman at this kind of math since the first computers were built. That was the reason computers were built. We needed more numbers crunched than we had time to crunch them by hand.

Procedural Mathematics?

“An even number plus an even number is always an even number.” Can you prove it? This kind of mathematics—asking questions about mathematical relationships in general—is trickier. You may have done proofs in high school geometry or algebra, and if you took serious mathematics courses in college proofs are a large part of what you did. They are the bread and butter of research mathematics. They are less procedural-feeling. How can you number-crunch your way through a statement about an infinite number of numbers? Instead, you say something like this:

• An even number can be written as 2 × n, because the whole definition of an even number is that it’s a multiple of 2.
• So if you have two even numbers, 2n and 2m, their sum is 2n + 2m.
• This can be factored into 2(n + m).
• 2(n + m) is a multiple of 2, and therefore an even number.
• Done.

I mean this is steps, right? They’re steps being expressed in English, but they’re steps that carry out a procedure. Unlike chess where there’s only a handful of legal moves in any given position, there seems to be a limitless number of possible ways each step might go. But once you have those steps, they can be checked for correctness just like you can check whether a chess move is legal.

The way you can check whether your “moves” in the game of math are legal traditionally boiled down to human expertise from the accumulated experience of centuries of mathematics. Eventually mathematicians decided that mathematics was supposed to be fully logical all the way down to its foundations, and the result was a rich variety of results in fields like set theory and type theory and model theory. Any valid mathematical proof can in principle be turned into a list of statements that you can check by turning a crank just like you’d do for long division. Working mathematicians have not traditionally done that because it’s a lot of pain for modest gain, but it was understandably important to them that it could be done if they wanted to.

Arbiters for the Rules of Mathematics

The advent of computers and modern formalization systems is changing that, fast. If you can write a proof in ordinary mathematical language and express it in a computer-checkable formal language like Lean or Coq, that provides a huge boost to the confidence one can have about the result. For instance, here’s the “even plus even equals even” proof in Lean. Like chess, it’s essentially a series of steps. You may not know the rules yet, but you can learn them just like you learned the rules of chess.

def IsEven (a : Int) : Prop := ∃ k : Int, a = 2 * k

theorem even_add_even {a b : Int} (ha : IsEven a) (hb : IsEven b) :
  IsEven (a + b) := by
  obtain ⟨n, hn⟩ := ha
  obtain ⟨m, hm⟩ := hb

  refine ⟨n + m, ?_⟩
  rw [hn, hm, Int.mul_add]

I don’t actually know Lean yet myself, so the above was computer-generated. It’s also more elaborate than necessary because Lean has the “even plus even” proof already baked into the standard library as a one-liner. But this version does follow the structure of the English version of the proof.

“Computer generated?” you may ask. “I can see how a computer could check it, since checking is just a follow-the-steps and turn-the-crank procedure. But how did the computer generate it?”

And that’s the connection to chess. In computer chess, the chess engine is trained by having a neural network play games. If the game is won, great, do more of those kinds of moves. If the game is lost, oh well, do less of those kinds of moves. In computer mathematics, a neural network like ChatGPT or Claude might have started off simply doing next-token-generation on human-written proofs in the mathematical literature, but once it was possible to define “win” or “loss” with reference to a computer-checkable formal system then this could be much more easily automated. If I work at Anthropic and I want to train Claude to be able to prove statements like “odd plus odd equals even”, I don’t have to sit there and grade all its outputs (many or all of which might be wrong). I can just have Lean do the crank-turning.

This isn’t trivial, by the way. There’s no procedural method for converting English into Lean. That itself is an active research area. In fact it’s one of the most popular and productive areas for amateurs and professionals alike. It’s what’s turning the entire historical literature of mathematics from a source of tokens to a full-blown formally verifiable training arena.

No one outside the labs knows exactly the proprietary strategy for training AIs at mathematics. But it’s generally believed that the methods in broad terms involve formal verification during training. Do that enough and the machine will learn to prove things better.

Right now, Claude and ChatGPT have blown past the “high school math test” level and are on the Ph.D. researcher level. It’s probably a position roughly comparable to computer chess in the years just before Deep Blue defeated Kasparov. Both are polishing off sort of “Elo 2000” mathematical research problems right and left. AI mathematics has been progressing much faster than computer chess, and indeed (often with assistance from human mathematicians), AI systems have solved some fairly high-tier problems like the Jacobian conjecture.

Are AI systems going to be able to break into the “grandmaster” level of mathematics and start polishing off Millennium problems? I phrase that as a question out of politeness. As much as I love the human spirit and the conquest of the human mind over the Platonic mysteries, I’ve played Stockfish and been systematically dismantled. I’ve given AI the hardest math problems I myself can do—and I’m a Ph.D. physicist, albeit not in the theoretical or mathematical fields—and I’ve seen AI polish them off in minutes. Yes, three years ago it would fall flat on its face and hallucinate totally wrong answers. Today, it doesn’t. It’s not flawless, but it has fewer flaws than me. It’s the real deal, today. If it doesn’t improve at all starting today, it would still revolutionize mathematical practice. I don’t know what else to tell ya.

But I suspect it won’t stop, in exactly the same way that computer chess didn’t stop after equalling Kasparov. Of course AI mathematics might eventually top out at some better-than-human but still human-comprehensible level, just as Stockfish hasn’t achieved infinite Elo. But I don’t see human mathematics surviving in its current form. Human mathematics may still survive and thrive in much the same way chess has survived and thrived in the Stockfish era, but it will be different. The fact that computer chess and computer mathematics are so closely related under the hood is why. We have a way to generate moves (the LLM) and a way to say whether we’ve won or lost (Lean). Those are the two ingredients for training an AI, and AI training with a verifiable reward function is the most reliable way of building an AI with superhuman performance in that field.

The Future of Mathematics-as-Chess

Fernando Borretti wrote an essay arguing that mathematics as a human endeavor is doomed. Much of it is correct in ways that are hardly arguable today, mere weeks after Claude achieved a new upper bound on the potential off-line zeros of the Riemann zeta function.

But although the methods of computer chess are in a sense why AI is so good at mathematics, his chess analogy misses what chess actually is to the people playing it. I’ll quote it in full:

“Computers are already superhuman at chess, yet we still play chess.”

Unlike most copes, I think this one is interesting. Computers are superhuman chess players, yet we don’t care, and continue playing as normal. Why should mathematics be different?

The main reason, I think, is that chess is self-contained: results from chess don’t help us understand the orbits of the planets or the binding of drugs to protein surfaces. But mathematics, famously, is the great dynamo of science, the best language and method for understanding the world. A machine that can replace a human mathematician, but better and faster and cheaper, is materially useful; a better chess engine is not.

If two computers which are superhuman at chess play against each other, who cares? There is little demand for this, so there is no-one to outcompete. A superhuman mathematician is different.

This mixes two different ways to think about chess. First, there’s chess as the production of good chess moves. Humans are obsolete in producing good chess moves. If you need to solve a chess puzzle or the Riddler is going to kidnap your dog, Stockfish will solve it. But “production of good moves” is not why people care about chess.

Chess is a game. If you compete honorably and enjoy yourself, the mere fact that you’re bad makes no moral difference.

If there had been an industrial application for good chess moves, there would be a computer in the good chess move factory generating them. People would still play chess, because people like chess.

It’s easy enough to imagine mathematics proceeding in much the same way. Perhaps the human theorem-provers in academia will be obsoleted as theorem-provers. In that case, if you need a theorem proven in order to make sure a bridge doesn’t fall down, rely on the computer. If you want to play with mathematics, play with mathematics.

The amateur astronomers figured this out long ago. The amateur with his backyard Schmidt-Cassegrain telescope still makes interesting discoveries and small contributions to astronomical science. No, it’s not competitive with the power of professional research astronomy, but it happens. Even if it didn’t happen, amateur astronomers would still do amateur astronomy. Professional astronomy takes billions of dollars of equipment. Amateur astronomy needs a human eyeball. Humans still look at the stars.

Matthew Springer
2026-08-15