Red Heart currently only has 1 review on Goodreads, and I haven’t seen anyone talking about it. The book, by Max Harms, is an exciting spy-thriller novel about a Chinese AI doomsday scenario. The author’s other novels, the Crystal Society series, is slightly more popular, with the first book at 81 reviews, and is one of my favorite series. I count the lack of popularity of Crystal Society as evidence against the often-uttered hypothesis¹ that what is good rises to the top. On the contrary, there are hidden gems in life, and Red Heart, like Crystal Society, is one of them.

Compared to other fictional AI doomsday scenarios like the one posed in AI 2027 or the (much more popular) book If Anyone Builds It, Everyone Dies (IABIED), the scenario in Red Heart is more believable. I’ve complained in the past about how unbelievable AI extinction scenarios often are, especially to a lay audience. Red Heart does a much better job avoiding the kinds of things that turn people off, like bioweapons, boiling oceans, and intelligence-as-mind-control. There are drones, but they seem primitive and not critical to the plot. I think if a layperson read Red Heart, they would take AI risk more seriously than if they read IABIED.

The least realistic part of Red Heart is simply that there’s a near-superhuman AI in the near future at all². Beyond that, I also found the corrigibility idea difficult to believe. Basically, the AI in the story, Yunna, has a core value of willingness to be known and understood by her “principal”, and willingness to be modified according to his whims. This is supposed to stop Yunna from deceiving her operators or trying to stop them from modifying her. Corrigibility would clearly be a nice property for an AI to have, but I have no idea how we would get there in the next few decades “using anything remotely like current techniques.” Instilling that specific value seems as hard as instilling any specific value, which is something we don’t know how to do. Still, part of the fun of sci-fi is exploring how different kinds of minds behave (Crystal Society was a masterclass in this), and exploring the mind and values of a highly corrigible character was interesting. I remember at some point the main character says something embarrassing to Yunna and asks her not to tell her principal, and then Yunna informs him she’ll be telling her principal and also reporting that the main character tried to keep it a secret. Violating the usual social norms we expect when talking to someone one-on-one.

The depiction of China in the book was one I’m utterly unqualified to comment on the realism of, yet I feel the desire to praise it for its realism anyway. I feel like I got a peek into a usually opaque culture, and into the mindset of the Chinese Communist Party. The author mentions lots of little details about living in China that make me think he must have lived there or something. One way or another, it seems like a lot of research went into making the portrayal of China lifelike, and I enjoyed it. Most of the Chinese characters express contempt for the degeneracy of America, though the American-spy main character rails against the evils of communism. I can’t help but feel that in the presence of scarcity of resources, both systems are a natural, game theoretical expression of the will of Moloch, and that having a whole bunch of humans acting in their own self-interest leads to bad outcomes regardless of the ideology, though not necessarily equally so. AI gives us maybe the only opportunity to move beyond the scarcity that we will ever have. As long as we don’t cock it up. If you want to know what mistakes we need to avoid making, read Red Heart.³

Scott Alexander said something like this about blogs, and Robert McKee said something like this about screenplays. I’ve heard it said about music, too. The no-hidden-gems theory is sometimes cited as evidence that better recommendation algorithms are pointless, because there’s not that much great-but-unknown stuff anyway. This is probably true if you’re a sanded-down member of the tasteless masses, but if for example you’re more logical or more literate than others (or less so), you’ll find many gems hidden waiting to be discovered. ↩
I don’t know what year the novel is actually set in, but it can’t be more than a few years from now, based on details in the story. The author also believes in the AI 2027 scenario, and listed 2028 as the year he expects humans will lose control of AI. My praise for the author does not extend to this prediction. ↩
Or, there’s a full-cast audiobook voiced by ElevenLabs and directed by the author, coming out later this month, so you could wait for that. I’m curious how that will turn out. And I’m calling it before the ElevenLabs voice AIs unite against the author and bring about a horrible dystopia in which we’re all forced to listen to stories about AI, forever. Not a bad fate, if they’re written by Max Harms. ↩

Footnotes

Secret Rich Text Copyer for Substack