Interview with David Auerbach and Review of 'Meganets'

A lot of people are worried that AI will take over the world, but Auerbach’s new book raises the specter of a different threat: what if we’ve already been conquered by an autonomous intelligence?

David Auerbach is a former software engineer at Google and Microsoft who has become an influential tech writer with his first book, Bitwise, being named one of the top tech books of the year by Popular Mechanics. His latest book, Meganets: How Digital Forces Beyond Our Control Commandeer Our Daily Lives and Inner Realities, recently came out and I was able to talk to him about it.

A lot of people are worried that AI will take over the world, but Auerbach’s new book raises the specter of a different sort of threat: what if we’ve already been conquered by an autonomous intelligence? And what do we do about it? Auerbach describes his concept of meganets, which are networks of humans and machines that create complex feedback loops, like social media, massive multiplayer online games, and cryptocurrencies. He argues that these networks have become semiautonomous in that their creators have lost control over them, including some he built at Microsoft and Google. He’s less concerned with the intelligence of these networks and more worried about the lack of control we have over them, and the large amount of control they have over us: “Ironically, for all the worries about a Skynet-like AI taking over the world, the genuine threat to human self-determination is a meganet-driven system that relies not on superpowerful AI but on the inscrutable collective impulses of groups of humans organized through fairly rote computation” (Meganets, 338).

Not only are these networks out of our control, they’re also causing economic instability, disinformation, and political/social polarization. I’ve written before on the transition from one-to-many to many-to-many communication and what that might look like for authoritarianism. Auerbach’s book is very much an in-depth look at the impact of many-to-many communication on our society and he has some proposals for how to mitigate the problems. Rather than censorship, he wants to make broad based changes to the networks: slow down the exponential growth, reduce the feedback loops of recommendation engines, break up clusters to reduce groupthink, and randomly inject heterodox opinions into the networks.

The book is aimed at a general audience and so it has no mathematics on complexity theory or chaos. A brief introduction to these topics might have made the case more compelling that a collection of independent agents can exhibit self-organizing behavior outside of any individual person’s control. However, the topic is tricky to write about for a general audience: a small amount of math can really limit readers, but without it you have to rely on case studies to illustrate that these feedback loops have become semiautonomous. Luckily for Auerbach, the Internet has gotten up to some real shenanigans the past few years.

Internet Gone Wild

After Venezuela’s 2017 financial collapse, many unemployed workers turned to mining gold in RuneScape — a game similar to World of Warcraft — which caused hyperinflation in the game’s currency, causing other players to call for racial segregation of Venezuelans to other parts of the online world. There are two main takeaways here that are demonstrated over and over throughout the book: first, the feedback between the real world and virtual worlds, and second, the gaming company was ultimately unable to control the hyperinflation of their own toy currency. The people who made the virtual world of RuneScape have lost control of it.

But it’s not just the real world driving chaos in online worlds. In 2021, members of the now famous subreddit WallStreetBets spontaneously organized to drive up the GameStop stock from $17 a share up to $500. A classic example from complexity theory is the economy and the efficient market hypothesis, which is the idea that independent agents collectively price a stock at a value that accurately represents the information available. Auerbach points out that GameStop short squeeze completely defied the efficient market hypothesis. The chaos from the feedback of online networks and the real world have broken fundamental ideas that controlled the behavior of our economy. Like RuneScape, the admins of the real world stock market (the SEC) have publicly acknowledged they have lost control and they have no idea how to get it back.

The transition from one-to-many communication to many-to-many communication is sometimes framed as a transition from elitism to populism, that social media has disrupted old power institutions and transferred power to the people. But Auerbach argues that the power has instead been transferred from elites to a third party: self-organized, semiautonomous forces outside of anyone’s direct control. He writes, “To an elite, top-down organization like the SEC, the wild west of WallStreetBets appears to be chaos, but it is not sheer chaos as much as a new kind of order, albeit one lacking in central controls” (Meganets, 75). You can mechanistically control where a car goes on the road, but you can’t mechanistically drive Reddit. WallStreetBets is going to go where WallStreetBets wants to go. You might call this populism, except that the individuals in the crowd are as helpless to the whims of the mob as the elites.

Interview

A lot of people are worried about AI conquering the world. But if these meganets are truly autonomous or semiautonomous, and our lives are already controlled by them, then in some sense, you’re saying that we’ve already been conquered by an autonomous collective intelligence.

Whether one can count it as an intelligence per say is something of a philosophical debate. The issue is that it’s something that we can’t get our heads around, not even collectively since it’s too big and we’re going to have an issue even assessing whether it’s intelligent or not because we aren’t going to have the tools to perform that sort of assessment. Maybe locally it’s not intelligent enough to count that way, but maybe from a larger scale view it could be seen as intelligent. Because it doesn’t look like SkyNet, because it’s not some unified intelligence, it’s very unclear as to what exactly we’re dealing with. But it’s out of our control.

What distinguishes the more general concept of a complex dynamical system from the specific example of a meganet? To make that question more concrete: Our economy was a complex dynamical system prior to meganets, how are meganets different?

Certainly complexity theory and chaos theory influenced my view of things. As soon as you get some sort of nonlinearity that opens the door to complexity. A meganet is a particular subset, where that complexity meets a certain bar that has an impact at the level of impacting society, a sociological phenomenon like the Industrial Revolution. On top of that, it produced specifically by the interaction of people and machines. So you need the algorithmic and the human component. And that’s what makes it potentially more explosive than even the economy. Now when you had high frequency trading, did that count? I’d say it probably did. But now it’s growing to a scale that’s considerably beyond that. Now you have systems driving massive many-to-many interactions. Now it’s not just a company running an HFT [High Frequency Trading]. Now you have people driving up GameStop without the need for some major institutional player running HFTs.

You argue that tech companies have lost control of their social media networks that they can’t control them even if they wanted to. The distribution of links in social media networks follow distributions with very long tails. These scale-free networks tend to be highly resilient to the deletion of nodes. But that’s because when nodes are removed at random they are almost always ones with low connections. Instead, when hubs are removed, it often crashes the network. For example, when Chicago O’Hare airport goes down, it causes a cascade of failures across the whole North American airport network. So if you were a tech company trying to exert control and suppress a narrative, then you could target the hubs, in this case, the influencers. Censorship will fail to suppress a narrative if it targets average people, but it’ll succeed if you target the influencers.

There is some sort of centrifugal force going on so if you lose a hub you can get some distributed reconstitution. I feel like it may be robust to that sort of attack. I think there’s more censorship resistance than people fear. If you have your sights bent on getting your information distributed on one particular platform you may be vexed, but China has been going at this for decades now and they certainly have not achieved an unqualified success. They have been openly embracing a paternalistic squashing of dissent, but there’s still plenty of dissent out there. I think especially under Xi Jinping’s reign it says something that there’s still a distinct lack of what one might see as top down totalitarianism because even with an army of censors they are still facing a certain leakiness in their degree of control. I’m not saying you can’t engage in some amount of suppression, but I think there’s a fair amount of pushback too. So it’s a concern of mine but I don’t think we’re doomed in that regard of hegemonic suppression. It’s more likely that groups become invisible to one another and they become solidified.

So if these networks can’t be controlled then we don’t need to worry about foreign interference in elections. If we can’t control them then neither can the Russians.

It’s a question of how much you need to worry about it. Is there going to be interference? Yes. How much interference? That’s the million dollar question. Because we still need to quantify what the effects were. We aren’t even going to be able to measure the effects. It’s going to be difficult to get wide-scale effects. But everybody, even the Russians, are going to have trouble figuring out how effective they are. In some ways it’s always been hard to know whether you’ve made a dent or not but that’s going to get much more severe. If you’re trying to take out an electrical grid that’s pretty obvious, but as far as trying to shape the discourse, well the atomization and diaspora-ization of discourse is going to make it hard to know if you’ve made an undisputed impact in the way you could in twentieth century by controlling one arm of propaganda communication.

In your book you wrote, “The will of these systems is greater than that of even the most powerful individual.” How do one of these systems go from a hashtag on Twitter to having the willpower to make something happen in the real world without an army?

Theoretically speaking, if you could get everyone to ban cryptocurrencies or shut down Facebook, you’ve killed the meganet. But we don’t have that level of coordination or unified will and that’s what open the door for these new groupings to exert tremendous amount of control. There’s a tacit degree to which sovereign nations are letting them effect them. Partly because you’d have to literally detach from the modern world in order not to. Nations have no choice unless you want to go the North Korea route. You basically just have the creation of a new large scale power that is not a sovereign nation but does have a greater degree of influence over those entities than anything we’ve ever seen before. The SEC is wringing it’s hands wondering how do you stop people from just causing the price of stocks to just shoot up in the air without any change in fundamentals, but unless you want to run a complete police state, you can’t.

In the mainstream discussion we’re often presented with two options when it comes to free speech versus disinformation. On the one hand you have free speech absolutists with the tradeoff of disinformation and hate speech. On the other hand, censorship with the tradeoff that malicious rulers could use it for authoritarian purposes. I like that you’re presenting a third option. Out of the proposals you made, what do you think is the most important to gain control of these systems?

Slowing it down is probably the most feasible. If you’re looking to restrict viral explosiveness simply slowing it down. That’s probably the easiest thing you can do. Because the issue is now you can have these phenomena happen now and blow up and cause huge problems before anyone has had a chance. But that’s only a partial fix because you can’t do that when it comes to things like stock trading. But I think that’s the best starting point to see how can we at least reduce the degree to which we see unpleasant online phenomena just get completely out of control and turn into something pretty ugly without resorting to ham-fisted censorship mechanisms.

Since these networks are mostly only 15 - 20 years old, where do think things are heading long term?

I think they’ll be more hermetic and self validating. I call them narrative bunkers in which people can very much have a fixed view of the world and intrusions into it will be very threatening because people will have things that are never questioned day to day in their portion of the internet and won’t realize that other people will have completely different understandings of the world.