Samo Burja and Wolf Tivy discuss the forces raising our civilization higher vs the forces pulling it down. Is techno-industrial progress made of social and intellectual components which will take it down when they go, or can it weather the decay?
Transcript
Intro (00:00)
Wolf: Welcome back to the Students. I’m here with Samo Burja again. He’s been a longtime colleague of mine. Both of us have maintained quite an interest in the problem of civilizational collapse, the rise and fall of civilizations, kind of looking at these ecosystem dynamics of social technologies, of institutions, and so on. And I think now in an era of simultaneously, we’ve got people really worrying about the collapse of Western civilization.
There’s all kinds of corruption, mass immigration, potential war. All kinds of things happening there that have people scared. On the other hand, we have rapid progress in artificial intelligence, in development, especially in Asia. It looks like there’s almost two worlds to contrast here. One of rapid industrial progress, rapid technological progress, maybe a takeoff technocapital singularity.
On the other hand, maybe a rapid decline into global favela world. These are sort of ideas that people throw around. I don’t know which of these is fully credible, but what I wanted to discuss with you, Samo, is just kind of the prospects for our civilization. Like how are we doing? Are we nearing kind of these crux points where these things are going to be rapidly decided?
Are we facing maybe a long drawn-out decline like the Roman Empire? Or are we going to take off into the stars and not decline at all? So I’d love to get your thoughts on that subject. And Samo, I know you’ve thought about this a lot more than I have and a lot more than most people have, so—so I’d love to hear your thoughts.
Samo: Yeah. First off, it’s a pleasure to be here again in the new studio. It looks excellent. Well, of course, I’ve written for this topic in Palladium Magazine. There’s an old excellent article, titled, “The End of Industrial Society.”
The favelaworld future (02:24)
Samo: And I think people have considered it, but their ideas or the causes of it or what could even cause a civilization as advanced as ours to collapse—I think thinking tends to be fairly muddled. To give an example, it’s completely imaginable for, say, the Western world to be overwhelmed by migration, and I think we should probably discuss migration at a certain point, as a very important topic. Yet global industry could continue. For example, if we move everything to China, many people have noticed that maybe China has a comparative advantage in everything. Maybe the spherical cow invoked in physics.
In economics, I think the spherical cow is that everything in the world is built in a single gigafactory, right? Economics kind of wants to super concentrate it, and then you can imagine this world, when people talk of favela world on Twitter, what they’re imagining is your house kind of has shitty wiring. There’s a lot of crime. You’re not dressed very well, but your home weirdly still has a flat-screen TV made in China. So we could imagine that, right?
We could imagine—the developed world going from the spectrum of Europe as Argentina and America as Brazil and every developed country on some line between these points, right? These formerly developed countries, maybe not quite Brazil, but I think Argentina definitely qualifies as a formerly developed country.
Wolf: Yeah. Brazil is a funny character because it’s simultaneously quite developed in parts and obviously maintains cutting-edge industry even. We’ve all flown in Embraer airplanes—which are built in Brazil, and on the other hand, some truly shocking poverty and decline in other areas. But I guess if we just sort of imagine globalized Brazil, that’s maybe a good model for in the absence of kind of transformative technological change or political change or something that’s maybe this default scenario, right? It kind of just looks like that.
Samo: I think that it is the result of basically political entropy and institutional decline. I think to maintain these sort of first-world conditions, you have to have these unusually effective governments. And then if one has, say, a democratic form of government or a democratic element where, say, voting matters, in that case, oh, actually the population also has to be disciplined. Now it gets a bit different when you start talking historically just in political science terms about a colonial empire, right? Where Britain in 1900 has hundreds of millions of people living in conditions that are much, much worse than today’s conditions, because sort of that’s what the state of development of these colonies often was when acquired, right?
And because of that, I don’t think first-world living conditions in London were impacted in 1900. Today it’s a different story. Maybe that’s a result of air travel. Maybe that’s just a result of democratic politics. Whatever is the case, I don’t think first-world conditions will be that easy to maintain, and it’ll sort of take live players to produce, let’s call them neo-first world conditions.
Wolf: Right. Yeah, there’s this great idea that occasionally gets brought up, which is I think very important, which is that you either have growth or decline, right? This is the way living systems work—whether they be a tree or an animal or a society or any particular model of a society if there isn’t going to be an end of history where things just stabilize and we kind of cruise along in stable first-world conditions. As soon as you have stabilization, that’s actually a form of decline. It’s stagnation and then therefore decline.
And so that’s a very important point, what you’ve just said there, that even to maintain first-world conditions is actually this kind of ongoing struggle to reinvent them, to rebuild first-world conditions against a kind of background of decline. Just by default, right? Never mind anything we’ve particularly done or any of our particular challenges, but just by default to maintain any given living standard is a process of kind of continual struggle for renewal.
Samo: Yeah. At Bismarck Analysis, where we’ve done hundreds of these industrial case studies, another Students guest, Marko Jukic, whose—episode will hopefully be released soon. We’ve had these deep discussions on demographics, and the conclusion is even sort of in the complete absence of, let’s say, a cultural globalization, where the culture of, let’s say, India, Brazil, and Europe, and the United States all converge on the same TikTokified slop. Even in a world where you were to avoid that through a great digital firewall—not to even talk about a demographic firewall, I don’t know, maybe in Ceuta near Spain. Even in the absence of that, just the fertility issues and the structural problems of fewer and fewer able-bodied working people—supporting a larger and larger class of older rentiers.
We talked a little bit about this on the—on the Youth Revolution—
Wolf: Gen Z
Samo: episode. Exactly. So truly, we’re going to have to have unprecedented developments. Now, maybe that’ll be labor-saving technology, but labor-saving technology, I think, in itself, does not resolve the fundamental contradictions of modernity. I think you can go pretty far, all the way to the near obsolescence of man, and all you are doing is increasing demand for labor, and then also, in theory, increasing the rewards of importing cheap labor from everywhere.
You know what I mean? Jevons paradox actually cuts both ways there.
Wolf: Yeah. Well, there’s another—interesting sort of paradox there, especially in a democracy, which is that as you sort of put people out of work or automate things and increase the ability of a smaller and smaller fraction of the population to maintain civilization, if you take that option and allow that population to become very small, you actually increase the voting base for other things. Non-productive—
Samo: For redistribution
Wolf: the non-productive redistributive voting base can become—very large. And so that may be actually this dynamic balance factor against development.
Samo: Well, perhaps we’ll have a deep discussion on democracy—
Wolf: Yes, we should
Samo: another time. But I think now we sort of reached the limits of the favela world analysis, and I may propose—
Competition and accommodation (09:00)
Wolf: Yeah. Let’s talk about the fundamental contradictions of modernity, which is, I think, what you were alluding to there, and that’s where I’m really interested in. But where were you going?
Samo: Oh, totally. Well, first off, we called them neo-first world conditions earlier, and we’ve chatted a little bit about this in some contexts. But the modern mode of life is not sustainable, even if the entire world were to reach first-world living conditions—and first-world modes of political organization, we would actually have a fairly stagnant world. It’s actually stuff from the outside, such as China, that has shocked us into redeveloping economics. Even when we talk about, say, innovation in oil or carbohydrates, it’s shocks from places like Russia, from Iran, that make fracking a godsend rather than an environmental hazard.
And not only fracking, other things like CO2 injection becoming economical, et cetera. It turns out, not only did we step away from a nuclear revolution in 1971, we didn’t even complete the fossil fuel revolution. We could have global energy consumption that’s 5X current energy consumption—just with better and more efficient fossil fuel usage. And I’m pretty sure we could find a way to reduce emissions as well, not eliminate, but reduce. So the fun question always has been, okay, the first world stagnated.
They had a kind of a Cold War that kind of prolonged dynamism a tiny bit. The internet showed up, and then China showed up. And then with these shocks together, we might just get robotics. We might just get very cheap GPUs, maybe cheap enough that if you assemble big enough piles of compute, we solve the problem of a, maybe you would put it artificial man. Because what we’re looking for is not just super ChatGPT.
I think what’s been under discussion is sort of this concept of an artificial will and an agency coming to be. Let’s call it an artificial live player.
Wolf: Yeah. Right. Artificial live player is actually this very interesting possibility, and that’s part of a more general, the hope, I guess, with a lot of this AI stuff or hope from depending on how you count this, but is—
Samo: Hope and threat.
Wolf: Yeah. Hope and threat from AI, is that you kind of get the whole technocapital apparatus, the industrial stack, able to manufacture its own intelligence—and able to manufacture its own labor, right? Rather than relying on this external input of natural human labor that can be consumed within the fires of industry, you produce it internally. And not only that, but it would also have to, I think part of what you’re alluding to is it would also have to produce sort of motive, this kind of squishier will to actually do things. Because I think as we’ve discussed previously, the profit motive alone or the sort of unfolding of the industrial process actually doesn’t seem to drive progress as much as I think people claim, as much as—the economists claim.
You can get long times of stagnation. I was talking to Marko, previous episode, about progress in space, and it just sat there for 50 years. Right? Between von Braun and Elon Musk. And because there was no live player with the motive.
Right? And so the artificial live player is very much this crucial technology that makes a big difference on, does industry manufacture its own reasons for existence in the absence of human input? And I just want to dwell on that notion about these external shocks as one of those motives, right, motives for actually developing.
China’s motive for developing is to escape from the humiliation of being a third-world colonized country. Explicitly, this is what they’re saying. Right? It’s like, “We don’t want to be colonized. We don’t want to be humiliated. We want to escape from this century of humiliation. We want to become powerful. We want to reclaim our place in the world stage.”
So it’s this very reactive—they’re reacting to the supremacy of the West, saying, “We want to catch up. We want to be supreme.” And so that’s driven China. Obviously, they have a ton of very competent people, but politically, that’s what’s driven them to accept the compromises of development. It is this very difficult thing to pull off, and that’s what drives them forward.
Meanwhile, in the West, like you’re saying, after the Cold War, we did stagnate quite a lot. And it’s only the rise of China, it’s like, “Wait a minute, someone’s actually challenging us here. We might be challenged in our way of life.” That’s what starts to get people talking about American dynamism, about reindustrializing America, and all this stuff, this new generation of tech companies and so on. And so it’s an interesting point that you bring up that if you did actually achieve global development, that might just also achieve global stagnation.
Samo: Well, if you achieve global development, it’s the case that not even political decentralization might save you. Because I think that the Soviet Union and the United States reached an accommodation from their own internal stagnation, arguably exactly at the moment of the Apollo-Soyuz mission. Before that, you have the space race. You’re racing, you’re competing, you’re developing weapons systems in space. The militarization of space was underway legally.
I’m not sure if you know, but there was a series of space stations developed on the Salyut model that were manned militarized space stations with primitive, by our standards today, anti-satellite weaponry. I think that’s kind of amazing and cool. And one of my beats has been recently in going around saying that actually maybe the militarization of space is a great thing, and I’m sure that’s been maybe discussed with Marko and others. But to point out that simply because the elites failed to produce the growth necessary to credibly keep up with the other side. Today we narrate the Cold War as like a failing Soviet Union sort of running out of steam and collapsing with America triumphant.
During the Cold War, in the ‘70s, the ‘60s, the perspective is that maybe we’re losing. Actually, look, our economic growth is slowing down. We can’t quite peek over the Iron Curtain and see what’s happening there. And likewise, on the Soviet side, they were freaked out by things like the space shuttle. They thought this was an incredible weapons-grade technology, like the reusable access to space, the ability to drop maybe 50 Marines anywhere in the world within 60 minutes.
That motivated their construction of their derivative Buran space shuttle. We of course know space shuttle economics were garbage. But the Soviets didn’t know the ‘70s and ‘80s when they were receiving this intel. They were expecting America to continue its rapid progress. Now, the only domain where the US, I think, truly was radically technologically outrunning the Soviet Union, I don’t actually think it was nuclear, I don’t think it was space, despite the moon landings, which were a great achievement.
It was basically the production of CPUs. But here the story is already complicated. Even by the late 1980s, the offshoring process of the manufacturing of the CPUs had begun. Right? People began to do, like Tokyo Electron was in ascendance.
And I know you’re a fan of anime, so in the ‘80s and ‘90s when you had these depictions of a hyper-competent, hyper-industrialized, hyper-information-enabled Asian power, they’re talking about Japan. And they’re not wrong. But Japan stopped growing in the ‘90s. So there’s an interesting situation here where we keep imagining the first zeroth world country, and it keeps failing to happen. And to me that suggests that maybe it’s going to fail to happen with China, too.
Getting stuck at 1st world development (17:04)
Wolf: Right. Yeah. Maybe China stays sort of stuck in the middle-income trap, maybe develops up to a Japanese level.
Samo: Wolf, maybe it becomes stuck at the American level of development. Which is actually a disaster—because the American level of development turned out it’s not a sustainable civilization.
Wolf: Yeah, it doesn’t seem to work.
Samo: We are doing our own version of involution. Offshoring mass immigration to replace a failing birth rate. The privatization of public goods, right, where instead of having a nice park, you just have a bigger backyard, and a bigger backyard and the park is like—basically unlivable for families with children. These are all methods—
Wolf: Yeah. It’s involution
Samo: Exactly. It’s involution with Western characteristics.
Wolf: Right. Yeah. This is what we’ve adopted. Yeah.
Samo: So, I’m even saying if China catches up to America, certainly a world-shattering development, and I think likelier than people think, but if it stops afterwards, what a disaster for mankind.
Wolf: Right. And they have not yet set their sights higher than that.
Samo: No. Even their goals for space, they’re not like, “We’re going to colonize space.” Their goal is, “We’re going to go to the moon to prove that we are equal to America.”
Wolf: Right. That’s all that—yeah. And it’s so interesting, this point about this de-escalation between America and the Soviet Union preceding their mutual decline.
Samo: Well, the elites—
Wolf: I’ve often been interested about how much of international politics, there’s nominally 190, whatever, countries in the world, but so much of the politics of those countries and what they actually do is so customary, right? It’s based on these customs of how governments are supposed to act. Obviously, the vast majority of those countries are fake creations of the UN—sort of theater. But some of them are real, and even the real ones, they don’t go all out doing crazy things to increase their birth rate or do eugenics or start nuking people. There’s so much that sort of sits as fears in our political imagination of what a rogue state could do.
Right? And so little of it is actually done. And I find that interesting as a data point towards that you can get these almost global scale kind of gentlemen’s agreements not to go there.
Samo: Well, first off, you get gentlemen’s agreements to not go there. Secondly, let’s put it this way. I think that there is a way, and I’ve written about this, that China sits in America’s Jungian shadow. We hallucinate onto China the things we know we could do but we dread to do. When people imagine secret programs to create genetically engineered super soldiers and how China will inevitably do this.
Wolf: Right. And so of course, those are things we should be doing.
Samo: And we thought of doing. We thought of doing it in the 1960s. You could take any episode of “Star Trek” and get everything, literally the most mass media form of science fiction imaginable, and it’s actually way more subversive and dangerous if you watch it with fresh eyes. Oh, a planet where everyone’s dead from biowarfare. Oh, a eugenicist spaceship from the past where supermen awaken—and threaten to overthrow politics.
Oh, a computer that keeps humanity in stagnation, an aligned supercomputer—that keeps a whole planet in an Amish state of existence. These are actually way better science fiction stories than we give them credit for, and they were mediocre back then. We, at this point, cannot write new science fiction because our fears have overtaken our curiosity, our imagination, and I think that we’re going to have to face these fears—to escape from our current trap. And I do think we are in a global industrial trap.
Wolf: So let—
Samo: China is not going to do that for us.
Wolf: Yeah. So let—
Samo: That would be my claim.
Wolf: Yeah. So let’s talk a little bit more about the nature of that trap. I sort of alluded earlier that I’m interested in this problem of the fundamental contradictions of modernity. I think this is the thing we’ve kind of given an overview here, but nothing—to really credibly challenge, let’s say, the techno-optimist kind of AI escape sort of view of the future. But—
Samo: It’s not the only techno-optimist view. Many economists—
Wolf: No, it’s not the only one. There’s many—such views. But it’s one that I think is particularly interesting to us here in San Francisco, at least to our audience.
Let’s get into the theory problem with modernity. Why is industrial society not just working? And is it even possible for it to actually just work?
How much intelligence does the consumer economy even need? (22:18)
Samo: I think we might hit limits of consumer capitalism. Wolf, are you ready for the anti-singularitarian black pill? I’m going to black pill you.
Wolf: Feed me the black pills, bro.
Samo: I’m going to feed you the black pill for why we’re not going to have ASI, artificial super intelligence recursively self-improving to take over the universe and turn us into paperclips. Okay. The world’s consumer economy, including all the plausible consumer needs of every individual human being on Earth, do not require that much intelligence to fulfill. Consumer demand for intelligence is very low, actually. I think that if you give everyone on Earth an IQ 140 guy, I think they’re pretty much set for life.
Wolf: Yeah. This is this argument that basically like, well, look, we’re not using the intelligence we already have. So is that actually the bottleneck?
Samo: But I think we could even produce enough intelligence to fulfill every material need and every entertainment need that everyone has. Imagine sort of a subcriticality singularity situation—where we pile up, I’m using the metaphor of piling up a lot of uranium or plutonium, getting criticality. Imagine an intelligence explosion that stays subcritical. We sort of pile up all of this compute we need for every conceivable need. It remains this kind of glass cannon.
It’s been called glass cannon of intelligence. It sort of fulfills and sits in our phones and answers all questions that human beings ask in the mode of normal consumer life. The culture
Wolf: And it does all our jobs for us. We keep collecting paychecks, or maybe we get unemployed and have to find new jobs or whatever.
Samo: For example, currently, we’ve plausibly automated a big chunk of math. Maybe that’s fully true, maybe that’s not fully true. Maybe young men stop trying to compete status-wise with each other to be the most brilliant mathematician that gets the Fields Medal. And maybe because of that, we actually lose a culture of mathematics, and we have still people at universities who call themselves mathematicians.
Wolf: It becomes a curiosity within the AI benchmarks or something.
Samo: Totally. And then note, as an immediate side effect, the demand for more advanced mathematical AI goes away because if humans—
Wolf: Right, it’s just an irrelevant benchmark now.
Samo: So what I’m—
Wolf: If no one actually cares
Samo: so what I’m proposing is we might infuse our world with vast amounts of artificial intelligence, coat the surface of the Earth in artificial intelligence, in the words of Jensen Huang. Because he still has a quite consumerist vision of AI, too. We all have GPUs in everything, in our car, in our phone. Why couldn’t your phone run a local instance of Claude Pro or whatever, right? It totally could.
Especially if we design custom hardware. Okay, we do all of that. Every human, every company, we all have massive material abundance. We’re entertaining ourselves, and the AI still doesn’t have agency.
Wolf: Yeah. I guess the—
The singularitarians will lose control of AI (25:16)
Samo: And by the way, the few live players who want to, say, achieve criticality, they just maybe don’t have that much power even in that situation. Even now, here’s my second take. The default trajectory of all major AI labs is that the singularitarians in them will predictably lose power to the non-singularitarians.
Wolf: Interesting. Why do you think that is?
Samo: Well, basically because that happens in these major organizations over time. You have a crazy original ideology. Eventually, succession is failed. Now, the counterargument is maybe the singularity happens so fast that there’s no such thing as succession. Yet, if we look at the key people driving it, again, Jensen and Elon are not like 20.
People like Sam Altman are younger. They’re our age, late 30s. But it’s quite plausible that they get outmaneuvered. For example, Sam Altman, I just mentioned him. He has had so many close calls.
I would still bet on him. He’s one of the live players in the industry. But would it truly shock you if, say, Dario or Sergey or others are just outmaneuvered by essentially bureaucrats at their own companies? In fact, if I mention Sergey, isn’t there a good argument that is precisely what happened at Google? Not today, not 20 years from now, but five or 10 or 20 years ago.
And that’s why OpenAI even exists. It was a circumvention of Google.
Wolf: Right. Yeah. It was Elon sort of responding to not being able to buy DeepMind basically.
Samo: And look, there’s maybe this kind of a black pill on accelerationism is inevitable because I think that human organizations and human-led industry, for now, everything we have seen is a side effect of that. I don’t yet think it’s fully—the loop has not closed on it being fully autocatalytic.
Wolf: Yeah. So this is a very strong and interesting thesis, and I want to challenge it on a few points. Because I think this is where the interesting discussion is to be had.
Samo: And I’ve never heard, by the way, this in any other discussion. I have not heard anyone propose this will happen. Yeah.
Wolf: Yeah. This is something I’ve been worried about. Just before I play devil’s advocate here—I’ve sort of proposed this danger as like, well, what if the AI world, we create AI world where we replace humans in all the critical industrial positions—and AI still has this enormous corruption problem, this enormous political problem, this enormous institutional problem. To put it sort of in a joking way, what if AI also has below replacement fertility? Right.
Samo: No. That is a very interesting position. And I think there are others who worried about other variants, like maybe all civilizations die, and the Fermi paradox exists because everyone retreats into these virtual pleasure worlds—and does not seize the matter and energy of the universe.
Wolf: Well, yeah. I’m very skeptical of these civilizational decline theses as an answer to the Fermi paradox—
Samo: I agree
Wolf: just because usually when things decline, it actually means range expansion for whatever else is around, right?
Samo: I would agree with that. I’m just crediting others who have come up with—similar hypotheses. Yeah.
Wolf: Yeah. But, okay. So let me throw some—
Samo: Let’s go. Yeah
The bull case for AI (28:30)
Wolf: more AI optimist takes at this problem. So, okay, let’s say we stagnate and we provide everyone’s post-scarcity human needs. Nobody needs anything anymore. We all just have our Claudes send each other emails all day. But the sort of transformative story with AI is that real companies doing things like expanding into space, like let’s say SpaceX.
What if SpaceX could effectively 100x its employee productivity or replace all its human employees? Now, I don’t think this is actually going to work, at least not in the short term, but this is sort of the idea that’s being tossed around is what if SpaceX could replace all its human employees with a big data cluster in space, and then, by the way, it’s concerning itself mostly with building another one, colonizing Mars robotically. Elon, or Cyber Elon, or whatever, kind of decides that spreading the light of consciousness through the stars works just as well with the computers doing it as opposed to actual humans. And the idea with AI and the AI sort of takeoff scenario is that it actually just kind of escapes all the human problems. It’s like we sit here and stagnate.
The AI goes and builds the Dyson sphere because it wants more compute, it wants more electricity, it wants to build solar panels out of the moon, whatever kind of scenario we’re imagining. And the only missing component there for that is like does the actual labor replacement technology exist? Can industry manufacture its own human-level generality, competent labor? And the second one is that artificial live player aspect. Can it generate from itself the sort of coherent demand for some kind of development?
And now those are hard problems, but they seem sort of, if AI development continues with the rate of progress that it’s had, it seems plausible that we could get those sorts of things. And many people think quite probable. That’s even the default. So I’m curious kind of what you make of that. Can AI escape out the side while we kind of decline into everyone watching TV?
Samo: This has been known as the breakaway civilization hypothesis, where there are various—
Wolf: Right. It’s AI breakaway civilization.
Samo: Exactly. There’s a variance of it, and I use the intelligence explosion concept. Wolf, I hope I don’t dox us too badly to the audience by saying that we originally met on a website called lesswrong.com.
Wolf: Ah, yes. Our classic shame.
Samo: Our classic shame.
Wolf: We meet in the dark.
Samo: Our deep, deep, deep dark secret.
Wolf: No. It was great. It was great.
Institutional decline vs technological progress (31:38)
Samo: It was great. It was a great forum. It still is a great forum. But I note that I had more hair back then. So it’s sort of like you can keep waiting for the singularity longer than your lifespan, and it can seem totally inevitable that it’s going to happen immediately.
And I sort of feel that institutions currently are declining at a rate faster than a human lifespan. So just for example, consider rule of law, and even basic ideas like, “Oh, we’re going to nationalize AI.” Well, that could easily happen if anti-AI sentiment were to rise rapidly. And the AI guys, they’re just not very good at politics. They didn’t realize you’re supposed to study human politics.
They’re like, “We’ll ask the AI how to solve human politics for us.” Now, maybe this works.
Wolf: Good luck. Yeah.
Samo: No, no. But they say that, right? And maybe this will work with the artificial live player, but by the time you have the artificial live player to answer your political challenges, at that point, you kind of don’t have any political challenges anymore, or at least not for very long. Right? So there’s this long period of vulnerability where they might get literally expropriated—just because of wealth politics.
They might get internally expropriated, where you stay in ownership of your company, but you lose operational control. I would highlight Jack Dorsey is kind of a weirdly based left libertarian, weed-smoking kind of guy. That was not the vibe of Twitter until Elon bought it, now was it? That was not the censorship policy of Twitter.
Wolf: And even Elon’s censorship policy is plausibly not quite controlled by him.
Samo: Let’s put it this way. X.com is to the left of Elon. “Arena” magazine is to the left of—
Wolf: Lonsdale
Samo: is to the left of Joe Lonsdale. We could joke since you are the founder of the magazine and maybe arguably Palladium is to the left of Wolf Tivy. No, I’m kidding. I’m kidding. Left and right, we’re exploding these concepts.
They’re disappearing. They’re dissolving.
Wolf: You’re not wrong.
Samo: Yeah. Well, I would say maybe even to my left, and I’m currently still running it. So, it’s kind of funny where you have to work with the material you have and the social assumptions you have to have. And I think that there’s this question of, well, live players can go very far and can create unique organizations, and just to be clear, as a disclaimer, a lot of the right is so stupid and anti-intellectual, it’s their own—damn fault. The right just actually is allergic to thought.
Yeah. But to go to a step back—
Wolf: Let me ask a real question—
Samo: This is not even a left or right thing
Wolf: about the—
Samo: Yeah, let’s go
Wolf: left, right sort of, or I hate talking about that stuff, too.
Samo: No, no, I agree. Yeah. I regret it
Wolf: how much of the existing, what feels like this rapid AI progress, this rapid—sort of takeoff thing, how much of that is just the current administration sort of having very business-friendly policies? Right? And maybe, I know the Trump administration was supposed to pour $500 billion into Stargate or whatever project. I don’t know if that ever happened, but there’s all these ways that, or it’s maybe coordinated with the trade war with China in various ways. There’s all these ways that just the current political administration, which everyone knows is extremely fragile, might just be kind of this critical component of what seems like the current development environment or industrial environment with respect to AI.
And imagine Neo-Biden 2029.
Samo: Emulated Joe Biden. Joe Biden the Second.
Wolf: Yeah. Or the autopen has gained some new features since last time, right? But it could look very different. I know one of the big reasons that Marc Andreessen, for example, went against the Democrats and flipped over towards Republican was simply because they perceived it to be more like the Republicans were going to be more business friendly. Okay, well, what if we get a Hasan Piker, Zohran Mamdani kind of presidency?
How much of that, can Dario work with that? I know he’s angling to work with that. But is that going to work? And if it’s just like, okay, Dario gets to sort of control the whole picture, we shut everyone else down as reactionaries and wreckers does that itself cause the stagnation of the AI development?
Samo: I don’t—
Wolf: Maybe the pause people get their wish simply by a change in politics. What I’m proposing here is just that the whole picture, the picture of this inevitable AI takeoff, may be predicated on political conditions that could rapidly change. I don’t know. Maybe the Chinese will continue without us. People certainly say that the Chinese are not as AGI-pilled.
They’re mostly interested in just using it to improve the power of the Chinese Communist Party and the quality of life of the Chinese people. Does building recursively self-improving artificial intelligence help with that or not? Maybe they’re not actually that interested.
Technocapital: Historical force or mere ideology? (37:08)
Wolf: There’s this myth, I guess, and maybe I should get Nick Land on here and talk to him about this because he’s sort of this originator of this idea that the kind of technocapital apparatus proceeds without your consent, right? And that it’s going ahead whether you like it or not, and it’s an arms race. You have to participate or the other guy’s going to win. It’s sort of this not optional thing. I don’t think I actually believe that.
I think to a large degree, even the idea of an arms race is just this meme that gets passed around to try to justify development that people already want to do.
Samo: The arms race was always an egghead psyop to give more money and some authority rather than zero authority to eggheads, right? When we’re talking about people—
Wolf: Right, von Neumann and—
Samo: Von Neumann and even von Braun, right? Von Braun is very clear that why he wants to go to space is nothing to do on Axis versus Allies or USA versus USSR. But of course, the threat of larger rockets from the other side, or the promise of a wonder weapon, right? I think currently the reason America is so invested into it, it’s not like the general population is AGI-pilled. It’s not like Wall Street is AGI-pilled.
Wall Street delights in shorting the AGI pilled when they can get away with it. It just happens so infrequently, right? So they celebrate the few cases where they can.
Wolf: Yeah. It’s just been going well recently, right? So everyone has to kind of go along with it. But how much of that is a friendly administration combined with a few live players who really want to do it?
Samo: Well, I would say that, I don’t know whether this concept has been raised on the show. It’s certainly been discussed internally at Bismarck quite a bit. Maybe this is the wonder weapon moment, where we’re facing total economic defeat because of demographics, because of external competition in China, and now we’re pouring all our resources, shooting the metaphorical equivalent of rockets at London, but not rockets at London.
Wolf: It’s AI.
Samo: We’re shooting Claude memos at Beijing.
And Beijing doesn’t seem very worried, actually. And maybe Beijing picks up AI and picks it up as a technology, but I think their belief is, we do not need AI to win. And I think that might be the most—
Wolf: Oh, yeah. I think—
Samo: fundamental reason why they’re not investing into it.
Wolf: Yeah. Right. That’s an interesting point. They don’t have to generate these kind of psychological—
Samo: Economic wonder weapon, right?
Wolf: complexes around it because, yeah, they don’t need an economic wonder weapon, right? Whereas I see the way Americans talk about AI, and it’s like, “If we don’t do AI, we’re totally going to lose. The Chinese are going to defeat us. We have to go all in on AI.” AI has to be—
Samo: No one in China talks this way
Wolf: has to be a wonder weapon. It has to be a nuke. It has to revolutionize the world, otherwise we’re totally screwed, right? And maybe that’s all true, it’s just that that “has to” doesn’t actually compel reality.
Samo: I even think that it would benefit if Americans had a longer view of the future and a longer view of history, right? The fall of the Roman Empire was not even the end of Italian genetics, right? If you look at the countryside of Italy, they basically stayed very Roman, and it was the cities that were filled with Syrians and so on, but the cities had below replacement fertility. So run through the Middle Ages, and suddenly Italians are not that genetically different, actually, from Romans.
Wolf: Yeah. That was a very interesting result, that medieval Italians are more early Roman—than late Romans.
Samo: Exactly. And perhaps that’s a good thing to discuss with someone like Razib Khan or other—very interesting geneticists who have looked at the past and looked at these things. But the reason I’m raising that is that it’s possible for the American empire to fall and America to be saved, and for Americans to persist, right? And then return again to world historical prominence a few centuries from now. While it sounds kind of ridiculous to imagine, I’m going to note that according to current demographics, the majority of Paraguay’s population in 200 years if trends persisted, would be Mennonite.
German speaking. Can you imagine German speaking Paraguay? It might yet happen. There might be other interruptions in history—
Wolf: I know
Samo: but if we had a global technological dark age, is that even such a bad outcome, Wolf?
Wolf: No, I suppose not. I know one of our friends went down there and lived among the Mennonites a little bit. And was disappointed as a philosopher, but I think the philosophers are always disappointed by the people who are actually having kids.
Samo: Look, the philosophers were disappointed when they went among the Russian peasants to find true Christianity. They were disappointed when they went among the Roman peasants to find true old fashioned Roman Republican virtue. The philosophers do not like the life of the raw material of civilization. Even if they might like the fruits of civilization.
Wolf: Yeah. I like to think that I would enjoy Puritan New England, but I might not.
Samo: I think you’d be advocating for socially disruptive technologies about five months in.
Wolf: I would definitely be one of their many—disruptive characters. But back to the matter at hand. Yeah, I still want to give more credit to this AI development view. Certainly, there’s this incredible economic progress on the ability. Again, this fundamental question is can industry manufacture its own intelligence?
And if it can, it almost feels like the rest of this is obsolete, especially if it’s able to sort of circumvent the political system and the various ways that the political system—could impinge on that, as we’ve been discussing.
In the limit, you get sort of actual breakaway civilization where the Ukrainians—maybe the Americans decide that AI becomes unfashionable. We’re a society of artists and bohemians, et cetera. San Francisco is forced to learn a hard lesson from New York about this. We get our own Zohran Mamdani. AI goes away in America.
But maybe in Ukraine, they have big interest in militarized artificial intelligence, and maybe artificial intelligence is real enough that they’re able to kind of keep that going. And now I don’t know if that’s economically a plausible picture, but you could imagine something like that where you get a country that is actively innovating, sort of weaponizing cutting-edge technology, and they achieve more and more sovereignty as a result of that, and they’re able to just insulate themselves and their own sort of economic development, their own industrial development from world affairs. Or maybe it’s China, right? Maybe China takes an interest in some war or—
Samo: Well, I am—
Wolf: you get somewhere around the world, to sort of indicate Nick Land, someone picks up the torch of technocapital development and benefits thereby. And this seems to me the crucial question is how much is technocapital development an ideology versus how much is it actually a historical force?
Samo: Yeah. I think the answer I would give is that it was a world historical force within the context of Anglo-civilization and the world-spanning oceanic trade circulation that it maintained. And much more so than the oceanic trade and circulation when the Portuguese breached containment to both Africa and India and explored and opened up the world, right? They were merchants, but they did not have this technocapital engine take off from their merchant empire. Though perhaps, of course, they provided a lot of the foundations for the British, but there was nothing inevitable about the English and the French having their own world war, right?
And their own competition that drove the English to the parts of the world and connected global pools of material capital with intellectual capital at home and the production capital that they had built. Right? Cheap wool from India going to Scotland to be spun into clothing and so on. That sort of dynamic. The thing we think of as globalization, actually.
I do think that was begun by Anglo-civilization, the British—
Wolf: Right. Globalization is just the British Empire.
Samo: Right. And then I think Nick Land would even agree with this, except he would say—
Wolf: Yeah. No, of course
Samo: that it’s a rupture from history, and I would say that, well, I don’t think it outlives the civilization that birthed it.
Wolf: Well, I think actually that’s one of Land’s—
Samo: So I’m not even saying it doesn’t birth from history. I’m just saying that—if the civilization dies, the ideology dies, and then what you end up doing with ocean-going ships or factories... Look, we could look at modern England. What happened to those factories? The ideology changed, and the factories went away.
And I don’t think it’s just efficient capital allocation. I bet even Nick Land, if we were to have a three-way debate, would agree that there was capital destruction that was economically irrational that happened in the deindustrialization of, let’s say, Britain or America’s Great Lakes area.
Wolf: Yeah. First of all, speaking for myself, I think that’s very true. I think the ideological changes that happened in Britain in the 20th century were very self-destructive, and some of it deliberately. But one of Nick Land’s—
Samo: So let’s say Nick Land’s machine god did not like that.
Wolf: Yeah. But one of the, I mean, to give credit to Nick Land. Not to have the conversation with him without him present, we should get him present.
Samo: No, we just have read so much of him. Yeah.
Wolf: He is very important. We’ve read him a lot. We’ve discussed with him a lot. But one of his most interesting threads in recent years is precisely this, that how much is the technocapital sort of capitalist unfolding of history just a feature of Anglo-civilization? He’s been thinking about that.
And he’s been moving more towards the side of this might just be this kind of technological development of the type we have right now is just an outgrowth of English influence in the world. And it sort of abstracted through many layers now of the Chinese kind of picking up our various memes and industrial techniques and trade technologies and market organization and all this. But notably, they wear English suits in -
Samo: They do
Wolf: China, right?
Samo: They do.
Wolf: There is some credibility to that hypothesis. But this seems to me like this crucial question.
Samo: Well, I would—
Wolf: I feel very wary of dismissing AI development. I think AI development and generally technological development, computer development, computerization, there’s this very strong underlying trend for the last 500 years of just industrial takeoff. And so I’m very wary of dismissing it as, oh, it’s a passing trend. We’re going to change ideology, or this thing’s going to decay and it’s going to go away. But it nonetheless seems like this crucial question is are we actually going to see that thing continue as it does look like we’re kind of onto this decline track with respect to what seemed to be the civilizational fundamentals underpinning it.
So, this is this question I really am very interested in, and obviously we’ve been exploring here. But just to reiterate, that I think is this heart of the matter that I want to get to the bottom of.
The most technologically transformative war in history (49:10)
Samo: And I think the question of criticality perhaps to this concept of the artificial live player, as I’ve termed it. And I’m sure—
Wolf: It’s a good concept.
Samo: It’s a good concept. Others have talked about it. But arguably the term AGI used to just mean this. But I feel like the AI industry waters down these terms over and over again.
Wolf: AGI’s been locked down to OpenAI makes 100 billion in revenue or something. It’s become this extremely crude measure, or it’s like it can answer your email or something, therefore it’s AGI.
Samo: But the euphemism treadmill will keep going, and given that we’re in Silicon Valley, I wouldn’t be surprised if an AI company is going to use, “We’re making the artificial live player,” as part of their marketing, and then we’ll need to retreat to a new euphemism.
Wolf: Yeah. We’ll have to come up with some new idea. Yeah. It’s not real AGI until it thinks it’s Napoleon.
Samo: Et cetera. Exactly. But, no, I’m not dismissing AI at all. I think it’s the most important technological development in my life—and in my lifespan, and arguably the most important technological development of the last 1,000 years, I think plausibly.
Wolf: If it fully plays out. I don’t think we’ve seen that yet, but if it plays out.
Samo: I would say it is possible for it to play out enough to be the most important thing in the last 1,000 years—and still fail to achieve ASI, right? It might be the steam engine of intelligence, which sounds comforting, but actually it’s kind of disappointing, right?
Wolf: Right. Because the steam engine was the steam engine of muscle, and it’s like, okay, did the steam engine actually... It replaced muscle power in the economy.
Samo: In many parts, but—
Wolf: But theorists of automation in the 19th century, notably Marx—and characters like this, they imagined a much more comprehensive social transformation as a result of industry than actually happened.
Samo: I think we are pinning a lot of our hopes on a better society and a better future on this transformation being guaranteed. And my position would be, and when I say we, I mean we as a civilization, not us at this table, because we’re having these discussions in the first place, and this is one of the only venues where such discussions happen. Believe it or not, the audience might not believe it, but there’s no secret room with better conversations. It’s actually terrible.
Wolf: That’s why we’re recording them.
Samo: That’s why we’re recording them, right? And—
Wolf: Because otherwise they weren’t being recorded.
Samo: If they are, please invite us, right? At Bismarck and Palladium and so on, over the years we’ve striven to find the best conversations about the future of our civilization, and we have to have them, because if we don’t have them—
Wolf: Yeah. No, these are the most crucial matters.
Samo: Exactly. Now, I sort of think that the industrial build-out of China is baked in. That does not mean America loses. America might revitalize under the leadership of live players. I am—
Wolf: Or we go to a sort of a more historically normal circumstance where China’s the economic center of the world, and various Western barbarians are—
Samo: Occasionally militarily innovative
Wolf: doing militarily interesting things.
Samo: Yeah. Your point on Ukraine and Russia was actually a good one. If there is a frontier of the Western world, it is a classic Western... Let’s put it this way. There’s a plausible case to be made that the last country, the last outpost of Western civilization, is Ukraine, because Ukraine is the only Western-aligned country using classical Western military, industrial, and scientific means to wage a total war.
And then if you’re extra spicy, you would say, “Oh, but Russia is also civilizationally a Western country.” And if you look at the Russia-Ukraine War, deeply tragic, of course.
Wolf: It’s terrible.
Samo: They’re destroying the best of their biology—
Wolf: It’s terrible
Samo: but the generative aspect is also there. The war has improved—the health of both states, right? And both states, let’s put it this way. The US has a space force. This is excellent.
Well, Ukraine and Russia have a drone force. Both have vertices of command where the entire bureaucratic structure is committed to the automation and roboticization of warfare. There’s not a single branch of the US military committed to this—
Wolf: Interesting. That’s a good point
Samo: and there’s no credible project, and because you know that I’m a big believer in studying bureaucracies and these sort of institutions, I think as soon as you call from the ether an institution with a mission, if it’s functional, that mission advances rapidly. Let me drop another bomb. Hopefully not literally. It is plausible that the Russia-Ukraine War, if it continues for two or three more years, might become more technologically transformative than World War II.
Wolf: Within two or three years. Interesting.
Samo: Within two or three years, if the sort of, let’s call it soft singularity hypothesis is true—because then the first fully automated, fully AI-controlled militaries would be fielded in Ukraine and Russia.
Wolf: Right. Yeah. And that’s sort of the scenario I’m alluding to, right? Is like if AI in fact is this incredibly powerful technology, not just because we’ve hyped it up as such or imagine it maybe becomes such, but because it actually is such, as proven on the battlefield—in terms of paying for itself and whatnot, in terms of state security.
Samo: I, of course, believe in peace and would prefer peace.
Wolf: Of course we all prefer peace.
Samo: I’m not even, let’s say, a soft singularitarian. I would say I’m a very friendly to the singularity skeptic, right? So I don’t actually think it will quite get there, but it’s certainly the most technologically transformative war of the last 60 years. Right? So everything since World War II, arguably less transformative than this war.
Because we’re eliminating the soldier. The soldier is being replaced, and the soldier is not just a military role, it’s a political role. And perhaps robot soldiers are a whole episode onto themselves and their consequences.
Can the industrial escape political decline? (55:21)
Wolf: We should definitely have that conversation. But yeah, I’m trying to think where we should go from here. But again, I guess to reiterate this crucial question, right? Is does AI actually result in an industrial stack somewhere in the world able to escape political decline? And I don’t want to also hype up political decline sort of more than—
Samo: Excessively, yeah
Wolf: more than it’s warranted, right?
Samo: It’s easy to exaggerate it.
Wolf: It’s like we’re talking in these apocalyptic terms like, “Oh, yeah, it’s all baked in. We’re going to end up in late Roman collapse.” But I think it’s a real possibility. It seems plausible based on the trajectory of the world. And what I’m really trying to do is kind of test the strength of these hypotheses against each other, right?
It’s like, does the strongest version of civilizational collapse defeat a strong version of AI? Does it defeat a weak version of AI? Does the weak version of AI defeat a strong version of civilizational collapse?
The interaction between these two hypotheses seems to me to be one of the big underlying stories of world history right now. And—
Samo: I would agree
Wolf: And so that’s why I’m yeah, trying to explore this topic in this way.
Samo: Well, what I would say is, I think the strong, harsh technological collapse that does historically occasionally happen, and I can make the case for that, we can debate it further or not. I think that does defeat any, let’s call it subcritical intelligence explosion, right? Because it stops the accumulation of compute or the accumulation of insights as to how to apply the compute to the artificial live player prematurely. And because of that, again, we already see what a collapse of an industrial society looks like. When the Soviet Union collapsed, sure, consumer living standards slowly increased after a pretty big shock because they were basically importing consumer products from elsewhere.
But the military and technological capacities of let’s call it the Russian state, continued to decline for 20 or 30 years. And I mentioned earlier the Soviet Space Shuttle, the Buran spaceship, right? We use that as a picture to illustrate my article on industrial decline, this sort of picture of the collapsed hangar over it. Well, imagine that there is 30 years of global GDP contraction because of, say, global aging. And as a result, the richest countries in the world get a little bit poorer every year, and the poorest countries in the world, for the most part, do not have their shit together enough to go and do the same thing China did.
So you might see a world where actually slowly there’s a decline in demand, and then eventually further AI build-outs do not make sense because there’s no capital to raise, and there is no reason then to continue the GPU build-out. And we end up with a situation where maybe every phone has a GPU, and maybe every car has GPUs, right? Maybe every car that’s self-driving has GPUs, and maybe your home has a few robots here and there, and they have GPUs. But we’re only talking actually about 10 times one order of magnitude more GPUs in existence than today, or maybe two orders of magnitude more.
Wolf: Yeah, I think it’s something like two orders of magnitude, yeah.
Samo: Perhaps, yeah. Perhaps two orders of magnitude more, and maybe that’s just not enough. And then you wait 30 years, and then the industry slowly adjusts to being like every other industry, an S curve. So what then happens is we stop climbing the escalating curve of compute that seemed inevitable. But climbing the curve of energy seemed inevitable in 1900, as you well know.
And we stepped away from that curve.
Wolf: And then in the early 1970s, we just stopped.
Samo: Exactly. And I think we stopped not because we even failed to jump to nuclear, I think we just stopped with just oil and coal and natural gas, right?
Wolf: Yeah. That’s a very interesting historical analogy here, right? It’s not that we made it all go away, though countries like Germany—eventually attempted this, but—
Samo: And might yet succeed.
Wolf: Might yet succeed, unfortunately, at doing the Morgenthau Plan on themselves. But you would think energy, the most important critical commodity for any economic development—obviously incredibly important to the power of a state, incredibly important to the returns of capital, incredibly important for standard of living. You would think everyone’s aligned on more energy and all the economic fundamentals are there for sort of capital development to just continue to return more and cheaper and better energy. And yet, a political sea change, 1960s, 1970s, and boom. Energy use flat-lined in the United States and across the West.
That’s fascinating. That can happen to energy, something as critical as energy. It was not like we were bottlenecked on intelligence or something, right? We were probably smarter then than we are now. But—
Samo: Global IQs were probably higher.
Wolf: Exactly. But But nonetheless, we stopped. And can we get the same thing with AI? It seems totally plausible, right? And that’s from the perspective of development, kind of this horror scenario, right?
It’s like the technological development, industrial development actually sits on this fickle basis of political development, political winds. And I guess I want to get back then to this crucial question of why does that sort of thing happen? What is it about civilization that can occasionally just stop like that? Or is it industrial civilization? Or are there particular contradictions within industrial civilization that do this particularly?
Is it about the exhaustion of the human capital? Is it about the growth of parasite classes? Is it about the loss of live players after setting up the initial institutions? What is it that causes that kind of stagnation and how are our conditions doing now against those measures?
Samo: First, I want to add the caveat that while we have used the term political in this sense, I wouldn’t say these phenomena are strictly political. Especially in our society, which is very politically centralized. A lot of them are top-down diktats, right? These are directives given, instructions given. And in all societies, in all times and places, it is true that it is only a very small intellectual and political and economic elite that decides things, and that either has the means through which to make their civilization prosper or to wreck it, no matter what the wishes are of the majority.
Now, having said that, I do think that there are sometimes also developments that are not well described as political. There are cultural developments, there are economic developments. Honestly, perhaps one way to think about it is that there was a coordination issue with industrial civilization from the very start. Where if you look at it, this whole relationship between factory owner and human labor, the principal-agent problem—known in economics is actually just the alignment problem in disguise. And in a way, the 19th century found no solution to it except this halfway compromise that sort of doesn’t work either for capital or labor 150 years later.
But maybe it worked, let’s say, from 1950 to about the year 2000, right? Or maybe optimistically for a country like Sweden, let’s say from 1930, because Sweden did not have a political blow-up like other countries, all the way to 2030, where for now, still no deep collapse, but we’ll see after 2030 if social democracy outlives 100 years. Yeah.
Wolf: But okay, so there’s various factors that it could have been, but what—
Samo: I’m saying this is perhaps some political or—basis or the foundations of society.
Exhausting human capital (1:04:17)
Wolf: There’s this sub-political, and to sort of give some flavor to what that could be, why it exhausts itself in that way. One thing that I’ve sort of looked at over time that’s jumped out to me as a plausible factor here is just the way industry is incentivized to scoop up human capital and use it to advance industry, or not just industry, but all the sort of powerful institutions of society. But there’s no modern, powerful institution whose job it is to produce human capital. Or rather, there is, there’s a whole education, et cetera, but I mean, whose incentive it is to produce human capital. I think the education systems broadly are badly incentivized.
They’re mostly engaged in the game of scooping up and consuming human capital just like everyone else. So there’s, let’s say, virtues that preexist industrial society or come from living in some fundamental lifestyle. Often you have these stories of farmers’ sons going off and creating whole industries or being fantastic workers or something. Well, it’s like, why is that a farmer’s son and not an nth generation tradesman? Or why is it not an industrialist’s son or an engineer’s son?
Isn’t that interesting, right? That the stereotype is often that the freshest human capital, the least industrialized human capital is the best at industry. And so one factor, this is very non-political, is just this consumption of human capital. There actually might just be a non-replacement consumption of human capital, even independent of birth rates and so on. But just something about the way people live or the way people are organized.
I think this is one of the hypotheses that we explored in your article, “The End of Industrial Society,” that there is just this exhaustion that happens at a cultural level in terms of the way people actually live and their actual virtues.
Samo: And it could be partially the result from the rationalization and concentration of knowledge, right? You have a great article, “Don’t Learn Values from Society.” Yet industrial society is predicated on everyone learning values from society.
Wolf: Right, which means that it’s opened up to all these vulnerabilities in terms of—
Samo: And exploitations
Wolf: exploitations and corruptions. Yeah, that’s an interesting point. Yeah, so there’s these sub-political elements. Do we think there are also political elements in terms of, I guess I would use political in maybe a bit of a broader sense, encompassing also things like corruption, coalitions of corruption, or just sort of broadly changes among the incentives—of the elite or things like that.
Forgetting how to do science (1:07:10)
Samo: I also will propose I tend to more use the civilizational or institutional language, though a lot of that—is political, like by easily half of it. One hypothesis I’ve had for a very long time is that there was an intellectual apocalypse around 1918, and this is different than Ben Landau-Taylor’s idea of Western elites becoming demoralized. I have had and have been exploring the possibility that we have forgotten how to do science, which sounds absurd if said out—Clearly, our scientific understanding of the world is superior today—
Wolf: We’re publishing more papers than ever.
Samo: Exactly. Well, first off, yes, we can make the differentiation between academia and science, and that’s worth multiple episodes with multiple guests in itself. So I’ll table that for now. But there’s an even crazier idea, which is that for most of human history, we were slowly accumulating what we can describe as knowledge about the world, even scientific knowledge about the world, occasionally losing it, occasionally regaining it, but we weren’t really pursuing a scientific process, and only a few times and places can lay claim to something like a scientific revolution. Maybe like Hellenistic Greece, maybe like 18th century England, maybe 19th century Germany.
It’s possible we had sort of slow progress in science, a scientific revolution, and then it stopped, and we continue, now that it’s stopped, to slowly accumulate knowledge through bureaucratic means or chance discovery, and this fake, what if all of academic science is like a cargo cult of the miracle of science that was done by crazy mystics in the 18th and 19th century?
Wolf: Ooh.
Samo: People who had truly bizarre ideas and would refactor their own minds and would, through a reputation system that we cannot formalize, gauge who was in better touch with reality—and therefore unlock new secrets of the universe. I think that’s quite possible because we have had, say, a stagnation in theoretical physics. We’ve had a stagnation in computer science. Wolf, most of these scientific things that are happening, because I will differentiate between AI science and engineering, most of AI science was probably solved in Moscow University in 1980, right? And the parts that weren’t were like, well, the transformers paper, sort of.
Right?
Wolf: Yeah, there’s been a lot of kind of elaborations of techniques that were hypothesized.
Samo: But we did not have the material means to implement.
Wolf: Proving that they actually work—and so on.
Samo: Yeah, but what if, for example, there is a scientific breakthrough needed in AI, and no amount of compute will actually produce that breakthrough for us? I note that Wikipedia did not produce 1,000 Einsteins. Now, if I say it that way, everyone’s like, “What do you mean?” Well, but science fiction writers in the 1980s, when they imagined something like Wikipedia, they imagined 1,000 Einsteins would emerge from Eastern Europe, from Africa, from India, from the backwaters of the West.
Wolf: Right. Because suddenly everyone has access to this knowledge.
Samo: To all the world’s knowledge, right. And isn’t current version of ChatGPT that answers any question that any human has asked—
Wolf: It’s even more accessible
Samo: it’s hyper Wikipedia. It’s hyper Wikipedia, right? But what if it has just as disappointing a result, and actually, there’s some sort of psychic lock that prevents most human beings from actually thinking new thoughts?
Wolf: Mm-hmm. Yeah, or specialized training in a particular environment—
Samo: Is needed to differentiate good or bad thoughts, yeah
Wolf: to crack out of conventionality.
Samo: Or maybe the problem isn’t even individual genius. Maybe we actually have 100 Einsteins, but they can’t talk to each other and cannot direct resources to each other and cannot share notes, right? Maybe, for example, there was this one point where we had the scientific issue where you were the editor-in-chief in Palladium and we had some article contributions on science needs sovereigns. There are political preconditions—for science, but also there are social preconditions for science. Things like the idea of an intellectual friendship as the basis of the discovery of truth, because truth is awfully socially and politically and economically inconvenient—actually.
But you kind of need a prioritization of the discovery of truth. And one of the very few ways in which you can do that is to have some sort of emotional bond to balance out the equally productive emotional bond of male competitiveness, right? Which also can be directed to truth. And I think with those two, maybe that’s what gets you a scientific community. Right?
And maybe we have all the Einsteins, but they’re subcritical, and no—AI glue between them can fix that.
Wolf: Right. So basically, you’re hypothesizing that there could be these sort of social conditions for an industrial revolution or an industrial transformation that are kind of almost a one-time happening in history where you assemble the right materials, boom, you get an industrial—
Samo: Well, partially
Wolf: scientific revolution—and then things go back to sort of stagnation as usual.
Samo: I was sort of hypothesizing a slightly different thing. It’s like, what if new science is needed for our industrial breakthroughs? Now, we already said we already stopped way before we needed even new science, right? We stopped with nuclear. We didn’t do it.
We did, like, 10% of it. We did like 60% of what we could have done with fossil fuels, et cetera. But I don’t know. Maybe you needed constant scientific breakthroughs opening new domains to keep old domains growing, and let me substantiate that. I think without computer science, there would not be compute.
And without the increased demand for compute, we wouldn’t have had this sudden wave of new energy build-outs, right? AI, despite political anti-energy stuff, it pushed us demand open up by what is basically a new branch of science birthing a new branch of engineering, pushed us further on old branches of engineering, right? And might even yet push us further on old branches of science, not through AI discovering new things, but you really have to—
Wolf: Just because it’s an open frontier.
Samo: Dude, you just need to, even to manufacture the chips we’re manufacturing, borderline you have to understand new things about light. Or new things about how to place atoms. It’s kind of insane how good that is. Maybe there’s a guy hidden in ASML who doesn’t even know it, but has actually new theory of photons developed and practiced in an engineer way, and there’s no way to recognize that or crack ASML open or crack open whatever company’s—doing work on mirrors, and to distribute that to the scientific community.
Wolf: Mm-hmm. Yeah. So we could require new breakthroughs, but we just don’t have the social—
Samo: Technology to ingest them, even if there’s economic demand. But I genuinely think scientific discovery is pre-economic. And you can go really, really far without any classical industrial revolution in a scientific revolution. And I think the ancient Greeks basically did a real version of it. I actually am willing to—
Wolf: We should have that conversation at some point as well.
Samo: Awesome. Yeah.
Wolf: Yeah. Well, why don’t we leave it here? I think we’ve talked around a very important set of questions here for quite a while, and to leave it in conclusion, I think there’s this crucial question of AI looks like it’s incredibly powerful. It looks like we’re in the midst of this kind of technocapital takeoff. How much of that is actually predicated on fragile social conditions that—might shift under it at any time?
There’s also what looks like, at least in the social realm, in the political realm, kind of ongoing stagnation and decline, various reasons to believe that we don’t necessarily have what it takes to undergo another sort of industrial revolution or scientific revolution. So the implied question there of how these things square with each other, I don’t think we’ve answered it, but certainly, we’ve explored it here, and I’d love to continue having conversations about that. But that seems to me, again, to be one of these crucial questions of future history is what exactly is going on with these trends in the early 20th century, or sorry, early 21st century. So Samo, thank you so much for coming on. I think this has been a lot of fun.
Samo: Yeah. Pleasure.