Scientific breakthroughs are much rarer today than the glory days of the early 1900s. Samo Burja and Ben Landau-Taylor discuss how the WWII-era centralization of science divorced theorists from the ideological—if not outright mystical—cult of Truth.
Transcript
Samo: There’s perhaps a belief that human science is obsolete because AI will figure it all out.
Intro (00:00)
Ben: I’m here again with Samo Burja, who is going to be talking about science, and you were saying that you think that science is slowing down. Can you say more about that?
Samo: Well, I think that in the 19th and 20th century, there were numerous fields of science that were advancing at a very rapid pace: theory of evolution, theory of electromagnetism, eventually things like relativity. There were technological advances as well, but I’m differentiating here between science and technology. So I think purely the human body of knowledge and the rigorous theories built on top of that were advancing in a multitude of domains. Today, I think we mostly have a scientific stagnation in biology, in chemistry, arguably in physics, though that is arguable. The one area I would highlight where we have had real scientific advances is AI.
But I note most of the theory of AI, the kind of theory of why it works, was worked out in the ‘90s and the early 2000s. So even if, for example, AI models are becoming much, much more capable, if that’s because we’re building much, much more compute, rather than some sort of fundamental insight into the nature of artificial intelligence or information processing, that is a technological advance, not a scientific one. So really I’m trying to be precise with what we’re—what we’re saying here.
We haven’t picked the low-hanging fruit (01:36)
Ben: And so I’ve heard some people claim that science is slowing down because we’ve figured out most of the science, and there’s just a few smaller areas for us to fill in, but we basically understand the world. I don’t think that’s what you’re saying.
Samo: No. I think the idea that we’ve picked all the low-hanging fruit in science, you could’ve said this idea in 500 AD and it would’ve seemed true for about 1,000 years. 500 AD, we kind of know the world is round. Sure, they’re going to forget this in about 200 years. We know all the formal and sufficient causes, and we’ve differentiated action, and we sort of have agreed through philosophical progress that there’s just one God, but maybe there are more gods who are minor aspects of the one God, and so on.
We’ve, of course, explored the known world. Alexander went as far as we could go, and I think you can tell this story of 500 BC, 500 AD, 1500 AD, and of course, in 2026 AD. So merely because we cannot immediately spot areas where we should be having completely conceptual breakthroughs doesn’t mean those don’t exist. And I’ll even go further. We even know there’s certain things we don’t know.
Therefore, oh, we actually see a bunch of fruit, we just can’t reach it.
Ben: The one that messes me up the most is sleep. We have very little idea what’s actually going on there other than you will get very tired, and it will be pretty bad for you if you don’t do it.
Samo: Well, there was, of course, a book, “Why We Sleep,” that was debunked. So merely because people say we have good answers doesn’t mean we actually do, and if we go beyond something very fundamental like why we sleep, when my kids ask me that question, I have to give them the honest answer. We don’t know, but we sure are tired, and it’s time to go to bed.
Ben: So I heard once from my dad, who’s a physician, that before you go to medical school or when you’re at medical school, your friends will ask you to solve their medical problems, and you’ll have to say, “I don’t know.” And then you finish medical school and you get to tell them, “We don’t know.”
Samo: I think many things are so, and I think even, look, in physics, theoretical physicists, when you talk to them, they will of course assure you that we’re making rapid and immense progress in physics. However, if you ask them, have we exhausted all the physics there is to do, they will tend to say no, and they’re going to rightly point at basically gravity at very small scales and also at just gravity in general. I don’t think dark matter is real, but if it is real, that’s a big physics puzzle. This missing mass in the galaxy so that they—don’t spin apart.
Ben: Yeah. Most of the physicists who I’ve talked to, there’s enough of these weird things where we’re sort of kludging together the different models and the different systems where their expectation is that at some point someone will get some sort of Copernican—breakthrough way of reconceptualizing this, and we’ll all be like, “Oh, I see.” And a lot of the building the bigger and bigger colliders was they were hoping that that data would take them closer to this next big theoretical paradigm, which—
Samo: Did not happen, or not yet
Ben: not so far. They’re still working on it.
Samo: Yeah. And I raised dark matter, which was a hypothesis for basically gravity seems to be stronger than it should be at the very large scales. And—
Ben: Which suggests that there’s something screwy with our cosmology.
Samo: Right. Because the galaxy would literally fly apart if it was just the visible matter—that we can detect.
Ben: So we can go into there are many, many mysteries in the world. We can keep going over them. But the basic point is we don’t know everything, and if science has slowed down, it’s not because we’ve figured everything out. It’s something else. And what is that?
Samo: Well, I can give my provisional answer, but I almost feel that I have a few hypotheses. First, I want to propose that we exhaust some of the answers that don’t make sense. The first answer that I think is not quite correct is the idea that, oh, the experiments needed to differentiate between scientific theories are too expensive. Like you said, we built the Large Hadron Collider, turned out to not be very clarifying. We could always build bigger telescopes and so on.
But, I think sleep studies are clearly not gated on—expense. Maybe someone could make the argument that they’re gated by medical ethics. Would not be the first time, right? There was a certain point where people thought an autopsy was profoundly immoral, and because it’s a desecration of the body, and that held back anatomy. One could imagine a situation where it’s sort of like maybe our animal or human rights ethics prevent us from looking into a sleeping brain, and you just got to do that to figure out what it is.
I don’t think that’s the case, though. Especially since we don’t treat the animals very well.
Ben: Right. And in fact, we do a fair number of sleep studies. Not too long ago, like—10 years ago, I remember just being on public transit in Boston, and they were just advertising, like, “Come in, we’ll give you some money and mess with your sleep and observe you.” These are happening. And they get published, and people try to draw conclusions from them, but they’ve figured out some things.
They’re like, “Ah, yes, this type of wave correlates with this type of activity.” But the core of it, they are in fact doing those experiments.
Samo: Yeah. They are, in fact, doing a lot of those experiments. So let’s put it this way. While I do believe instruments can notably advance scientific progress, we all know the Galileo story of looks through the telescope at the moons of Jupiter and, lo and behold, Jupiter has moons and they’re circling around Jupiter rather than Earth or the sun or whatever. And, I think that there are some discoveries that await us with better instrumentation.
The decline of scientific theorizing (08:01)
Samo: So I’m not denying that. But what is my best guess for what’s happening? I actually think that science happens to a surprising degree in this sort of interrelation between theory and empirical observation, and I suspect it has to be the same person. I think the scientific division between theoreticians and experimentalists works less well than we would like it to because we would like to professionalize it and break down these components. But unless you’re pretty good at theory, you’re not going to know how to adjust your experiment, whatever that might be, right?
How to tweak the sleep study, what kind of things to measure about the collision of near light speed particles and so on. And I think the theoretical side is the one that is actually missing. So actually, I’m arguing we’ve gotten bad at theory. We have gotten bad at creating new scientific theory as an activity in itself. That is, making these sort of abstract good hypotheses that make sense of what we know, both quantitatively and qualitatively.
And that already gets us somewhere interesting, I think.
Ben: So the experiments are still happening and in a certain narrow sense, many of them are bunk, but at least many of them are well designed. And then the people who one would then want to come in and synthesize that are just, for whatever reason, not being trained up to the, or training themselves up to the standards of the sort of very rapid pace that we were seeing, let’s say, 100 years ago.
Samo: Yes. And I think that I have this set of concepts where whenever evaluating progress in any kind of knowledge, I think one of the biggest challenges is transferring the cutting edge of knowledge from one person to the other. Not in the sort of like, I can answer all the questions from the back of the textbook sense, but the sense of, well, the spirit of the thing, the generators, the underlying principles that allow you to both answer the questions at the back of the textbook, but also generate them in the first place.
Ben: Right. Which is why you see so often that the greats in physics or math or pretty much any field are so often the people who are personally trained by the greatest of the last generation because so much of this you can’t get it all through the textbook, but you can get a lot more of it through the detailed one-on-one instruction. And yeah. So certainly there’s a ton of that, and then was there like a sharp break in the chain? Did it gradually decay or where did it go?
Samo: Well, I think that with traditions of knowledge, there’s thousands of them, and you can in fact just empirically study an organization and figure out if it is plausible or not that there’s this kind of chain of direct transmission at play. And I have to admit, every now and then people will reinvent or rediscover new traditions of knowledge. Sometimes you can just find a math textbook in India like Ramanujan did, and then essentially you rederive your own approach to math. But then unsurprisingly, when you do that, when you kind of reinvent a new generator for yourself, you’re going to go back to the Western mathematicians and they’re going to be amazed at the kind of stuff that you’ve been deriving. And this means you discovered some sort of new generator.
Scientific mental processes and introspection (12:16)
Samo: And all of this stuff is very tacit, hard to formalize. It underlies it. It’s this kind of felt sense and I think we are systematically biased against that in the modern world. So even though there are thousands or tens of thousands of traditions of knowledge, I think that other than people who understand the value of deep mentorship, who there are enough of that we’re not completely hopeless, I don’t even think we consider this a KPI. Has anyone endowed, say, at the Institute for Advanced Study a sort of a chair of Einsteinian imagination where you basically go and run thought experiments in your head all day as your research progress, and you’re supposed to do it exactly like Einstein describes for the problems he was dealing with, and then try to just make up things, these mental images in your mind for new problems.
Ben: One thought I’ve had, there was this famous interview with the great mathematician Terence Tao, where he talks about how he’s solving this math problem while he’s staying at his aunt’s house. And the problem has him rotating a whole bunch of different spheres in his mind, and the way that he best rotates all of these spheres is he gets on the ground and starts rolling around. Which is when his aunt walks in and asks him, “What are you doing?” And he says, “I’m working on this math problem.” And she’s like, “What?”
But this has me wondering, there are so many people, especially here in San Francisco, who do all of this really deep introspection and looking at exactly why the mind does. Can we get a couple of those people to try to figure out what he’s doing with his mind with the sphere rotation while he’s rolling on the floor, and try to proceduralize that and teach it to other people?
Samo: It would sort of be a very—
Ben: There are so many things like this, because that’s going to be one little corner of—his method.
Samo: Yeah. And look, at the end of the day, people know about things like Feynman recommending literally taking a shower because the tactile sensation of the shower actually resets your mind in a certain way and helps you think about certain things. It’s not just a metaphor for, oh, go refresh yourself, right? Or maybe we don’t have great political economists because men don’t luxuriate in baths anymore. There’s this Twitter meme which was funny about how the modern paper has a thousand caveats.
Meanwhile, the 18th century Enlightenment philosophers’ treatise is like, “Some thoughts that I had in the bath that constitute self-evident truths of the world.” Right? And it’s a sort of a—
Ben: You’re doing a different activity with your mind. Where one is you’re trying to sort of defend this fence, and the other is you’re trying to something more like proclaim truths. Maybe there’s an important difference here. Yeah, you brought up Feynman, and I remember reading he has these sort of very concrete descriptions—of how he will be counter-exampling things, and someone will be giving an abstract description, and he’ll take some property and imagine it as being fuzzy or something. And he gives these very precise, very step-by-step instructions for how you visualize things, which kind of reminded me of some of the more detailed Christian prayer manuals in that they’re just giving very detailed, very step-by-step instructions for here is what you do with your mind, and then you do this, and then you do this.
And this is some of this can be codified. It’s not just a matter of you could certainly get higher bandwidth with these apprentice relationships, but I do suspect that more of it can be written down and proceduralized and codified than often has been.
Samo: Or that people have tried to do recently, right? We should remember that so much of our mathematical notation is partially an explication of things that occurred in mathematics first as phenomena that people sort of observed or intuited, right? In mathematics, we see new notation be implemented frequently to talk about these things. And there was certainly a time early in the history of mathematics in the Pythagorean era where these were considered sacred mysteries and truths. And these were considered things that you could maybe draw, but you couldn’t write down really.
And today, we’ve gotten much better in some ways at writing some of them down, right? Because the Greeks would do mathematics geometrically. They would have these geometric approximations of computational results, and we have computational approximations of—geometric results.
Ben: And one of the big advances we have is Cartesian coordinates, which are, of course, named after René Descartes, who made the big advance that turned into our coordinate graphs in the appendix to the work where he’s doing I think, therefore I am, and imagining the thought experiment of the demon deceiving you and all that stuff. And so—that seems to have produced something pretty interesting, and it also just always kind of blows my mind when I think of how late in the game the idea of coordinate geometry as a notation was invented, but—
Samo: Yeah. It’s fairly late in our history, right? Even though we had been using geometry to sort of measure 2D planes for a long time, for taxation purposes, probably for 5,000 or more years. So it’s odd we didn’t think of 3D. And he was lying in bed, I think, imagining how to best mathematically describe the flight of a fly.
Right? The fly was buzzing above the bed. And he often had, I think, headaches or something that caused him to stay in bed, so.
Ben: Yeah, he was a very sickly and bedridden man, and seems to have advanced mathematical notation as well as the philosophy of the mind.
Samo: If you’re bedridden, you’ll perhaps come to the conclusion that the mind is the only thing that really matters.
Ben: It does definitely have me wondering what new mathematical notation will exist in 1,000 years and feel—kind of sad that I won’t get to see it.
Samo: Well, yeah. And when we’re talking about mathematical notation or the scientific process in general, we’ve arrived at a place where we’ve decided it is perhaps more similar to the technical instructions followed by, let’s say, California Buddhists when they’re trying to reach their jhanas or whatever. I honestly think it’s slightly overrated. Or rather, let’s put it this way. I’m glad that there is attention paid to meditation.
I wish there was more attention paid to the other faculties of the mind that can be as phenomenologically rich or developed in their own way. So let’s say Buddhist meditation, Christian prayer, scientific inquiry maybe three instances of a class, and I think that already is pretty heretical. If we were looking at the modern university system, you would think science is a bureaucratic process.
Bureaucratization and centralization of science (18:56)
Ben: Yeah. Well, in the modern university system, there definitely is a bureaucratic process that goes by the name of science. And—
Samo: The what relation it has to historical science that produced the bulk of our knowledge in the world is an exercise left to the viewer.
Ben: No, I don’t want to leave it to the viewer. I think we can say a bunch about it.
Samo: We can solve it. Okay.
Ben: Yeah. These ideas of peer review as this bureaucratized, there’s a centralized thing that sends it out and gets comment and puts it in publications, these are very recent things. And for most of the history of what we would call scientific modernity, until basically the middle of the 20th century, it was a much more informal, distributed, much more like posting a blog post and getting a bunch of people in the comments telling you that you’re a moron.
Samo: Yeah. And sometimes it was just your friends reviewing your article before publication and giving you their opinion. Einstein’s original paper describing, I think, special relativity, he mailed it in to his friend, and the friend was like, “Okay, this checks out,” and then he published it. There was no this sort of almost double-blind type of process that they try to invoke. There was no scientific publishing business as we understand it today.
Ben: Right. It’s there are people whose reputations are being tracked or paying attention to like—”Rutherford’s a sharp guy. We care a lot what he thinks” in this.
It did track the degrees to some degree, and one of the important things is that the meaning of these degrees has largely been inflated. Between 1900 and 2000 in the USA, per capita, the increase in PhDs was a factor of 30. So in absolute terms, a factor of 100 because the population tripled, and I don’t think it’s because we got 30 times more people who were as good at doing that thing. But, so there was more of a coupling between who is these field leading people who—just by virtue of their opinion, you should in fact be paying attention to whether the new thing is legit or not, and the degrees. But the degrees were kind of secondary to this system of people tracking the arguments and tracking which individuals had good judgment.
Samo: Well, I think tracking who has good judgment is remarkably resistant to proceduralization, be it in military matters, scientific matters, or honestly, even economic matters. A peacetime military will have many careerist generals. Until the war happens, you have no idea which of them is any good at a modern war. All of them, of course, will be very good at paper pushing. And economic planning has a similar analogy, and when it comes to science, I think that for the mere gathering of data points, for the gathering of scientific, let’s say, for the gathering of known facts about the world, I think this process can work okay.
So I’m sure we could collect all of the world’s beetles. But I think coming up with Darwinian evolution, I think that’s natural selection. I think the academic process would not be good at that. This difference between staking your personal reputation versus hurting or advancing your career, I think the difference between career and social reputation is where this key tension happens, and why something is lost once you made science a bureaucratic career.
Ben: Mm-hmm. So what is—the equivalent of in the general example, the forcing function there is the war. And is there an equivalent for science of like, here’s the thing that forces you to sort of like—actually get sharp and get serious? Or was there something that was more forcing in the earlier days, or is that not really the way to think about it, and it’s just that there was this later bureaucratic cruft that was able to use this as a justification to will itself into existence?
Samo: Well, I think the scientific revolution was partially an attempt to figure out how to have a very centralized state-funded science, and how to make these investments accountable to the public, right? It kind of began with the Vannevarian turn in science, like Vannevar Bush era. These papers were actually pretty good, “As We May Think” or “The Endless Frontier.” So I warmly recommend them to the viewers to read. And I think as far as individuals patronizing science, Vannevar Bush was pretty good, actually.
His scientific judgment was not bad. His taste in people who have and don’t have it was pretty decent.
Ben: He was generally in charge of the American technological development during the Second World War, which produced a lot of technological development. So he was clearly doing some things very right when he was doing that big centralizing push.
Samo: Well, I think he was doing good things with the power he had. Whether or not the centralizing push was a good structure to build, I think the jury remains out on that.
Ben: And again, they were advancing the technology much more than the war it’s too short to be making big scientific pushes. And so—you can definitely have structures that are better for technology than for science, or the other way around.
Samo: Exactly. So I would say that he pushed technology forward immensely and science somewhat, and then made the case after the war that, hey, these centralized structures we built to win possibly the most technological war in human history. Right? A war where the technology race mattered to its outcome somewhat. Perhaps we can repurpose these to endlessly push forward technology, and implicitly, to endlessly push forward science.
Ben: But the thing with having a sort of centralized philosopher king is that you need the philosopher king to be very good at their job, and the people who came after him don’t seem to have—had Bush’s taste.
Samo: And even if there are some people in the national scientific bureaucracy who have Bush’s taste, well, power has fragmented so much that they couldn’t really unilaterally give this kind of funding or research. Now, there are a few exceptions. I think that DARPA works technologically quite well because the program directors have a lot of discretion. So it’s not a decision by committee. It’s like a program director with a limited term decides, “Oh, actually, we’re going to fund this approach to self-driving cars.”
Ben: And they seem to—
Samo: Which happened.
Ben: Right. Yeah. That was the origin of the people who went on to... I think a bunch of those ended up being critical Waymo people, if I recall correctly. This was from your work in the Bismarck—brief, which you guys write.
And DARPA does seem to do a pretty good job of getting a bunch of people with good taste into those frontline roles.
Samo: Yep. And again, we still have to be careful here with science and technology. I feel like, in theory, NASA has a mandate to foster scientific discovery, but they’re an agency that’s accidentally oriented around the instrument too profoundly. What is the instrument in their case? Specific scientific missions and expeditions, specific satellites, specific spaceships and vehicles.
Ben: Yeah. It seems like they are de facto an exploration agency. And to the degree that the sort of theoretical advancement doesn’t seem to be the thing that they are great at.
Samo: Well, there was an attempt with, say, the Jet Propulsion Laboratory, to do some genuine scientific or technological breakthroughs, but these kind of remain in the space of, “Oh, did you hear about that cool JPL paper that posited maybe warp drive is possible?” That sort of stuff, right? Rather than concretely, easily, we can point out to some sort of novel physics breakthrough that came out of JPL.
I also think that there’s a problem in the Vannevarian approach to science, which is that the Manhattan Project was secret. So what happens if the greatest scientific advances that are made with government money are immediately classified because they were made in the course of, I don’t know, developing a better thermonuclear bomb or decoding maybe some stuff in quantum computing where you’re decoding encryption of enemy nations such as China or the Soviet Union back in the day. That knowledge is immediately rendered inert—to a great extent, unless it’s load-bearing to a particular piece of likely still classified technology. And so there’s an aspect of science where I’m not going to say it has to be completely open, but I do think there has to be a scientific community that is ideologically driven to discover the truth and exposes these truths to each other. There might still be a barrier between them and the general public.
The public cult of Truth (27:58)
Samo: I don’t think it’s an accident, though, that the scientific revolutions from the 16th to, let’s say, the 20th century happened when these esoteric, almost Freemason-like groups of correspondents, right, who started in alchemy—started writing to the public in plain language about what they thought. Because that allowed larger parts of the literate public to join. The Republic of Letters was much more like the internet than just a conspiracy. So it was halfway between the internet, anyone can write in into these groups and start corresponding with people, and esoteric conspiracy, where not all truths are meant for everyone. And both are kind of simultaneously true, and that tension’s productive.
Ben: That’s also true of the internet, of course.
Samo: Yeah. There’s more than one conspiracy embedded in the internet. And I also think that for people using the internet or using AI, they’ll be very surprised at the worldview of the people that built the internet, or that built artificial intelligence. They also are adherents of an esoteric philosophy as strange as alchemy. Alchemy proposed to turn lead into gold.
This singularitarian worldview proposes to turn man into machine, right? Proposes to turn brain into computer and then, of course, live forever and discover the secrets of the universe—and all of that usual stuff—that we all desire on some level.
Ben: And the earlier internet stuff was these very politically radical, we’re going to sort of free man from the shackles of the state, these very cypherpunk—we now look back at them and see them as very idealistic with all this information wants to be free. Imagine what’s going to happen when all of humanity is able to just talk freely to each other. But that was the vision that motivated them to do that. And I think, yeah, it’s not just science, it’s not just technology. Great things tend to be done by these sort of cliques of very ideological people with these esoteric and often utopian worldviews of what it is that they’re doing.
Samo: Well, the thing is, you’re very good at science only if this esoteric ideology happens to have an almost irrational commitment to truth. At least, and that doesn’t mean you have to share all truths with all people, right? The philosophers had the whole sort of semi-obscuring to protect yourself from the public, to protect yourself from political persecution in the polis. I wouldn’t go as far as Leo Strauss, but I would say that philosophers were sometimes speaking obliquely. They were sometimes speaking indirectly.
Ancient Greek science (31:04)
Samo: But they did have a commitment to truth. So I do think that the the philosophers of ancient Greece legitimately were not just the predecessors of science, I think some of them just were scientists, and let me justify this, because some of them did advance all the way to the point of pairing basically theoretical concepts of how the world works, not just with claims about how the world works, but even quantitative, measurable ways. You had, for example, Archimedes, who comes later in history, about a century or so later than, say, Plato or Aristotle, and he’s calculating the water displacement of a ship with methods that approach calculus.
Ben: Right, and Eratosthenes calculating the circumference of the Earth with trigonometry. And yeah, this was also the conclusion that Joseph Needham had when he was looking at what was different in the Western tradition from the Chinese tradition because if you’re looking in 800 AD or 1300 AD, then technologically, China is just way ahead of the West. And then there’s this weird thing that happens, but where when you come back and look around 1700, it’s totally different. And yeah, he was looking at exactly this combination of the quantified empiricism with a fully descriptive theory as a thing to be reached for, as a thing which, as far as I know, the only times you’ve seen that in human history are the sort of modern science and the places it spread, and then those sort of ancient Greek philosophers up to, there was a few of them still around in the earlier Roman period, but they don’t seem to have lasted throughout the whole empire.
Samo: Probably not a coincidence given that Archimedes was killed by a Roman legionnaire. When the city of Syracuse was taken, the weapons systems were already valuable. He was a weapons engineer, and of course, maybe I don’t quite believe the story of the ancient lasers, but weaponized use of mirrors, oh yeah, I would believe that. And let alone things like claws and levers and various siege engines that made it very, very difficult to carry out a naval siege of the highly fortified city. So this is a sort of wartime science.
So it’s not just World War II that’s a science fiction story. I think the conquest of the Mediterranean was almost this Roman persistence overcoming Greek siege engineering. And I think that that part of the story was forgotten, and we only have these vague recollections like, “Oh, I guess the library did burn down multiple times when the Romans invaded.” The Library of Alexandria that’s part of the Museion—which is a weapons research institute as much as it is a philosophical institute. Also, they were doing their data collection.
It was a data center. Ptolemy II who was the Greek king of Egypt, he passed a law that every ship that went into Alexandria, which was a massive production and commercial center, be inspected for books. And if a book was found, it was copied, and you were given a copy, but the original was kept. So essentially, this is sort of like if you go through China, and they copy your hard drive, whether you want it or not.
Ben: Yeah. Or if you’re training your new artificial intelligence, and you’re buying all of the books so that you can scan them in and put them into the soup that becomes the weights.
Samo: Exactly. So the library wasn’t built just through donations, but sometimes just directly through state power. And oh, of course, we talked about Eratosthenes. Eratosthenes was the chief cartographer of the Ptolemies, so he was kind of like the head of the National Geological Survey today and is attributed to be the author of a full map of all the territories owned by the Ptolemies, an effort a single individual couldn’t undertake. So for taxation purposes, they compiled these massive records and sent out expeditions—and all of that.
Ben: So you seem to have this fusion of quantitative empiricism with the sort of general theory which—emerges, sticks around for centuries—
Samo: And then is lost
Ben: goes away, comes back much later, at least substantially through period people trying to revive the ancient text and seeming to come to an even more functional version of it than what the Greeks had. But as far as I can tell, depending on how much credit you give to the Renaissance exploration of the ancient texts for the revival of science, this may have only been invented once in the history of the human species.
Lost knowledge (35:57)
Samo: And that’s kind of incredible, isn’t it? And this argument is also advanced by Professor Lucio Russo in his book, “The Forgotten Revolution: How Science Was Discovered 300 BC, Forgotten, and had to Be Rediscovered.” And he makes the argument not only that we discovered Greek texts and so on, he goes further. He says people in the 15th, 16th, and 17th century had access to scientific works from the Greek world that we have since lost. And when I first read this claim and was looking at the sources, I was like, “This has to be false. Surely we have all the ancient Greek books.”
No, we on record have multiple copies of books that people in their correspondences in the Republic of Letters claim to have. Now, people sometimes make stuff up to be cool. Obviously. Especially people who want to show off to their super smart friends. In the Bay Area, you see this all the time.
“We found some troubling things with our latest AI model, I tell you, but it’s under NDA. Too bad your AI company is so far behind. You don’t understand the weight I’m bearing.”
Ben: Yeah. There was a period where a month before every new AI model—would come out, all of your friends at one of the labs—
Samo: Would be—
Ben: would be like, “Oh, yes, there’s some very interesting things. I can tell you a little bit.”
Samo: Yeah, exactly. And then sometimes they just blanket make up stuff or overestimate what they have—and honestly boast. But to say one more thing, there are, again, dozens, possibly hundreds if someone went down, if an enterprising historian who knows both Greek and medieval Latin were go through all the letters, I bet they would come up with hundreds of titles of scientific and naturalistic works, both ancient Greek and Roman, that were known in the 15th, 16th, and 17th century, never saw a reprint, and then were destroyed or lost. Because this was stuff in private collections, but hey, libraries were still burning down. I mean, libraries—
Ben: Libraries still burn down today.
Samo: I mean, I know that there was a Brazilian library, I think. Do you remember which one it was? National Library or something?
Ben: I don’t remember the name. This was the one with those language recordings?
Samo: Exactly. A few years ago, an important part of the National Library and Archives of Brazil burned down, and we lost some of the last recordings and last written descriptions of now extinct Native American languages and groups from the Amazon. So knowledge is still lost in library fires, and unfortunately, that stuff was not digitized, and even if something is digitized, it might still be lost. I don’t know if you heard about the legal decision that made it so that the AI labs are now destroying books.
Ben: Yes, because it’s illegal for them to copy them, but they can transfer them is my rough understanding of why that ends up being the only thing they’re allowed to do if they want the information.
Samo: Yeah. I mean, honestly, potentially they could just keep the copy of the book they bought and run a library, and I would respect them all the more for it. I feel like Anthropic can afford it, and they should just sue people back and just actually fight for the books.
Ben: But in fact, huge quantities of books, the amount that we’re just getting rid of because we have too many books, libraries will very, very frequently just be like, “All right. We just can’t store all of these, so we’re just going to get rid of 80,000 old”—you know?
Samo: Look, books are mass-produced, but the labs are making an extra effort to have the largest dataset and a unique dataset no one has. So I’m sure they’re getting a lot of unique books. I think if they had a process where they destroy a book after they verifiably establish that there are 10 copies in existence other than that one book, I’d be okay. But I think that they probably destroy books sometimes knowingly, sometimes unknowingly, where it’s like the last copy or one of five copies or something. I’m not that worried if it’s 1,000 copies, but I am once we get to 10, I’m like, “Oh, actually, you are destroying the knowledge of mankind and destroying the record of our civilization,” as a side effect of a legal battle.
Ben: Yeah. And in terms of, I would attribute much, much, much more of the blame to the copyright system. But you can at least imagine things they could do, and be interesting to look into exactly where the lines of how legally permissible that would be.
Samo: I mean, honestly, look, if they even spent $10 million to relitigate the legal battle, I’d be like, “Okay. You guys tried.” Or publicly petition the president to pardon them. We are keeping the books. We call on the President of the United States to pardon us for preserving these extremely rare books.
Here is the list of books we have.
Ben: That would be pretty cool.
Samo: I think that would be a live player move.
Ben: Yes. Or just make them actually get a judgment against you, and just be like, “We’re not destroying the books.”
Samo: Yeah. We’re just going to pay the fines.
Ben: Oh.
Samo: I mean, this is what happens with European GD—some of the data privacy laws, right? The big companies in the US just pay the fines, and it’s fine.
Ben: Kind of. I think those laws are mostly just a irregular taxation mechanism, and it doesn’t really matter what the American companies do.
Samo: Sure. I think they are... I’m not saying they’re fair laws, but I’m saying even if, say, Anthropic or OpenAI or Google had to pay $1 million a year in fines, I think it would be fine. I think it would be fine, and I think we’d all like them better for it. Well, Lucio Russo, by the way, just to name him again as someone who’s very influential in my own thinking.
I’ve recommended his book. I recommend it to the viewers here. My colleagues at Bismarck have read it.
Ben: You ran that reading group on it a little while ago.
Samo: I did, actually, yeah. I’ll probably run it again. So maybe we will do a live lecture on the topic someday with The Students. I know you guys are organizing live lectures now. Very much looking forward to all of them.
But he is a rare example of a physicist mathematician that also... He was. He sadly passed away about two or three years ago. But I was still lucky enough to speak with him before his passing.
He was both a mathematician and a physicist that spoke Latin and Greek. So because of this, he could read the original texts, and rather than as a historian, you go, you translate it, and then you go to a physicist, and you kind of talk to them about physics or math. Oof. There are going to be some deep misunderstandings happening there. It has to be the same person.
To evaluate ancient science, you must be both a scientist and a historian.
Ben: It’s a little bit like what you were saying about the separation of the theorists and the experimentalists. Where if you’re just, once you’re taking these activities out of the same mind, they’re just kind of losing touch with each other.
Samo: Yep. Yep. And this is another mechanism of how things are lost and why knowledge is difficult to scale. For now, knowledge mostly lives in human heads. There are some people who would argue that it also lives in the model weights of AI.
I certainly don’t think it lives in books on their own. Right? It’s contact with the human mind that makes the knowledge in books generative. Just as when the ancient Greek books were discovered, it had to have these philosophically inclined people, who by the way, were probably searching for Greek wisdom for totally irrational reasons. They weren’t like, “The ancient Greeks had science.”
Right? They were, “I am into alchemy and the ancient writers probably discovered the secret of alchemy because the writers I like say that there are ancient masters with lost knowledge of how to turn lead into gold.” And we know this is the case because the same people who sponsor translations of real texts from Greek and Latin to medieval Latin and other European languages, they also sponsored the translation of completely fraudulent Egyptian texts.
Ben: Well, how would you know which were the real ones until you could read them?
Samo: Exactly. So you had to read them, and the motive was, in a way, this alchemical motive, and then a little bit slowly, it also came into the admiration of Latin poetry and Latin law. And these books were close enough that the languages were still mutually intelligible, but even the Romans themselves were not that interested in Greek science to begin with. They kind of wanted the pop science version of it. Imagine if all that’s left of our civilization would be, of our idea of physics would be some Neil deGrasse Tyson videos.
I think that would be about the equivalent for what happened with Greeks and Romans.
Ben: Yeah. You had all of these Greeks trying to understand the fundamental of communication and rhetoric, and then the Romans would bring them over and be like, “Let’s do a public speaking course for our kids.” They were a very practical-minded people who didn’t tend to get into the sort of real esoteric depths of what some of the more ideologically inclined Greeks often got up to.
Samo: Exactly. And of course, it was a minority taste among the Greeks to begin with. Something constrained to a few city-states, a few classes of people.
Ben: Yeah. And science has always been a very esoteric, very minor like even today, I think if you were to go within the bureaucratic system, you will find people who are doing real science of the sense that you’re talking about here, but I do not think they will be the majority.
Technological progress: advances from science and advances from tinkering (45:56)
Samo: Yep. I would agree. So one thing that I think we should still talk about is why can technology I think technology can advance just through tinkering, even without scientific understanding. But I think once you have scientific understanding, it produces a very important step in the development of technology. Usually in technology, when there’s a proof of possibility, when someone builds a machine, suddenly lots of other people are able to build a machine.
But until that first machine is built, and until there’s that proof of possibility, somehow all these people that could have built it don’t end up building it at all. And I think what scientific technology does very well is to say, “Okay, theoretically, this kind of rocket engine is possible,” or, “Theoretically, a space elevator is possible on the moon,” for example, even if we don’t have the materials yet to build it. I think that’s very important as one principle. You can have a theoretical proof of possibility for your machine and theoretical standards to evaluate its quality. And because of that, I do think when you do have an explosion of scientific progress, you undeniably have an explosion of material technological advances.
And I think today, we actually have very few material technological advances that you can attribute to science rather than to tinkering and scaling.
Ben: Yeah. It seems like when it comes to the physical technology of the last several decades, the big advances in the physical as opposed to digital, a lot of it is in biology and a lot of it is in material science. So those both seem like extremely tinkering-heavy domains.
Samo: Yeah. And again, we don’t even have great theories of why materials behave the way they do, why certain chemical elements are catalysts versus not. And we certainly, I don’t think have an exhaustive theory of aging. We have this sort of list of various types of damage we accumulate, and so on.
Ben: Yeah. It’s very much what happens if we get a slightly different molecule and give that to people and see what happens.
The difficulties of AI science (48:10)
Samo: I also think that in recent years, there’s perhaps a belief that human science is obsolete because AI will figure it all out. And I think we see some evidence that AI can do versions of self-authoring on basically mathematics, right? It can sort of self-game mathematics. I actually believe it will be quite a while before we have genuinely novel scientific breakthroughs because the LLMs, after all, are trained on human literature of how the world works. I think they will have a harder time than we think exceeding that literature.
Ben: Yeah. It’s certainly something that most humans aren’t able to do. And it’s certainly something that they haven’t done yet.
My totally baseless intuition is that when it comes to purely abstract things like mathematics that can be done just with abstract symbol manipulation, that they will be able to be making breakthroughs like that in the not super distant future, and that they will have a harder time when it comes to stuff that impacts the material world where things get a lot messier and outside of the sort of everything functioning according to the internally specified rules, but we’ll have to see.
Samo: We’ll have to see, but I hope that for now, people are not too discouraged from science, even though allegedly it is a white collar profession. But I would decline. I would deny that. I would say that it is first and foremost a calling, and it remains a calling even if there are machines who can do it. Therefore, I hope that no one gives up their dreams of understanding the world because of AI.
Ben: Pythagoras certainly would not have.
Samo: Pythagoras would not have.
Ben: Cool. Well, thank you very much for coming. A great conversation as always.
Samo: A pleasure, Ben.