The internet can feel like a part of the landscape, but all our software was intelligently designed—and so were the social systems which produce that software. Christine Peterson of the Foresight Institute tells Ben Landau-Taylor about the struggle to define “open source software”.
Transcript
Intro (00:00)
Ben: For people who grew up with the internet, it’s easy to think of it as a natural feature of the world, like a mountain or something. But in fact, the internet was built and designed by people who were making choices about the technology and the culture to try to get a particular shape of society. And I’m here with Christine Peterson, who your most sort of famous and legible role in this was inventing the term open source, and I understand that you are also very active in putting together a lot of the social scene and curating many of the spaces where this sort of collaboration and discussion behind a lot of this technological and cultural and legal design could take place.
Christine: Yes. Through my nonprofit, Foresight Institute, we’ve been able to interact with a lot of the movers and shakers here in Silicon Valley for decades. In fact, this is our 40th anniversary year, so we’ve been pretty close to the center of the culture for that long.
The hackers who built the internet (01:11)
Ben: Yeah. On a lot of these different technologies and cultural issues, you guys have been in it. So, for a lot of our younger viewers, can you just tell us what was going on in this sort of, I think, late ‘80s throughout the ‘90s period where all of this was happening? What was sort of the background cultural context in which this work had to be done?
Christine: So, this was a time where there were many fewer software engineers. Many of them came originally out of math. Often, in the early days, there wasn’t a computer science degree you could get. So, these are people who, they were very technical, of course, very quantitative, but they didn’t have a computer science degree, per se. They came out of other fields, picked this up on the fly, which means they’re super bright.
Ben: It’s a little bit like how with the artificial intelligence technology we’re seeing now, there just haven’t been very many preexisting experts as it’s ramped up.
Christine: Exactly. It’s a time of great opportunity if you grab it, right? If you can jump into a field when it’s new, and you can still do that right now with AI, for sure, you can ride that wave, and there’s no one to tell you you can’t be very quickly an expert in the field. And that’s how it was back then for the early days of the internet, and the early days of the personal computer in general. So, it was an exciting time.
These are super bright people, highly technical, of course. Many of them very libertarian-oriented, for whatever reason.
Ben: Yeah. A lot of these anarchist manifestos floating around in the space.
Christine: And a lot of hopes for increased freedom, increased communication around the world, which certainly we’ve gotten, not always leading to the wonderful beneficial results we had hoped for. But today, it’s unimaginable to think that we wouldn’t have these abilities, right? It’s hard to imagine a world without them. So, it was wonderful fun to see them come in, to play a role, however small, in shaping that, to see the explosion. It was similar to the AI explosion, from no personal computers, to personal computers, to internet, and then to watch the web expand.
Ben: Yeah. And I understand that a lot of the sort of freedom and ability to just be connecting to anyone, a lot of that was deliberately built in at the technology level as this was getting built out, right?
Christine: There were some fundamental architectural decisions that were made by folks designing the internet to make it very decentralized. Now, there have been at least two explanations for this. One is, and this is probably what they told the people who were paying them, which is this will make it hard for a nuclear war to interrupt the internet, and that is true. However, it also makes it pretty challenging to censor the internet. But perhaps they didn’t mention that to the folks who were paying them to design it.
Ben: Yeah. I remember hearing the term, “The internet interprets censorship as damage and routes around it.”
Christine: That is the classic phrase, and it’s true. And it’s been a wonderful thing that the internet is so hard to censor. Now nation-states can do it, but other than that, it’s pretty darn hard.
Ben: Yeah. And so tell us about who was this group that was coming together around Foresight?
Christine: So, at this point, it was early in the internet. The standards were very simple, very basic, and pretty flawed.
They didn’t meet the criteria that, for example, earlier hypertext projects such as Xanadu had hoped for. One of the features that was missing is called backlinks, where who has linked to my document? In these days, it was very hard to find that out. You couldn’t see who has linked to my document. So, we felt, well, we’ll put this in.
We will add this as a feature to the internet. Foresight Institute wanted to add this, and we had a group of programmers who were all brilliant people, and series of meetings, and the primary architect of our backlink software was Ka-Ping Yee, a brilliant programmer. And we wanted this software to be what they called at the time free software. That was the term and that’s how Foresight and I got involved in dealing with this movement called the free software movement, and tried to help it become what it is today, which is continue to be a movement, but also an industry.
Ben: Mm-hmm. And so the free software movement, what were they up to?
Christine: So, I think probably your viewers are aware that there’s two kinds of software. One is closed source, proprietary software, where you buy it in a box or over the internet, comes these days over the internet. Something like Microsoft Word. And if it has a bug or a flaw in it, there’s nothing you can do except complain to Microsoft. There’s no way to fix it.
You could hire the best programmer in the world, and he or she cannot fix this. There’s no way to do it. You cannot get at the source code, which is what you need to do the repairs. That’s one kind, and the other kind is what was called at the time, free software, where you can get the source code and you can fix any bugs that are in there. You can make the software better, you can redistribute it.
It’s a very nice form of software if you’re a programmer and have those abilities. So, these were the two kinds. So we felt we wanted ours to be the free kind so that people could improve it, distribute it, repair any bugs. And there were many other projects similar to ours where people felt strongly that they wanted it to have these abilities.
Ben: Yeah, it was—
Christine: Including the first browser. You probably know that, right?
Ben: This was—
Christine: The Netscape—browser. Before it was Netscape, it had come out of the academy. I think that was Marc Andreessen. Yep. And—
Ben: He’s certainly one of those people who got in early and rode the wave, as you were saying.
Christine: Absolutely, and also brilliant. So, he’d made this company and, if I recall properly, they were having trouble competing with Microsoft, so they had decided to make their browser into free software.
So this was part of the movement. So there was a general feeling of, okay, there’s this whole category of software, it’s a very important category. How do we get people to understand why it’s important? How do we get people to take advantage of it? How do we spread this model?
Even though the people involved were very pro-free enterprise, it’s not that this was an anti-corporate movement in any way. It’s just that, if you’re a software programmer, this ability to fix the code is super important to you.
Ben: Right. And so you guys got involved in it because you were trying to build this piece of software and, as the infrastructure and architecture of the internet was still being worked out, it was just like, “Oh, yes, we can just figure out an improvement to this architecture, push it, people will adopt it.” This was still happening all around at the time.
Christine: It was at the time, and of course, in many ways it still is happening. So that’s why Foresight, as an organization, was interacting with the free software folks and also the Netscape folks, because they were in the process of rolling out this change, which involved communications, marketing, education of the consumers. It was a whole push they were doing. So this question came up, if you call something free in English, it refers not just to freedom, but to price, right?
The “Free software” problem (09:53)
Ben: Right. People would be like, “Huh, it’s pretty weird that Andreessen got so rich off of free software. That doesn’t sound like that’s what the word should mean.”
Christine: Well, so the challenge was that there was a lot of wonderful free software available, but to the non-programmer, it’s impossible to use because the non-programmer, such as myself, we need the software installed. We need support. We need automated updates. We need a place to report bugs and get them fixed. And those come with corporate support.
If you want those kinds of services, you need a company to do them, which means there needs to be a way to pay for it. And—
Ben: Because there has to be someone who’s building all this and keeping it updated and as the other infrastructure it depends on changes or as bugs are discovered, to be like, “Oh, I’m going to drop all of my other plans and get on fixing this,” and—
Christine: Right. There’s just no way to provide this level of service that the non-programmer needs without some way to compensate them. So there were startup companies. Tiny little companies were beginning to provide those services. And so they would go out and try to promote this product and explain that it had to do with free software, and immediately the customer would say, “I don’t understand. Why are you charging me money for free software?”
And it just was a non-starter. So this problem would never arise, say, in Spanish, because they have two different words for these two meanings, free as in freedom and free as in price, gratis and libre. But in English, these are the same words. So these tiny little support companies would go out trying to sell their services using thinking they’re referring to free as in freedom, but all the customer can hear is free as in price. And this was just going nowhere.
You can see why, right?
Ben: Yeah. It’d be very easy to get tripped up.
Christine: No matter how many times you explain to the customer that, “No, no, we mean free as in freedom,” to the customer, to the non-programmer, there is no freedom. They don’t have the skills to change the software. So arguing that freedom is a benefit to this person is not something they will understand. All they’re going to hear is free as in price. And once they’ve anchored on that, you can’t get them to pay anything for it.
Ben: So you’ve got this movement of technical people who are getting deep into it. They want to control their computers. They have this ideological view of digital sovereignty and how they should control all of their stuff, and they should be able to understand and modify it. And then they’re using this to, in dialogue with each other, be building these things. And then the reason they’re building it is because there’s some customers out there who at the end get value from it.
And so they have this idea of how their sort of production area where they’re making all of this is going. And then over here, the customer’s like, “Look, I don’t care what’s going on in the place where it’s all being built. I just want to get your software, and I’ll pay money for software that helps me, but why are you telling me that I’m supposed to pay you for free stuff? That doesn’t make any sense.”
Christine: It was a marketing disaster, basically. It was a communication problem, a labeling problem. It was holding us back terribly. And I was not the only one noticing this. Others were noticing this as well.
And I recall a conversation with the folks at Netscape saying, “Hey, we have to fix this.” And they said, “You’re right. We have to fix this.” But then nothing happened. I hadn’t heard any proposals.
You can’t fix it until somebody says, “Well, here’s another idea.” You have to have an idea of another name or else it never changes, right? So it just seemed like we were stuck for some period of months on this. Right.
The genesis of “open source” (14:37)
Christine: So then what happens is what always happens, which is you’re in the shower or on a walk, and an idea pops into your head like, “Oh. Oh, that would work,” or in my case, it was like, “That’s good enough.” In other words, you don’t need the perfect term. All you need is a term that isn’t a disaster, which is what we had. So the term was, well, how about open source software?
That seems good enough. It’s clear. It’s about, okay, the source is available. It’s open. You can get it, and you can change it.
It’s not a perfect term, but we didn’t need a perfect term. We needed any other term.
Ben: Right. And one of the advantages is if you hear and you don’t know what it means, you’re like, “What’s that? I don’t know what that means.”
Christine: Right. At least it’s a different term.
It makes you ask, what does this mean?
Ben: Right. And then when you hear it, you’re like, “Oh, yeah, I see what you mean by that.”
Christine: To a technical person, a software person, it’s a pretty clear term. They would understand this term immediately, I think. To a non-software person, they might not even care, actually. They often don’t care. Yeah.
Ben: Yeah. And so then how did you, upon being like, “Okay, yeah, this is at least a better term”? I don’t imagine you’re just like, “Aha, I have it. I’m sending out the email,” then everyone just starts using it.
Christine: That’s definitely not how to do it, right. So, I was in kind of a tricky situation because this community, the free software movement, that community, was exclusively programmers, and I am not a programmer. And they have a very clear sort of status hierarchy within the community, and it makes sense, which is if you produce a lot of excellent code that millions of people use, you get a lot of status. And if you produce only a little bit of code that not many people use, you have lower status. And if you don’t produce code at all, like me, you have no status at all, okay?
And this—
Ben: Feels like a pretty sensible way for that to be set up.
Christine: Absolutely. It makes perfect sense, and I fully support this. So, I realized that, okay, I am not in a position to send out an email announcement saying, “Guess what? I’ve solved your problem.” Absolutely not.
So I thought, “All right. We have to do this carefully.” So first, I ran it by a few close advisors. Most of them said, “It’s fine. It’s good enough.”
My PR guy said, “No, open has been overused,” which was true, but I decided that the other people were enough to overrule that view, and we would go ahead.
So, I knew that I needed a front man, someone who had status within the community, at least more than I did, which was zero, to do this introduction. Fortunately, I had a lot of friends in the movement.
Ben: Which was not a coincidence. You’d been—
Christine: Not at all
Ben: and you guys still are at Foresight, just cultivating and being a center for where a lot of these people congregate and providing a lot of the infrastructure so that they can just be in dialogue and keep everything flowing.
Christine: Yeah. We were well-connected in the community, as we still are here in the Valley. So, it wasn’t too hard to come up with a strategy. In fact, one of the leaders of the movement, many people at the time would say he was the leader of the movement, was Eric Raymond, who was in town for meetings. To try to help this movement move forward.
The power of doing the legwork (18:20)
Christine: And he has vision problems, so he needs a chauffeur, so I was driving him around.
And because I was a chauffeur, I happened to be in the room with a group of the leaders of the movement. I was only there as a chauffeur really.
Ben: This is a very common pattern of this sort of thing. If someone is just willing to show up and start doing the legwork, it gets you into a lot of surprising places. One of the famous examples of this is Garry Tan, who’s the current CEO of Y Combinator, initially got in the door because he just sort of showed up with a camera and appointed himself their volunteer photographer, and then just kept getting his foot in the door and...
Christine: That’s a perfect example, and something for young people watching to keep in mind, which is just start helping out. People appreciate it, people notice it, and the next thing you know, they start throwing things at you, and then you’re getting paid out of nowhere. It happens.
Ben: Yeah. And it just sets you up to get lucky in this sort of way of like, “Oh, I just happened to be his chauffeur because I just do a bunch of these things to facilitate things when people are coming to town.”
How the name took off (19:44)
Christine: Exactly. You want to be in the room under any capacity. But I was sitting at the table with everyone else. I’ll point out a humorous fact, which is that almost certainly many of the people in the room assumed that I was Eric Raymond’s girlfriend. Which is not the case, but we were both married to other people, but I’m pretty sure that’s what they thought.
But that didn’t cause a problem. So, this was a meeting which had nothing to do with this issue of the name. In fact, that was not on the agenda at all. It was a very high-level strategy meeting covering issues of which I had no knowledge of. Fortunately, in addition to Eric, who I had not mentioned this to at all, the name issue, I think he was aware of the problem, but I hadn’t given him any thoughts about what the new name proposal was, that I recall.
However, one of the other friends of Foresight was in the room. I believe his name was Todd Anderson. And I had discussed this with him, and he liked it. So here I am at this meeting, zero status, and not really in a position to bring up the topic. It was not on the agenda, so it didn’t seem like the right time to do anything, and for me to do anything.
So Todd already knew. He knew the name. So Todd did something absolutely brilliant. Did he say, “Hey, guys, I want to change the name”? No, no.
No. While discussing the general topic, which was the correct topic of the meeting, he just used the name. Just used it, like this is the name of what we’re talking about. Just threw it out there. And I’m like, “Oh.”
I’m watching. And then he and I look at each other like, “Okay.” And then two minutes later, somebody else used the name, and I’m like, “Yes, the meme has started.” I look at Todd, we’re both like, “Yes.” And then it just took off like a rocket from there.
It was really impressive. And it’s not because it’s a brilliant name, it’s because the problem was so large. Right? You don’t need a great name if the old name is disastrous. Right?
So I think that’s why it took off. And there were other suggestions. Once the name started to catch on, there were other suggestions for, “Gee, if we’re going to change the name, what should we change it to?” Michael Tiemann, who’s a leader in the industry now and has been for decades, suggested Sourceware, which is great also. That would’ve been fine.
It would’ve worked just as well, maybe better. And there was a later meeting of the leadership of the movement to vote between these two names, and I was not there because I am not a leader of the movement, right? I was not in that room. I didn’t have a vote, which is perfectly fine. I should not have been there.
And the vote apparently went for open source for whatever reason. But I think Sourceware would’ve been just as fine. But the point is that when there’s a tremendous pressure of a problem, then the solution, if you can come up with any solution, may catch on very fast. As long as you don’t handle it in a really ham-fisted way. And I think Todd’s way of doing it, rather than getting into some kind of discussion, just throwing it out there and seeing if it takes off, was perfect.
It was the perfect way to do it.
Ben: Yeah. And so it seems like a lot of what happened was you’re repopulating the wild, you take your breeding pair animal and just let it out there to do what it’s going to do, and then because this term just actually worked better for describing things, it just got picked up and seeded into the right place to then just spread because it worked better.
Christine: Right. And so what are some lessons? One lesson that I feel very strongly about is names are important. They’re very important. And you can shoot yourself in the foot for a long time if you keep using a name that is limiting.
So whenever you’re starting something new, think pretty hard, and if it turns out you’ve guessed wrong, change it. Don’t stick with a bad name just because you started with it. If it’s not working for you, change the name.
Ben: Yeah. This reminds me, I used to be pretty deeply involved with what at the time was calling themselves LessWrong, and then they now call themselves the Rationalists. They’ve been another pretty influential online group. And that choice to switch from what had been the name of the old blog to this new name was very deliberate. And I remember Oliver Habryka, who’s the current head of Lightcone Infrastructure, sort of their most central community organization, was going around and just talking to everyone and being like, “I think we should be using this name instead of that name.”
And I know he was incredibly diligent because I had, at that time, 60% left the group, and nevertheless, he talked to me. So I’m like, okay, he must have approached everybody because he was literally like, yeah, the name is in fact a very big deal.
Christine: Well, he was right. Names are important, so it’s worth the effort. And he put in the effort. I think the change is right. It was probably a good move.
It’s possible we could come up with a better name than the Rationalists. There may be a better name out there for it, but it’s good enough for now.
Ben: Yeah. And it’s certainly caused its own problems, and people are like, “Oh, that happens to be the name of this philosophical movement from several hundred years ago.” But it doesn’t seem to have people stumble over it as much as LessWrong does, so
Christine: Exactly. And in the case of open source software, the term open source per se, just those two words, is not a new term. It’s used in the intelligence community, so-called intelligence community.
Ben: Another case of a very deliberately chosen name.
Christine: Indeed. Indeed. The spy crowd is really who they are. And they’ve used this term for many, many years, decades. So the name isn’t even original in that sense.
But if you look at the definition that the so-called intelligence community uses, it fits pretty well. So it’s not a contradictory meaning.
Cultures are intelligently designed (27:00)
Ben: Yeah. And another thing that I think is very important about how this scene of people works is, on one level, the scene, the thing that’s being done here consists of all of these software patterns and software standards, and people are pushing code. But on another level, a huge part of it is just in order to have all of that code moving that way, it depends a lot on the cultural practices of this very specific and not that big group of people. And this isn’t just a matter of sort of Darwinian evolution of someone tries something and something else happens. It’s very intelligently designed.
Even you, who’s a fairly peripheral participant, are sort of sitting down and thinking, “What are the mechanisms? How do the gears work? Can we slot in a different gear here?” And in order to get a functional thing that’s working, you have to have a lot of people being like, “What should the cultural practices be? To what extent should we be according status only to the people who actually push code?”
I think there were a bunch of arguments around this. “What types of infrastructure do we need? Oh, I’ve got an idea.” And then people thinking about what the missing pieces are, producing infrastructure like GitHub, infrastructure like HTTPS, and all these sorts of things. It is a pretty small group of people who has a very specific and very ideological vision of what the ecosystem should look like, figuring out how to make that real, and having all of these meetings.
Funding open source software (29:04)
Christine: Yes, that’s right. There were many very competent, very serious, very deliberate choices made and implemented by quite a few people. And the process hasn’t stopped, right? So we had the free software movement, now we have the open source movement and industry, and now the next stage is happening with, again, this whole very deliberate choice, which is one big problem that we have in open source is that because you can just use the code, there is very little money that flows back to the developers and maintainers, which means there’s a huge amount of open source code out there that either is not supported at all anymore or is widely used but is being maintained by one 75-year-old Nebraska guy with a heart condition, right? And he gets $0 for it, but he does it out of love.
Now, this is not a good way to run a system, right? So there’s now a new proposal that is beginning. One of the original proponents of open source was Bruce Perens. He uses the term Post-Open, which is a start. But there is now a crowd of folks who have a very specific proposal.
There’s a website up, there’s a term, and here’s the proposal, which is we take the current open source definition or set of licenses and add one thing to it, one change, which is if you’re a huge, successful corporation and you use the code you do have to pay something, okay? Because we have to get some money from somewhere for this stuff. So that’s the proposal. It’s called Prosperous Software. That’s the term they’re using.
Is it an ideal term? Maybe not. Maybe we’ll come up with a better term, but that’s the term they’re using right now. So if you go online and search on Prosperous Software, you will find their homepage and their whole proposal. So this is the next evolution, which is we have to fix this problem of complete lack of resources for these maintainers.
Ben: Right. And as I understand it, a lot of the big corporations have been among the most willing to write checks to support a lot of these open source things, but it’s in a very ad hoc way and often depends on is there a specific guy who’s the vice president of whatever, who wants to really make the case internally.
Christine: Yeah. It’s just very slapdash, very occasional. You just can’t count on it. It needs to be institutionalized. We need something clear, and I think these companies would be happy to chip in.
They want this software to be maintained, right?
Ben: Right. And the big barrier, as I understand it, is mostly just the internal coordination of them figuring out, “Oh, I guess this is our job. Oh, which budget is this going to come through?” And so if there is something like this that just happens to make it sort of more automatic—like—
Christine: I think they would love it. I think they would love it if they knew that, gee, there’s someone out there who is maintaining the code, looking for bugs, continually improving the security, especially in this day of AI, where software, any security bugs are being found and exploited immediately by AI. It’s urgent that all the open source software be properly supported if you are using it for any serious purpose.
Ben: Which it very frequently is.
Christine: Yes, it very frequently is, especially the internet. So, I think companies will welcome this. I think users should welcome this because the idea that critical software is being poorly supported in an age of AI is unacceptable.
Succession (32:53)
Ben: Yeah. Well, I hope they manage to figure it out and get to the next stage of the infrastructure here. Another thing I wanted to ask you about with respect to this, because I know you’re very interested in mentoring the young and how do we have this sort of institutional continuity and handing things off, and I’ve been extremely impressed with how you did that with the Foresight Institute and making sure to hand that off while you were still around, to be around so that the younger generation could ask questions and do it sort of gradually rather than the sort of sudden, the person in charge dies with their shoes on and everyone else has to scramble thing that happens kind of unfortunately often. So, in this open source community or whatever’s the more up-to-date term, how well has that sort of succession and acculturating the younger people been going in there?
Christine: Well, I think it needs to be much better. And the reason I was able to hand off Foresight is that there’s a budget. There’s money to pay people, so we are able to hire good people. And we have a fantastic staff. It’s a lot of work to raise that money, but we have it, right?
Whereas in open source, because the software, there’s no charge for it, it’s really hard for that 75-year-old guy with the heart condition in Nebraska to find anyone who wants to help deal with this, right? How is he even going to find anyone? Why should a young person take that job on, right? For free. It’s a volunteer position, right?
Ben: Right. And like you said—
Christine: Support an old piece of software for free. How is this appealing to young people?
Ben: Right. And that old Nebraska guy, he started doing it because he was just building his own thing. And so he’s sort of in love with this thing he’s been tinkering with for decades. But that’s not going to apply to some other person coming in to continue it.
Christine: Absolutely not. I think they need and deserve to be paid for their work. So, that’s why I’m very interested in this whole prosperous software concept, and I very much hope that this catches on because we have an urgent need.
AI and software security (35:20)
Ben: Yeah. And so, one of the other big things going on, which I’m sure a bunch of our viewers have been thinking of as we’ve been talking about is, the big sort of question in the software world is all this artificial intelligence. And there’s a bunch of people, it’s in a similar sort of position as you mentioned. It’s very new. No one’s really sure what it’s going to be.
Partly this is a question of just how does the underlying science of this work, but a lot of it is just a question of what infrastructure we build, what software infrastructure, what cultural infrastructure, what legal infrastructure. A lot of people are running around trying to think about this, trying to build it. They have their particular visions of what it all should look like, and how they’re going to go about doing that. So especially as it comes to the implementation, which often comes down to these very messy details, what should they be thinking about as they’re doing that?
Christine: Ah, yes.
Well, let’s see.
We have two big issues with AI. One is what they call alignment which is can we get this AI to follow rules and act in an ethical manner? So far the answer is not really. The alignment problem is not going well. I think it will eventually be solved, but it may require major architectural changes.
And by solved, I mean improved to the point where we can use the software. So that’s one issue. But many people are working on this. Many brilliant people are working on this. So I have great hopes that they will make progress.
And meanwhile, there’s the security issue, which is that we have these powerful tools. They are already being deployed, and the vast majority of software that our society runs on is insecure. It has immense numbers of flaws, and one can make the case that it is not just insecure right now, but it is insecure-able. In other words, it cannot really be fixed.
Ben: Yeah. Is it possible to make software for most of these things that would be robust to a very dedicated attacker to lots of resources?
Christine: It is possible. You can do formal verification of software that will show mathematically that this is secure. So it is possible to write secure software. It’s just that the vast majority of software we currently run on has not been written this way. So one can make the case that in an age of AI, we’re going to have to redo all the software, which is a huge amount of work.
But AI can help, right? AI writes code.
Ben: It sure does.
Christine: It sure does. But we need to get going on this project right now.
Ben: Mm-hmm. And so for the people who are trying to get these projects up and running, get the infrastructure, the social infrastructure and the way that you guys have often done at Foresight, what lessons should they be taking, either from this incident or just from the general experience of these earlier days in the internet of how practically one should organize to achieve goals like these?
Christine: Right. So excuse me.
So the two problems. The first one is the alignment problem.
I think they’re doing a good job of working on that. There are many people working on this in many different ways, including starting from scratch with totally new technologies, which I think may be necessary to fix this. But there’s money flowing in, there’s brilliant minds flowing in. So that’s all going well, I think.
Ben: Yeah, a lot of money in the last few years, and a lot of talent, as you would expect when you have the money to pay the talent.
Christine: Right. And a lot of super dedicated, very altruistic, very serious people are working on this. So I won’t say it’s well in hand, but I’d say that it’s being tackled by brilliant folks, which perhaps is all we can ask. The other one, though, is being completely ignored. So that is the urgent one.
Ben: And why is it being ignored?
Christine: I think it’s because, and even very smart people like David Sacks, who until recently was our AI and crypto czar for our current administration, very smart fellow. He has this idea that there’s just going to be a short period of time when there’s going to be all these bugs and then we’ll fix the bugs, and then everything will be fine after that. He keeps saying this.
Ben: The sort of passive voice, they will be fixed.
Christine: Well, I think the fundamental disagreement is not that we aren’t going to work really hard and fix these bugs. People will work very hard, and a bunch of them will be fixed. The question is: Is there an ending point to that? He seems to think that we will reach a point where the software is pretty secure and it won’t take too long and it won’t be too disruptive for too long. Our position at Foresight is that’s probably not the case, that the people I trust the most on this convinced me, persuaded me that that is not the case, that our software infrastructure is mostly insecure-able currently, and that we need to rebuild it on secure foundations from scratch.
Now, we can’t always do that, but there are things we can do to mitigate the problem. There are technologies out there to do this. But we haven’t really started, right? So that’s the problem. I would say the alignment folks are rolling.
Good for them. Keep going. The security folks have dropped the ball. And we need to get started on it
Ben: and so the alignment side, this is the result of a 15-year—push to argue that this is a big problem and for 10 years they were wandering in the wilderness with no one paying attention, and then over, especially, I guess the last six or seven years, it’s gained more and more momentum. And I guess it sounds like in the security side, there hasn’t been an equivalent of that, or perhaps it’s just getting going.
Christine: There has been an equivalent of that, which goes back even farther, but it turns out that people are just not interested. It’s a boring topic. Security is a boring topic. Nobody wants to hear about security issues. They don’t want to hear about security solutions. It’s boring.
So, the only way to get people interested is if there’s an emergency. Right? So, when the internet only handled data, there really wasn’t much of an emergency. Now the internet handles money, and that got to be a bit of an issue. Like in crypto, people lose millions of dollars.
So we in the security community, and I put myself adjacent to that, we thought, well, maybe people will care now because of these millions of dollars being lost. And they care a little more, but still not enough. But now increasingly I think with AI attacking our infrastructure, which it is, we’re going to see problems skyrocket. And eventually, I don’t know when, but eventually people will wake up and go, “We have to fix this problem,” and then we’ll see some action.
The first software security crisis (43:42)
Ben: Mm-hmm. Yeah, and I guess I have vague memories. I was still pretty young, but I recall something sort of like this happening when the internet was getting really big and starting to show up in everyone’s homes. And then there were these new things, there were computer viruses that would—replicate on your computer and then be sent to a bunch of other computers. And for a period of something like 10 years, I would just read in the newspaper that some new self-replicating computer virus had gone in and destroyed a bunch of major hard drives at Microsoft or wherever and caused millions of dollars of damage, which at the scale that the internet was operating on at the time was a very big deal.
And then eventually people were like, “Oh, so you should be running an antivirus. It’s just kind of irresponsible not to.” And then eventually that got good enough that those kind of stopped spreading through the internet like that, and then they just got incorporated into—the operating systems, and you no longer even need a specific, dedicated, separate antivirus program on your computer because it’s just part of the foundations. But that did take a bunch of these things just sort of running wild and causing a bunch of chaos. Is that the sort of thing that you’re envisioning at perhaps a larger scale?
Christine: Yes, I think so. And even though we no longer need a separate program, what we do need to do is to do continual updates. If you stop doing your updates on your software, you will very soon get in big trouble because there’s a continual war going on, right? There’s an arms race going on all the time.
Ben: The same reason they have to keep updating the flu shots.
Christine: Right. Because in the software world, there’s an offense and a defense, and it’s a continual war that is going on, and that’s going to continue probably forever. What we need to do is take the most important software and rewrite it to be extremely hardened and secure as best we can. Things like the software that runs the electrical grid, the software that runs the water supplies, the software that runs the banking system. Everything has to be locked down.
And I’m sure people, especially on the banking system, they’re probably working on this, and I hope they are everywhere. But we need to get much more serious about it and put a lot more money in.
Ben: All right. Well, this has been very interesting. Thanks for coming on the show.
Christine: Oh, I loved it. Thanks.