Josh chats with Daniel and Stefan from curl about their summer of bliss. Curl stopped taking vulnerability reports for a month and nothing much happened really. Daniel and Stefan have a really pragmatic view of all the new LLM powered vulnerability detection tools. The cost of finding a vulnerability has dropped dramatically, but the cost of fixing those bugs hasn’t changed. Taking some time off is important for anyone in the middle of these reports. Daniel and Stefan have some great experience and ideas on how to make this all happen. It’s great advice for anyone working on software, not just open source.

This episode is also available as a podcast, search for “Open Source Security” on your favorite podcast player.

Episode Transcript

Josh: [00:00:00] Today, Open Source Security’s talking to Daniel Stenberg and Stefan Eissing.

They are both Curl maintainers, everyone’s favorite weekend project. So Daniel and Stefan, welcome to the show.

Daniel Stenberg: Hi, thanks

Stefan Eissing: Hi, nice to be here

Josh: So I’ll, I’ll let you do some intros just so anyone who might not be familiar. I know Daniel’s been on before, but Stephan has not been here yet, and I’m extremely excited to have him here. So, uh, Stephan, let’s start with you for kind of a quick intro, and then we’ll go from there.

Stefan Eissing: Okay. Um, yeah, name’s Stefan Eissing. I’m, uh, from Germany. I’m in software since forever, I think. Um, sometime before the internet. Um, working in the last five, five years or so together with Daniel on, on cURL. We met like 11 years ago on some HTTP workshop. Uh, I was doing HTTP/2 for the Apache web server before this as open source work, and this Let’s Encrypt integration and, and this [00:01:00] kinds of fun stuffs.

But I decided I switch from the server to the client side. It’s much nicer there. So that’s me.

Daniel Stenberg: The good side

Josh: All right, Daniel, who are you? Why should we know you?

Daniel Stenberg: We’re celebrating cURL 30 years later this year, actually.

Josh: Wow!

Daniel Stenberg: I found this tool that started everything called HTTP GET back, back then. So then, uh, I, I became maintainer. I, I renamed it cURL, and I’ve been working on cURL since then.

Josh: 30 years. Wow, that’s crazy, man. Holy cow. I mean, congratulations. That’s like, what a ride. Wow. Wow. Amazing. Okay, so I wanted the two of you to come chat with us because this is… I know this is way late, but I remember you did your, your Summer of Bliss for, for kind of pausing security vulnerability reports this summer, and of course, the internet melted down. then I was like, “I’m gonna wait a couple months,” and a couple months turned into more than a couple. And then I wanna, I wanna talk to Daniel and Stefan and, and just [00:02:00] understand, like, what, what actually happened? Like, the world obviously didn’t end. We’re still here talking, but this feels like an amazing experiment.

But then even since then, there have been a bunch of wild just security things going on in the universe. So I will let… You can fight amongst yourselves. Who wants to explain what the cURL Summer of Bliss was?

Daniel Stenberg: You can start, Stefan. Go ahead

Stefan Eissing: Yeah, I think we were hit like in, in, in May, June with like immense loads of security reports, which were like no longer total nonsense like the year before. And we yeah, we were like– we really felt bruised and a little abused maybe also. And I think I suggested in, in our security chat that, well, let, let’s take the Swedish summer, let’s take it off, let’s shut, shut our inboxes off.

And Daniel turned that into the Summer of Bliss

Josh: [00:03:00] Awesome

Daniel Stenberg: Yes. I th- I think we all in, in the current security team felt a little bit, uh, just exhausted and, and, uh, uh, tired from, from that onslaught of vulnerability reports that we had there late spring, early summer. And, uh, I, I don’t know. Uh, um, I mean, it really, we don’t… I mean, in, in principle, you would, we could just pretend they don’t exist, right?

And, and do it anyway. But it’s something like, it’s, it’s, it’s a mental burden to just know that they’re there, they’re waiting,

Josh: Yes

Daniel Stenberg: and, and we know the severity and, you know, it’s a problem. We know it. It’s not that easy to just ignore and, and move on. And I think, I think all of us immed- when we started the bliss and we sort of, uh, just a few days in, I, I th- I think we all felt it, sort of, it was the bliss.

It was really, ah, that was, it brought the joy back and we could just ignore that, and we could do whatever we thought was fun. I mean, we didn’t exactly take off and, and go on [00:04:00] vacation all of a sudden, but we could then spend some development time on things we wanted to do or, It was a little bit like going back to the old days when we had to– we didn’t really had all of those security reports hanging over our, our heads all the time. So it was a, a bliss, and I th- I think we all enjoyed that month

Stefan Eissing: Yeah. You re- you realize how much it was sitting on the back of your head all the time when it was gone. You just realize the difference then, just like Daniel said, like two, three days in, it was just great

Josh: I mean, that, that’s shocking it only took a couple of days, right? That, that… I, I understand the mental anguish you speak of because I, you know, I’ve been doing security work forever, and there’s always more security work. It’s like, you know, delivering mail is one of the better analogies I’ve heard, where it doesn’t matter how much mail you deliver, there’s more mail tomorrow. security work feels like that sometimes, where, like, it doesn’t matter how many vulnerabilities [00:05:00] we squash today, there’s gonna be more vulnerabilities tomorrow. And it, it definitely weighs on you mentally, like unquestionably, I’m impressed it only took a couple of days to, to kinda lose the pressure, which is, which is awesome.

I mean, that’s good. And just from, from your perspective, the, the, you two, just, like mentally, recharge for a month, and when you came back, was there like existential dread or were you like, “I’m ready to work on this again”?

Daniel Stenberg: I, felt kind of a little bit refreshed a-and sort of, uh, ready for a, for a retake on this. And I think at least I anticipated that we would get another storm once, you know, open the floodgates again, and there would be a, a whole bunch of people waiting to submit a lot of reports to us when, when we did that.

So starting August, we– I, I was at least prepared to now we’re gonna get a busload of new ones. But, and to, to [00:06:00] my surprise, it didn’t start like that. So we opened up, we opened the check, the little s-checkbox on HackerOne and said, “Hey, bring it on.” actually took a few weeks until something happened.

I– for a brief moment there, I, I was thinking that maybe we have sort of, you know, gotten over the peak. Maybe, maybe the wave is a little bit… But, uh, that– I was misled. There was sort of– I was just, you know, optimistic.

Stefan Eissing: Yeah

Daniel Stenberg: not the reality.

Stefan Eissing: Daniel is the optimist here.

Daniel Stenberg: And then it, it then, so, so after a few weeks we got the, the, the, the, I don’t know how to call them, the researchers with the big guns, with the good tools and the, the big AIs and, and it said wham. And I think we have gotten, I don’t know how many reports since then, 70, 80. I don’t, I, I don’t

Josh: Wow

Daniel Stenberg: We’ve just drowned in reports the last few weeks, I would say

Josh: So you, you say [00:07:00] that, but you also put a toot on Mastodon saying, “Oh, we’re down to what? Eight vulnerability reports for this release.” Like this is… And then of course they come flooding in. Like I don’t know what you expect. This is like the meme with the dead pigeon of like, “I don’t know what I was expecting.”

Like

Stefan Eissing: Yeah

Daniel Stenberg: I, I kind of wanted that. But, but it also, you know, never done until we’re done. So it doesn’t matter even if I say we’re done, right? If someone can find more, we’re not done. So

Stefan Eissing: Yeah, but, uh, when, when I saw one morning or when I saw Daniel’s toot about, “Oh, uh, Mythos or some other model didn’t find anything,” I said like, “Uh-oh.”

Daniel Stenberg: I, I know exactly. That comment, when I comment on message, uh, I know that triggered the, the AISLE

Stefan Eissing: Yeah, exactly. Exactly

Daniel Stenberg: to compare themselves like that. So that, that was a– They saw that as a challenge. Sure, sure they can find a l- And they, dug up so many after that. So I don’t know. It’s, it’s like [00:08:00] th-they…

it never ends. And it is, it is– I don’t know. I can’t really explain it, but how, how many bugs can you actually find? So there should be a limit.

Stefan Eissing: Yeah, that’s the new thing about these tools, right? They can really– I mean, the, the human brain shuts down after a certain complexity. You– But they keep, just keep on, uh, how many iterations until they find something.

Daniel Stenberg: Y-

Stefan Eissing: Where you get, as a human, you get bored. And then-

Daniel Stenberg: what, what also what I talked about good people, I think AISLE is an example of a team with a toolset and harness and, and, uh, they’re really capable of extracting more problems and finding more things. So even think it’s getting hard to find new things for ordinary people with– If you just throw a, a model on, on Curl now and ask it to find something, it might not find anything at all.

But if you’re, you know, scratching the surface and digging and controlling and, and, and, and [00:09:00] I mean, direct it a little bit better, they can still find a lot of things quite clearly because they do

Josh: Yeah, yeah. And I, I, a couple episodes ago, I had, uh, Jaya Baloo from AISLE talking about some of this, and, like, their secret is people, right? Like, they have smart people doing smart people things. And, um,

Daniel Stenberg: Yeah, yeah. A-a-and it’s really useful for us to have smart people in the other end, so when they find something, sure, they– you have, crazy AI tools to find stuff, but we can have a, intelligent human in the other end to discuss the

Josh: Ja

Daniel Stenberg: you know,

Stefan Eissing: Yeah

Daniel Stenberg: can rerun it or repurpose it or try to do another reproducer and, you know, we can bounce that back and forth, so we don’t have to have a, a stupid AI in there.

We can have u-useful AI and a really skilled person on the other end. So it’s really a productive way to extract f-f-these kind of vulnerabilities.

Josh: So le- let me ask the two of you about that, because I think this is one of the things we’re starting to see, and there’s like, what is it? Uh, [00:10:00] ChainGuard has Athena, uh, Linux Foundation has Project Akrites, there’s Lightwell from Red Hat, where there’s this assumption that the tools can find vulnerabilities, so let’s just write tools that can parse the vulnerabilities on the other end, right?

‘Cause when we think of like vulnerability disclosure, there’s kind of the discovery, there’s a coordination in the middle, and then there’s the, the I guess the, the developer on the other side that gets these things. And I feel like all of the evidence I’ve seen, everything I’ve experienced over the course of my career, and especially in the last year, is that the, the end of this, the developer stage, not very automatable

Daniel Stenberg: I, I think that’s our experience so far as well. So, um, sure, uh, much all of these, uh, really fancy AI tooling, they can produce patches as well, right? They find a flaw, and they can offer a patch. “This is how we patch this flaw.” But it’s very rarely that’s the patch we want, and it’s, it’s more of the sort [00:11:00] of, yeah, they can paint over the symptom somewhere, but it’s, know…

It’s like, it’s like in an ordinary bug. When a normal person finds a bug too, where do you actually fix the bug, right? Is it where it happened, where it triggered that null dereference, or is it somewhere in the, some layer above or, you know? It’s, it’s typically a, a , you have to know something about the architecture, about the

Josh: Yes

Daniel Stenberg: and a little bit where is the best place to do this fix. Often, I mean, often we take a s- few step back, maybe rearrange some codes. Maybe we should, we shouldn’t do it this way because it’s too easy to do this mistake again. We should just refactor this little piece and, and do it in a more clever way so that we try to avoid repeating this mistake ri- right? One of the most important things to, to take back from one of these vulnerability reports is, of course, sh- we should do everything we can to not repeat it, right, so that we don’t have the same bug again i- in four or 12 years.

So it’s a lot about that and, [00:12:00] and the AIs are not good at that, so we very rarely have merged fixes from the AIs. So that’s one of… what’s the, the real benefit of having a really clever guy in the other end that does this reporting because we can have that back and forth, and, and they can then work as humans, right?

“That’s not the correct place to do the fix. Maybe it’s over there.” And they can work, and then they maybe can use the AI to do it if you have a little back and forth and can guide it properly to do the, the proper fix

Stefan Eissing: Yeah. I think we had, we had a lot of weaknesses in our like, uh, connection reuse where you match like authentication to the correct connection. If you take the wrong one, you leak some authentication there which should not go there, blah, blah, blah. And it was not just like a few lines of patch. What we did then, we, we rearranged our internal architecture in that way.

We introduced new abstractions so we can better reason about when a connection reuse is allowed or not, which makes the [00:13:00] code then simpler once we have that. And all this kind of work you don’t get from, from an AI or something. Like, like there’s a missing concept here, you don’t see that. Um, so you can’t do this.

And the other, I think is the, the categorization of, of when, when is something a bug or a security vulnerability? I mean, it’s like when it crashes or it’s a use after free, like what fuzzers do, um, that’s easy. You can iterate until you have a crash or use after free. That- that’s automatable. But if you find some weird thing like is it, is it a bug?

Is it a security vulnerability? And that’s really where the, the, the, the expertise of, for example, the AISLE people come in, uh, which can judge that much better than– You need the human there and, and, um, it doesn’t scale if it’s all put into the project. If, if the AI outputs is just forwarded to the [00:14:00] maintainers, I mean, it’s also the security reporter’s job to make a first assessment, right?

And that’s for some endeavors that is missing. They just– I just today, like the Python, one Python project got like 82 vulnerability reports in, in one email. It’s just like total, I mean, what

Josh: Wow

Stefan Eissing: is this?

Josh: I mean, that’s, yeah, that’s bananas. I mean, how do you even deal with 80 reports? That, and you know a ton of them are gonna be the same bug, right?

Stefan Eissing: Yeah

Josh: that hi- high comes in, I guarantee it

Daniel Stenberg: Probably, yes

Josh: Yeah. I mean, ‘cause n- whoever r- is reporting those almost certainly has not, like, properly vetted those reports, ‘cause no, no one can vet 80 reports. I mean, that’s like, that’s like months of work, right? I

Stefan Eissing: Yeah. And, uh, uh, and what’s, what’s the, what’s the purpose of this? I mean, you could– If you, if they take the 10 most [00:15:00] serious ones and forward them as a first batch to the project, that would be constructive, right? But just putting the whole stuff, throwing it over the fence, that’s, that’s not co-cooperative.

That’s bullshit

Daniel Stenberg: E-exact– it, it’s, it’s really just emphasizing that it’s really easy to run those scans, and it might even be very cheap and quick. So it, it takes a few hours. It produces a busload of reports. Just throw it over the fence. “Here you are, 82 reports. Now you go deal with it.” And it takes hours or even days per report in the other end to deal with it

Josh: Yep, yep. I mean, th- this is one of the challenges, right? I, I don’t know the answer to this. I mean, I, I’d love it if you guys have thoughts, but, like, historically, finding vulnerabilities was really hard, and fixing vulnerabilities was really hard, so I think the scale was kinda balanced, right? And now finding vulnerabilities is really cheap, but fixing vulnerabilities is still really hard, and so we have this, like, huge imbalance of resources and, and it’s like you folks, [00:16:00] like the developers, especially open source teams, are just getting hammered by this stuff.

And I mean, I, I, I don’t know how this ends because it’s also really… You can say, “Don’t send us a billion crap vulnerability reports,” and that’s easy to say, but, like, a lot of people just don’t listen or they don’t know, and they keep sending crap vulnerability reports. Even if they’re not crap, right? Just 80 is a, a ridiculous number.

Daniel Stenberg: Yeah, it’s, it’s actually kind of a worse problem when they’re not crap. I mean, in some ways, because it, it’s a lot of work and, and the better they are, and sometimes they’re in sort of in weird combinations and, and they challenge the, the architecture maybe or your solution. So it’s not just back to the question about how do you fix it.

And some of them, they’re really, “Oh man, how do I actually fix this?” Because it seems to sort of point out a pro- a sort of fundamental problem in, in the solutions we have here. Maybe we need to rethink everything, and that could… That’s a big thing to do. And when every other report ends [00:17:00] up in that discussion, it takes forever, and

Josh: Yeah

Daniel Stenberg: to do.

And also, even if you ask someone to just, yeah, deliver me the 10 top ones, there, there can still be more guys, right? Then they’ll show up three other teams that all deliver their top 10

Stefan Eissing: Yeah, the the economics of this thing is, of course, because of the billions that are in play somewhere on the, on the stock markets, everyone wants to share of that. And suddenly you have lots of people who are being paid to find these things, and the people who are paid to fix these things are a little bit fewer.

Josh: Yes

Daniel Stenberg: and in reality, I mean, they’re all bugs. We, we need to fix them eventually. And so, yeah, we just need to figure out how to surf the wave instead of, instead of, uh, drowning under it, right?

Josh: I mean, it, it’s the modern-day trolley problem in open source, right? Like, yes, these are bugs, and yes, we want to fix them, but also we have two very [00:18:00] overworked and very underpaid developers, you know, being inundated by sometimes, you know, randos just doing it for fun, sometimes companies trying to make a name for themselves.

Who knows what it is? And that’s like, yeah, it’s, it’s, it’s a trolley problem. I, I– There probably… I mean, today there is no good answer for this, and I hate that. I hate it so much

Stefan Eissing: . We are still experimenting how we do this. I mean, this, this Summer of Bliss, I, I think we definitely repeat it or maybe do it another way. Daniel had the idea of the vulnerability window. Maybe that we’ll try out that one. Could be

Josh: is the vulnerability window?

Daniel Stenberg: Well, similar to how we feature window. Basically, when we have a, a period of the res- release cycle when we allow them to get managed. So basically saying we only manage vulnerabilities maybe for half the release cycle and then ignore it for the other half or something. I think it… I mean, I think we’re all toying with different ideas and, I mean, we could do another summer or winter of [00:19:00] bliss as well.

I think, I think we’ve just sort of waited for– to see exactly how this wave and, and the storm turns out. Right now it feels really intense, but I don’t know how it’ll feel in, in a couple of more months. May- maybe, you know, I can hope that it’ll be better over time. At some point it has to get better, you’d imagine at least

Stefan Eissing: Yeah.

Josh: Well, I mean, stop taunting people. Like, that’s probably number one.

Daniel Stenberg: Yeah, but that’s not a solution either. I mean, the bugs are there. As long as they’re there, someone’s gonna find them. Even if I don’t taunt them, they will, they

Josh: All right. I w- I won’t argue with that

Stefan Eissing: Yeah, I think the, the– I mean, the, the, the product curl improves. We’re talking about, about the suffering, and I mean a little bit about our ability to develop the product further beyond what is security, w-what is usability, what is, what [00:20:00] is feature, what is performance, and all the other dimensions. I mean, and they get, they get, uh…

I mean, that’s what the summer of bliss was. There was some time to look at things again and, and experiment around with possible changes, blah, blah, blah. And that’s– Well, that’s the most fun part, but also, like, f-for the product is very important. And, and the security avalanche is just exempting all this kind of work in other project as well.

So the, the whole development, so to say, comes to a stall eventually. You, you just sit around and doing busy work with these reports. S-

Josh: Right, right. And look, no open source project was started to fix security reports, right? You have

Stefan Eissing: Yeah

Josh: mind and, and things you want to do that are actually fun. And vulnerability reports is the, you know, it’s like cleaning the bathroom of open source. Like, you know you have to do [00:21:00] it, but no one wants to do it

Daniel Stenberg: exactly. And no users, they, they don’t want it either really,

Josh: Yeah

Daniel Stenberg: want new fun features or,

Josh: Right

Daniel Stenberg: or whatever. They just assume that the security is there. They– So you’re right, that’s really the cleaning the bathroom

Josh: Yeah. Yeah. Right. I mean, like, uh, my favorite example, right? Like, I’ve been running Linux on my desktop for a million years, and a new version of GNOME comes out. Uh, I think it, I think it just came out or it comes out in a couple days. I can’t remember which it is. But like, that’s exciting, right? I wanna see that.

But if they were like, “Oh, we have a new release of GNOME that fixes 40 security vulnerabilities,” I’d be like, “Well, who cares?” Like, I don’t wanna see that

Stefan Eissing: Exactly. So if it– someone comes and has like some, some deployment of cURL and some weird combination and blah, blah, blah, and so, and then there’s a vulnerability, how exactly is that our vulnerability? You put it into your product

Josh: Oh man, that’s a whole other discussion, isn’t it? I

Stefan Eissing: No, uh, so, so why is that our vulner– I mean, we maybe we are willing to help. [00:22:00] Sure, and we want to have a better project. We want to remove bugs, sure. But I mean, the risk of the vulnerability, and that’s always connected to a vulnerability that there’s risk, that’s not the curl project that has the risk, right?

That’s the people who, who deploy it to use it. And-

Josh: feel like that is starting to change a little bit, where I think maybe a year or two ago it was very much like open source project, I’m using your software, you suck, fix my bugs. And I, That still exists, but I do think there’s a little more ownership where, where some of these vendors are figuring out, like, if I ship it, I have to support it.

This is actually my problem, it’s not Curl’s problem. I think what recently you guys just had some customers sign up who wanted back ports and older versions of Curl, right? For se- security fixes. So like that’s progress. Well, I hope it’s progress, but

Daniel Stenberg: It is progress. I think so too. Uh, uh, and i-in my particular case, I think it’s progress that they come that to us instead of buying it from some middleman, because then the money doesn’t [00:23:00] end up in our pockets

Stefan Eissing: Yeah, true. And for example, there was this one person, I don’t know what company was behind it, who said like he’s ripping all out his, his proprietary network code and replacing it with libcurl. I mean, why is he doing that? You can speculate. My, my thinking is like they don’t want the scanners to hit their proprietary stuff because it will just explode.

And by replacing it with something that is, is already scanned a lot and fixed a lot, they get an ad- advantage because they, in their proprietary stuff, they, they cannot bring that to the same level. Not–

Josh: I, guaranteed. I mean, look, every time I have interacted with a proprietary system that was not open source software, the code was terrible because it’s hidden, right? No one sees it. You don’t have the public hammering it for bugs and vulnerabilities and whatever. ‘Cause the thing, I mean, the thing to remember is, like, even if your [00:24:00] software is, is not necessarily being like, you know, pen tested or something like that, just the normal use of, like, millions of people are going to uncover bugs, and some percentage of them are going to report the bugs, which obviously then you, you, your team gets to fix as bugs, and everyone gets a benefit of.

Versus I have a proprietary system, I hit a bug, we’ll add it to the, the bug tracker, and maybe someday product will let us fix this bug,

Stefan Eissing: Yeah. Right.

Josh: that’s how it works.

Daniel Stenberg: Definitely

Stefan Eissing: If the guy who wrote it is still in the company

Josh: Yes. I– Don’t laugh. I’ve seen that before too. I remember one of my first jobs out of college, we had, um, a binary, it was for zlib, and I’m like, “Where did, what, what, why do we have a binary?” And it was getting built into the, the application. I’m like, “We should just recompile the code.” They’re like, “We don’t have the code.”

I’m like, Yeah, they just had a, literally a .o file

Stefan Eissing: Yeah

Josh: carrying around for years, and no, no one knew what it was. I’m like, “This is zlib. It’s a compression library.” They’re like, “Oh, we didn’t know what that did.” I’m like, “Oh my goodness!” Like, [00:25:00] “What? This is crazy.”

Stefan Eissing: Yeah

Josh: But it, it… I bet every, every proprietary application probably has a story like that.

I would, I would be shocked if it doesn’t

Daniel Stenberg: Oh, for sure

Stefan Eissing: Greg HK, he once said, like from Linux, Greg, he, he said like, “Only in open source you have people who will have like 10 or 15 or 20 years experience in a s- on a certain subject.” You don’t get that in, in a company

Josh: Yes. Yes, 100%. Right. Yeah, I mean, well, what, Daniel has 30 years of experience with

Stefan Eissing: Yeah, yeah

Josh: it’s mind-boggling, right?

Daniel Stenberg: I actually, that’s, I, I often have that discussion with people. So when, when you’re talking about doing anything that’s not cURL, that competing with the age and maturity and, and, you know, not breaking ABIs ABIs for a long time, it’s hard to s- to sort of get close to cURL. But we haven’t broken the ABIs in 20 years, right?

Josh: Wow!

Daniel Stenberg: most other projects are younger than 20 years.[00:26:00]

Josh: Seriously. Wow. Well, the big Rust rewrite will change it, right?

Daniel Stenberg: For sure. For sure

Josh: Oh my goodness. we’re– I want, I wanna wind this conversation down ‘cause we’re kinda coming to the end. So I know, like, we, we’ve talked a lot about, you know, just vulnerabilities and, and all the scary things about them. I think, you know, there, there’s some, some sprinkles of hope the two of you have added in here.

But, you know, kind of, uh, what’s next? Like, what, what do you see happening? What do you think is coming? What are some things we can maybe look forward to in this universe? Because I will say, like, I’ll, I’ll just… I’m very grateful for the two of you because I think a lot of open source has been… Well, just, uh, in general, a lot of organizations and open source I don’t think have been quite, quite as just transparent and, and, like, honest and, and reasonable with all of this new technology, right?

I think you often get, “I hate it all,” or it’s, it’s, “Jesus, come, come sit [00:27:00] at our altar,” versus I feel like Curl has been extremely pragmatic in all of this, which has been super helpful, I think, for, like, just anyone paying attention.

Daniel Stenberg: Yeah, I, I kind of like that we’re, uh, in so- independent in so many ways, so that we, we usually don’t have anyone that contr- I mean, we don’t have any master company or master organization or w-

Josh: Sure

Daniel Stenberg: even a master person dictating what to say or not, so we can speak up, uh, speak our minds of exactly how we feel about it, and that’s goes for anyone in the project because that’s how we do this. So it, it’s really, really hard, I think, to Tell how, where we’re going and, and where, what’s gonna happen be- because things are changing so much and,

Josh: Yeah

Daniel Stenberg: the tools are improving and they’re… Some of the reports we get now, they’re, they’re, they’re so fascinatingly detailed and accurate and, and find, the [00:28:00] findings are, uh, u- use, it, it’s mind-boggling how they can figure this out and, and turn this into… you know, now, now the, one of the of the best things I think with the modern tooling is that they’re so good at doing reproducers. So now basically everything comes with a full, you know, source code to the server, to the client, and a Docker image to run it, and everything so that you can sort of pinpoint exactly the crash and all of that.

So it’s, it’s amazing tool, to, to find the problems and work on it. But it also, we can also feel and sense that they are the other side, if you call it sort of all the scanners and, and they are also improving all the time because they’re finding things now that they certainly didn’t find just a few months ago.

Josh: Yeah

Daniel Stenberg: And, and if you just, if you just, we just keep, they keep doing that for a while more, they’re going to f- I mean, s- back in the day when you, when humans found all these problems, you could only go to s- one level of sort of complexity for a, for [00:29:00] even a skilled, uh, pen tester, someone who actually did this for, uh, you know, for years and years, they could find really complicated things. Now, the AIs can find that complicated thing combined with another complicated thing, and that sort of, that is what the current set of errors are. But they’ll be just going to add another dimension soon, right? There’s going to, going to be three different things and, you know, and add time and weird stuff.

So it’s hard to say when this is going to end and how, h- how, where we’re going to go and how long it’s going to last. So from my point of view, and I, I pretty much my own, at least my personal view to all of development in general, I’ve been doing for this for 30 years. I don’t try to sort of tell how the f- future is going to be.

I, I just work on the few months, and that’s what

Josh: That’s fair

Daniel Stenberg: me, and that’s where I, where I am sort of I’m looking at my toes and where to put the next step. And, I think that’s where, we are in the team as well quite a [00:30:00] lot because it’s hard to s- see beyond all of this. There’s, there is a forest here and I, I can’t see beyond the forest.

There’s too many trees.

Josh: Yeah,

Stefan Eissing: I would, I would say like two things. Um, the one is like what Daniel talked about, the technical, what, what will be possible in a year from now or in half a year. That’s hard to say. I, I personally believe that the model capabilities are a little bit maxed out, but the harness capabilities, there’s a lot of development going on.

I think that’s what also the AISLE guys are doing correctly. They– And I think they’re not on the front-end models, they are on the, on smaller models and do excellent work with that. So that’s an indication that the, the hype the big AI companies are doing, that’s, that’s marketing, that’s… And if that dies away at some time, I don’t know.

I– That’s not my expertise to speculate about that. But I would [00:31:00] expect like it doesn’t, it– For me, it doesn’t make much sense what is currently happening. But the technology is definitely very interesting, and like companies like AISLE, what they can do with it, with, with cheaper models, that’s super. And that will probably not go away even if the big companies will all collapse.

Josh: Yes.

Stefan Eissing: that will stay. Um, the second thing I want to, um, as, as an outlook into the future, I think this Summer of Bliss has encouraged some other projects to also do that, and I hope it will spread. I hope people will find the, the courage to do that for themselves. It will be good for them. Um, and we are looking for other approaches.

What can we do even better there? Maybe we can make an example here for others because we are a little bit public and known that, that others can also argue may-maybe with their bosses or whatever to, to go a [00:32:00] similar path. And that would be very, very good if, if we could influence by giving an example how to manage this madness, um, and, and encourage other people to, to do that also.

Daniel Stenberg: Yeah, I would second that too. So that we need to make sure to take care of all the maintainers. So a little bit bliss here and there to make sure that we all survive, uh, that’s, it’s certainly a good

Stefan Eissing: Yeah

Josh: Yes. Yes. An abandoned project with burnt out maintainers is bad for everybody.

Daniel Stenberg: exactly

Josh: Yeah. And, and yes, I mean, I think Stefan, you, you kinda hit the nail on the head. Like Curl is a great example here, and Curl’s a widely used project, very respected project. You know, you’re, you’re very sane and reasonable.

So yeah, I’m hopeful more projects take this to heart and, and do things like, I don’t know, months of bliss here and

Stefan Eissing: Yeah. Yeah.

Josh: it, man. Like this stuff is

Stefan Eissing: Yeah. And it’s so good

Josh: Awesome. Awesome. All right, gentlemen, this has been an absolute treat. I wanna thank you so much for the time. I can’t [00:33:00] wait to have you back after maybe your next bliss or next crazy security thing or, or we’ll see. I mean, th- that’s one of the things that, that boils my mind right now is like a year ago, this conversation would’ve been about how garbage all these reports are, and now we’re having a conversation about how good they are, and I wonder in a year what that con- I mean, Dan, you hint at it like, you know, more levels of like understanding, but yeah, like,

Daniel Stenberg: So

Josh: don’t know

Daniel Stenberg: to sort of connect to that. So I did this talk in August 2025, about the slop in, in, in, uh, detection of AIs. And the– and there was this question from the audience, had we ever had a s- vulnerability report by an AI that was correct yet? And I said, no. That, that was August 2025, not a single one. Now, you know, a year later, and there’s like, I don’t know, 250 or something to… I don’t know how many since then. So it’s, yeah

Josh: Yeah. Yeah, you have, there’s a graph you put on [00:34:00] LinkedIn that I don’t have… Oh, I do have it pulled up. Um, 200, it’s 200 and change. I’ll, I’ll put a link in the show notes to the, the graph. But yeah, I, I know. It’s crazy. So all right, I guess until next time, thank you both. I, I truly appreciate it. This has been a lot of fun.

Stefan Eissing: Thanks a lot

Daniel Stenberg: Thank you