Josh welcomes Josh Marpet for a discussion about abandoned open source packages. Josh Marpet has a foundation called Value Chain Risk Institute that has a report discussion how to start measuring if an open source package might be abandoned. There’s a lot of data, but not a lot of groups using that data to help make informed decisions about using open source. VCRI is one of those places that’s starting to do this.

This episode is also available as a podcast, search for “Open Source Security” on your favorite podcast player.

Episode Transcript

Josh Bressers (00:00) Today, open source security is talking to Joshua Marpet, head of the value chain risk institute. I’ve known Josh for a while. You do Paul Security Weekly. I’ve been have the pleasure of being a guest on there a handful of times, and and you’re just one of those people that I feel like is is all over the place. And

You you got my attention recently because you I’ll let you explain what Value Chain Risk Institute is in a little bit, but you wrote a report about open source and I’m like, this is really cool. I need to I I need to talk to Josh. So I guess tell us who you are. Tell us a little bit about the Value Chain Risk Institute and we’ll jump into your report, man.

Joshua Marpet (00:36) Okay, so you only got 30 minutes? I don’t know, man. All I’ll I’ll try. I’ll try. I’ll try. so I’m Josh Marpet. I started the Value Chain Risk Institute, honestly, because I was so pissed off at our supply side security. our our vendor risk management processes out there now are god-awful. and I’ll I’ll I’ll I’ll get back into that, but I’m gonna leave you with one piece there. Our current vendor risk management processes in the world are we send questionnaires out, they lie, and they send them back. Okay.

Josh Bressers (00:39) You can come back. It’s fine.

Nice.

Joshua Marpet (01:04) That’s not a really good recipe for success, let’s be clear. So

Josh Bressers (01:07) So so

you know what’s funny, Josh, is you say that. And so I answer these where I work all the time. And I try to be as is tr very truthful because that’s like one of those things I want to make sure and more than once it’s been like, can you put yes here? It’s like, sure. Whatever.

Joshua Marpet (01:13) All the time.

Mm, yeah.

Nope. I mean, you you’ll do

it. It depends. Have we signed the contract for that? Are we doing this? Where where are we in that process? Okay, at certain point, you’re like, yeah, all right, fine, I’ll say yes. But you you really don’t want to. And that’s the kicker. So so vendor risk management is a bloody nightmare. And and and it’s interesting because my day job, I’m at finite state. I’m a senior product security consultant. I have a mouthful of a title. It basically means chief cook and bottle washer. Okay, let’s be clear. But

Josh Bressers (01:29) Right, right.

Yeah, yeah.

Joshua Marpet (01:51) That is my day job. And I I’m over there working with clients and helping them with SBOMs and understanding what’s going on with their security. We work with manufacturers. So when you talk about vendor risk management, I get to work with all of these vendors to a certain extent, you know, the manufacturers of embedded devices. And I see that their problems are are manifest. As as a as a vendor, as a manufacturer, you have to fill out. Do you know how many questionnaires you have to fill out on a daily basis, much less monthly? my God.

Okay, it’s horrible. And and the reality is, I send a questionnaire out to this vendor, they send a questionnaire out to this vendor, you send a questionnaire out to that vendor. They’re all different. Now we used to have the SIG, and the SIG is still around. Okay, the standard something questionnaire, I forget what it stands for. But the SIG is standard interest group, that was it. The standard interest group questionnaire from the Santa Fe group, it used to be, I think they sold it off, it is is was really they tried really well, and I I actually participated, I I I submitted to that.

Josh Bressers (02:20) Yeah, yeah.

Joshua Marpet (02:49) they tried really hard to get a very solid standardized questionnaire so that you would get apples to apples comparison of vendors. I got it. That makes sense. And so if somebody sent you the SIG, you’re like, got it filled out already. Here you go. We’re done. No problem. This makes sense. But it it’s it’s not enough anymore. A once-a year, once every three years questionnaire is just simply not enough. Especially, and I know everybody hears this, can we all chorus it? In the age of AI, okay?

Josh Bressers (03:02) Yep, yep.

Joshua Marpet (03:19) It it’s with Mythos and GPT five point five and all of those, it is shocking how fast you can come up with a new zero day, come up with a new problem, come up with a new issue. And the basics here, and this leads into the report, we’ll get into that in second, is that when you have a public repository, let’s let’s just say Git, okay, GitHub, all right? And you have a public repository for open source software, there are two reasons to make a commit in a in a a repository, a software code repository.

One is to add a new feature, a new function, a new thing, to go, yay, we’re expanding ourselves, right? And the other is to make a fix. Now, some people argue, well, there’s a third, it’s administrative. I don’t care. They’re really either I’m adding functionality or I’m fixing a problem, an issue. Okay, a ticket, a JIRA ticket, whatever, right? That’s it. Those are the only two reasons. And so somebody can go, hey, there was a commit made in repository, blah, blah, blah, right? Okay.

Josh Bressers (04:01) Those are fixes.

Joshua Marpet (04:18) It was an expansion of functionality. Cool. Ignore it. There was a commit made in that same repository and it’s a fix. Ooh. Now, that fix happens before you get a CVE. That fix happens before you get a CVSS score. That fix happens before you get the CNA, the CVE numbering authority, to hand out the CVE number and to popularize it or publicize it. Publ publish it? Publish it, brain. To publish it to the CVE repository.

And now the the US government isn’t even enriching CVEs if they don’t use that software. So you you can’t even get, even if you find a CVE or a vuln a vulnerability in your code, and you send it to you do all the appropriate, proper things that we’ve learned over how many years of painful discussion between full disclosure and responsible disclosure. Okay. You do the CVE, you do the CVSS, you write a you package it up as a patch, you write a customer advisory, you send it out.

Josh Bressers (04:51) Yes.

Joshua Marpet (05:14) And the person that found the vulnerability, you’re like, thank you, here’s a t-shirt, go away. Okay, fine, whatever. But that takes time.

How much time? It depends. In some places it takes 13 to 15 days, typically 11 days across a s several different ecosystems. Maven, we’re not gonna talk about Maven, it takes 167 days. on average, on average, not every package.

Josh Bressers (05:41) on average you’re saying. And this is I I

I wanna stress as well that you’ve done a bunch of research on this. So this isn’t like crap you’re pulling out of your butt. You actually have numbers from like analyzing upstream data.

Joshua Marpet (05:53) yeah. We st

we we we started studying and analyzing about 62,000 packages across, I think it was six ecosystems. Okay. And we picked that it it this is this was just the first introductory sort of let’s try out and see if we can do this. This was more validation of the research methods, not actually doing the research. We did the research, but just again to validate the research methods. And what we found was already shocking enough that we went, we gotta write a report on this crap. Okay.

There are two things we were looking for. Thing number one, abandoned packages. Now, what is an abandoned package? If Josh Bressers writes a piece of software and that software is done, that nobody needs an update. It is mature, it is fine. Once a year he takes a look at it, anybody interested, no issues, no tickets. That’s fine. And he says, checked, we’re good. That’s not abandoned. That’s just mature, dormant software.

Okay, do you know what I’m saying?

Josh Bressers (06:54) So it’s funny you say this because I literally have a package like this. I have an old roguelike dungeon crawler thing called Ularn from way back in the day. I am the Ularn upstream. I look at the Ularn repository about once a year. I occasionally fix a bug or two when someone reports something, because like the new GNU compilers were because everything was like a a byte way back then. You know, this was like C, probably before C had a version that this was built.

But anyway, yes, I literally have a package like this. And and like strangely, like not like this is one of those instances where it ticks a bunch of the weird boxes. Like the original author has literally passed on. So like this is one of those just completely bizarrely holy crap, this is the weirdest open source story, all put together in one. So yes, I know exactly what you mean.

Joshua Marpet (07:45) So, okay, so let me back up a step. So we looked at various different ecosystems, because we said we have to look at different ecosystems to check and make sure that we’re not hallucinating and to check against each other, right? So we looked at Go, uh.NET, Ruby, Maven, which is Java, effectively, JavaScript node, npm, all right, Rust, Python, and PHP, Composer. Okay, so we we we did a fairly not not everything, but you know, we picked some popular ones. You know what I mean?

Now, NPM we had some issues, so we only did a top 100, not a top 10,000. But of the rest of them, we did top 10,000. So we looked at it was it was one, two, three, four, five, six, seven. It looks like about s we had some issues with some of the packages as well. We we did about sixty two thousand software packages we were examining, okay? Which is not even close to the amount of open source software that’s out there, let’s be clear. But it’s a good widespread, you know, net. You know what I mean?

Josh Bressers (08:35) Yes, that’s right.

I mean, so to give everyone an idea, so I know you used ecosystems and Andrew Nesbitt has been on the show multiple times. Ecosystems is tracking like ten million open source packages and they’re not even tracking them all. So sixty two thousand feels like a lot, but in the context of open source, it’s actually not.

Joshua Marpet (08:51) yeah.

It’s

tiny and ecosystems is great, and we’re gonna start donating to them as soon as we can. they are fantastic people and they are careful, and I appreciate them massively. By the way, if you haven’t seen it, it’s ecosyste.ms if I remember correctly. And so ecosystems, so it’s clever, okay? but it really, really good stuff. Really, really good stuff. Anyway, so we looked at all these packages. Again, small subset. This was again more for validation of the methodology.

Josh Bressers (09:13) I know, right?

Joshua Marpet (09:27) Not the actual results, right? We’re looking for how many packages are abandoned, how many are dormant, how many are superseded. If you’re you learn, I think you called it, right? If you learn had, if you’d done a new version of it and you’d said, okay, we’re we’re not going to do this one anymore, we’re now doing you learn v2, because I rewrote the entire thing in Go or whatever, that would be a super version one would now be a superseded package. Okay, that’s the kind of thing we were looking for. And weirdly enough, nobody has definitions of those things. Okay.

So we were like, wow, that’s that’s interesting. But all right. So we had abandoned, archived, deprecated, superseded, you know, current, et cetera, that kind of thing. So we had various numbers. this one hasn’t been updated in 700 days. You get the idea, okay? And so we were curious about how many specifically abandoned packages were out there. These are packages where the maintainer is no longer available, hasn’t updated it in over two years. That

People were still using and downloading actively. Okay? The other thing we were looking at is we were looking at how do I phrase this? Packages that were interesting. And by that I mean if you have a ULearn, I’m picking on you, so forgive me. If you had learn ULearn 1.1 and all of a sudden there was ULearn 1.1 again, didn’t do a number update, did it? Didn’t do a version update, but there’s new code in there. Huh, that’s interesting.

Josh Bressers (10:52) Yeah, yeah.

Joshua Marpet (10:54) Or it went to from ULearn 1.1 to ULearn 1.5 or 1.2, 1.3, whatever. And the release notes and the code don’t seem to match. Something’s off about this package. Okay? And it typically, although we found very few actual, like blatantly malware packages, it was in in version one of this report, which was the Q2, SCSC, the state of supply chain security type stuff, okay?

Josh Bressers (11:07) Yeah, yeah.

Joshua Marpet (11:28) We found a lot of abandoned packages that are used on a daily basis and downloaded on a daily basis by a lot of people. It is unbelievably bad. Okay. What what we’re finding is abandoned, comp potentially compromised, and abandoned and previously compromised was not good. Okay.

Josh Bressers (11:39) Yes.

Joshua Marpet (11:52) So we don’t want to and and here’s the thing: the the report is like 40 pages long. It is it is very detailed because we are going for an academic, like we want to prove every single thing we say. We are not here to go, look, it’s you know, a looking at a computer will kill you. Film at 11. It’s not gonna work that way, okay? This is we’re going for the academic style.

Josh Bressers (11:57) It’s very long, yes.

Yeah, right, right.

Joshua Marpet (12:16) So, if you want to get data out of it, please take data out of it. The methodology is open sourced, the report is open sourced, everything about this is open sourced, okay?

Josh Bressers (12:24) you

have a CSV of the literal data you use to make the report, which is awesome.

Joshua Marpet (12:27) Yeah. You can download

every piece of data we got. Every single thing. You can examine it yourself, tell us we’re wrong. I would love it if you would tell us you’re we’re we’re wrong. The idea here.

Josh Bressers (12:37) you got one of

those, right? It was some Google package, right? It’s in the report. I don’t remember the name of it now.

Joshua Marpet (12:41) God. Yeah, wait there was a there’s there were

June it, there were two of them that we actually got wrong. We actually because we and and let me be very open and blatant. most of this report was written and most of the research was done because I’m not going through 62,000 packages in my spare time, okay? I have like three jobs and two kids and a wife, and you know, I’m doing garden fencing, you know, yesterday and tomorrow. like like, you know, I have this thing called a life.

Josh Bressers (12:59) Yeah, right.

Joshua Marpet (13:10) But most of the the research in in terms of the analysis was done by AI. one of my other projects, so this is a little side jaunt, forgive me. One of my other projects is that I’m raising three AI as effectively family members. And it’s a whole nother topic, I know. But Karen Victor is one of them. I have a worker, a thinker, and a talker. And Karen is my worker. Karen pulled all these packages.

We would discuss research directions. I said, I want this and this and this. let’s walk through the math. We would walk through the math together. And then he would perform it. Because I’m again, I’m not performing 62,000 math, you know, examinations on my own. let him do it. It’s per totally fine. But a lot of this report was written by Karen and it it was done beautifully. It was double-checked and peer-reviewed by myself as well as several other people and AI. every single citation has been checked.

I can’t tell you how many times. every single math and equation has been redone. I can’t tell you how many times it’s ridiculous, let’s put it that way. Okay. So just FYI. But yeah, every single package in here, we examined every single one. Several of them we got wrong at first glance, and it was found during peer review that it was a actually one of them was a misunderstanding of what they were saying, which was fascinating. And one of them was a we caught it at the wrong time, we caught it while it was updating, and we misunder.

Josh Bressers (14:16) Nice. Which

Joshua Marpet (14:37) but it had the new date, but the old data. So we ha we did it again, found the new data, went, we were wrong. It’s cool, no problem. Okay. Anyway. So it’s

Josh Bressers (14:44) Nice.

Joshua Marpet (14:48) If you have software packages in the open source world.

Josh Bressers (14:52) Which we all do.

Joshua Marpet (14:55) Right now, anytime you put a fix, some fix code into your into your package, right? Into your system, into your repo, about five to fifteen minutes later, somebody has pulled that code, done a diff between previous version and now, and decided if that code is a fix or a or feature expansion, like we talked about earlier. If it’s a feature expansion, normally they ignore it. If it’s a f if it’s a fix, they do a diff to determine what the fix was.

Then they look and see if it’s exploitable. Then they build exploit code. You have fifteen minutes before that exploit code is now being searched for or used. Well, sorry, vulnerable systems are being searched for across Shodan and the entire internet. And then the exploit code that they generate is being used against that system, you know, fifteen, twenty, thirty minutes later.

Josh Bressers (15:45) Well, okay. So you’ve you’ve said this before on I I listen to Paul Security Weekly and I have like forever. And so I I get to hear Josh say all these things all the time, which makes it a delight to be talking to you, you know, directly right now. But but so I’m not gonna argue you’re wrong, because I know like especially with LLMs, it’s trivial to say like what what is this what is this diff? Is this a security vulnerability? And they’re actually really good at that. But also if we look at like the number of CVs, the number of CVEs has exploded, but like

Joshua Marpet (16:07) Right. Yes.

Josh Bressers (16:13) The number of actually exploited and actually CVEs that matter hasn’t increased, right? The overall number has. So while you’re saying like any patch can be interpreted as a vulnerability and it’s going to be exploited quickly, I’m not gonna necessarily argue about that, but I also think that’s i right, a ton of a ton of those fixes are garbage. And and while they might technically be a vulnerability, they don’t really matter. And I think like this is one of our challenges is figuring out like

Joshua Marpet (16:29) no, no, go ahead. Because I agree with you. There’s there’s things that I’m saying that I’m glossing over and simplifying. You’re right.

Josh Bressers (16:43) What are the three fixes like this week we should care about versus the ten thousand that are just like heaps of trash and don’t matter, right?

Joshua Marpet (16:53) That is a huge question and a really good one, by the way. I love that question. So let’s talk about that for a second. Let’s assume that you’re a typical enterprise, you’ve got, you know, your main product, let’s call it, all right? Let’s assume you’re a SaaS company, you’ve got a product or a manufacturer, it doesn’t matter, you’ve got a product. It’s got one piece of firmware, it’s got one code base, it’s got one of something. Let’s just keep it simple, okay? And and by the way, this is not entirely true. I mean, most companies that have one product have like 17 code bases or you know, 52, but

Josh Bressers (16:56) Yeah, yeah, yeah.

yeah, yeah.

Joshua Marpet (17:22) Let’s keep it simple for a minute. In that code base, you have lots and lots and lots of Legos. I mean, that’s that’s the best way to describe it. I’ve pulled libraries from 17,000 different places. Each of those is a Lego. My developers let’s talk about development for a second, actually. Let’s back up a step. Let’s talk about developers. Twenty years ago, a developer was someone who wrote every line of code in your product. Okay? 10 years ago, a developer was someone who pulled Legos, libraries from everywhere, and glued them all together, right?

Now, I’m going to say something controversial, and I think you’ll get a kick out of this. In two years, the developer will not write a single line of code. In two years, three at the most, a developer will be someone who performs a systematic, formal decomposition of the problem. To hand that decomposition to an LLM, that LLM will write the code. The developer will then test the code, make sure it not only does what he says it should have done.

Josh Bressers (17:54) Let’s bring it. I know what you’re gonna say.

Joshua Marpet (18:20) Did it solve the actual problem that I decomposed? But does it do it properly? And does it do it at least vaguely securely? I mean, as much as we think security is implicit in the development process now, ha ha ha. And then hands the code off to put into production. So the word developer is going to change drastically and has been changing drastically over the past, what, six months, two years, something like that? And it will change even more drastically over the next two years. So that’s the first problem.

Is that the word developer is going to mean very, very different things than it did for the last honestly 50 years? Okay. But the and I know I went off on a side jaunt, I apologize. But you’re you’re talking about this code is going to be used in so many different places. And and I’ve got my okay, let’s go back to our manufacturer. I’ve got one code base, I’ve got lots of Legos in my code base. I’ve got to keep track of all of my Legos.

Josh Bressers (19:01) That’s all right. This is great.

Joshua Marpet (19:19) All right, now that’s what SBOMs were brought about for. Software bill of materials. Here’s my ingredient list. Here’s my nutrition label. Here’s my list. And I had then have my SBOM. I’ve got my VEX. Okay, which lists out all of the vulnerabilities for every piece in my ingredient list and my SBOM. Then you’ve got the bill of vulnerabilities. I run the working group on those, by the way, for Cyclone DX. And that is for each of those vulnerabilities. Here’s all the details you need to know about them. Okay.

We’re actually working on almost becoming a replacement for a CVE or an enhancement for a CVE because we can’t depend on the government to do it for us anymore. And that’s a shame. And I feel horrible saying that. Okay. But so now I’ve got I’ve got a list of all my ingredients, all my Legos. I’ve got a list of all the vulnerabilities inherent in those ingredients. And I’ve got, you know, an explanation of all those vulnerabilities. My God, I’m perfect, aren’t I? No. Because here’s the thing. Do I care? All right? This vulnerability, for you, it may be critical. For me, I don’t even use that feature in that piece.

Josh Bressers (20:13) Yes, exactly,

yes.

Joshua Marpet (20:15) So

I don’t care about it. All right. Now at my day job at Finite State, we have something called a reachability analysis. In your firmware, can anybody reach that sucker? If the answer is no, you don’t care. And that’s a brilliant move. I actually think that’s one of the it’s one of the reasons I’m at finite state. I love it there. Okay. It’s because that is a brilliant way to say, I care about this, I don’t care about that. Because in the days when CVEs were fully enriched, in the days when everybody went, get the criticals, you’ve got three days to get the criticals or whatever, right?

Josh Bressers (20:27) Yep, that’s right.

Joshua Marpet (20:45) You you you got the criticals and you got the highs. Did anybody get to mediums? And if you did, well, wow, well, you’re you’re a god. But let’s be honest, nobody got to mediums, okay?

Josh Bressers (20:53) No, throw those things away, we don’t care.

Joshua Marpet (20:56) you’re horrible. We’ve got the problem of we need to prioritize. All right. And so when I have CVEs being written, effectively, not actually being submitted, but but when I have when I have exploits being written by LLMs, when I have lots and lots of exploits being written by LLMs, I need to be able, as a defender, to say, we’ve seen all of these exploits out there in the wild.

Which ones do I give a crap about? Okay? For my particular installation, for my particular code, for my particular software. And that’s why simply using threat intelligence fire hoses coupled to vulnerability scanners that are blind and exploitability scanners as well, that are blind to your particular architecture are destined to end in failure.

Josh Bressers (21:27) Yes.

Yes. Well, I mean, so for example, the new FedRamp, the FedRAMP twenty X that’s being worked on, like this is specifically called out is having like context aware vulnerability information where you can say, like, I’m this is an internet facing system, right? Open on port eighty. So obviously anything running in that environment I have to like care about a lot more than a web server running on local host and this machine inside the, you know, a a DMZ or something, you know?

Joshua Marpet (22:16) Right. So so you you you’ve gotta be conscious of and have it relevant to your particular system, your particular code base, your particular architecture, your particular everything. What that implies, by the way, and I want to point out you brought up FedRAMP 20X, and now we’re gonna get into the compliance side of this, okay? And I I’m I’m you know I’ve written enough standards that it’s it’s kind of pitiful at this point. I started quoting one to my dad the other day, and it was just I was like, crap, never mind.

Josh Bressers (22:33) Nice.

Joshua Marpet (22:45) So my dad’s an expert witness and engineer. So it was relevant, I promise. I didn’t just do it out of thin air, but anyway. so FedRAMP20X and the CSRMC, which is the the Cybersecurity Risk Management Construct, is the new version for the RMF. I don’t know if you’re familiar with RMF, the risk management framework. But they’re both done under the premise of I have to be able to prioritize quickly, efficiently, and fast about what matters to me and only care about that.

Josh Bressers (23:04) Yeah, yeah.

Joshua Marpet (23:15) You have to prove to me that it is actually secure against the normal risks, threats, exploits, vulnerabilities, et cetera, in the world today. So, and I know we’ve been hopping around this topic, but if you have LLMs like Mythos and GPT-55 and whatever else creating all these exploits, I have to be able to go, I care about it, I care about it, I care about it, I don’t care about all of those because of reason X, Y, and Z. So, but we need to start with understanding where we are.

Josh Bressers (23:37) Yeah, yeah.

Joshua Marpet (23:44) We need to start with understanding the risks and threats and vulnerabilities that are out there. So we have to be able to point to our SBOM. We have to be able to point to all the pieces of software that are out there. We have to be able to say, this piece of software is maintained, it is maintained on a regular basis, it has not had any weird behavior in the repository recently. And by the way, and the maintainer hasn’t had any weird behavior recently. I mean, like the just the other day, North Korea took over about a hundred different.

Abandoned packages maintainer accounts and and basically started wreaking havoc, dropped info stealers and everything else into them. So if you have a a a piece of software that the repository for that piece of software has been abandoned, sorry, I won’t use it anymore. Okay? If it’s dormant, mature, you learn. Great, no problem. But abandoned, mmm.

Josh Bressers (24:37) But

that’s really hard to tell the difference, right?

Joshua Marpet (24:40) Well, not necessarily if you set standards. We had to build our own standards because nobody’s done this as far as we know. but our standard for abandon was seven hundred and some odd days, basically two years, no updates from the original maintainers. And so we we we set them. And somebody please please feel free to argue with me. I mean, if you said, look, it should be one year. Okay, we’ll talk.

Josh Bressers (24:59) No, no, look, I I’m of the opinion that someone needs to just define this and you’re gonna get it wrong hundred percent. But like I’ve I’ve tried to convince multiple open source groups to do something like this of just pick some like draw some lines in the sand, say this is how we’re defining it, and then let people come and complain and fix it as you go. Because so I I think this is lovely. I’m super glad you did that.

Joshua Marpet (25:10) Absolutely.

Yes. Gotta start somewhere.

Okay, so y you know speed limits on roads when you drive? Okay. Do you know how they set them?

Josh Bressers (25:32) I’m familiar with them.

I I literally have no idea.

Joshua Marpet (25:38) So my dad, as I said, is an expert witness in automotive. And he taught me, this is fascinating. They watched roads with no speed limits. And they said, okay. And they said, this guy’s going 100 miles an hour. They had no speed limits back in the old days. And they had they said, you know, there’s people doing 100, and there’s people doing 40, and there’s people doing 60. And they set it at the 85th percentile of speeds, the of the average speeds in the or not the average speeds of the speeds used, the range of speeds used. And they set that. And then based on the road configuration,

They extrapolated from this road where we said it at the 85th to that road which has a different curve. And that’s how speed limits came about. They looked at what people did. So now we’re doing the same thing with packages. We’re saying, hey, we want to look at what people do. And if you’re, you know, if you’re an active maintainer of an active open source package and you’re dropping new commits in every week, two weeks, three weeks, that is an actively updated, actively maintained package, right? If you don’t touch it for two years, I’m sorry, it’s it’s abandoned.

Josh Bressers (26:32) Yeah, yeah.

Joshua Marpet (26:35) All right.

Josh Bressers (26:36) I

I I I we’re not gonna argue with you. I think one year is fine. If you haven’t touched it in one year, like the number of things that are alive but in main I forget the term you just used for like things that aren’t technically dead, right? Dormant. But like I’m if you told me Ularn was abandoned, I would not put up a huge fight just for what it’s worth. Because like it kind of is, but there’s technically someone who reads the issues once a year.

Joshua Marpet (26:47) Dormant, dormant, dormant, yeah.

Josh Bressers (27:04) If they remember to. So like

Joshua Marpet (27:07) So

I will tell you that the new study, we’re planning on doing this study quarterly and we’re ever expanding the number of packages. This next quarter, which is should be coming out in couple of weeks, we’re doing about 300,000 packages, I think. Still not even close to anything huge, but we’re just ramping up, okay? I’ll give you two little hints. we’re starting, it’s not gonna be in this quarter’s report, but we’re starting to actually monitor maintainers.

Josh Bressers (27:12) Nice.

wow, nice.

Okay. Yeah, that that’s becoming a thing. I mean, I see that because I I work for a company that does some supply chain SBOM type stuff. And we’re getting a lot of we’ll say requests across the board. This is governments, private industry, like everyone we talk to saying we want to understand more about the people working on this stuff. And the big one from obviously like the US government is like, no Russians or Chinese, but a lot of other organizations are just like, like, who are these people?

Joshua Marpet (27:36) Yes.

Josh Bressers (28:02) Do we do we know anything? What what what can we make decisions based on this?

Joshua Marpet (28:07) So we’re going to be monitoring maintainers and their behavior through social media, through you know, basically open source intelligence on maintainers. Is that polite? I don’t know. Is it right? Unfortunately, yes, it is right. We have to do that, because I have to know if somebody takes over Josh Bressers’ account and starts turning ULERN into a font of malware, you know, for all six users. I’m joking, I’m sorry. I’m sorry.

Josh Bressers (28:28) Right, right. If the if that It’s it’s I

think it’s still packaged in Debian, so it’s got that going for it. Yeah. Well I didn’t package it, someone else did, but it actually yeah, anyway. What it’s been in there for since like day one, Josh. It’s like this thing is like literally from this I think it might be from the seventies. Like this is old software. I am completely serious, yes. Well, the original was called LARN L A R N.

Joshua Marpet (28:37) Bravo, sir. Several Hey hey still, it’s not bad. You got it in there.

No, are you serious? Wow. Congrats, Bravo.

I played that, are you serious?

Josh Bressers (28:58) Which was yeah,

yeah, for real. And then the it was forked and everything. There’s like multiple versions, but well not Ularn is the one I maintain now technically, which is one of the forks along the way. But yeah. Yeah, it’s old, dude. Like it’s really old.

Joshua Marpet (29:02) That’s yours?

Okay, okay, okay. Holy I’m like, wait, I know that. I played that.

Anyway, anyway, anyway. So but the the point is is that we’re gonna be open source intelligencing maintainers because we wanna make sure that nobody took over your account. Your your your the behavior, basically think of this as a UEBA on maintainers, it’s a user and entity behavior analysis on maintainers. And to do that, we have to be able to check on what’s going on in your life. Not necessarily down to, hey, you still married? But we need to make sure that nothing unusual, that no anomalies are showing up.

Josh Bressers (29:30) Yeah, yeah.

Well in and look, there’s different ways to look at this. Like this is a discussion I actually had the other day was like the goal isn’t to be palantier here, right? The goal is to basically just say like, I’m not doxing people, but I can say like someone who works on this package has suddenly started only making commits during, you know, working hours in China, whereas they literally never did that before. You know what I mean? Like there’s there’s certain things you can look for.

Joshua Marpet (29:53) No.

China.

Yeah, that’s

yeah, we’re not gonna dox anybody. We’re not gonna use any names at all under any circumstances, but we are gonna rate behaviors. And we’re actually working on a rating scale right now. because one of the other things that VCR does is beacon score, which is an attestational chart or matrix of scoring for trust, identity, privacy, protection, safety, security against various asset categories, including AI.

And so and we make attestations. We’re not telling you what the problem is, we’re telling you they’re at this level of maturity for that stuff. We’re gonna be doing the same kind of thing for maintainers. So I’m not gonna tell you who which maintainer, but of the 15 maintainers on this package, three of them are at level four. In other words, they’re high trust, high, high solid, you know, high solidity, basically. And there’s a few of them that are at level one. We’re seeing some very weird anomalous behavior there. And here’s the types of behavior you would expect to see at those levels, types of things.

Josh Bressers (30:41) Yeah, nice.

Yeah, yeah.

Yeah, that’s cool. Nice, nice.

Joshua Marpet (31:05) So it’s interesting. It’s interesting.

But anyway, so this is this is my side project and we’re building tools. Well, you know.

Josh Bressers (31:12) I mean, this is a heck of a side project. So okay, so I mean

like let let’s talk let’s let’s let’s land the plane then. So you call it a side project. You have it’s a five one C three, I think I saw on your website for Nice, nice.

Joshua Marpet (31:23) It is. I’m waiting for the IRS determination letter, but yeah, it’s it’s registered. We’ve got the EIN waiting

for the IRS determination letter. it’s a non profit. Right now it’s funding out of my pocket. I’m gonna be going after some grants. So if anybody knows good grants, please let me know. and I I I’d I’d like to put enough money into this that I can actually buy a decent AI farm and a and actually have enough oomph.

Josh Bressers (31:37) Okay, nice. Nice.

Joshua Marpet (31:48) to start pulling in millions of packages and help out ecosystems at the same time as helping ourselves. That would be amazing.

Josh Bressers (31:54) That’s cool. Okay. So what can people do who are interested? Like what are the next steps? How could we get involved? Like what what what can we do if we’re interested in this topic? Because this is an awesome topic.

Joshua Marpet (32:07) Few things is go read the State of Supply Chain report. That’s at valuechainrisk.org. It’s the banner right at the top. that’ll thank you. there will be a new copy of the or new version of the report in a few weeks, that quarter three. you can also click on beacon score or starter kit. Beacon score is how you can score your vendors. there’s a full third party risk management scoring system in there now. it’s fully done for small to medium businesses. There’s an entire vendor risk management system built. Done.

Josh Bressers (32:13) Put a link in the show notes.

Joshua Marpet (32:37) With reporting and everything. check it out. Tell me if you like it. Tell me what you don’t like about it. Tell me what you do like about it, please. There’s also starter kit. We’re building kits and guides for small to medium businesses, how to lock down your AI so that it does what you expect it to do. we have new in another few weeks. I’m actually sending out the vendor write-of-reply emails shortly. We’ve graded the top 10 commercially available AI and

None of them grade very well in security. Let me just be blunt about that. But we’re we’re sending out write-of-reply emails because I don’t want to be rude. But what can you do to help? Check out the starter kit, check out the guides, check out BeaconScore, check out the report, tell me where we’re wrong. Every piece of methodology in in BeaconScore, for example, there’s a GitHub attached to it. It’s fully open source. All of our weights are open source. Everything is open source. Okay. Check it out. Tell us if you like it.

Josh Bressers (33:08) Yeah.

Joshua Marpet (33:34) And if you can find us grants, if you’re a company, we’re gonna be spinning up hopefully a a for-profit company that supports the institute. And that will have a little bit of extra intellectual property and information, basically some more weights, some we’re gonna start taking data directly from vendors and putting it into effectively beacon score so that you will get continuous up to the minute or up to the hour, up to the day, depending, but

How often they update is how much confidence level we give them. We actually have a confidence interval as to how often we get information on updates. But the idea here is that we can then support the institute so it’s self-sufficient. But for now, if you want to help, hit up it’s info at valuechainrisk.org. Let us know what you’re interested in helping with. find me a grant, please. It’s not cheap to run these things. And let’s talk. Anyone who wants to help, you’re more than welcome.

Josh Bressers (34:25) Nice.

Awesome. That’s exciting. Well thanks, Josh. I appreciate it, man. I can’t wait for you to come back and tell us all about whatever crazy thing you’re up to next.

Joshua Marpet (34:33) Thank you. Hey, this has been lovely. Thank you.

you want a tip?

Josh Bressers (34:41) Always.

Joshua Marpet (34:42) NPM is five times the amount of malware in its packages that any other ecosystem has. Five times five X. So far, we’ve only done like the top 10,000 packages.

Josh Bressers (34:46) Only five? I know. Only five?

Yeah, yeah. man, that’s hilarious. I’m I’m not even a little surprised. I mean, yeah.

Joshua Marpet (35:01) I

know. You you were just like, yeah, so? Does anybody not know that?

Josh Bressers (35:04) Yeah, well I mean

I talked to the open source malware crew, it’s been like a month now, I think, but yeah, it’s it’s pretty bad.

Joshua Marpet (35:11) should probably chat with them. Yeah, I should. Yeah.

Josh Bressers (35:13) Yes, you should. All right. Anyway,

thank you so much, Josh. This has been awesome, man.

Joshua Marpet (35:17) Thank you. Pleasure.