Josh welcomes back Andrew Nesbitt to discuss some recent blog posts he wrote about the challenges of new ecosystems as well as challenges of no ecosystems like C. There aren’t very many people who look at multiple ecosystems in the way Andrew does. He has thoughts on why it’s so hard to create a new ecosystem as well as some of the reasons we don’t see a C language ecosystem. Andrew has a ton of interesting ideas and insight for us about both existing, new, and nonexistent ecosystems.

This episode is also available as a podcast, search for “Open Source Security” on your favorite podcast player.

Episode Transcript

Josh Bressers (00:00) Today, open source security welcomes back Andrew Nesbitt, open source developer and package manager nerd. I am always ecstatic to talk to Andrew. So welcome back to the show, man.

Andrew Nesbitt (00:09) Thank you very much for having me back.

Josh Bressers (00:11) I can’t wait. this is, I told Andrew this right before we hit record and this was unbeknownst to him, but this is kind of a three-part package management special we’ll call it. Cause we had Brian Fox talking about the sonotype report. Then we had Michael Scovetta talking about Alpha Omega and package management. Everything was kind of the underlying theme there. And now I’ve got Andrew who, runs eco. How did you just call it ecosystems? I don’t actually know how to pronounce this in public. Okay.

Andrew Nesbitt (00:37) Yeah, yeah,

the domain name is unpronounceable and I love seeing people struggle to say it.

Josh Bressers (00:43) I always, I

always say ecosyst dot ms is what I tell people and then they’re like, what? So.

Andrew Nesbitt (00:50) Yeah, even more fun with people with different accents as well from different countries.

Josh Bressers (00:55) ⁓ sure, sure. But, but Andrew has been blogging prolifically for like the last probably two months or so. And I’ve picked out two that again, this is not on purpose, but they are very package management, like ecosystem related. And I think they fit in nicely with my chat with Brian and Michael. So, so we’ll kind of keep it going. So you have, you have two posts. I’m going to do my best to read the titles of those screen up. the first is.

Zig and the ⁓ X N like multiplied and supply chain problem. And the second is the C shaped hole in package management. I feel like I got those right. So that makes me happy. ⁓ okay. So anyway, here’s what I was thinking and why I find this so interesting. And I’ll let you kind of take it from here, Andrew, but I feel like

your post specifically about Zig and the like ⁓ times N supply chain problem is not something I’ve thought about before, but I will let you explain it because as much as I would love to, you are literally the author of the post, but kind of explain to us what you were talking about and we’ll take it from there because I love this one.

Andrew Nesbitt (02:00) Yeah, so straight off the top, it is not actually about the Zig package manager at all. And I left out a lot of opinions and some of those might be quite spicy about how Zig has implemented its package manager. We can save that for another conversation. But instead, it is basically saying like, there are lots of pieces to a package manager and a package manager ecosystem and the…

Josh Bressers (02:07) Yes.

Andrew Nesbitt (02:29) the whole ecosystem for a given language that all kind of like have to be rebuilt over and over. And the Zig kind of example is excellent because it’s like the newest kid on the block and a new language with a new package manager that’s not reusing something else. It’s like a lot of the JavaScript package managers all ride along on the NPM registry, ⁓ but

Zig is kind of like a whole new thing. So then it’s kind of an interesting case to say, right, well, what does it take for Zig to be kind of accepted into the higher echelons of the world of package management? What different steps does it has to go through? Either things that they are in control of or things that they’re not in control of, which requires like other ⁓

companies and other kinds of tools to be able to kind of like recognize Zig and the specifics of how that package manager works to be able to integrate with them so that anyone who’s using Zig can kind of then follow all of the package manager.

norms that they would expect to be able to do with any other package manager that they already can do with Zig as well. And then when another programming language comes out with another package manager, they have to go through the same thing again. And some of these things take a long time and can actively harm the adoption of those languages as well. Like if you can’t have Dependabot and

security advisories actually recognize your, any of your code, you’re going to be like, ⁓ that’s kind of annoying. But then you might also have your boss being like, well, if, like, if we’re not getting any visibility into any of the, like the packages that we’re using, like, how can we get sign off from our security teams? I mean, you would know much more than that about me. That’s, that’s a sentence. but there’s like, there’s

Josh Bressers (04:35) there.

Andrew Nesbitt (04:40) I basically listed in that post here are all of the different places where Zig is going to have to reach out and encourage or build support in things like Dependabot, all the vulnerability databases, all of the like source code analysis tools, the artifactory and package proxying, package hosting services, platforms like ecosyste.ms and deps.dev and Libraries.io.

integrations into GitLab and gitea ⁓ a PURL type, and then like ⁓ acknowledgement in the SBOM formats. Like you can’t make an SBOM for a Zig program right now particularly well. They need, they have to have something like trusted publishing and kind of integrate all of the OSSF repository security best practices. Like there’s loads of different pieces and you multiply that by how many package managers there are. feels like

every one of these has to do the same thing over and over rather than being able to kind of slot into an existing system.

Josh Bressers (05:46) Yes. And I know you have another post. don’t, I’ll dig up a link to it, but you kind of talk about trying to standardize some of this because like this is, I never thought of any of this before. And I work for a company that makes a tool that builds SBOMs and scans for vulnerabilities. And like, we are adding ecosystem support all the time for package registries and like container providers and you know, Linux distros like pick, pick a topic and it’s there all the time. I don’t know if we have Zig support yet.

But like

Andrew Nesbitt (06:16) I suspect

not because the files are written. I said I wouldn’t talk about this, but their files are written in Zig. So it’s actually really hard to parse the manifest files right now.

Josh Bressers (06:21) ⁓ okay.

Then probably not yet, but it’ll come. Like I guarantee it. It’ll happen at some point. And this is fascinating to me.

Andrew Nesbitt (06:30) But then your work

is reproduced by then everyone else as well, right? It’s not like we all depend on Syft I do depend on Syft for some ecosystem services, so I get that for free. But everyone basically ends up having to do the same thing over and over.

Josh Bressers (06:35) Yes.

Yes. And I have a good example of this actually. So when Chain Guard created their, their Wolfi distribution, as well as then what do they call it? Chain Guard images. think they’re like, you know, they’re small hardened images, your container images. They went, and I give Chain Guard a ton of credit for this. They went to like Syft and Grype and Trivy and dependency track. And I don’t know exactly all the things, but they went to them and gave PRs to support their ecosystem, which is obviously an obscene amount of work.

But like that’s, I mean, that’s one of the ways to help bootstrap this, but you know what I mean? Like an open source project can’t do something like that when it’s a couple of people.

Andrew Nesbitt (07:23) Hmm.

And also like you, probably just mentioned all tools that are open source and able to accept PRs. Like you can’t go to GitHub and say, please add support to my, to your dependency graph feature for, for Zig. Like this is just not going to happen. And GitHub isn’t exactly prioritizing adding support for new ecosystems to a lot of those tools right now, because they’re distracted by other ⁓ semi-intelligent things. ⁓

Josh Bressers (07:31) That too.

Yeah.

Andrew Nesbitt (07:54) But there’s like, I guess the parallel is ⁓ the language server protocols that kind of spread quite quickly with the explosion of different editors to be able to say like, if these languages can implement a protocol that can basically go like, okay, well, can you give us syntax highlighting and ⁓ various kind of linting warnings, potentially even security problems or like…

don’t do this kind of things through the protocol, then each editor doesn’t need to implement its own support for each different, like we used to have textmate bundles and the textmate bundles ended up getting reused by other totally different editors so that they could basically get all of the support for different syntax highlighting for different languages as quickly as possible. But LSP then said like, okay, well if each

language implements an LSP server, then the editor basically can go and look at a known location for that thing and doesn’t need to then implement something specific. It’s just like, the protocol tells me everything that I need. Like I wrote about what would it be like if we had a protocol for package management, which would then enable Zig to basically declare I’m this kind of package manager. I do these kinds of things. I have these kinds of rules.

my dependency resolution happens like this and my manifest files are named like that and you can find them over here. Here’s where my registry is. Here’s the API for the registry. Of course, Zig doesn’t have a registry. So that’s always a fun one. And they will come back to regret that later, I think. But there’s like trying to move those things forwards in the world of package management as probably Brian and Michael had talked about is

difficult because it’s so little actual like resources available to do new work rather than just keep like tending to other fires all the time.

Josh Bressers (09:54) Yeah.

And there’s another piece of this, I think, that doesn’t get talked about very often, but I thought about when I was reading your post. So like, I’ll use ⁓ a Syft and Trivy as an example, right? Where Trivy is one open source SBOM slash vulnerability scanner. Syft is an SBOM only scanner, but then there’s a tool called Grype that does the vulnerability stuff. It’s just the, I don’t know, more Unix-y. We could argue about that. But here’s the thing is if you run Trivy and then you’re on Syft or really any combination of SBOM scanners,

you generally get different results because there’s certain opinions every scanner has about what is being conveyed by the package managers that it sees, right? And this is where like what you’re talking about, this isn’t opinionated necessarily. This is the package ecosystem kind of setting their rules versus today where these scanners are deciding on the rules. And there’s things like, do we include, you know,

Andrew Nesbitt (10:41) Mmm.

Josh Bressers (10:58) development dependencies. How do we catalog like multiple copies of the same package that we find? How do we handle nested dependencies? And it’s like, no one knows.

Andrew Nesbitt (11:04) ⁓ yeah, that’s a great one.

Yeah, as soon as you go to implement this, you’re like, my God, what is going on with like these two particular package managers that have a very strange behaviors that I didn’t expect or like the workspace features are totally different in this place. And or this manifest file is Turing complete and I can, I have to execute something to be able to get a result out of it. All of those things are like then require, okay, well,

This scanner has just uses regex. This one actually shells out to the underlying package manager tool, but then which version of the package manager were you using? Because you might get different results based on a big upgrade that changed the dependency resolver inside of it. All of those things make for an absolute mess, but everyone solves the mess in their own particular way. There’s no specs for any of this stuff.

Josh Bressers (12:07) So I was reading a paper from an academic the other day that was looking, they were doing this, right? Where they were comparing all of the, well, not all, but a bunch of the open source SBOM scanners. And there was one, I can’t remember which one it was now, but it would actually try to run like NPM install or pip-r install on the like requirement, right? Like it’s literally doing like arbitrary code execution during SBOM creation. like, I don’t even have to say about this. Like this feels horrible to me.

Andrew Nesbitt (12:36) I mean, especially if you’re using it for investigating security problems, it’s like, we are the security problem now.

Josh Bressers (12:39) Yeah. Right.

And, but at the same time, like no one said, don’t do that. You know, it’s, ⁓ this is so bananas, but okay. But now I want to tie this then into your C shaped hole post, which I feel like is super related to this. And I have long wondered why no one has tried to build a C package management system, even just badly. And I feel like this is why.

Right. Because like these are the problems that no one wants to solve and, really are almost unsolvable in the C world. Cause we already have, no, they’re bad, but we have mechanisms for getting C libraries to the places we want them when we’re building our C stuff. Right.

Andrew Nesbitt (13:10) You

Yeah. And you get this kind of strange, it’s hard to write about it without angering someone. ⁓ When the term package manager means different things to the different groups of people where it’s like, this is for managing my applications in my Debian box. ⁓ Versus this is for managing the dependencies of my Python web application that I’m building. The problem is that

Josh Bressers (13:34) It’s okay. I’m fine with that.

That’s true.

Andrew Nesbitt (13:57) those two things get really mixed together where it’s like, so we ship some applications in PyPI as a single binary, or we have some Python packages repackaged in our Debian package manager because we needed them as dependencies rather than just like it. It’s where you get that kind of murky middle of it’s not an app store where you download a single binary because

that single binary might need patching and we don’t want to have to keep rebuilding and republishing every version of everything all the time. So instead you make it so that you can bump individual pieces of dependencies in your system package manager. But now your system package manager is acting as a dependency manager and you mix the two things together because all the language package managers are like, well, we don’t really do the C stuff. We’ll just…

that’ll happen elsewhere, but never actually define it because of, mean, there’s various problems there, but the main one is like most package management is about naming. You’ve got names all the way down. You’ve got the package name, maybe a namespace, the name of the registry, then your names of versions, which then ⁓ also will have names of builds, names of platforms, which turns out like they have a whole nother blog post about that, which is like, wow, these platform names are

crazy in that you can’t get things across. Like I love that Zig’s who Zig again, the most recent people to investigate this came away deciding that the default name of a binary for Windows should have GNU in it because not everyone has the ⁓ proprietary Microsoft ⁓ C build chain, whereas the GNU one is always available.

Josh Bressers (15:27) Yeah.

Andrew Nesbitt (15:52) So they fall back to that because the binaries then have GNU in them for Windows, like as the name of the platform, which feels great to be like, yes, this is how you know it’s for Windows. It’s like, that hurts my brain.

Josh Bressers (15:58) Awesome.

Right, long

history of GNU and Microsoft getting along, right?

Andrew Nesbitt (16:09) Yeah. So you have like all of these names. And one of the hardest things for the language package managers to do is to say, I’m going to depend on this C library. It’s like, well, pick your name and guarantee that name will not be the same in every different place that packages ⁓ these C libraries that you want. So what are you going to do? Like they just end up going like, okay, well, that’s just user’s problem now.

and throw their hands up unless you have the kind of like nix and geeks where they say, OK, well, what you’ve got to do is repackage everything into our space. We support C and we support the language package manager is just fine. But you then are like, we have to go wholesale into one of these other ⁓ these other package managers and you lose all of the things that you’re used to in your particular package manager. ⁓ Or you have the kind of

HPC package managers, SPAC, terrible name, and EasyBuild, which are both kind of like designed for building for high performance supercomputers. And so they have to have like the whole tool chain, including the compilers and the versions of compilers in their dependency chain. And so they have the like the best picture of the whole dependency tree, but they still have to end up going like…

We’re going to have to name some things, which is like, oh, great. And you imagine you go back to this kind of like, if we had a protocol for package managers and you could say like, oh, I’m going to depend. I’m going to jump across ecosystems. I want a dependency on something in another ecosystem, which could reasonably work. Say like, I’ve got this JavaScript project that depends on back in the day, it used to be the SAS Ruby library.

and it was called SAS and RubyGems. And you could say, depend on this. Everyone hated it because they had to install Ruby to be able to compile their CSS for their JavaScript applications. But at least, like, there was a name that you could point at and like a RubyGems, you could turn it into a package URL these days. But with C, you’re just left holding like a big pile of different names. How are you supposed to try and work that out?

Josh Bressers (18:07) Yeah.

Okay. I’m

going to push back on that though, because package URLs are one of my favorite examples where you can say you can turn something into a package URL, but the package URL includes the distribution mechanism. So depending upon how you got something can change the package URL. So you can have the same thing with two different package URLs, even though the artifact on the machine is literally identical.

Andrew Nesbitt (18:43) Yes.

Yeah, it’s really not ideal.

Josh Bressers (18:57) I’m not I’m not bashing

PURL mind you I understand why they did that but it is one of the difficulties

Andrew Nesbitt (19:02) Yeah, PURL is very practical for the language package mergers, which ends up being quite impractical a lot of the time when you try to use it for things that it wasn’t designed for. Of course, when an SBOM says we need a PURL for absolutely everything, you’re like, okay, right, well, luckily this accepts arbitrary extra bits in this string, so we can just slap anything we like in there to be able to end up in a place. And going back to that platform URL.

Josh Bressers (19:20) Yes.

Andrew Nesbitt (19:32) platform strings also ends up being like, this is one PURL, but we have a single string which then represents like all of the specifics of the binary that was ⁓ built from that thing. In theory, that’s a whole nother topic we can go into another time around the publishing of those binaries and whether or not you actually like can reproduce from start to finish. But the like

when you get to the kind of end result of I’d like an SBOM, often the SBOM is not something that can be reversed back into like, here’s how to produce a real build environment because there’s this gap in the middle that’s like, how did you get here? And the only place I found that’s been close to being useful to find that out is a Docker file, which is really not particularly good, right? You have like, okay, well, at least this Docker file is like, we declared our operating system.

So we know what the package manager was and we can see that there’s some lines that install some top level packages here for that package manager. Because that doesn’t actually help if you’re not on that operating system. ⁓ You often see in a readme like, if you’re on a Mac, then you can brew install this. Or if you’re on Ubuntu, you can apt-get install that and then it finishes. There’s no other distros ever included in the readme for how you might.

Josh Bressers (20:58) right.

Andrew Nesbitt (20:59) Definitely not Windows as well because everyone’s just like, no, no, definitely not getting near that. I don’t have a Windows machine to test on that.

Josh Bressers (21:07) ⁓ I know, I know. We have this problem. So Syft and Grype, we have Windows builds for them and they break all the time because we don’t have reliable access to Windows to do the testing on. And so often it’s like users tell us we broke something and then it’s like, ⁓ we’ll go fix it now. But yeah, I get it. Like it’s definitely, it’s a bizarre world.

Andrew Nesbitt (21:16) You

And of course,

yeah, you’re then assuming that all of the dependency chain also all testers against Windows, which a lot of language package managers also probably not doing a lot of Windows testing ⁓ where they’re just running in GitHub CI on Linux, for example. ⁓ The other problem you get with having this kind of gap is when you try to do anything around ⁓ kind of…

Josh Bressers (21:43) Yes.

Andrew Nesbitt (21:52) supporting open source and sustainability because you quickly realize like, we have this Python package. It looks like it’s heavily used, but actually like a lot of the heavy lifting that goes on is from a Fortran package or an R package or a C library inside of it that gets built, but it never like is declared in a machine readable way.

that allows you to follow that dependency chain down to go like, okay, well, we can give transitive credit to these lower level pieces of software. It just stops at the, that kind of like the chasm of like, we reached the system and we, and exactly the same for the kind of security problems where it’s fine if you already have the built machine, you can run a scanner on it and go and see what’s there.

Josh Bressers (22:33) Yes.

Andrew Nesbitt (22:45) Or you have the binaries and you can go and do some inspection there. But if you wanted to go from source and say, does this end up pulling in libxml2 and what version at some point when you build it later so that we can kind of be proactive rather than reactive, you again run into that chasm of like, OK, well, we’re just going to go look for matching of strings in this source code somewhere.

Josh Bressers (23:09) Yes, that’s, well, Syft does that. We look for binary strings, in fact, in things. I mean, and you talk about this in your C ⁓ post. You call them, well, you call them phantom dependencies. You credit Endor Labs for the title. But then I know like this is something Seth Larsen has talked about as well from the Python universe where like there are many Python packages that build binary stuff that’s in the wheel.

But when you do like an SBOM scan, you just get the Python package. doesn’t include, and I know they’re trying to now do like SBOMs or Python packages and things like that to start capturing some of this, but like, this is just turtles all the way down, man. Like everywhere you look, all of these problems are just like, feels insurmountable sometimes.

Andrew Nesbitt (23:54) Yeah. And there’s people that are trying to kind of come at it from the other direction as well, where they’re scanning the binaries. Vlad from Open Source Pledge is trying to do this, partly again, from the kind of open source sustainability side of things to go like, can we scan all of the wheels on PyPI to try and find the symbols that indicate which ⁓ system libraries are on there that weren’t declared or aren’t, you know…

in the Python metadata that has kind of exactly the same problems of like it’s really lossy and it’s a kind of best attempt where you’re really hoping the compiler is like, actually, can we leave more notes inside of these things? I remember at FOSDEM this year, someone mentioned at the end of Vlad’s talk that they had built, I think it was the guy was from Sony had built a tool.

that would compile things and actually leave almost like an SBOM like thing inside of the binary to say like, here’s the things that were there. And I was like, great, can you just rebuild everything as well, please, using that? Like, it doesn’t help with any existing tools and trying to get that lift of getting everyone to kind of acknowledge. It’s even difficult to get that kind of like problems acknowledged. As you see, if you go following the comments of that C-shaped hole,

A lot of people will say, actually, this is fine. I don’t need to solve this any further. Especially people who are quite happy just living in their distro package manager without ever touching the language package managers.

Josh Bressers (25:31) Yeah.

Yeah. Yeah. Okay. So I, first of all, your, your point about the compiler adding this information, this is something that go and rust compilers actually do now is they add that metadata. So it’s a marvelous thing to do. And I do hope we see some of the C compilers start doing that. Your comment about, you know, a comment or saying, this isn’t my problem. It is not their problem. I mean, this is one of the challenges with a lot of this sort of stuff is

The problem actually belongs to the person you’re giving it to. And so it’s not something that affects me. I mean, this is one of the challenges I think I’ll never forget. I had this discussion one time at an OpenSSF thing where I, they were talking about getting open source projects to build their own SBOMs, right? And then distribute them. like, what is the advantage to an open source project doing this? And they’re like, what do mean? I’m like, why should they do this? It literally gives them zero benefit. And it was like deer in headlights. They’re like, well, there’s a benefit. I’m like,

Andrew Nesbitt (26:17) you

Josh Bressers (26:30) What is it? And they’re like, more people will use their project? No, no, they won’t. That’s not a benefit.

Andrew Nesbitt (26:35) Yeah, as someone

who indexes millions of open source projects, no one is using SBOMs in open source, like zero. There’s nothing there. The only people that have SBOMs are companies that are… We won’t go as far as saying open washing, but they are, know, like a project happens to be open source, but it’s basically a enterprise product. And so we have SBOMs, but there are no packages that are doing this.

Josh Bressers (26:43) Yeah, I know, know. Yep.

Yeah. Right.

Andrew Nesbitt (27:04) unless their infrastructure does it automatically for them.

Josh Bressers (27:08) Right. And we’ll see if the CRA makes it happen. I hope it does, but I’m not entirely convinced yet. Like we’re going to find out. Okay, Andrew, I want to, we got to bring this one home. So here’s my question for you. And like, this is the big question. How do, how do we start fixing this? I know you’ve done some talking about some sort of standardized like metadata for this, but, like the reality is this comes back to the, isn’t my problem, right? Like

Python and NPM and Go and Maven and Cargo and everywhere. Actually, does the Go repo have a name other than Go? Do you know? You’re the person who know. I don’t think it does. You need a fancy name. Yeah, anyway, okay. That’s not important right now. So how do we convince some of the existing, like it could be easy to convince Zig.

Andrew Nesbitt (27:46) I mean, no, it’s not like there’s a bit of infrastructure there, but yeah, it’s basically go.

Josh Bressers (28:02) to do this, right? Cause they don’t have a ton of infrastructure, but a lot of these other package repositories. And as we talk, I talked about with Michael Scovetta, you know, a week ago, there’s not like these, these ecosystems aren’t flush with cash to do this kind of work. like, give me hope, like fill me with hope and optimism, Andrew.

Andrew Nesbitt (28:21) that’s not really my thing. the package management studying is often like, wow, look at all these different shapes of mess that we have and somehow they work. and I don’t have necessarily any particularly good, ⁓ concrete solutions. I think teasing out more of these patterns and being able to say like, it’d be great if you have more kind of high level

Josh Bressers (28:22) Stop.

haha

Okay, that’s fair. That’s fair.

Andrew Nesbitt (28:51) descriptions of things actually like getting package manager registries to provide an open API spec file for their registry APIs is already a big stretch for some, especially if you don’t have a registry, then you know, everything falls to pieces. But thinking about some of those things, slightly higher level and building tools that work with multiple package managers often shakes out that quite quickly and goes like, oh, okay, right. This stuff is

Josh Bressers (29:00) That would be so nice.

Andrew Nesbitt (29:20) harder. The other thing I guess we could look at, and this is kind of like, I guess a 10 year kind of forward thinking thing is could you look to start to have some standardization as maybe a second layer or an alternative API to these package managers, things like using Homebrew is just about to switch or is in the middle of switching to using OCI as the format for their archive files.

you could push more package managers to also provide their packages like that, which then gives you at least a way of saying like, okay, at the file level, I’m just talking about the individual archives. They come with some predictable chunk of metadata and I can reason about them. And then like, okay, well now how do we go about fetching the list of those archive files? Like what versions are available in this registry? There’s no good standard around those things.

because the resolvers use that data to then decide, like, how do we do dependency resolution? you can… And the thread through my blog, because there’s… I am way too scared to try and write this up as one humongous kind of manifesto of how to solve all of package management is like just slowly tapping away at little cracks in it to be like, maybe eventually we could end up going like, here’s the grand, like, whole picture of how this stuff works.

And here are places that we could upgrade it independently to get to a point where things are starting to see those kind of reductions of the amount of ⁓ times N implementations that have to happen. And also then what kind of comes out the other side of that. But it is a long and painful road, especially with people continually bringing out new stuff that doesn’t ever go looking at

you know, the existing world of package management, which is kind of why I was trying to write this stuff down to just be like, look, let’s just describe the state of the world right now because it’s a mess. And if you can see it’s a mess, you can maybe go like, ⁓ maybe I can not add a different kind of mess to the pile. Instead, I can use one of the existing messes. ⁓ So there’s less number of messes.

Josh Bressers (31:42) Well, I mean, XKCD, common conclusion here, right? Like now we have 15 standards, but okay. You have given me some optimism here, I will say. And I’ll also point out, I’m going to put a link to Andrew’s blog, just his, like the post we talked about, as well as the overall blog, like just go read them all. They’re amazing. And one of your recent posts, when I say recent, I don’t, in my brain, it feels like it wasn’t that long ago.

Andrew Nesbitt (31:49) Exactly.

Josh Bressers (32:06) You wrote a post about like the different archive formats from the package managers and how like hilariously bizarre they all are for a variety of ridiculous reasons, which is, was awesome as well. But anyway, here’s what kind of gives me some hope is number one, you’re pointing all this out. I don’t know of anyone other than you who exists in a place where you kind of straddle all of these ecosystems. think almost everyone I know is firmly rooted in one ecosystem. And of course their ecosystem.

is either fine or the worst thing in the world. And they don’t know anything about anything happening outside of that universe. Right. So that’s kind of a big deal. But I think you starting to chip away at this, like you said, and just starting to point these things out. And not that I think it’s going to necessarily create change in the short term. You’re helping build a foundation that then maybe hopefully we can have things like, know, Alpha Omega type projects in the future. You know, Michael Scoveta, who I just talked to, they have the ability to start funding some of these things because

you know, funding a second level API in all the package registries would be an amazing feat and would give, you know, obviously all of the downstream people like you and, and SBOM generators and all of that stuff, an easy way to get the data we need. Cause as you know, extracting useful metadata out of package repositories is like the hardest thing in the world. In fact, before I found your project, I had my own tooling that would go into

Andrew Nesbitt (33:25) You

Josh Bressers (33:34) know, PyPI and NPM and things and extract all the data. And the scripts are just completely ridiculous because you have all these like insane nested if statements, which I guess it’s not that they’re gone. It’s just your problem now instead of mine. But, but it’s super hard. Oh, they’re there.

Andrew Nesbitt (33:48) Yeah, I tried to hide them away deep down. And as someone

who just implemented support for geeks today, feel, ⁓ feeling that especially because the manifest files are Scheme, which is, you know, it’s just lovely to try and extract information out of that. ⁓ Yeah, we have, there’s many layers to this. ⁓ And as Alpha Omega, ⁓

Josh Bressers (34:05) Wow.

Andrew Nesbitt (34:15) like to say and Mike Windsor especially that package managers are, especially the registries are big points of leverage to be able to have a outsized impact by improving the registries can then have kind of knock on effects to going like, ⁓ wouldn’t this be interesting if we could do this? And it turns out most package managers are similar, have some, they’ve got to have like there’s some kind of law of package management says.

that you’ve got to have some kind of weird, unusual edge case that no other package manager has. Like there’s not one that isn’t like, ⁓ why is that? Like you almost got a full scorecard worth of ticks and then you run into something that’s like, ⁓ there you go. You lost it. The perfect 10. ⁓ But if you look at the kind of like, if you strip it all away, most of these package managers go from a given name, work out like where

Josh Bressers (34:45) You

Yeah.

Andrew Nesbitt (35:09) to download a thing and then put it in a place and then tell the runtime or the runtime has a known list of places, I will load stuff from there by its name. Like that’s basically all they do. It’s just that they kind of have a load of different kinds of scaffolding around those things. If you had enough of a shared like commonality of APIs and registries and all these different pieces,

Obviously there’s all kinds of interesting problems around governance that start to shake out where you’re like, if you remove the scaffolding and the infrastructure, you kind of like, actually we still have to deal with naming and takedowns and security problems. Like that doesn’t go away, but the infrastructure could become very kind of like uninteresting rather than now where it’s kind of too interesting for what it is.

Josh Bressers (36:03) Yes.

Andrew Nesbitt (36:04) You like

you don’t want your roads to kind of just have this like, ⁓ well, if you’re in this kind of car, then your road occasionally does a, ⁓ like a loop, the loop in the middle of the highway.

Josh Bressers (36:18) Right.

Yeah, yeah. No, I agree. Yes. Yes. That is a good analogy. And I’m going to end us on that because you are completely correct. Like this is the sort of thing that should be extremely boring and it should be the roads and power distribution of open source. But unfortunately it isn’t really, but.

Andrew Nesbitt (36:37) Yeah,

if I can run out of things to write about because it’s all got so boring, then we’re in a good place.

Josh Bressers (36:42) I love it. That’s so good. All right, Andrew, until next time, my friend. Thank you so much.

Andrew Nesbitt (36:48) Thanks, bye.