I keep seeing commentary about AI making open source dependencies obsolete. The idea is that instead of using an open source dependency, the AI will just write all the code you need. No more need for that random person in Nebraska. They can finally take a well deserved break!
Some people think this is inevitable, some think it’s hogwash. I like to take the stance of disliking everything equally. But to better understand all of this, let’s break it up into a few possible outcomes. There are 4 basic things that could happen if we take these arguments to their ridiculous extremes.
- AI writes 100% of the code. You tell it you need something served up on port 80, it writes it all from the web server on up. Maybe even the whole operating system, why not!
- You use AI to replace some of the open source you use. Rather than write everything, some open source is used and some code is written by the AI
- AI is very aggressive about using open source. It tries to reuse as much code as possible
- AI writes almost none of the code, and you use very little open source. This would be development the way 1975 intended
There would be good and bad for each of these possibilities. But the thing I want to talk about is the second order effects.
The second order
Calling something “the second order” sounds ominus. Like some sort of secret society. Everyone wants to be part of a secret society! It’s not really a secret, but at least it sounds cool.
Let’s think about the second order of these possibilities. What I mean is the things not immediately obvious, but will be problems we have to deal with eventually. There are no solutions that won’t have second order downsides.
AI writes 100% of the code
You’re now responsible for the security and maintainability of all your code. You can’t actually properly audit it all. There will be millions of lines of code and as we’ve seen, the AI relies on the current corpus of code to do this. Common mistakes in public samples propagate to the suggested code it writes.
The biggest challenge will be understanding what the code does and auditing the changes in a sane manner. Even if you vibe code everything some level of understanding is needed. Or at least it should be needed. If you expect to maintain something longer than a few hours.
The first thing member of the church of AI will say is “bUt tEh AIzzz wIlL aUdIt c0dEz!!!” This is wishful thinking today. There are AI tools that can help audit code, but I’ve yet to see anything operate on the level we would need. We could now start an argument that this is unquestionably the future, but I’m not a ghost in a Dickens novel. I care about what exists today. Oh wait, that’s also a Dickens ghost.
The second order problem with this one will be the unknown maintenance. We don’t really know how much work it will be take care and feed for a codebase that’s 100% AI written. And even if we had a flawless code auditor (human or robot), we will be responsible for 100% of the security flaws and bugs. AI cannot write flawless code today, we don’t know if it will ever write flawless code. We know humans can’t write flawless code. The amount of responsibility that falls on the developers in this scenario will be overwhelming. If you think keeping your dependencies updated is hard, try also keeping all the code behind those dependencies updated.
AI replaces some of the open source
Maybe you could replace unmaintained or unexciting libraries. Fewer dependencies would mean less of a chance for a supply chain attack. Think ’left pad’ type code that you don’t need a dependency for. This one is plausible, but we probably don’t yet know how to draw this line.
I’m sure in the next few years there will be many research papers and conference talks about this topic. Some might even be end up being true!
Our second order problem here will be getting to keep track of your dependencies as well as the code that could be a dependency but isn’t. That’s going to be a lot of work.
AI uses as many dependencies as possible
Basically this is today, or at the very least it will be the near future. If you use AI to help you write code, this is what you get now. It will use a lot of existing dependencies. Some of the second order challenges revolve around an AI picking a popular dependency that is out of date, abandoned, or EOL. There’s minimal regard for the safety and security of the dependencies. You asked the stochastic parrot to stochast, not to write secure code! Unless you added “make sure you don’t write insecure code” to the prompt, then I’m sure it’s very secure.
While lots of dependencies could mean attacks on the upstream projects, we’ve gotten pretty good at figuring out the dependency list and how to track updates to those dependencies. But updating dependencies isn’t free, so this isn’t a perfect solution. It has costs, but the problems are more solvable than the other options today.
Minimal AI and minimal dependencies
This ship sailed a long time ago. The idea of writing all your own code isn’t a realistic option. I only include it here so the chart below doesn’t have a blank square.
The big reveal
Speaking of charts, I made a chart that should help explain the possible outcomes. It’s clear a great deal of time and money was spent creating this professional looking graphic.
As these sort of charts go, most of us do not exist at the extremes. It’s just as silly to claim AI will write 100% of the code in the future as it would be to claim you’re going to write 100% of the code yourself. Well, unless your goal is to get quoted in a news story, then it’s just the sort of sensationalist comment that will get you attention.
At this point I think making the claim that AI can write 100% of your code and never need a dependency again is about as intellectually honest as claiming the current LLMs will create AGI. There’s just no evidence it’s a plausible outcome. You are literally making crap up.
The thing I think about often is that I know quite a few really good developers (I am not a really good developer). Most of them don’t do all that much development with AI, almost all of them think AI code is pretty bad. While I’m not a good developer, I do write a lot, and I think AI sucks at writing. I say this because while most people don’t live at the extremes of the above chart, some do. Those are the people to keep an eye on.
And lastly, if you’re an open source developer, I think you can rest easy knowing your hard work will continue to be thankless and underpaid for the foreseeable future.