I got to chat with Lars about a new CI/CD system he’s been working on called Ambient. It sounds really cool and does some very clever things today, with even more things planned in the future. We also spend some time discussing a project he works on called Radicle, a distributed Git forge. It feels like having decentralized infrastructure might be more important than it’s ever been, for some reason.

This episode is also available as a podcast, search for “Open Source Security” on your favorite podcast player.

Reimagining Git and CI/CD with Ambient and Radicle

The world of CI currently resembles the batch processing environments of the mainframe era. Lars had a great story about waiting hours for a CI pipeline to return a simple syntax error, and you’ll hear the same frustration that programmers experienced decades ago. It’s this shared pain that brought Lars to create Ambient – a project aiming to fundamentally rethink how we approach CI/CD. We also hear from him about a distributed git system called Radicle he works on.

The Frustrating State of CI/CD Systems

If you’ve worked with modern CI systems, you’ve likely experienced the cycle: push code, wait for CI to run, get a cryptic error log, make changes, push again, and wait… again. Lars has a story about waiting two full days for a CI job, only to discover a simple bash syntax error. After fixing it, he had to wait, again, for confirmation.

This experience highlights a few problems with today’s CI/CD systems. Developers lack hands-on access during builds, when something fails you’re left with a log blob to decipher without the ability to interact with the environment. And most interesting, if you run a CI system, it runs arbitrary code from contributors – essentially enabling remote code execution by design.

Ambient: Virtual Machines, TAR Archives, and Security

Ambient is a project Lars created, it tackles some of these challenges with a fundamentally different approach. At its core, Ambient runs code in isolated virtual machines without network access. This design choice immediately addresses many security concerns that plague container-based solutions.

The technical implementation is fascinating in its simplicity and security focus. When Ambient prepares to run a job, it first downloads all dependencies (currently supporting Rust dependencies). These dependencies, along with the code to be tested, are packaged into a virtual machine that runs with specific resource constraints – limited CPU cores, RAM, disk space, and runtime.

Ambient uses TAR archives via virtual block devices to move data into and out of the CI runner. Mounting a filesystem image can cause the system to panic, tar has some nice advantages. For logging and monitoring, Ambient provides two virtual serial consoles: one for kernel boot logs and another for CI run output. This approach gives developers the feedback they need while maintaining the security boundary.

The Distributed Vision: CI in the Atmosphere

The name “Ambient” is supposed to be CI that’s just there, in the atmosphere, without requiring explicit setup. The idea is you can have a distributed system of CI runners. It’s a bold goal, and a really interesting one.

Imagine a network of volunteer nodes donating compute capacity to run CI jobs. Your laptop sitting idle during work hours could contribute to the CI capacity of projects you care about. Because Ambient runs everything in secured virtual machines, the security risks of running arbitrary code are significantly mitigated. It’s not zero, but this is a good start I think.

One of the more interesting aspects of distributed tools at the time of writing is the geopolitical one. Everything is pretty weird right now, it could get weirder, it might be time to put serious effort into more distributed architectures.

While the full distributed vision is still just a vision, Lars has integrated Ambient with another of his projects, Radicle, which provides a practical implementation of many of these ideas.

Radicle: Git Collaboration Without Centralization

Radicle complements Ambient by offering a peer-to-peer, distributed Git collaboration system. Rather than relying on a centralized forge like GitHub or GitLab, Radicle works through interconnected nodes that share information about repositories, issues, and patches.

Each Radicle node decides which repositories to track, with repositories, issues, and patches all stored within Git (though in different namespaces from branches and tags). When nodes connect, they gossip about the state of repositories they know about, gradually propagating information through the network.

New nodes bootstrap by connecting to well-known public seed nodes, similar to how other peer-to-peer systems operate. From there, they discover other nodes and begin participating in the network.

The integration between Radicle and Ambient means we can start to envision a distributed development workflow. When a developer pushes changes to Radicle, then nodes configured to run CI for that repository notice the change and trigger Ambient to run tests in isolated virtual machines. The results then propagate back through the Radicle network. In theory of course, the devil is in the details.

Radicle allows node operators to define policies, for example, running CI for their own repositories, repositories they contribute to, or even all repositories in the network if they have the resources to spare.

Current State and Future Directions

Both Ambient and Radicle are working systems, though still evolving. Lars has been using Ambient as his primary CI system for about six months, while Radicle serves as his Git forge of choice for most projects.

Ambient currently exists as a command-line program integrated with Radicle. While the full distributed network vision isn’t yet implemented, the integration with Radicle provides many of the same benefits. The core security model, isolated virtual machines exchanging data through TAR archives, works and seems like a good start.

Moving Forward: The Potential of Distributed Development

The ideas behind Ambient and Radicle represent more than just technical innovations, they’re responses to the changing landscape of software development. As concerns about centralization, vendor lock-in, and geopolitical constraints grow, distributed approaches seem like a good idea.

If you’re intrigued by these ideas, consider exploring these projects. Both welcome contributions – from code to documentation feedback. Even something as simple as reviewing the documentation and providing feedback on unclear areas would be valuable. And for those interested in the future of distributed development, these projects offer glimpses of what might be possible.