Dimitri Stiliadis, CTO from Endor Labs, discusses the recent tj-actions/changed-files supply chain attack, where a compromised GitHub Action exposed CI/CD secrets. We explore the impressive multi-stage attack vector and the broader often-overlooked vulnerabilities in our CI/CD pipelines, emphasizing the need to treat these build systems with production-level security rigor instead of ignoring them.

This episode is also available as a podcast, search for “Open Source Security” on your favorite podcast player.

Okay, here is a draft blog post based on your discussion with Dimitri and the writing style analysis.

Our CI/CD Pipelines Are Under Attack, Again

We find ourselves talking about CI/CD security breaches yet again. It feels like a recurring theme, doesn’t it? This time, the focus was on a popular GitHub Action known as tj-actions/changed-files. As Dimitri explained, this action does a simple job, it checks if a pull request modified specific files. Its popularity made it a prime target.

Attackers managed to inject malware into this action. The goal was pretty simple, steal any secrets configured in the CI/CD pipeline environment and expose them publicly by printing them in the action’s log files. For any public repository using the compromised action during the attack window, their pipeline secrets were potentially leaked into a public accessible file.

The immediate impact was projects using this action whose pipelines happened to run while the malicious version was live. Dimitri sumed up how most of the recent supply chain attacks have gone for us: We got lucky. Thanks to quick detection within about 24 hours by the folks at Step Security the damage was contained. GitHub acted swiftly pulling the compromised action, which stopped future secrets leaking.

Research conducted by the team at Endor Labs, Dimitri’s company, dug into the aftermath. They used GitHub’s APIs to identify repositories using the action, check their run histories, and determine if secrets were exposed. While thousands of repositories could have been affected, their analysis found only around 200 actually leaked secrets. Thankfully in many of these cases the exposed secrets appeared to be ephemeral or short-lived. Endor Labs did the responsible thing and notified the maintainers of the affected repositories. It’s a lot of work to track down and notify even 200 maintainers, a task that deserves appreciation.

It should be noted, getting lucky, shouldn’t be our primary defense strategy. The potential impact could have been far worse. Imagine if leaked Docker Hub credentials were used to overwrite popular container images, or cloud credentials were used to pivot deeper into infrastructure. The interconnected nature of our development ecosystems means a breach in one pipeline can rapidly cascade, amplifying the damage. Attackers understand this; they target CI/CD specifically because it’s a critical junction with the potential for widespread impact.

This focus on attacking the build process rather than just the final artifact is a pattern we’ve seen before, with incidents like Codecov several years ago or the more recent XZ Utils backdoor. Dimitri pointed out the obvious, we know how these attacks happen, yet we struggle to prevent them.

The forensic analysis of the tj-actions attack, detailed by researchers like those at Unit 42, revealed a pretty bananas attack. It apparently started months prior, back in November. The attackers identified a vulnerability in how the SpotBugs Java static analysis tool’s repository ran CI/CD for pull requests. They used a PR to trigger the pipeline and discovered a maintainer’s Personal Access Token (PAT) stored as a secret. Stealing this PAT gave them access. They then identified another project where this maintainer had commit rights, used the stolen PAT to push a malicious change there, compromising a different GitHub Action used by tj-actions. Finally, leveraging this transitive dependency, they compromised tj-actions itself, manipulating Git tags to point legitimate-looking versions (like v1.2.3) to their malicious code commit instead of the real one. It was a multi-stage hop, apparently aiming for a Coinbase repository, though we can’t know the full scope of the intentions.

This intricate attack chain exploited weaknesses we knew could happen. Running CI workflows triggered by external PRs, especially in open source, without severely limiting their permissions is risky. Allowing overly broad PATs as secrets in CI/CD is asking for trouble. Relying on mutable tags for actions instead of pinning to immutable commit SHAs is a known vulnerability vector. We know these things are a good idea, but we don’t do them (even us security folks often don’t).

Why? Usually it’s the classic tension between security and usability. Making things easy often means less secure defaults. GitHub could enforce stricter defaults, like disallow PATs as secrets, mandate commit pinning, restrict PR workflow permissions. But this would inevitably add friction for developers, potentially hindering adoption. It’s a difficult balance. The “temporary fix” often becomes permanent because secure practices aren’t the default path of least resistance.

Perhaps the most crucial takeaway we discussed is the mindset shift required. We often treat our CI/CD pipelines and build environments as dev systems, implicitly less critical than production. We apply rigorous security controls and compliance checks to production, while the systems that build and deploy to production get less scrutiny. This has to change. These build systems are easily our most critical production systems because everything flows through them. They deserve the same, if not stricter, security controls.

We lack visibility into these complex build chains. Actions depend on other actions, creating transitive dependencies just like in our application code. We run scanners on our final artifacts, but what about the sea of code running inside the build runner itself?

We need to elevate the security posture of our build systems. It requires education, awareness, and pushing vendors towards more secure-by-default configurations even if it introduces some friction. We need to treat the systems that create our software with the critical importance they deserve.