Case study
The first day of a source repository compromise
A software company found commits in its source repositories that nobody on the team had written, and developers had already pulled them. The first day was containment: cut off access, stop further changes landing, establish what had changed and which machines had it, then plan the rest deliberately.
Based on a real engagement. Client anonymised and details changed.
At a glance
- Engagement
- Emergency incident response
- Timeframe
- The first day of the response, which is all this account covers
- Scope
- The source repositories, every credential able to write to them, and the developer machines that pulled the unauthorised changes
- Outcome
- Write access revoked and reissued, main branches behind review, history preserved, affected machines identified, next stage scoped
The situation
A software company noticed changes in its source repositories that none of its engineers could account for. By the time anyone looked closely, developers had already pulled those changes during ordinary work, so the question was no longer only who had reached the repositories but which machines now held the changes. We were brought in for emergency incident response. This account covers the first day only.
What we did
- Revoked and reissued the credentials that could write to the repositories, including access tokens and deploy keys, rather than only the account under suspicion, because the entry route was not yet known
- Put the main branches behind protection rules that require review, so no further change could land unreviewed while the investigation ran
- Reviewed the recent commit history to separate the team's own work from changes nobody could account for, preserving that history as evidence rather than rewriting it away
- Worked out which developer machines had pulled the unauthorised commits, from the repository platform's own records and from the developers' accounts of their work
- Closed the day with a written, scoped follow-up plan instead of an open-ended investigation
What we found
The repository was not the whole scope
Because the changes had been pulled during normal work, containment could not stop at the repository. Every machine that fetched them, and anything that runs automatically when code is checked out or built, had to be treated as potentially affected until shown otherwise.
Revoking one account would not have been containment
With the entry route still unknown, any credential able to write to the repositories had to be treated as exposed. Revoking only the account that looked suspicious would have left every other route open, including the one that may actually have been used.
Protecting the branches mattered as much as revoking access
Revocation stops a known credential. Requiring review on the main branches stops the next unreviewed change whichever credential it arrives with, which is the control that holds while the investigation is still working out what happened.
What changed
By the end of the first day the credentials that could write to the repositories had been revoked and reissued, the main branches required review, the recent history had been reviewed and preserved, and the affected developer machines were identified. The next stage was proposed as a separate, scoped plan: examine the affected machines, establish how access was first obtained, and check whether anything built from the affected code had gone further.
The lesson
In a source repository compromise the repository holds the evidence and the developer machines hold the exposure. The first day is for making sure nothing else can change, not for naming a culprit: revoke broadly, protect the branches, preserve the history, find the machines, and then plan the investigation deliberately rather than improvising it at speed.
What to check in your own environment
- List every credential that can write to your repositories: personal access tokens, deploy keys and automation accounts, not only people
- Require review before anything merges to your main branches, and check who is able to bypass that rule
- Confirm you could tell which developer machines fetched a given commit, and how long that record is kept
- Know what runs automatically when code is checked out or built, such as hooks, install scripts and build steps
- Decide in advance who may revoke credentials during an incident, so containment does not wait on a meeting
- Preserve history before anyone rewrites or tidies it, because it is the evidence
Recognise your own systems in this?
A short conversation is the quickest way to find out whether the same thing is true of yours, and what it would take to check.