Deleting the file does not remove the secret: git history, and why Toyota's key sat exposed for five years
A committed secret lives in git history and in cached copies on the host, not in the file you deleted. Why a new commit changes nothing, what actually removes it, and why the only safe assumption is that the credential is already burned.
By Ahmed PingerPublished
9 min read13 views
A developer notices an API key hard-coded in a config file, deletes the line, commits "remove secret," and pushes. The file is clean. The secret is not gone. It is sitting one command away in the repository's history, and depending on where that repository has been, in cached copies the developer cannot reach at all.
The one distinction that this whole problem turns on: git does not store files, it stores commits, and a commit is immutable. Deleting a secret in a new commit adds a commit that removes the line. It does not alter the earlier commit that added it. Anyone who can read the history can check out the earlier commit and read the secret exactly as it was. The working tree looks clean; the object database remembers everything.
Why the exposure varies from "rotate and move on" to "assume total compromise"
How bad a committed secret is depends on where the history has travelled and how long it sat there:
- Local-only, caught before push: contained, but still present in your local object database and reflog until you rewrite and prune.
- Pushed to a private repository: exposed to everyone with read access, and, as recent research shows, sometimes beyond that. GitGuardian's State of Secrets Sprawl 2025 reports that 35% of the private repositories it scanned for customers contained at least one plaintext secret, so "it is private" is not the reassurance it sounds like.
- Pushed to a public repository, even briefly: treat it as compromised the moment it landed. Automated scanners crawl public GitHub continuously. The same GitGuardian report counted 23.8 million new hardcoded secrets added to public GitHub in 2024, a 25% year-on-year increase, of which 58% were generic secrets like passwords and custom tokens that no vendor scanner will catch for you. It also found that 70% of the secrets leaked in 2022 were still valid at the time of the report, which is the real reason this matters: leaked credentials are not rotated, so they keep working.
The severity, in other words, is set by exposure time and reach, and the safe default is always the pessimistic one.
The anchor: Toyota's T-Connect key, exposed December 2017 to September 2022
This is a technique-and-consequence topic rather than a single CVE, so the anchor is a real, documented incident with a primary source. In its own disclosure, Toyota stated that part of the T-Connect source code was mistakenly published to a public GitHub repository, and that it contained an access key to a data server. According to Toyota's notice, the code was accessible from December 2017 until 15 September 2022, roughly five years, and the exposure put the email addresses and customer management numbers of 296,019 customers at potential risk. Toyota made the repository private on discovery and changed the access key on 17 September 2022.
Two details make it the perfect anchor. First, the exposure window is measured in years, not hours, which is what happens when nobody realises a secret is in history because the current files look fine. Second, Toyota could not rule out unauthorised access, only say the logs did not confirm it, which is exactly the position you are left in when a live credential has been publicly readable for an unknown fraction of five years. The credential has to be considered burned regardless of what the access logs seem to say.
A lab you can build in a scratch directory
No servers, no Docker, just git. Create a throwaway repository and reproduce the trap so the behaviour is concrete.
mkdir secret-lab && cd secret-lab && git init
printf 'API_KEY=AKIA_EXAMPLE_PLACEHOLDER_00\n' > config.env
git add config.env && git commit -m "add config"
# "Fix" it the wrong way: delete the line and commit.
printf 'API_KEY=\n' > config.env
git add config.env && git commit -m "remove secret"
# The working tree is clean. The history is not:
git log -p -- config.env | grep -n 'AKIA_EXAMPLE_PLACEHOLDER_00'
git grep 'AKIA_EXAMPLE_PLACEHOLDER_00' $(git rev-list --all) | head
The last two commands find the placeholder inside the earlier commit even though it is absent from the current file. That is the entire lesson, made visible: the "remove secret" commit is cosmetic, and any clone carries the value.
Finding secrets already in your history
Point a scanner at the full history, not just the current tree, because the current tree is exactly where the secret is not.
- gitleaks runs
git log -punder the hood and scans every patch across history, and installs as a pre-commit hook to stop the next one from ever landing. - TruffleHog scans git history and, for many credential types, verifies whether a found secret is still live by testing it against the provider's API, which turns a wall of findings into a ranked list of the ones that still work.
# Scan the entire history of a repository, not just HEAD.
gitleaks detect --source . --log-opts="--all"
Sort whatever you find by one question: is this credential still valid? A verified-live key that has been in public history is an incident. A long-dead test token is cleanup. TruffleHog's verification step exists precisely to make that triage fast.
Fixing it: rotate first, then rewrite, then accept the cached copies
The order is not optional, and the first step is the one people skip.
1. Rotate the secret. This is the actual fix. GitHub's own guidance is explicit: if the sensitive data is a credential, revoke or rotate it, because once revoked it can no longer be used for access. History rewriting is housekeeping; rotation is what closes the exposure. If you only have time for one action, it is this one. Toyota's five-year window is the cautionary version: the rewrite is worthless if the key still works.
2. Rewrite the history to remove the value. The git project itself no longer recommends git filter-branch, whose documentation warns that its "safety and performance issues cannot be backward compatibly fixed" and points users to git filter-repo instead. Run git filter-repo in a fresh clone to purge the secret from every commit. No manual cleanup follows: its README says it will "automatically remove old cruft and repack the repository for the user after filtering," and it refuses to run outside a fresh clone unless forced, because a clone you can throw away is the recovery mechanism. The refs/original/ deletion and git gc --prune=now sequence belongs to filter-branch, not to filter-repo.
Keep the original repository untouched until the rewrite is verified; every commit hash downstream changes.
3. Accept that copies you do not control still exist. This is the part that makes rotation non-negotiable. On a hosting platform, the pushed object can survive your rewrite:
- GitHub documents that removed commits can remain reachable directly by their SHA-1 hash in cached views, and through any pull request that referenced them. Removing them requires contacting GitHub Support to dereference or delete the affected pull requests and cached views.
- Truffle Security's 2024 research on Cross Fork Object Reference showed that commits pushed to a fork can remain reachable through the upstream repository by hash even after the fork is deleted, and that short hashes are guessable because the four-character keyspace is only 65,536 values. Once a secret has been pushed to a public forked repository, "I deleted it" is not a security control.
Rewriting history is worth doing to stop the secret propagating into fresh clones, but it does not retroactively un-expose anything. The rotated credential is what protects you.
Preventing the next one
Turn on push protection. GitHub's secret scanning push protection blocks pushes that contain recognised secrets before they reach the repository, and user-level push protection is enabled by default for pushes to public repositories on GitHub.com. It catches the mistake at the last possible moment before it becomes history.
Add a pre-commit secret scanner. A gitleaks pre-commit hook stops a secret at the earliest possible moment, on the developer's machine, before it is even committed. Push protection and a pre-commit hook are complementary: one guards the commit, the other guards the push.
Keep secrets out of the tree entirely. Environment variables, a secrets manager, or a git-ignored local config file mean there is nothing to commit. Ship a committed config.example with placeholder values so the real one never needs to exist in the repository.
Detecting it in the wild
If you operate at any scale, assume secrets are already leaking and instrument for it.
- Scan continuously, not once. A one-time history scan is a snapshot; a scheduled scan across all repositories catches the secret committed next week. Wire gitleaks or TruffleHog into CI so every push is scanned.
- Prefer verified findings. Verification against the live API separates the credential that still works from the thousand dead ones, and the live ones are the incidents. This is the difference between an actionable alert and alert fatigue.
- Watch for use, not just presence. The credential that has been public and is still valid should be monitored at the provider for anomalous use, because a leaked-and-live key is exactly the one an attacker is trying. Toyota's inability to rule out access is what happens without that monitoring.
What to take away
Deleting the file does not remove the secret, because git stores immutable commits and the secret lives in the earlier one, in every clone, and often in cached copies on the host that you cannot reach at all. The Toyota disclosure is the canonical version: a key readable for five years, discovered by a researcher rather than the owner, rotated only after the fact. So invert the instinct. Rotation is the fix, and it comes first. History rewriting with git filter-repo stops the secret spreading into new clones but retroactively un-exposes nothing, and on a public host cached views and forks can outlive the rewrite. The only safe assumption for a secret that reached a repository is that it is already compromised, and the only reliable remedy is a new secret.
Further reading
- Toyota: Apology and notice regarding possible leak of customer email addresses (2022-10-07): the primary disclosure, exposure window and affected count
- GitGuardian: The State of Secrets Sprawl 2025: 23.8M public-GitHub secrets in 2024, 70% of 2022 secrets still valid
- GitHub Docs: Removing sensitive data from a repository: rotate first, git-filter-repo, cached views and pull requests
- git-filter-repo: the tool the git project recommends over filter-branch
- Git documentation: git filter-branch: the warning and the refs/original, reflog, gc cleanup steps
- Truffle Security: Anyone can access deleted and private repository data on GitHub: Cross Fork Object Reference and guessable short hashes
- GitHub Docs: About push protection: blocking secrets before they reach the repository
- gitleaks and TruffleHog: full-history scanning and live-credential verification
Was this useful?
Get the next write-up by email
New technical write-ups by email when we publish. No sales sequence, and every email has an unsubscribe link.
Prefer a feed? RSS. How we handle your address: privacy notice.
Comments
Loading comments…