How One Unrotated Token Gave Hackers Access to Grafana's Codebase
Grafana has traced its recent data breach to a single GitHub workflow token that nobody got around to rotating. One token. That's all it took.
The incident connects back to the broader Shai-Hulud malware campaign, attributed to a group called TeamPCP, which spent the past couple of weeks seeding the npm registry with poisoned TanStack packages. The packages carried credential-stealing payloads, and Grafana's CI/CD pipeline swallowed one of them. When it executed, it quietly shipped GitHub workflow tokens off to the attackers.
Grafana caught the malicious activity on May 1 and kicked off its incident response process, which included rotating those tokens. Most of them got rotated. One didn't.
That missed token was enough. The attackers used it to walk into Grafana's private GitHub repositories, and the company has since confirmed that source code was stolen in the intrusion.
The latest update adds a further wrinkle: it wasn't just code. The investigation found that the attackers also made off with operational data, described by Grafana as business contact names and email addresses of the kind exchanged in normal professional dealings. Crucially, none of this came from production systems or the Grafana Cloud platform.
"A subsequent review confirmed that a specific GitHub workflow we originally deemed not impacted had, in fact, been compromised," the company acknowledged in its update.
Grafana is emphatic that customer production environments were not touched. The codebase itself was not modified during the breach, so software downloaded during the incident window is considered clean. No action is required from users at this point, though Grafana has promised to reach out directly if that assessment changes.
The company has also made clear it will not be paying any ransom.
For everyone else in the industry, the incident is a fairly pointed reminder that token rotation under pressure is one of those tasks that really can't afford gaps. Incident response checklists exist for a reason.