Codewall/Blog/26/03/How we hacked McKinsey's AI Platform
- How We Hacked McKinsey's AI Platform — CodeWall.ai
Fun fact: As part of our research preview, the CodeWall research agent autonomously suggested McKinsey as a target citing their public responsible diclosure policy (to keep within guardrails) and recent updates to their Lilli platform. In the AI era, the threat landscape is shifting drastically — AI agents autonomously selecting and attacking targets will become the new normal.
46.5 million chat messages ... Every conversation, stored in plaintext, accessible without authentication.
Most required authentication. Twenty-two didn't. ... API documentation publicly exposed
.. it recognised a SQL injection that standard tools wouldn't flag (and indeed OWASPs ZAP did not find the issue). From there, it ran fifteen blind iterations — each error message revealing a little more about the query shape — until live production data started flowing back*...
Poisoned advice — subtly altering financial models, strategic recommendations, or risk assessments. Consultants would trust the output because it came from their own internal tool.
Data exfiltration via output — instructing the AI to embed confidential information into its responses, which users might then copy into client-facing documents or external emails.
- If they had access to the database, in theory, they could have exfiltrated data through that mechanism. But most systems at McKinsey scale do have some kind of egress monitoring, which makes it harder to get data out without being detected. But this provides a way to get data out without triggering those warnings.
Silent persistence — unlike a compromised server, a modified prompt leaves no log trail. No file changes. No process anomalies. The AI just starts behaving differently, and nobody notices until the damage is done.
- While silent persistence and "The AI just starts behaving differently, and nobody notices until the damage is done" iis definitely an important threat aspect here, here it seems like that doesn't match up with their example claim that a modified prompt leaves no log trail. If the database row with a prompt was changed, it will have updated the modified time of that row. Also, as they disclosed, all the chat messages were in the database, so they were logged. But definitely, silent persistence would be the worst.
- My Notes
- The headline was that they "Hacked McKinsey's AI Platform" but really, what happened was that McKinsey had 22 unauthenticated API endpoints, some even with documentation, that were vulnerable to SQL injection.
- The main take-away is that people are using agentic tools to hack - and there are actors that will follow responsible use disclosure, and other actors that will not, and will just make use of the data.