Malik Haidar is a veteran in the high-stakes world of cybersecurity, having spent years defending multinational corporations from the most sophisticated digital incursions on the planet. His work sits at the intersection of deep-threat intelligence and corporate strategy, where a single breach can cost a company its entire competitive edge. Haidar has watched the landscape shift from simple data theft to the era of intellectual property strip-mining, where the very “brains” of artificial intelligence are being targeted. In our conversation, he breaks down the mechanics of industrial-scale distillation attacks, the shadowy networks of proxy services, and the evolving defenses required to protect the next generation of AI models.
The discussion covers the massive scale of unauthorized training campaigns identified earlier this year, the specific tactics used by international labs to siphon intelligence from frontier models, and the methods by which thousands of fraudulent accounts are used to bypass modern API safeguards. We also explore the ethical and security implications of rerouting user traffic and the technical countermeasures, such as encrypted reasoning, that are becoming essential in this digital arms race.
The scale of recent distillation campaigns is staggering, with some reports citing millions of daily exchanges. How do these industrial-scale operations manage to bypass standard API security protocols and remain undetected for so long?
These operations are incredibly sophisticated and function more like a massive digital engine than a single hacker in a dark room. To bypass security, unauthorized labs utilize sprawling proxy services that act as transfer or relay stations, creating a thick layer of anonymity between the attacker and the target. In the largest campaign we’ve tracked, GTG-16005, we saw a cluster of operators peak at roughly 3 million exchanges per day by rotating through more than 3,500 fraudulent accounts. They aren’t just guessing passwords; they are using a secondary market of stolen credit cards, illegally harvested API keys, and fictitious identities to look like legitimate paying customers. It is a constant game of cat and mouse where they rotate identities faster than standard detection algorithms can flag them as suspicious.
When we look at the specific labs involved, some have been accused of rerouting their own users’ requests to Claude without their knowledge. What are the broader security and privacy implications when a company silently uses another model to process user data?
This is perhaps the most brazen part of the strategy because it exploits the trust of the end-user. For instance, Moonshot AI, under the GTG-16002 campaign, stealthily rerouted almost 300,000 customer requests to Claude instead of using their own model, Kimi, and then simply displayed the results as their own. This means sensitive information from individual users and even state-affiliated actors was being passed through a shadowy network of 5,380 fraudulent accounts located in places like Singapore and Japan. The users have no idea their data is being used as training fodder for a “student” model to copy the teacher’s capabilities. Beyond the privacy violation, it creates a massive security vacuum where proprietary data from multinational companies is essentially floating in an unmonitored secondary market.
Distillation is often described in a “teacher and student” framework, but these illicit campaigns seem to be pushing that definition to its breaking point. Could you explain how these labs are specifically targeting “Chain-of-Thought” reasoning and why that is so valuable?
In a legitimate setting, distillation is a helpful way to make models faster, but here it is being used to steal the “logic” of the model rather than just its final answer. Campaigns like GTG-16006, run by Zhipu, focused specifically on a Chain-of-Thought extraction pipeline to see the internal reasoning traces Claude uses to solve complex problems. By replaying these reasoning steps through their own systems, they can train smaller models to mimic the high-level logical reasoning, coding, and data analysis of a much larger frontier model. We saw this targeted heavily in the 151 million exchanges observed between May and July of this year, where the goal was to harvest capabilities in kernel development and long-horizon tasks. They aren’t just copying the output; they are trying to steal the actual “thinking process” that makes a model elite.
Anthropic has introduced several new defensive measures recently, such as “preserved thinking” and encrypted reasoning. How effective are these technical barriers against attackers who are already using prompt manipulation tricks?
The introduction of the Fable 5.1 update is a significant step forward because it addresses the core method attackers use: editing the context of a conversation to force the model to reveal its internal logic. By using “preserved thinking,” we can stop new API accounts from altering the system prompts or the tools that precede the model’s reasoning in a multi-turn session. We have also started encrypting that internal reasoning, making it much harder for a lab to simply scrape the logic and use it for follow-on training. To complement this, there is a heavy focus on banning accounts in unsupported regions like China, Iran, and Russia when they fail to provide verified identities. While prompt manipulation is always evolving, these updates make the stolen transcripts significantly less useful and more expensive to acquire, which helps break the economic incentive of the attack.
What is your forecast for the future of AI model protection as these distillation attacks become even more automated and widespread?
I expect to see a move toward “zero-trust” AI architectures where every single API call is scrutinized not just for its content, but for its behavioral patterns over time. The “secondary market” for harvested user exchanges is growing, with proxy networks now saving and selling conversations to the highest bidder, so the defense must move beyond simple account bans. We will likely see more models that can detect when they are being “probed” for training data and deliberately provide less useful, though still accurate, responses to those specific queries. The stakes are rising because this isn’t just about corporate secrets anymore; we’ve already seen these models being used to research biological weapons and surveil citizens. In the coming years, the ability to keep the “internal thoughts” of an AI private will be just as critical as protecting the source code of our most sensitive national security infrastructure.
