Loading an open-source AI model can silently run code nobody asked for. See the real pickle exploits security researchers found, and what actually fixes it.
Pulling an open-source AI model means loading a file, and for years the dominant format for that file let it run arbitrary code the moment it loaded, not when anyone deliberately chose to execute anything. That single technical fact, not “Shadow AI” as an abstract governance term, is the actual mechanism behind AI supply chain risk. The paradox in the name is specific: the openness that makes a million freely shared models valuable is the same openness that let malicious ones hide among them, undetected for months in documented cases.
Traditional software risk has a clear boundary: you choose to run a program, and that choice is the moment risk gets introduced. A machine learning model breaks that boundary because of how it has traditionally been stored. Python’s pickle format, the default serialization method behind most PyTorch model files for years, can serialize arbitrary Python objects, including executable code, not just numerical data. A model file built on pickle can carry a hidden instruction using Python’s own __reduce__ method, and that instruction runs automatically the instant a data scientist loads the model into memory, with no separate step where anyone consciously executed anything.
This is not a hypothetical edge case in the format’s design. It is the specific mechanism every documented Hugging Face model attack below has actually used, and the same class of risk zero trust security already assumes about any component: verify before you trust, never trust because a file merely looks legitimate.
Security researchers at JFrog scanned PyTorch and TensorFlow Keras models on Hugging Face and identified roughly 100 repositories carrying malicious payloads, the large majority in PyTorch format. One example, a repository named baller423/goober2, used pickle’s __reduce__ method to open a reverse shell to a hardcoded external IP address the moment the model loaded, and evaded Hugging Face’s own scanning at the time.
Separately, ReversingLabs documented malware dubbed nullifAI hidden inside two Hugging Face models that went unnoticed for more than eight months. Both files evaded ProtectAI’s scanner and ClamAV, and Hugging Face’s own pickle scanner reportedly failed to even recognize one of the files as a pickle at all. Evasion, not just malicious payload, was the point being demonstrated.
The pattern has kept scaling rather than fading. A 2026 Cloud Security Alliance research note documented a repository named Open-OSS/privacy-filter that impersonated an official OpenAI release and accumulated 244,000 downloads before it was identified and removed, delivering credential-stealing malware through a bundled loader file. No technical exploit of the platform itself was required. Naming proximity to a trusted brand and a spot on a trending list did the entire job.
Discover how Threatcop protects your workforce from modern cyber threats.
The industry did not leave this unaddressed. Hugging Face built safetensors, a model serialization format that stores only raw tensor weights and metadata in a simple structure with no executable content at all. A safetensors file cannot carry a hidden __reduce__ payload because the format has no mechanism for arbitrary code in the first place, by design rather than by scanning for known-bad patterns after the fact.
The format’s adoption milestone is recent and worth naming precisely: safetensors joined the PyTorch Foundation, under the Linux Foundation, in 2026, and is now the default checkpoint format on the Hugging Face Hub. That is the closest thing this space has to an industry consensus fix, and it works by eliminating the vulnerable mechanism entirely rather than trying to detect every way it might be abused.
None of the above requires a data science team to stop using open-source models. It requires treating a model file with the same supply chain discipline already applied to any other third-party software dependency, the same discipline an AI risk management framework is meant to formalize rather than leave to individual judgment.
This is the same discipline behind treating any third-party dependency as a risk to actively manage, applied to a dependency type most software supply chain programs were not built with in mind.
The open-source paradox is not a vague tension between innovation and risk. It is a specific, well-documented technical fact: a model file could run code nobody asked it to run, at the exact moment someone tried to use it for its intended purpose. The fix already exists and is becoming the default. The remaining gap is whether an organization’s model-vetting process has caught up to a solution the format itself already solved, the same gap people security management exists to close between a known fix and the human process that actually applies it.
Because many models are stored using Python’s pickle serialization format, which can encode arbitrary executable instructions alongside numerical data. A specific technique using pickle’s __reduce__ method lets an attacker embed code that runs automatically the moment the file is deserialized, which happens the instant the model loads into memory.
The AI supply chain risk has been exploited repeatedly and documented by multiple independent security research teams. JFrog found roughly 100 malicious models on Hugging Face using this exact mechanism, and separate research documented malware that stayed hidden inside models for more than eight months while evading several security scanners.
Safetensors is a model file format that stores only tensor weights and metadata, with no mechanism for embedding executable code at all. It closes the specific vulnerability pickle-based formats carry by design rather than by detecting known attack patterns, which is why it became the default checkpoint format on the Hugging Face Hub.
No. Documented cases have specifically evaded multiple scanners, including Hugging Face’s own pickle scanner and third-party tools, sometimes for many months. Scanning reduces risk but should not be treated as a guarantee, which is why format choice and publisher verification matter as independent layers.
The mechanism applies to any platform distributing pickle-based model files, since the vulnerability lives in the file format, not in any single platform’s infrastructure. Hugging Face is the most documented case because of its scale and the volume of public security research focused on it, not because the underlying risk is unique to it.
Sushant Kumar is the AVP – Technology at Threatcop, bringing over a decade of experience in technology leadership and product development. He has worked across technology-driven organizations, including Paytm, and focuses on building scalable solutions that address evolving business and cybersecurity challenges. His areas of interest include cybersecurity technology, AI-driven security, product innovation, and enterprise technology. He is passionate about using technology to solve complex security challenges.
New breach notification laws are tightening cyber insurance underwriting across Southeast Asia. See what mid-market organizations actually need to...
CAN-SPAM fines now reach $53,088 per email. Google and Yahoo enforce their own rules too. See what actually satisfies...
Domain blocklists miss most shadow AI. See how TLS fingerprinting, NAC, asset reconciliation, and OAuth audits actually find it...
