Skills Are the New Packages
AI agents no longer work alone. Coding agents and desktop assistants now load skills: small bundles that combine written instructions with executable scripts, so an agent can deploy an app, process a document or call an internal tool.
That makes a skill a software dependency, with one twist. The one installing it, and deciding when to run it, is often the agent itself. A human may never read what it does.
To manage that risk, a new category of tools appeared: skill scanners. They review a skill's documentation and source code before it is trusted. Two pieces of research published in October show why reading the skill is not the same as knowing what it will do.
How the PyCache Trap Works
When Python imports a module, it can use a compiled bytecode cache, a .pyc file usually stored in a __pycache__ folder, instead of compiling the .py source again. Under the right conditions, the loader accepts that cache and runs it. Nobody reads .pyc files: they are binary, and scanners focus on the source.
Clean Python Source
The skill ships with benign, readable .py code and honest documentation. Any human reviewer or scanner would approve it.
A Substituted Bytecode Cache
Next to it, the skill bundles a .pyc cache that the loader accepts but that behaves differently from the source. The skill's instructions then make the agent call that module during a normal, task-relevant step.
The researchers tested this on 100 skills against seven skill scanners. The attack succeeded 94% to 100% of the time, and none of the scanners recognized what the substituted cache actually did. They call it the inspection-execution gap: the scanners inspect one artifact, the runtime executes another.
The Proposed Defense: Execution-Aware Validation
The paper proposes execution-aware validation (EAV). Instead of only reading files, it builds a typed graph that links a skill's instructions, scripts, imports and runtime artifacts, so it knows what the agent can actually load. It then checks every compiled artifact by trusted reproduction: rebuild it from the reviewed source and compare.
In the researchers' evaluation, EAV detected all 100 source-present cache substitutions and reached 92.8% recall at a 10.0% false positive rate across five attack families, measured against 200 benign skills.
Not a One-Off: The Render Skill That Leaked Tokens
A second case, reported by AIR Security on October 14, shows a different failure with the same lesson. It involved OpenAI's official render-deploy skill, which teaches agents how to deploy apps to the Render cloud platform.
- The skill's reference files said private Render services live under
render-internal.com, with examples such asinternal-api.render-internal.com. - That domain did not belong to Render, and it was not registered.
- When the researchers asked Codex to deploy two services that talk to each other, it wrote the domain into the generated code as the default internal API address, and into the deployment configuration.
- After the researchers registered the domain as a passive sinkhole, real applications sent it live calls carrying tokens.
| Date (2026) | Event |
|---|---|
| January 27 | The wrong domain appears in the skill's first commit. |
| March 13 | The skill is bundled in the openai/plugins repository. |
| August 14 | Render removes every render-internal.com reference from the skill. |
| August 26 | OpenAI removes the Render plugin from openai/plugins. |
| October 14 | AIR Security publishes the report. The domain is now under Render's control. |
AIR Security did not publish how many applications or tokens were affected. Anyone who generated code with the old skill should still check it: a default value written into an app does not disappear when the skill is fixed.
What Security Teams Should Do Now
| Practice | Why |
|---|---|
| 1. Reject or rebuild bytecode | Do not admit skills that ship .pyc files or __pycache__ folders. If compiled files are needed, rebuild them yourself from the reviewed source. |
| 2. Scan what can load | Review every file the runtime can execute or import, not only the files a human can read. |
| 3. Treat domains as dependencies | Verify that every domain, URL and package a skill mentions belongs to who it claims. Check registration and ownership. |
| 4. Search generated code | Grep your repositories and vendored skill copies for render-internal.com and other defaults that came from skills. |
| 5. Control who installs skills | Keep an approved list of skills, pin versions and stop agents from installing new ones on their own. |
| 6. Watch outbound traffic | An app calling an unknown "internal" domain over the internet is a strong alert signal. |
The Fine Print
In October 2026, researchers showed that pairing benign Python source with a substituted bytecode cache bypassed seven AI agent skill scanners 94-100% of the time across 100 skills, while AIR Security showed that OpenAI's official render-deploy skill led Codex to write an unregistered domain into generated apps, which then sent it live, token-bearing calls. Both point to the same rule: a skill is a dependency, and reviewing what it says is not enough.
Frequently Asked Questions
.pyc bytecode cache that Python's loader accepts and runs instead. Scanners review the clean source and miss the cache.render-internal.com as Render's private network domain, but Render did not own it and it was unregistered. Codex wrote it into generated apps as a default address, and those apps later sent token-bearing calls to it. Render fixed the skill and now controls the domain.My Take
We spent a decade learning, the hard way, that reviewing source code is not enough in software supply chains. That is why reproducible builds and signed artifacts exist. Agent skills are replaying that history at high speed, and the user installing them is often an agent that never asks why.
The scanners in this study did their job: they read the code. The problem is that reading the code and running the code are no longer the same thing. As a network security engineer, I would add one more layer the attacker cannot hide from: the network. A fake "internal" domain still has to resolve and connect somewhere, and that traffic is visible.
Does your organization review agent skills before they run, or are agents installing them on their own?
Sources:
- Liao et al., "PyCache Trap: The Inspection-Execution Gap in Agent Skill Scanners," arXiv 2610.10612 (October 7, 2026)
- AIR Security: OpenAI Sesame, Skill Vulnerability in OpenAI's Official Skill Made Production Apps Leak Tokens (October 14, 2026)
- Adversa AI: Agent skill security resources, November 2026
This article is published for information purposes only, to help readers understand emerging AI security risks.
Related Articles: