JoySafeter
๐ JoySafeter: An enterprise AI Agent PlatformโNot just chatting. buildingใrunningใtesting, and tracing autonomous Agent Teams with visual orchestration...
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/jd-opensource/JoySafeter with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/jd-opensource/JoySafeter'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/jd-opensource--joysafeter)HTML
<a href="/listings/jd-opensource--joysafeter"><img src="/api/listings/jd-opensource--joysafeter/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
One-Click Launch (Recommended)
./deploy/quick-start.sh
The script provides an interactive menu to choose your startup mode and customize ports (with conflict detection):
| Mode | Description | Ports Configured | |------|-------------|-----------------| | **(1) Docker Compose Full Stack** | All services in containers, supports localhost or remote server IP/domain | Frontend, Backend, PostgreSQL, Redis | | **(2) Local Frontend Only** | `bun run dev`, supports connecting to remote backend | Frontend (can specify remote backend address) | | **(3) Local Backend Only** | `uvicorn --reload`, supports remote DB/Redis | Backend (can specify remote DB/Redis/frontend address) | | **(4) Local Frontend + Backend** | Auto-starts middleware, supports exposing via non-localhost address | Frontend, Backend |
All modes support remote deployment scenarios: - **Docker Compose Full Stack** โ choose deployment address (localhost or IP/domain) + http/https - **Local Frontend Only** โ optionally connect to a remote backend API (enter backend IP + port + protocol) - **Local Backend Only** โ optionally connect to remote PostgreSQL, Redis, and frontend (enter each address and port) - **Local Frontend + Backend** โ optionally expose services via a non-localhost address - Non-localhost deployments automatically update `frontend/.env` CSP whitelist (`NEXT_PUBLIC_CSP_CONNECT_SRC_EXTRA`)
./deploy/quick-start.sh --skip-env # Skip .env file initialization ./deploy/quick-start.sh --skip-db-init # Skip database initialization
Launch by Scenario
# โโโ Development โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/scripts/dev.sh # Docker full-stack dev (containerized frontend + backend) ./deploy/scripts/dev-local.sh # Local dev prep (start middleware, run backend/frontend on host) ./deploy/scripts/dev-backend.sh # Local backend only (requires middleware running) ./deploy/scripts/dev-frontend.sh # Local frontend only (requires backend running) # โโโ Production โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/scripts/prod.sh # Production deploy (pre-built images + docker-compose.prod.yml) ./deploy/scripts/prod.sh --skip-mcp # Production without MCP service ./deploy/scripts/prod.sh --skip-pull # Skip image pull, use local images # โโโ Middleware / Infrastructure โโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/scripts/start-middleware.sh # Start middleware (PostgreSQL + Redis + MCP) ./deploy/scripts/minimal.sh # Minimal startup (PostgreSQL + Redis only) ./deploy/scripts/minimal.sh --with-mcp # Minimal + MCP service ./deploy/scripts/stop-middleware.sh # Stop middleware # โโโ Test / CI โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/scripts/test.sh # Test environment (minimal deps, automation-friendly) # โโโ Install / Check โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/install.sh # Interactive installation wizard (generates config files) ./deploy/install.sh --mode dev --non-interactive # Non-interactive install ./deploy/scripts/check-env.sh # Environment preflight (Docker, ports, config files) # โโโ Image Management โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ./deploy/deploy.sh build # Build frontend + backend images ./deploy/deploy.sh build --all # Build all images (including OpenClaw) ./deploy/deploy.sh push # Build and push to registry ./deploy/deploy.sh pull # Pull latest pre-built images
Default Ports
| Service | Port | URL | |---------|------|-----| | Frontend | `3000` | http://localhost:3000 | | Backend API | `8000` | http://localhost:8000 | | API Docs | `8000/docs` | Swagger UI | | PostgreSQL | `5432` | Database | | Redis | `6379` | Cache |
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md ยท README ยท 352 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- <h1 align="center">
- <img src="docs/assets/joysafter.png" alt="JoySafeter" width="80" /><br/>
- JoySafeter
- </h1>
- <p align="center">
- <strong>The AI-native platform for building, orchestrating, and running security agents at scale.</strong><br/>
- <sub>From idea to production-grade security automation โ in minutes, not months.</sub>
- </p>
- <p align="center">
- <a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache 2.0"></a>
- <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB?logo=python&logoColor=white" alt="Python 3.12+"></a>
- <a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-20+-339933?logo=nodedotjs&logoColor=white" alt="Node.js 20+"></a>
- <a href="https://github.com/langchain-ai/langgraph"><img src="https://img.shields.io/badge/LangGraph-1.0+-FF6F00?logo=chainlink&logoColor=white" alt="LangGraph"></a>
- <a href="https://fastapi.tiangolo.com/"><img src="https://img.shields.io/badge/FastAPI-0.122+-009688?logo=fastapi&logoColor=white" alt="FastAPI"></a>
- <a href="#"><img src="https://img.shields.io/badge/MCP-Protocol-purple" alt="MCP Protocol"></a>
- <a href="#"><img src="https://img.shields.io/badge/DeepAgents-v0.4-red" alt="DeepAgents v0.4"></a>
- </p>
- <p align="center">
- English | <a href="./README_CN.md">็ฎไฝไธญๆ</a>
- </p>
- ---
- ## Why JoySafeter
- Traditional security tooling hits a ceiling: scripts are brittle, single agents lack context, and complex scenarios require 2โ3 engineers working in parallel. JoySafeter breaks that ceiling.
- | Challenge | Traditional Approach | JoySafeter |
- |-----------|---------------------|------------|
- | APK vulnerability analysis | Manual MobSF + engineer review | Autonomous agent: upload โ analyze โ report |
- | Penetration testing | Fixed scripts, static playbooks | Dynamic DeepAgents that adapt to findings in real time |
- | Tool integration | Custom glue code per tool | 200+ tools via MCP Protocol, zero glue |
- | Scale | Linear headcount growth | Agent teams that multiply capacity |
- > JoySafeter defines a new paradigm: **AI-driven Security Operations (AISecOps)** โ where multi-agent collaboration, cognitive memory, and scenario-matched skills replace manual coordination.
- ---
- ## Real-World Cases
- ### Case 1 โ APK Vulnerability Detection Agent
- > Upload an APK. Get an OWASP Mobile Top 10 report. No engineer required.
- <p align="center">
- <img src="docs/assets/APK-case.gif" alt="APK Vulnerability Detection Demo" width="800" />
- </p>
- **How it works:**
- 1. User uploads the APK file
- 2. Agent invokes MobSF for static analysis
- 3. Extracts critical risk signals โ permission abuse, hardcoded secrets, insecure network config
- 4. Deep-validates high-severity findings via Frida dynamic instrumentation
- 5. Auto-generates a structured report aligned to OWASP Mobile Top 10
- The entire flow โ from upload to report โ requires zero manual intervention, covering work that traditionally takes 2โ3 security engineers.
- ---
- ### Case 2 โ Penetration Testing Agent
- > Describe the target and scope. The agent plans, executes, and adapts โ then delivers a report.
- <p align="center">
- <img src="docs/assets/pentest-case.gif" alt="Penetration Testing Agent Demo" width="800" />
- </p>
- **How it works:**
- 1. Open the Workbench and create a new agent
- 2. Enable **DeepAgents mode** โ select penetration testing skills
- 3. Provide an authorized target URL and test requirements
- 4. Agent runs autonomously โ if it discovers a login page, it automatically triggers auth bypass testing
- 5. Download the final report when the run completes
- > **Note:** Requires sandbox image `swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/jd-opensource/joysafeter-sandbox:latest` configured in Sandbox Settings.
- This dynamic decision-making โ where the agent adapts its next step based on what it finds โ is what fixed scripts cannot replicate.
- ---
- ## Core Capabilities
- <table>
- <tr>
- <td width="50%">
- ### Visual Agent Builder
- - **No-code workflow editor** โ drag-and-drop nodes with loops, conditionals, and parallel execution
- - **Rapid Mode** โ describe in natural language, get a running agent team in minutes
- - **Deep Mode** โ visual debugging and step-by-step observability for complex security research
- </td>
- <td width="50%">
- ### 200+ Security Tools, Ready to Use
- - Pre-integrated **Nmap, Nuclei, Trivy**, and more
- - **MCP Protocol** โ extend with any tool via Model Context Protocol
- - **30+ pre-built skills** โ penetration testing, document analysis, cloud security, and more
- </td>
- </tr>
- <tr>
- <td width="50%">
- ### DeepAgents Orchestration
- - **Manager-Worker multi-level** agent collaboration
- - **Memory evolution** โ long/short-term memory for continuous learning across sessions
- - **Skill system** โ versioned, reusable capability units with progressive disclosure
- - **LangGraph engine** โ graph-based workflows with full state management
- </td>
- <td width="50%">
- ### Enterprise Ready
- - **Multi-tenancy** โ isolated workspaces with role-based access control
- - **Full audit trail** โ execution tracing and compliance governance
- - **SSO integration** โ GitHub, Google, Microsoft, OIDC (Keycloak, Authentik, GitLab), JD SSO
- - **Multi-tenant sandbox** โ per-user isolated code execution, zero state leakage
- </td>
- </tr>
- </table>
- ---
- ## Quick Start
- ### One-Click Launch (Recommended)
- ```bash
- ./deploy/quick-start.sh
- ```
- The script provides an interactive menu to choose your startup mode and customize ports (with conflict detection):
- | Mode | Description | Ports Configured |
- |------|-------------|-----------------|
- | **(1) Docker Compose Full Stack** | All services in containers, supports localhost or remote server IP/domain | Frontend, Backend, PostgreSQL, Redis |
- | **(2) Local Frontend Only** | `bun run dev`, supports connecting to remote backend | Frontend (can specify remote backend address) |
- | **(3) Local Backend Only** | `uvicorn --reload`, supports remote DB/Redis | Backend (can specify remote DB/Redis/frontend address) |
- | **(4) Local Frontend + Backend** | Auto-starts middleware, supports exposing via non-localhost address | Frontend, Backend |
- All modes support remote deployment scenarios:
- - **Docker Compose Full Stack** โ choose deployment address (localhost or IP/domain) + http/https
- - **Local Frontend Only** โ optionally connect to a remote backend API (enter backend IP + port + protocol)
- - **Local Backend Only** โ optionally connect to remote PostgreSQL, Redis, and frontend (enter each address and port)
- - **Local Frontend + Backend** โ optionally expose services via a non-localhost address
- - Non-localhost deployments automatically update `frontend/.env` CSP whitelist (`NEXT_PUBLIC_CSP_CONNECT_SRC_EXTRA`)
- ```bash
- ./deploy/quick-start.sh --skip-env # Skip .env file initialization
- ./deploy/quick-start.sh --skip-db-init # Skip database initialization
- ```
- ### Launch by Scenario
- ```bash
- # โโโ Development โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/scripts/dev.sh # Docker full-stack dev (containerized frontend + backend)
- ./deploy/scripts/dev-local.sh # Local dev prep (start middleware, run backend/frontend on host)
- ./deploy/scripts/dev-backend.sh # Local backend only (requires middleware running)
- ./deploy/scripts/dev-frontend.sh # Local frontend only (requires backend running)
- # โโโ Production โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/scripts/prod.sh # Production deploy (pre-built images + docker-compose.prod.yml)
- ./deploy/scripts/prod.sh --skip-mcp # Production without MCP service
- ./deploy/scripts/prod.sh --skip-pull # Skip image pull, use local images
- # โโโ Middleware / Infrastructure โโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/scripts/start-middleware.sh # Start middleware (PostgreSQL + Redis + MCP)
- ./deploy/scripts/minimal.sh # Minimal startup (PostgreSQL + Redis only)
- ./deploy/scripts/minimal.sh --with-mcp # Minimal + MCP service
- ./deploy/scripts/stop-middleware.sh # Stop middleware
- # โโโ Test / CI โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/scripts/test.sh # Test environment (minimal deps, automation-friendly)
- # โโโ Install / Check โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/install.sh # Interactive installation wizard (generates config files)
- ./deploy/install.sh --mode dev --non-interactive # Non-interactive install
- ./deploy/scripts/check-env.sh # Environment preflight (Docker, ports, config files)
- # โโโ Image Management โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ./deploy/deploy.sh build # Build frontend + backend images
- ./deploy/deploy.sh build --all # Build all images (including OpenClaw)
- ./deploy/deploy.sh push # Build and push to registry
- ./deploy/deploy.sh pull # Pull latest pre-built images
- ```
- ### Default Ports
- | Service | Port | URL |
- |---------|------|-----|
- | Frontend | `3000` | http://localhost:3000 |
- | Backend API | `8000` | http://localhost:8000 |
- | API Docs | `8000/docs` | Swagger UI |
- | PostgreSQL | `5432` | Database |
- | Redis | `6379` | Cache |
- > **Prerequisites:** Docker + Docker Compose. See [INSTALL.md](INSTALL.md) for detailed installation guide, [deploy/PRODUCTION_IP_GUIDE.md](deploy/PRODUCTION_IP_GUIDE.md) for production deployment.
- ---
- ## Architecture
- <p align="center">
- <img src="docs/architecture-diagram.png" alt="JoySafeter System Architecture" width="900" />
- </p>
- > Full architecture details: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- **Key design principles:**
- - **Graph-based execution** โ every agent workflow is a stateful LangGraph, enabling pause, resume, and branch
- - **Unified Run Center** โ Chat, Copilot, and Skill Creator share a single event-sourced run lifecycle (Run โ Event โ Snapshot)
- - **Unified WebSocket layer** โ BaseWsClient abstract class; Chat / Run / Notification clients share lifecycle, auth (ws-token), and reconnect logic
- - **Full-chain trace_id propagation** โ contextvars-based request tracing from HTTP/WS entry through LangGraph to persistence
- - **Glass-box observability** โ real-time Langfuse tracing of every agent decision and state transition
- - **RAII sandbox isolation** โ per-user Docker containers with automatic handle release, zero state leakage
- - **Canonical model identifiers** โ full-stack (provider_name, model_name) resolution via ModelService โ ModelFactory
- - **Layered skill system** โ skills are versioned units that compose into workflows without coupling
- ### User Journey โ Quick Start in 9 Steps
- <p align="center">
- <img src="docs/user-journey-quickstart.png" alt="JoySafeter Quick Start User Journey" width="900" />
- </p>
- > **Login** โ **Configure Models** โ **MCP Tools** โ **Skill Management** โ **Build Agent** โ **Self-Test (Langfuse Trace)** โ **Publish** โ **Chat UI** โ **Run Center**
- ---
- ## Tech Stack
- | Layer | Technology | Purpose |
- |-------|------------|---------|
- | **Frontend** | Next.js 16, React 19, TypeScript | Server-side rendering, App Router |
- | **UI** | Radix UI, Tailwind CSS, Framer Motion | Accessible, animated components |
- | **State** | Zustand, TanStack Query | Client & server state |
- | **Workflow Editor** | React Flow | Interactive node-based builder |
- | **Backend** | FastAPI, Python 3.12+ | Async API with OpenAPI docs |
- | **AI Framework** | LangChain, LangGraph, DeepAgents | Agent orchestration & workflows |
- | **MCP** | mcp 1.20+, fastmcp 2.14+ | Tool protocol support |
- | **Database** | PostgreSQL, SQLAlchemy 2.0 | Async ORM with migrations |
- | **Cache** | Redis | Session cache & rate limiting |
- | **Observability** | Langfuse, Loguru | Tracing & structured logging |
- ---
- ## What's New
- > Full history: [CHANGELOG.md](CHANGELOG.md)
- | Tag | Feature | What it means |
- |-----|---------|---------------|
- | **NEW** | **Run Center Architecture** | Chat & Copilot fully integrated into Run Center โ run details, session recovery, and live event replay on page refresh |
- | **NEW** | **Dark Mode & Preferences** | System / Light / Dark theme switching; redesigned profile page with language & theme preferences |
- | **NEW** | **Unified WebSocket Layer** | BaseWsClient abstract class โ Chat, Run, and Notification clients share lifecycle, auth (ws-token), and reconnect logic |
- | **NEW** | **Full-Chain trace_id Propagation** | End-to-end request tracing via contextvars for complete observability |
- | **NEW** | **Ollama One-Click Integration** | Local Ollama model provider added out of the box |
- | **NEW** | **Version Display** | In-app version info tied to bump-version.sh release pipeline |
- | **NEW** | **Unified Model Identifiers** | Full-stack (provider_name, model_name) canonical form with data migration โ no more legacy field ambiguity |
- | **UPGRADE** | **Design Token Overhaul** | Hardcoded colors, font sizes, and border radii replaced with CSS variables and Tailwind tokens; z-index and typography scales unified |
- | **UPGRADE** | **Sandbox Overhaul** | RAII handle management, adapter API uploads, security hardening |
- | **UPGRADE** | **Frontend Component Extraction** | ConfirmDialog, UnifiedDialog, InlineRenameInput, SidebarContextMenu, AgentListContext โ less prop drilling, more reuse |
- | **UPGRADE** | **i18n & Code Quality** | Backend error messages internationalized; email templates moved to Jinja2; LLM prompts externalized to Markdown; 129 unused SVG icons removed |
- ---
- ## Documentation
- ### Getting Started
- - [INSTALL.md](INSTALL.md) โ Installation guide (Docker / manual / pre-built images)
- - [DEVELOPMENT.md](DEVELOPMENT.md) โ Local development setup
- - [deploy/README.md](deploy/README.md) โ Docker deployment
- - [deploy/PRODUCTION_IP_GUIDE.md](deploy/PRODUCTION_IP_GUIDE.md) โ Production deployment
- ### Deep Dive
- - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) โ Architecture overview
- - [backend/README.md](backend/README.md) โ Backend guide
- - [frontend/README.md](frontend/README.md) โ Frontend guide
- ### Tutorials
- See [docs/tutorials/](docs/tutorials/) for step-by-step guides on model setup, MCP integration, skill development, and more.
- ### Governance
- - [CONTRIBUTING.md](CONTRIBUTING.md) โ Contributing guide
- - [SECURITY.md](SECURITY.md) โ Security policy
- - [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) โ Code of conduct
- ---
- ## Community
- Join the WeChat user group for questions and discussion:
- <p align="center">
- <img src="docs/assets/wechat-group-3.png" alt="JoySafeter User Group 3" width="280" />
-
- <img src="docs/assets/wechat-group-4.png" alt="JoySafeter User Group 4" width="280" />
- </p>
- ---
- ## Contributing
- ```bash
- git clone https://github.com/jd-opensource/JoySafeter.git
- git checkout -b feature/amazing-feature
- git commit -m 'feat: add amazing feature'
- git push origin feature/amazing-feature
- ```
- See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
- ---
- ## License
- Apache License 2.0 โ see [LICENSE](LICENSE) for details.
- Third-party component licenses: [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md)
- ---
- ## Acknowledgments
- <table>
- <tr>
- <td align="center"><a href="https://github.com/langchain-ai/langchain"><img src="https://avatars.githubusercontent.com/u/126733545?s=64" width="48"/><br/><sub>LangChain</sub></a></td>
- <td align="center"><a href="https://github.com/langchain-ai/langgraph"><img src="https://avatars.githubusercontent.com/u/126733545?s=64" width="48"/><br/><sub>LangGraph</sub></a></td>
- <td align="center"><a href="https://fastapi.tiangolo.com/"><img src="https://fastapi.tiangolo.com/img/icon-white.svg" width="48"/><br/><sub>FastAPI</sub></a></td>
- <td align="center"><a href="https://nextjs.org/"><img src="https://assets.vercel.com/image/upload/v1662130559/nextjs/Icon_dark_background.png" width="48"/><br/><sub>Next.js</sub></a></td>
- <td align="center"><a href="https://www.radix-ui.com/"><img src="https://avatars.githubusercontent.com/u/75042455?s=64" width="48"/><br/><sub>Radix UI</sub></a></td>
- </tr>
- </table>
- ---
- <p align="center">
- <sub>Made with โค๏ธ by the JoySafeter Team</sub><br/>
- <sub>For commercial solutions, contact JD Technology Solutions Team at <a href="mailto:org.ospo1@jd.com">org.ospo1@jd.com</a></sub>
- </p>
backend/app/core/agent/backends/pydantic_adapter.py ยท Bundled script ยท 1,028 lines ยท 2 findings marked
- """Adapter for pydantic-ai-backend DockerSandbox to SandboxBackendProtocol.
- This module provides an adapter layer that bridges pydantic-ai-backend's
- DockerSandbox implementation with deepAgents' SandboxBackendProtocol interface.
- Supports advanced features from pydantic-ai-backend 0.1.5+:
- - RuntimeConfig for pre-configured environments (python-datascience, python-web, etc.)
- - session_id for multi-user session management
- - idle_timeout for automatic container cleanup
- - volumes for Docker volume mounting
- """
- import uuid
- from datetime import datetime
- from pathlib import Path
- from typing import List
- from deepagents.backends.protocol import (
- EditResult,
- ExecuteResponse,
- FileDownloadResponse,
- FileInfo,
- FileUploadResponse,
- GrepMatch,
- SandboxBackendProtocol,
- WriteResult,
- )
- from deepagents.backends.utils import format_read_response
- from loguru import logger
- from pydantic_ai_backends import DockerSandbox
- from app.core.agent.backends.constants import (
- DEFAULT_AUTO_REMOVE,
- DEFAULT_COMMAND_TIMEOUT,
- DEFAULT_DOCKER_IMAGE,
- DEFAULT_IDLE_TIMEOUT,
- DEFAULT_MAX_OUTPUT_SIZE,
- DEFAULT_WORKING_DIR,
- )
- from app.core.agent.backends.runtime_config import (
- BUILTIN_RUNTIMES,
- RuntimeConfig,
- get_builtin_runtime,
- list_builtin_runtimes,
- resolve_runtime,
- )
- from app.utils.backend_utils import create_execute_response
- # Re-export for backward compatibility
- __all__ = [
- "PydanticSandboxAdapter",
- "RuntimeConfig",
- "BUILTIN_RUNTIMES",
- "get_builtin_runtime",
- "list_builtin_runtimes",
- ]
- class PydanticSandboxAdapter(SandboxBackendProtocol):
- """Adapter that wraps pydantic-ai-backend's DockerSandbox for deepAgents.
- This adapter implements SandboxBackendProtocol by delegating to
- pydantic-ai-backend's DockerSandbox, providing a seamless integration
- with deepAgents' FilesystemMiddleware.
- **Lifecycle Management:**
- - `start()` is called automatically in `__init__` to start the container
- - `stop()`: stop the container only (no remove); use for stop/restart semantics
- - `cleanup()`: stop and remove the container; use for rebuild/teardown only
- - With auto_remove=False, stop() keeps the container so start() can restart it
- **Features:**
- - Full SandboxBackendProtocol compatibility
- - Delegates to pydantic-ai-backend's DockerSandbox
- - Explicit lifecycle management (start/stop)
- - Automatic resource cleanup
- - Error handling and logging
- - Idempotent start/stop operations
- - RuntimeConfig support for pre-configured environments
- - Session management with session_id
- - Idle timeout for automatic container cleanup
- - Docker volume mounting support
- **Pre-configured Runtimes:**
- - python-minimal: Basic Python 3.12 environment
- - python-datascience: Pandas, NumPy, Matplotlib, Scikit-learn
- - python-web: FastAPI, Uvicorn, SQLAlchemy, httpx
- - python-ml: PyTorch, Transformers
- - node-minimal: Basic Node.js 20 environment
- - node-react: TypeScript, Vite, React
- Example:
- ```python
- from app.core.agent.backends.pydantic_adapter import (
- PydanticSandboxAdapter,
- RuntimeConfig,
- )
- # Basic usage
- adapter = PydanticSandboxAdapter(image="python:3.12-slim")
- # Using pre-configured runtime
- adapter = PydanticSandboxAdapter(runtime="python-datascience")
- # Using custom runtime
- custom_runtime = RuntimeConfig(
- name="ml-env",
- base_image="python:3.12-slim",
- packages=["torch", "transformers"],
- )
- adapter = PydanticSandboxAdapter(runtime=custom_runtime)
- # With session management and volume mounting
- adapter = PydanticSandboxAdapter(
- runtime="python-web",
- session_id="user-123",
- idle_timeout=1800,
- volumes={"/data": "/app/shared"},
- )
- # Use as context manager
- with PydanticSandboxAdapter(runtime="python-minimal") as adapter:
- result = adapter.execute("python --version")
- ```
- """
- def __init__(
- self,
- image: str = DEFAULT_DOCKER_IMAGE,
- working_dir: str = DEFAULT_WORKING_DIR,
- auto_remove: bool = DEFAULT_AUTO_REMOVE,
- max_output_size: int = DEFAULT_MAX_OUTPUT_SIZE,
- command_timeout: int = DEFAULT_COMMAND_TIMEOUT,
- runtime: RuntimeConfig | str | None = None,
- session_id: str | None = None,
- idle_timeout: int = DEFAULT_IDLE_TIMEOUT,
- volumes: dict[str, str] | None = None,
- ):
- """Initialize PydanticSandboxAdapter.
- Creates and starts the Docker sandbox container following the
- pydantic-ai-backend lifecycle pattern. The container is automatically
- started via `start()` method.
- Args:
- image: Docker image to use (default: python:3.12-slim).
- Ignored if runtime is specified.
- working_dir: Working directory in container
- auto_remove: Auto-remove container on exit
- max_output_size: Maximum command output size in characters
- command_timeout: Command execution timeout in seconds
- runtime: Pre-configured runtime environment. Can be:
- - str: Name of builtin runtime ("python-datascience", "python-web", etc.)
- - RuntimeConfig: Custom runtime configuration
- - None: Use image parameter directly
- session_id: Session identifier for multi-user scenarios.
- Used to track and potentially reuse sandbox instances.
- idle_timeout: Time in seconds before idle container is cleaned up (default: 3600)
- volumes: Docker volume mappings {host_path: container_path}
- Raises:
- ImportError: If pydantic-ai-backend is not installed
- RuntimeError: If DockerSandbox creation fails
- Note:
- The sandbox container is automatically started during initialization.
- Call `cleanup()` when done to stop and remove the container.
- Example:
- ```python
- # Using builtin runtime
- adapter = PydanticSandboxAdapter(runtime="python-datascience")
- # Using custom runtime
- adapter = PydanticSandboxAdapter(
- runtime=RuntimeConfig(
- name="custom",
- base_image="python:3.11",
- packages=["requests"],
- ),
- )
- # With volumes
- adapter = PydanticSandboxAdapter(
- runtime="python-web",
- volumes={"/host/data": "/container/data"},
- )
- ```
- """
- # Store parameters
- self.runtime = runtime
- self.session_id = session_id
- self.idle_timeout = idle_timeout
- self.volumes = volumes or {}
- # Resolve runtime to get effective image and config
- effective_image, self._runtime_config = resolve_runtime(image, runtime)
- self._id = session_id or str(uuid.uuid4())
- self.image = effective_image
- self.working_dir = working_dir
- self.auto_remove = auto_remove
- self.max_output_size = max_output_size
- self.command_timeout = command_timeout
- self._started = False # Track sandbox start state
- self._saved_container_id: str | None = None # For restart: reuse same container after stop
- # Log initialization
- runtime_info = f", runtime={self._runtime_config.name}" if self._runtime_config else ""
- volumes_info = f", volumes={len(self.volumes)}" if self.volumes else ""
- logger.info(
- f"Initializing PydanticSandboxAdapter: id={self._id}, "
- f"image={self.image}, working_dir={working_dir}{runtime_info}{volumes_info}"
- )
- # Prepare runtime parameter for pydantic-ai-backend
- pydantic_runtime = None
- if self._runtime_config:
- pydantic_runtime = self._runtime_config.to_pydantic_runtime()
- # Create DockerSandbox with pydantic-ai-backend API
- try:
- sandbox_kwargs: dict = {
- "image": self.image,
- "work_dir": working_dir,
- "auto_remove": auto_remove,
- "runtime": pydantic_runtime,
- "session_id": self.session_id,
- "idle_timeout": self.idle_timeout,
- "volumes": self.volumes if self.volumes else None,
- }
- self._sandbox = DockerSandbox(**sandbox_kwargs)
- logger.info(f"DockerSandbox created: id={self._id}, image={self.image}")
- except Exception as e:
- logger.error(f"Failed to create DockerSandbox for adapter {self._id}: {e}", exc_info=True)
- raise RuntimeError(f"Failed to create DockerSandbox: {e}") from e
- # Start the sandbox
- logger.debug(f"Starting sandbox {self._id}...")
- self.start()
- @property
- def id(self) -> str:
- """Unique identifier for this backend instance."""
- return self._id
- def is_started(self) -> bool:
- """Check if the sandbox container is started.
- Returns:
- True if the sandbox is started, False otherwise.
- """
- return self._started
- def get_runtime_config(self) -> RuntimeConfig | None:
- """Get the current runtime configuration.
- Returns:
- RuntimeConfig instance, or None if no runtime config is in use.
- """
- return self._runtime_config
- def get_container_id(self) -> str | None:
- """Get the Docker container ID for persistence/reconnection.
- Returns:
- Docker container ID string, or None if not available.
- """
- container = getattr(self._sandbox, "_container", None)
- if container is not None:
- return getattr(container, "id", None)
- return self._saved_container_id
- @classmethod
- def from_existing_container(
- cls,
- container,
- session_id: str,
- image: str = DEFAULT_DOCKER_IMAGE,
- idle_timeout: int = DEFAULT_IDLE_TIMEOUT,
- working_dir: str = DEFAULT_WORKING_DIR,
- max_output_size: int = DEFAULT_MAX_OUTPUT_SIZE,
- command_timeout: int = DEFAULT_COMMAND_TIMEOUT,
- ) -> "PydanticSandboxAdapter":
- """Create an adapter that wraps an already-running Docker container.
- Used for app-restart recovery: reconnect to a container that was
- created in a previous process lifetime.
- Args:
- container: A docker.models.containers.Container instance (already running).
- session_id: Session/sandbox ID.
- image: Image name (for metadata only).
- idle_timeout: Idle timeout in seconds.
- working_dir: Working directory inside the container.
- max_output_size: Max command output size.
- command_timeout: Command timeout in seconds.
- Returns:
- PydanticSandboxAdapter wrapping the existing container.
- """
- # Create a minimal DockerSandbox and inject the container
- sandbox = DockerSandbox(
- image=image,
- work_dir=working_dir,
- auto_remove=False,
- session_id=session_id,
- idle_timeout=idle_timeout,
- )
- # Inject the existing container so DockerSandbox uses it
- sandbox._container = container
- # Build adapter without calling __init__ (which would start a new container)
- adapter = cls.__new__(cls)
- adapter.runtime = None
- adapter.session_id = session_id
- adapter.idle_timeout = idle_timeout
- adapter.volumes = {}
- adapter._runtime_config = None
- adapter._id = session_id
- adapter.image = image
- adapter.working_dir = working_dir
- adapter.auto_remove = False
- adapter.max_output_size = max_output_size
- adapter.command_timeout = command_timeout
- adapter._started = True
- adapter._saved_container_id = None
- adapter._sandbox = sandbox
- logger.info(
- f"PydanticSandboxAdapter.from_existing_container: "
- f"id={session_id}, container={getattr(container, 'short_id', 'unknown')}"
- )
- return adapter
- def start(self) -> None:
- """Start the Docker sandbox container. Idempotent. Restart reuses same container if available."""
- if self._started:
- return
- upstream_container = getattr(self._sandbox, "_container", None)
- if self._saved_container_id and upstream_container is None:
- try:
- from app.core.agent.backends.docker_check import get_docker_client
- client = get_docker_client()
- container = client.containers.get(self._saved_container_id)
- container.start()
- self._sandbox._container = container
- self._started = True
- self._saved_container_id = None
- logger.info(f"Sandbox {self._id} restarted (same container)")
- return
- except Exception as e:
- logger.warning(f"Failed to restart existing container {self._id}, will create new: {e}")
- stale_id = self._saved_container_id
- self._saved_container_id = None
- if stale_id is not None:
- self._force_remove_container(stale_id)
- try:
- if hasattr(self._sandbox, "start"):
- self._sandbox.start()
- self._started = True
- logger.info(f"Sandbox {self._id} started (image={self.image})")
- except Exception as e:
- logger.error(f"Failed to start sandbox {self._id}: {e}")
- raise RuntimeError(f"Failed to start sandbox {self._id}: {e}") from e
- # Dangerous command patterns (defense-in-depth, not sole security boundary)
- _DANGEROUS_PATTERNS = [
- r"rm\s+-rf\s+/\s*$", # rm -rf /
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 373 on GitHub
- r"mkfs\.", # format disk
- r"dd\s+.*of=/dev/", # write to device
- r":\(\)\s*\{", # fork bomb :(){ :|:& };:
- Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
- Line 376 on GitHub
- ]
- _DANGEROUS_RE = None # Lazy-compiled combined regex
- @classmethod
- def _get_dangerous_re(cls):
- if cls._DANGEROUS_RE is None:
- import re
- cls._DANGEROUS_RE = re.compile("|".join(f"(?:{p})" for p in cls._DANGEROUS_PATTERNS))
- return cls._DANGEROUS_RE
- def _exec_command(self, command: str) -> tuple[str, int]:
- """Execute command in sandbox with safety checks.
- Args:
- command: Shell command to execute
- Returns:
- Tuple of (output, exit_code)
- """
- if self._get_dangerous_re().search(command):
- logger.warning(f"[{self._id}] Blocked dangerous command: {command[:100]}")
- return "Error: command blocked by security policy", 1
- logger.debug(f"[{self._id}] _exec_command START: {command[:100]}")
- try:
- result = self._sandbox.execute(command)
- # 1. Handle ExecuteResponse from pydantic-ai-backend (output/exit_code/truncated)
- # This is the primary format returned by DockerSandbox.execute()
- if hasattr(result, "output") and hasattr(result, "exit_code"):
- output = result.output if isinstance(result.output, str) else str(result.output or "")
- exit_code = result.exit_code
- logger.debug(f"[{self._id}] _exec_command END: exit_code={exit_code}, output_len={len(output)}")
- return output, exit_code
- # 2. Handle ExecutionResult format (stdout/returncode)
- if hasattr(result, "stdout") and hasattr(result, "returncode"):
- output = (
- result.stdout.decode("utf-8", errors="replace")
- if isinstance(result.stdout, bytes)
- else str(result.stdout or "")
- )
- logger.debug(
- f"[{self._id}] _exec_command END (ExecutionResult): exit_code={result.returncode}, output_len={len(output)}"
- )
- return output, result.returncode
- # 3. Handle dict format (TypedDict or plain dict) - supports both naming conventions
- if isinstance(result, dict):
- output = str(result.get("output", result.get("stdout", "")))
- exit_code_raw = result.get("exit_code", result.get("returncode", 0))
- exit_code = 0
- if exit_code_raw is not None:
- try:
- exit_code = int(exit_code_raw)
- except (TypeError, ValueError):
- exit_code = 0
- logger.debug(f"[{self._id}] _exec_command END (dict): exit_code={exit_code}, output_len={len(output)}")
- return output, exit_code
- # 4. Fallback - should rarely happen now
- logger.warning(
- f"[{self._id}] _exec_command: unexpected result type {type(result).__name__}, returning as string"
- )
- return str(result) if result else "", 0
- except Exception as e:
- logger.error(f"[{self._id}] _exec_command FAILED: {e}")
- return f"Error: {e}", -1
- # SandboxBackendProtocol implementation
- def ls_info(self, path: str) -> list[FileInfo]:
- """List files and directories in the specified directory.
- Args:
- path: Absolute path to directory.
- Returns:
- List of FileInfo dicts for files and directories.
- """
- try:
- # Use ls command to list files
- output, exit_code = self._exec_command(f"ls -la {path}")
- if exit_code != 0:
- return []
- infos: list[FileInfo] = []
- lines = output.strip().split("\n")[1:] # Skip "total" line
- for line in lines:
- parts = line.split()
- if len(parts) < 9:
- continue
- permissions = parts[0]
- size = int(parts[4]) if parts[4].isdigit() else 0
- name = " ".join(parts[8:])
- # Skip . and ..
- if name in (".", ".."):
- continue
- file_path = f"{path.rstrip('/')}/{name}"
- is_dir = permissions.startswith("d")
- infos.append(
- {
- "path": file_path + ("/" if is_dir else ""),
- "is_dir": is_dir,
- "size": size,
- "modified_at": "",
- }
- )
- return infos
- except Exception as e:
- logger.error(f"Failed to list directory {path}: {e}")
- return []
- def read(
- self,
- file_path: str,
- offset: int = 0,
- limit: int = 2000,
- ) -> str:
- """Read file content with line numbers.
- Uses DockerSandbox.read() which leverages Docker's get_archive API
- for reliable file reading with intelligent encoding detection.
- Args:
- file_path: Absolute file path
- offset: Line offset to start reading from (0-indexed)
- limit: Maximum number of lines to read
- Returns:
- Formatted file content with line numbers, or error message.
- """
- logger.info(f"[{self._id}] Reading file: {file_path}")
- try:
- content = self.raw_read(file_path, offset=0, limit=100000)
- if content.startswith("[Error:") or content.startswith("Error:"):
- return content
- # Format with line numbers using deepagents utility
- lines = content.splitlines()
- file_data = {
- "content": lines,
- "created_at": datetime.now().isoformat(),
- "modified_at": datetime.now().isoformat(),
- }
- result: str = format_read_response(file_data, offset, limit)
- return result
- except Exception as e:
- logger.error(f"[{self._id}] Failed to read file {file_path}: {e}")
- return f"Error: {str(e)}"
- def raw_read(
- self,
- file_path: str,
- offset: int = 0,
- limit: int = 100000,
- ) -> str:
- """Read raw file content without injecting line numbers.
- This is intended for UI/API consumers that need the original file text
- and will handle presentation concerns such as line-number gutters.
- """
- logger.info(f"[{self._id}] Raw reading file: {file_path}")
- content_raw = self._sandbox.read(file_path, offset=offset, limit=limit)
- content = content_raw if isinstance(content_raw, str) else str(content_raw)
- if content.startswith("[Error:") or content.startswith("Error:"):
- return content
- # Remove upstream pagination footer to keep API consumers on raw file text.
- if "\n\n[..." in content:
- content = content.split("\n\n[...")[0]
- return content
- def write(
- self,
- file_path: str,
- content: str | bytes,
- ) -> WriteResult:
- """Create a new file with content.
- Uses DockerSandbox.write() which leverages Docker's put_archive API
- for reliable file writing without shell command length limits.
- Accepts both str and bytes (upstream DockerSandbox supports both).
- Args:
- file_path: Absolute file path
- content: File content (text or binary)
- Returns:
- WriteResult with success or error.
- """
- logger.info(f"[{self._id}] Writing file: {file_path}")
- try:
- # Check if file already exists
- check_result = self._exec_command(f"test -f {file_path}")
- if check_result[1] == 0: # exit_code == 0 means file exists
- return WriteResult(
- error=f"Cannot write to {file_path} because it already exists. "
- "Read and then make an edit, or write to a new path."
- )
- result = self._sandbox.write(file_path, content)
- if hasattr(result, "error") and result.error:
- return WriteResult(error=result.error)
- return WriteResult(path=file_path, files_update=None)
- except Exception as e:
- logger.error(f"[{self._id}] Failed to write file {file_path}: {e}")
- return WriteResult(error=f"Failed to write file: {str(e)}")
- def write_overwrite(
- self,
- file_path: str,
- content: str | bytes,
- ) -> WriteResult:
- """Write a file, overwriting if it already exists.
- Uses DockerSandbox.write() which leverages Docker's put_archive API
- for reliable file writing without shell command length limits.
- Accepts both str and bytes (upstream DockerSandbox supports both).
- Args:
- file_path: Absolute file path
- content: File content (text or binary)
- Returns:
- WriteResult with success or error.
- """
- logger.debug(f"[{self._id}] write_overwrite: {file_path}")
- try:
- # Use upstream DockerSandbox.write() which uses Docker put_archive API
- # This handles large files and special characters reliably
- result = self._sandbox.write(file_path, content)
- # Convert upstream WriteResult to deepagents WriteResult
- if hasattr(result, "error") and result.error:
- return WriteResult(error=result.error)
- return WriteResult(path=file_path, files_update=None)
- except Exception as e:
- logger.error(f"[{self._id}] Failed to write file {file_path}: {e}")
- return WriteResult(error=f"Failed to write file: {str(e)}")
- def delete(self, file_path: str) -> bool:
- """Delete a file inside the container.
- Args:
- file_path: Absolute file path inside the container.
- Returns:
- True if deleted successfully, False otherwise.
- """
- import shlex
- logger.debug(f"[{self._id}] Deleting file: {file_path}")
- try:
- _, exit_code = self._exec_command(f"rm -f {shlex.quote(file_path)}")
- return exit_code == 0
- except Exception as e:
- logger.error(f"[{self._id}] Failed to delete file {file_path}: {e}")
- return False
- def mkdir(self, dir_path: str) -> bool:
- """Create a directory (and parents) inside the container.
- Args:
- dir_path: Absolute directory path inside the container.
- Returns:
- True if created successfully, False otherwise.
- """
- import shlex
- logger.debug(f"[{self._id}] Creating directory: {dir_path}")
- try:
- _, exit_code = self._exec_command(f"mkdir -p {shlex.quote(dir_path)}")
- return exit_code == 0
- except Exception as e:
- logger.error(f"[{self._id}] Failed to create directory {dir_path}: {e}")
- return False
- def edit(
- self,
- file_path: str,
- old_string: str,
- new_string: str,
- replace_all: bool = False,
- ) -> EditResult:
- """Edit a file by replacing string occurrences.
- Uses DockerSandbox.edit() which performs in-memory string replacement
- and writes back using Docker's put_archive API.
- Args:
- file_path: Absolute file path
- old_string: String to replace
- new_string: Replacement string
- replace_all: Replace all occurrences (default: False)
- Returns:
- EditResult with success or error.
- """
- logger.info(f"[{self._id}] Editing file: {file_path}")
- try:
- # Use upstream DockerSandbox.edit() which:
- # 1. Reads file using Docker get_archive API
- # 2. Performs in-memory string replacement
- # 3. Writes back using Docker put_archive API
- result = self._sandbox.edit(file_path, old_string, new_string, replace_all)
- # Convert upstream EditResult to deepagents EditResult
- if hasattr(result, "error") and result.error:
- return EditResult(error=result.error)
- return EditResult(
- path=getattr(result, "path", file_path),
- files_update=None,
- occurrences=getattr(result, "occurrences", 1),
- )
- except Exception as e:
- logger.error(f"[{self._id}] Failed to edit file {file_path}: {e}")
- return EditResult(error=f"Failed to edit file: {str(e)}")
- def grep_raw(
- self,
- pattern: str,
- path: str | None = None,
- glob: str | None = None,
- ) -> list[GrepMatch] | str:
- """Search for a pattern in files.
- Args:
- pattern: Search pattern (literal string)
- path: Directory to search in (default: working_dir)
- glob: Glob pattern to filter files
- Returns:
- List of GrepMatch dicts or error string.
- """
- logger.info(f"[{self._id}] Grepping for pattern: {pattern}")
- search_path = path or self.working_dir
- grep_cmd = f"grep -rn '{pattern}' {search_path}"
- if glob:
- grep_cmd += f" --include='{glob}'"
- output, exit_code = self._exec_command(grep_cmd)
- if exit_code != 0 and not output:
- return []
- matches: list[GrepMatch] = []
- for line in output.strip().split("\n"):
- if not line:
- continue
- parts = line.split(":", 2)
- if len(parts) >= 3:
- matches.append(
- {
- "path": parts[0],
- "line": int(parts[1]) if parts[1].isdigit() else 0,
- "text": parts[2],
- }
- )
- return matches
- def glob_info(self, pattern: str, path: str = "/") -> list[FileInfo]:
- """Find files matching a glob pattern.
- Args:
- pattern: Glob pattern (e.g., "*.py", "**/*.txt")
- path: Directory to search in
- Returns:
- List of FileInfo dicts for matching files.
- """
- logger.info(f"[{self._id}] Globbing for pattern: {pattern}")
- # Use find command with pattern
- find_cmd = f"find {path} -name '{pattern}'"
- output, exit_code = self._exec_command(find_cmd)
- if exit_code != 0:
- return []
- infos: list[FileInfo] = []
- for file_path in output.strip().split("\n"):
- if not file_path:
- continue
- # Get file size
- size_cmd = f"stat -c %s {file_path}"
- size_output, _ = self._exec_command(size_cmd)
- size = int(size_output.strip()) if size_output.strip().isdigit() else 0
- infos.append(
- {
- "path": file_path,
- "is_dir": False,
- "size": size,
- "modified_at": "",
- }
- )
- return infos
- def execute(self, command: str) -> ExecuteResponse:
- """Execute a shell command in the container.
- Args:
- command: Shell command to execute.
- Returns:
- ExecuteResponse with output, exit code, and truncation flag.
- """
- logger.info(f"[{self._id}] execute() called: {command[:100]}...")
- try:
- # Execute command with timeout
- output, exit_code = self._exec_command(command)
- # Create response with automatic truncation
- response = create_execute_response(
- output=output,
- exit_code=exit_code,
- max_output_size=self.max_output_size,
- )
- if response.truncated:
- logger.debug(
- f"[{self._id}] Output truncated: {len(output)} -> "
- f"{len(response.output)} chars (max={self.max_output_size})"
- )
- logger.debug(
- f"[{self._id}] Command execution completed: exit_code={exit_code}, truncated={response.truncated}"
- )
- return response
- except Exception as e:
- logger.error(f"[{self._id}] Error executing command '{command}': {e}", exc_info=True)
- return ExecuteResponse(
- output=f"Error executing command: {str(e)}",
- exit_code=-1,
- truncated=False,
- )
- def download_files(self, paths: list[str]) -> list[FileDownloadResponse]:
- """Download multiple files from the Docker sandbox using native API.
- Args:
- paths: List of file paths to download.
- Returns:
- List of FileDownloadResponse objects, one per input path.
- """
- responses: list[FileDownloadResponse] = []
- for path in paths:
- try:
- result = self.execute(f"cat {path}")
- if result.exit_code == 0:
- responses.append(FileDownloadResponse(path=path, content=result.output.encode("utf-8"), error=None))
- else:
- responses.append(FileDownloadResponse(path=path, content=None, error="file_not_found"))
- except Exception as e:
- logger.error(f"Failed to download file {path}: {e}")
- responses.append(FileDownloadResponse(path=path, content=None, error="permission_denied"))
- return responses
- def _collect_file_paths(self, dir_path: str, base: str, out: List[str]) -> None:
- """Recursively collect all file paths under dir_path (relative to base)."""
- infos = self.ls_info(dir_path)
- for info in infos:
- path = (info.get("path") or "").rstrip("/")
- if not path:
- continue
- is_dir = info.get("is_dir", False)
- if is_dir:
- self._collect_file_paths(path, base, out)
- else:
- if path.startswith(base):
- rel = path[len(base) :].lstrip("/")
- if rel:
- out.append(path)
- def export_working_dir_to(self, target_dir: Path) -> int:
- """Export all files from the container working directory to target_dir.
- Preserves directory structure. Returns the number of files written.
- """
- target_dir = Path(target_dir).resolve()
- target_dir.mkdir(parents=True, exist_ok=True)
- base = self.working_dir.rstrip("/")
- if not base:
- base = "/"
- paths: List[str] = []
- self._collect_file_paths(base, base, paths)
- if not paths:
- return 0
- responses = self.download_files(paths)
- written = 0
- for resp in responses:
- if resp.error or resp.content is None:
- continue
- path = resp.path
- if path.startswith(base):
- rel = path[len(base) :].lstrip("/")
- else:
- rel = Path(path).name
- dest = target_dir / rel
- dest.parent.mkdir(parents=True, exist_ok=True)
- try:
- dest.write_bytes(resp.content)
- written += 1
- except Exception as e:
- logger.warning(f"[PydanticSandboxAdapter] Failed to write {dest}: {e}")
- return written
- def upload_files(self, files: list[tuple[str, bytes]]) -> list[FileUploadResponse]:
- """Upload multiple files to the Docker sandbox using base64 encoding.
- Args:
- files: List of (path, content) tuples to upload.
- Returns:
- List of FileUploadResponse objects, one per input file.
- """
- import base64
- responses: list[FileUploadResponse] = []
- for path, content in files:
- try:
- encoded = base64.b64encode(content).decode("ascii")
- _, exit_code = self._exec_command(f"echo '{encoded}' | base64 -d > {path}")
- if exit_code == 0:
- responses.append(FileUploadResponse(path=path, error=None))
- else:
- responses.append(FileUploadResponse(path=path, error="permission_denied"))
- except Exception as e:
- logger.error(f"Failed to upload file {path}: {e}")
- responses.append(FileUploadResponse(path=path, error="permission_denied"))
- return responses
- @staticmethod
- def _force_remove_container(container_id: str) -> None:
- """Force-remove a Docker container by ID, ignoring all errors."""
- try:
- from app.core.agent.backends.docker_check import get_docker_client
- get_docker_client().containers.get(container_id).remove(force=True)
- logger.info(f"Force-removed stale container {container_id[:12]}")
- except Exception as e:
- logger.warning(f"Could not remove stale container {container_id[:12]}: {e}")
- def stop(self) -> None:
- """Stop the Docker container without removing it. Idempotent.
- IMPORTANT: This only stops the container. It does NOT remove it.
- After stop(), start() can restart the same container via _saved_container_id.
- Never falls back to cleanup() which would destroy the container.
- """
- if not self._started:
- return
- # Save container_id BEFORE stopping so we can restart later
- upstream_container = getattr(self._sandbox, "_container", None)
- if upstream_container is not None:
- self._saved_container_id = getattr(upstream_container, "id", None)
- try:
- if hasattr(self._sandbox, "stop"):
- self._sandbox.stop()
- elif upstream_container is not None and hasattr(upstream_container, "stop"):
- # Direct Docker API: stop container without removing
- upstream_container.stop()
- else:
- # No stop method available; just mark as stopped
- logger.warning(
- f"Sandbox {self._id}: no stop() method available, marking as stopped but container may still run"
- )
- logger.info(f"Sandbox {self._id} stopped (container kept, id={self._saved_container_id})")
- except Exception as e:
- logger.warning(f"Failed to stop sandbox {self._id}: {e}")
- finally:
- self._started = False
- def cleanup(self) -> None:
- """Stop and remove the Docker container. Idempotent. Use for rebuild/teardown only.
- Sequence:
- 1. Save container_id from live container reference
- 2. Stop the container (sets _started=False, saves _saved_container_id)
- 3. Remove the container using saved references
- """
- # Save container reference BEFORE stop() clears it
- pre_stop_container = getattr(self._sandbox, "_container", None) if getattr(self, "_sandbox", None) else None
- pre_stop_container_id = getattr(pre_stop_container, "id", None) if pre_stop_container else None
- self.stop()
- if getattr(self, "_sandbox", None) is None:
- self._saved_container_id = None
- return
- # Determine which container ID to remove
- container_id_to_remove = self._saved_container_id or pre_stop_container_id
- try:
- # Try direct container reference first (may still be on _sandbox after stop)
- container = getattr(self._sandbox, "container", None) or getattr(self._sandbox, "_container", None)
- if container is not None and hasattr(container, "remove"):
- container.remove(force=True)
- logger.info(f"Sandbox {self._id} container removed (direct ref)")
- elif container_id_to_remove:
- from app.core.agent.backends.docker_check import get_docker_client
- client = get_docker_client()
- client.containers.get(container_id_to_remove).remove(force=True)
- logger.info(f"Sandbox {self._id} container removed (by id={container_id_to_remove[:12]})")
- elif hasattr(self._sandbox, "remove"):
- self._sandbox.remove()
- logger.info(f"Sandbox {self._id} container removed (sandbox.remove)")
- else:
- logger.warning(f"Sandbox {self._id}: no way to remove container")
- except Exception as e:
- logger.warning(f"Failed to remove sandbox container {self._id}: {e}")
- finally:
- self._saved_container_id = None
- # NOTE: __del__ is intentionally NOT implemented.
- # GC-time cleanup of Docker containers causes unpredictable behavior:
- # - May delete containers still tracked by the pool
- # - May run after event loop is closed
- # - Interferes with the pool-based lifecycle management
- # All cleanup must be explicit via cleanup() or the SandboxPool.
- def __enter__(self):
- """Context manager entry."""
- return self
- def __exit__(self, exc_type, exc_val, exc_tb):
- """Context manager exit."""
- self.cleanup()
backend/app/core/agent/code_agent/executor/router.py ยท Bundled script ยท 315 lines ยท 1 finding marked
- #!/usr/bin/env python
- """
- Executor Router for CodeAgent.
- This module implements intelligent routing between different Python executors
- based on code analysis and security requirements.
- """
- import re
- from typing import Any, Callable, Optional
- from loguru import logger
- from .base import CodeOutput, PythonExecutor
- from .local_executor import LocalPythonExecutor
- # Patterns that indicate potentially dangerous code
- DANGEROUS_PATTERNS = [
- # System/OS access
- (r"\bimport\s+os\b", "os module import"),
- (r"\bimport\s+subprocess\b", "subprocess module import"),
- (r"\bimport\s+sys\b", "sys module import"),
- (r"\bimport\s+socket\b", "socket module import"),
- (r"\bimport\s+shutil\b", "shutil module import"),
- # File operations with write mode
- (r"\bopen\s*\([^)]*['\"][wa]", "file write operation"),
- (r"\bopen\s*\([^)]*mode\s*=\s*['\"][wa]", "file write operation"),
- # Network requests
- (r"\brequests\.", "requests library usage"),
- (r"\burllib\.", "urllib library usage"),
- (r"\bhttpx\.", "httpx library usage"),
- (r"\baiohttp\.", "aiohttp library usage"),
- # Code execution
- (r"\beval\s*\(", "eval() call"),
- (r"\bexec\s*\(", "exec() call"),
- (r"\bcompile\s*\(", "compile() call"),
- (r"\b__import__\s*\(", "__import__() call"),
- # Shell commands
- (r"\bos\.system\s*\(", "os.system() call"),
- (r"\bos\.popen\s*\(", "os.popen() call"),
- (r"\bsubprocess\.", "subprocess usage"),
- # Dunder access
- (r"__\w+__\s*(?:\[|\.)", "dunder attribute access"),
- # Pickle (arbitrary code execution)
- (r"\bpickle\.load", "pickle deserialization"),
- (r"\bcPickle\.load", "pickle deserialization"),
- ]
- # Patterns that suggest data analysis (safe for local execution)
- DATA_ANALYSIS_PATTERNS = [
- r"\bimport\s+pandas\b",
- r"\bimport\s+numpy\b",
- r"\bimport\s+matplotlib\b",
- r"\bimport\s+seaborn\b",
- r"\bimport\s+sklearn\b",
- r"\bimport\s+scipy\b",
- r"\bpd\.",
- r"\bnp\.",
- r"\bplt\.",
- ]
- class SecurityError(Exception):
- """Exception raised when dangerous code is detected."""
- pass
- class ExecutorRouter:
- """
- Routes code execution to appropriate executor based on analysis.
- The router analyzes code to determine:
- 1. Is the code safe for local AST interpretation?
- 2. Does it require Docker isolation?
- 3. Should it be blocked entirely?
- Features:
- - Pattern-based danger detection
- - Configurable routing policies
- - Fallback handling
- - Execution metrics
- Example:
- >>> router = ExecutorRouter(
- ... local=LocalPythonExecutor(),
- ... docker=DockerPythonExecutor(),
- ... )
- >>> result = router("import pandas; df = pd.read_csv('data.csv')")
- # Routes to local executor (safe data analysis)
- >>> result = router("import os; os.system('rm -rf /')")
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 92 on GitHub
- # Routes to Docker executor (dangerous but allowed in sandbox)
- """
- def __init__(
- self,
- local: Optional[PythonExecutor] = None,
- docker: Optional[PythonExecutor] = None,
- allow_dangerous: bool = False,
- prefer_docker: bool = False,
- dangerous_patterns: Optional[list[tuple[str, str]]] = None,
- ):
- """
- Initialize the executor router.
- Args:
- local: Local Python executor (AST-based).
- docker: Docker Python executor.
- allow_dangerous: Allow dangerous code (routes to Docker).
- prefer_docker: Always prefer Docker when available.
- dangerous_patterns: Custom dangerous patterns to check.
- """
- self.local = local or LocalPythonExecutor()
- self.docker = docker
- self.allow_dangerous = allow_dangerous
- self.prefer_docker = prefer_docker
- self.dangerous_patterns = dangerous_patterns or DANGEROUS_PATTERNS
- # Metrics
- self._local_count = 0
- self._docker_count = 0
- self._blocked_count = 0
- logger.info(
- f"ExecutorRouter initialized: local={type(self.local).__name__}, "
- f"docker={type(self.docker).__name__ if self.docker else 'None'}, "
- f"allow_dangerous={allow_dangerous}"
- )
- def analyze_code(self, code: str) -> dict[str, Any]:
- """
- Analyze code for routing decisions.
- Args:
- code: The Python code to analyze.
- Returns:
- Analysis result with danger level and details.
- """
- dangers = []
- for pattern, description in self.dangerous_patterns:
- if re.search(pattern, code, re.IGNORECASE):
- dangers.append(description)
- # Check for data analysis (considered safe)
- is_data_analysis = any(re.search(pattern, code, re.IGNORECASE) for pattern in DATA_ANALYSIS_PATTERNS)
- return {
- "is_dangerous": len(dangers) > 0,
- "dangers": dangers,
- "is_data_analysis": is_data_analysis,
- "danger_level": len(dangers),
- }
- def route(self, code: str) -> PythonExecutor:
- """
- Determine which executor to use for the code.
- Args:
- code: The Python code to route.
- Returns:
- The appropriate executor.
- Raises:
- SecurityError: If dangerous code is not allowed.
- """
- # Always prefer Docker if configured
- if self.prefer_docker and self.docker is not None:
- logger.debug("Routing to Docker executor (prefer_docker=True)")
- return self.docker # type: ignore[return-value]
- # Analyze code
- analysis = self.analyze_code(code)
- if analysis["is_dangerous"]:
- dangers = ", ".join(analysis["dangers"])
- logger.warning(f"Dangerous code detected: {dangers}")
- if self.docker is not None:
- logger.info("Routing dangerous code to Docker executor")
- return self.docker # type: ignore[return-value]
- elif self.allow_dangerous:
- logger.warning("No Docker available, executing dangerous code locally")
- return self.local
- else:
- self._blocked_count += 1
- raise SecurityError(
- f"Dangerous code patterns detected: {dangers}. "
- "Configure a Docker executor or set allow_dangerous=True."
- )
- # Safe code - use local executor
- logger.debug("Routing to local executor (safe code)")
- return self.local
- def __call__(
- self,
- code: str,
- additional_tools: Optional[dict[str, Callable]] = None,
- ) -> CodeOutput:
- """
- Execute code using the appropriate executor.
- Args:
- code: The Python code to execute.
- additional_tools: Optional additional tools.
- Returns:
- CodeOutput from the execution.
- """
- try:
- executor = self.route(code)
- if executor is self.local:
- self._local_count += 1
- else:
- self._docker_count += 1
- return executor(code, additional_tools)
- except SecurityError as e:
- return CodeOutput(error=str(e))
- def send_tools(self, tools: dict[str, Callable]) -> None:
- """Send tools to both executors."""
- self.local.send_tools(tools)
- if self.docker:
- self.docker.send_tools(tools)
- def send_variables(self, variables: dict[str, Any]) -> None:
- """Send variables to both executors."""
- self.local.send_variables(variables)
- if self.docker:
- self.docker.send_variables(variables)
- def reset(self) -> None:
- """Reset both executors."""
- self.local.reset()
- if self.docker:
- self.docker.reset()
- def cleanup(self) -> None:
- """Cleanup both executors."""
- self.local.cleanup()
- if self.docker:
- self.docker.cleanup()
- def get_metrics(self) -> dict[str, int]:
- """Get routing metrics."""
- return {
- "local_executions": self._local_count,
- "docker_executions": self._docker_count,
- "blocked_executions": self._blocked_count,
- "total_executions": self._local_count + self._docker_count,
- }
- def __repr__(self) -> str:
- return (
- f"ExecutorRouter("
- f"local={type(self.local).__name__}, "
- f"docker={type(self.docker).__name__ if self.docker else 'None'}, "
- f"metrics={self.get_metrics()}"
- f")"
- )
- def create_router(
- enable_docker: bool = True,
- allow_dangerous: bool = False,
- prefer_docker: bool = False,
- docker_kwargs: Optional[dict[Any, Any]] = None,
- local_kwargs: Optional[dict[Any, Any]] = None,
- ) -> ExecutorRouter:
- """
- Factory function to create a configured ExecutorRouter.
- Args:
- enable_docker: Enable Docker executor.
- allow_dangerous: Allow dangerous code.
- prefer_docker: Always prefer Docker.
- docker_kwargs: Arguments for DockerPythonExecutor.
- local_kwargs: Arguments for LocalPythonExecutor.
- Returns:
- Configured ExecutorRouter instance.
- """
- local = LocalPythonExecutor(**(local_kwargs or {}))
- docker = None
- if enable_docker:
- try:
- from .docker_executor import DockerPythonExecutor
- docker = DockerPythonExecutor(**(docker_kwargs or {}))
- except Exception as e:
- logger.warning(f"Failed to initialize Docker executor: {e}")
- return ExecutorRouter(
- local=local,
- docker=docker,
- allow_dangerous=allow_dangerous,
- prefer_docker=prefer_docker,
- )
- __all__ = [
- "SecurityError",
- "ExecutorRouter",
- "create_router",
- "DANGEROUS_PATTERNS",
- "DATA_ANALYSIS_PATTERNS",
- ]
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If itโs helped, pass it on.