Skip to content
Back to search

dagu

Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/dagucloud/dagu with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/dagucloud/dagu'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/dagucloud--dagu/badge.svg)](/listings/dagucloud--dagu)

HTML

<a href="/listings/dagucloud--dagu"><img src="/api/listings/dagucloud--dagu/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

Install

**macOS/Linux:**

curl -fsSL https://raw.githubusercontent.com/dagucloud/dagu/main/scripts/installer.sh | bash

**Homebrew:**

brew install dagu

**npm:**

npm install -g --ignore-scripts=false @dagucloud/dagu

**Windows (PowerShell):**

irm https://raw.githubusercontent.com/dagucloud/dagu/main/scripts/installer.ps1 | iex

**Docker:**

docker run --rm -v ~/.dagu:/var/lib/dagu -p 8080:8080 ghcr.io/dagucloud/dagu:latest dagu start-all

> This command does not expose the host Docker daemon to Dagu. Workflows that > use `container:` or `action: docker.run` need the > [container-step Docker setup](https://docs.dagu.sh/getting-started/installation/docker#run-container-steps-when-dagu-runs-in-docker). > Mounting the Docker socket grants workflows control of the host daemon.

**Kubernetes (Helm):**

helm repo add dagu https://dagucloud.github.io/dagu
helm repo update
helm install dagu dagu/dagu --set persistence.storageClass=<your-rwx-storage-class>

> Replace `<your-rwx-storage-class>` with a StorageClass that supports `ReadWriteMany`. See [charts/dagu/README.md](./charts/dagu/README.md) for chart configuration.

The script installers run a guided wizard that can add Dagu to your PATH, set it up as a background service, and create the initial admin account. Homebrew, npm, Docker, and Helm install without the wizard. See the [Installation documentation](https://docs.dagu.sh/getting-started/installation/) for all options.

Create and run a workflow

Create `hello.yaml`:

steps:
  - id: hello
    run: echo "hello from Dagu"

Run the workflow with:

dagu start hello.yaml

Start the server

dagu start-all --dags .

Visit http://localhost:8080

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,182 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. <a href="https://dagu.sh">
  3. <img src="./assets/images/hero-logo.png" width="720" alt="Dagu: built for teams whose main work is not orchestration">
  4. </a>
  5. <p>
  6. <a href="https://docs.dagu.sh">Docs</a> ·
  7. <a href="https://docs.dagu.sh/getting-started/cli">CLI</a> ·
  8. <a href="https://petstore.swagger.io/?url=https://raw.githubusercontent.com/dagucloud/dagu/main/api/v1/api.yaml">API</a> ·
  9. <a href="https://docs.dagu.sh/writing-workflows/examples">Examples</a> ·
  10. <a href="https://dagu-demo-f5e33d0e.dagu.sh">Live demo</a>
  11. <code>(username/password: demouser)</code> ·
  12. <a href="https://discord.gg/gpahPUjGRk">Discord</a>
  13. </p>
  14. </div>
  15. <h1>Dagu</h1>
  16. Dagu is a local-first workflow engine for operations and internal automation. It is open source and self-hostable: a single binary with a built-in Web UI, no external database or message broker, running on Linux, macOS, and Windows. Define [DAGs](https://en.wikipedia.org/wiki/Directed_acyclic_graph) in a declarative YAML format. It natively supports shell commands, Docker containers, Kubernetes Jobs, remote commands via SSH, and more through Dagu Actions.
  17. Dagu turns existing scripts and runbooks into production workflows with scheduling, retries, human tasks, and run history. It runs where your data and credentials live: on-prem, air-gapped, edge, or cloud, and scales from a single node to a fleet of workers.
  18. **Highlights:**
  19. - Single binary installation.
  20. - Self-contained: no external DBMS or message broker required.
  21. - Runs on Linux, macOS, and Windows.
  22. - Declarative YAML format for defining DAGs.
  23. - Run existing shell commands, Docker containers, Kubernetes Jobs, and remote commands over SSH without modifications.
  24. - Compose reusable Sub-DAGs and run work in parallel with concurrency controls.
  25. - Schedule workflows with cron syntax, timezones, overlap policies, and catch-up windows.
  26. - Keep logs, run history, retries, notifications, and webhook triggers in one place.
  27. - Built-in MCP server for inspecting workflows and runs, maintaining Wiki pages, applying changes, and controlling runs.
  28. - Optional monitoring in Dagu Console: every server's runs and trends in one place, and an email when a workflow fails or a server stops reporting. Opt-in, outbound HTTPS only, metadata only.
  29. ## Quick Look
  30. For a quick look at how workflows are defined, see the examples.
  31. <div align="center">
  32. <a href="https://dagu.sh/dagu-film.mp4">
  33. <img src="./assets/images/dagu-film-poster.jpg" width="720" alt="Watch the Dagu film: your code is the engine, Dagu is the cockpit">
  34. </a>
  35. </div>
  36. | Run Details | Step Logs | Wiki |
  37. |---|---|---|
  38. | ![Run details in dark mode](./assets/images/readme-run-details-dark.png) | ![Workflow logs in dark mode](./assets/images/readme-logs-dark.png) | ![Workflow Wiki in dark mode](./assets/images/readme-wiki-dark.png) |
  39. **Try it live:** [Live Demo](https://dagu-demo-f5e33d0e.dagu.sh) (credentials: `demouser` / `demouser`)
  40. ## Why Dagu?
  41. Orchestration is not your main work. You have scripts and containers that already work. You want a schedule, retries, dependencies, and a place to see logs. The usual options each have a cost:
  42. - **cron** runs commands, but gives you no dependencies, no retries, no history.
  43. - **Airflow** orchestrates, but you operate a platform for it (scheduler, metadata database, workers, a Python environment), and your jobs get rewritten as `@dag`/`@task` framework code.
  44. - **Temporal** gives durable execution, but your business logic moves into its SDK and programming model.
  45. You wanted to schedule some jobs. Now you operate a second system, and the orchestrator lives inside the code it was supposed to serve.
  46. Dagu treats workflow structure as configuration, not code. Order, dependencies, retries, schedules, and human tasks go in one YAML file next to your scripts; the engine that runs them is a single process:
  47. ```sh
  48. Traditional Orchestrator Dagu
  49. ┌────────────────────────┐ ┌──────────────────┐
  50. │ Web Server │ │ │
  51. │ Scheduler │ │ dagu start-all │
  52. │ Worker(s) │ │ │
  53. │ PostgreSQL │ └──────────────────┘
  54. │ Redis / RabbitMQ │ Single binary.
  55. │ Python Runtime │ Self-hosted.
  56. └────────────────────────┘ Adds scheduling, retries, and human tasks around existing automation.
  57. 6+ services to manage
  58. ```
  59. Your scripts never import the orchestrator. Delete the YAML and they run exactly as before. Keep it, and every run gets a dependency graph, retries, per-step logs, history, and a Web UI.
  60. ## Performance
  61. Dagu stores state in local files and reaches production throughput without external services.
  62. - **Throughput:** A single machine can run thousands of workflow runs per day. Actual capacity depends on CPU, memory, disk, and workflow shape.
  63. - **Load control:** Queues, concurrency limits, and resource limits control how many runs execute at once and where they run.
  64. - **Scale out:** Workers spread execution across machines when one node is not enough.
  65. ## Real-World Use Cases
  66. | Use Case | How Dagu Helps |
  67. | --- | --- |
  68. | ETL and data operations | Turn data extraction scripts, SQL queries, dbt commands, and data-processing runbooks into observable pipelines with durable execution. |
  69. | Legacy scripts and scheduled jobs | Turn interdependent scripts into maintainable DAGs with a UI, automatic logging, retries, and notifications instead of opaque cron jobs. |
  70. | Media conversion | Run `ffmpeg` for video transcoding and format conversion. File-backed state allows workers to run heavy conversions in parallel without single-machine bottlenecks or external databases. |
  71. | Infrastructure and server automation | Run any command or script over SSH on remote servers, keeping logs, results, and notifications in one place. |
  72. | GitHub-driven workflows | Trigger workflows from GitHub events to run automation on private infrastructure without exposing servers to the public internet. |
  73. | Container and Kubernetes workflows | Run Docker containers and Kubernetes Jobs as steps in your workflows without building a custom control plane around containers. |
  74. | Customer support automation | Provide self-service workflows that non-engineering teams can run for diagnostics, database queries, and routine operations without escalating to engineering. |
  75. | IoT and edge workflows | Run sensor polling, local ML inference, data preprocessing, backups, offline sync, and health checks close to the data source with Web UI visibility. |
  76. ## Quick Start
  77. ### Install
  78. **macOS/Linux:**
  79. ```sh
  80. curl -fsSL https://raw.githubusercontent.com/dagucloud/dagu/main/scripts/installer.sh | bash
  81. ```
  82. **Homebrew:**
  83. ```sh
  84. brew install dagu
  85. ```
  86. **npm:**
  87. ```sh
  88. npm install -g --ignore-scripts=false @dagucloud/dagu
  89. ```
  90. **Windows (installer):**
  91. Download and run [dagu-windows-amd64-setup.exe](https://github.com/dagucloud/dagu/releases/latest/download/dagu-windows-amd64-setup.exe).
  92. **Windows (PowerShell):**
  93. ```powershell
  94. irm https://raw.githubusercontent.com/dagucloud/dagu/main/scripts/installer.ps1 | iex
  95. ```
  96. **Windows (Scoop):**
  97. ```powershell
  98. scoop bucket add dagu https://github.com/dagucloud/scoop-bucket
  99. scoop install dagu
  100. ```
  101. **Docker:**
  102. ```sh
  103. docker run --rm -v ~/.dagu:/var/lib/dagu -p 8080:8080 ghcr.io/dagucloud/dagu:latest dagu start-all
  104. ```
  105. > This command does not expose the host Docker daemon to Dagu. Workflows that
  106. > use `container:` or `action: docker.run` need the
  107. > [container-step Docker setup](https://docs.dagu.sh/getting-started/installation/docker#run-container-steps-when-dagu-runs-in-docker).
  108. > Mounting the Docker socket grants workflows control of the host daemon.
  109. **Kubernetes (Helm):**
  110. ```sh
  111. helm repo add dagu https://dagucloud.github.io/dagu
  112. helm repo update
  113. helm install dagu dagu/dagu --set persistence.storageClass=<your-rwx-storage-class>
  114. ```
  115. > Replace `<your-rwx-storage-class>` with a StorageClass that supports `ReadWriteMany`. See [charts/dagu/README.md](./charts/dagu/README.md) for chart configuration.
  116. The script installers run a guided wizard that can add Dagu to your PATH, set it up as a background service, and create the initial admin account. The Windows installer can add Dagu to your PATH and run it as a Windows service. Homebrew, npm, Scoop, Docker, and Helm install without a wizard. See the [Installation documentation](https://docs.dagu.sh/getting-started/installation/) for all options.
  117. ### Create and run a workflow
  118. Create `hello.yaml` in an empty folder:
  119. ```yaml
  120. steps:
  121. - id: hello
  122. run: echo "hello from Dagu"
  123. ```
  124. Run the workflow with:
  125. ```sh
  126. dagu start hello.yaml
  127. ```
  128. ### Start the server
  129. From that folder, run:
  130. ```sh
  131. dagu start-all --dags .
  132. ```
  133. Open http://localhost:8080. On first launch, create the admin account unless a script installer already created one.
  134. If an installer set Dagu up as a background service, it is already running (on port 8080 by default), so add workflows from the UI or put `hello.yaml` in its DAGs directory: the one the script installer printed, or `C:\ProgramData\Dagu\dags` for the Windows installer.
  135. ## How You Run Dagu
  136. Dagu runs on one machine, on temporary workers your platform creates for each run, or on workers you keep running. All three are self-hosted, and the same workflow YAML runs on any of them. Any of them can report to Dagu Console, which shows your servers together and emails you when a workflow fails or a server goes silent. See the [Deployment Models guide](https://docs.dagu.sh/overview/deployment-models).
  137. <table>
  138. <tr>
  139. <td width="50%" align="center" valign="top">
  140. <strong>Single server</strong><br>
  141. <img src="./assets/images/deployment-model-single-server.svg" width="100%" alt="Single server: one machine runs dagu start-all; the Dagu server runs your steps and writes DAGs, run history, and logs to local files.">
  142. </td>
  143. <td width="50%" align="center" valign="top">
  144. <strong>Temporary workers</strong><br>
  145. <img src="./assets/images/deployment-model-temporary-workers.svg" width="100%" alt="Temporary workers: your launcher starts a temporary worker that runs dagu start and writes state to a shared volume; the always-on Dagu server reads the same volume, with no network path between the two.">
  146. </td>
  147. </tr>
  148. <tr>
  149. <td width="50%" align="center" valign="top">
  150. <strong>Distributed workers</strong><br>
  151. <img src="./assets/images/deployment-model-distributed-workers.svg" width="100%" alt="Distributed workers: workers on your hosts poll the coordinator in the Dagu server over outbound gRPC, so they need no inbound port; the server writes status, logs, and artifacts to Dagu data.">
  152. </td>
  153. <td width="50%" align="center" valign="top">
  154. <strong>Monitoring with Dagu Console</strong><br>
  155. <img src="./assets/images/deployment-model-console-monitoring.svg" width="100%" alt="Monitoring with Dagu Console: three servers, one per topology, report metadata to Dagu Console over outbound HTTPS; Dagu Console shows them together and emails you when a workflow fails or a server goes silent, while logs, outputs, and secrets stay on each server.">
  156. </td>
  157. </tr>
  158. </table>
  159. | Topology | Execution | Best for |
  160. |----------|-----------|----------|
  161. | **Single server** | `dagu start-all` runs the server, scheduler, and steps in one process on one machine. | Development, single-machine scheduled workloads, edge jobs, and internal automation. |
  162. | **Temporary workers** | Cloud Run Jobs, Kubernetes Jobs, or CI provision a worker per run that invokes the binary and is destroyed when the run ends. The server reads run state from a shared volume. | Ephemeral compute, capacity that falls to zero between jobs, and launchers you already operate. |
  163. | **Distributed workers** | Workers you keep running poll a coordinator over gRPC and are routed work by label. | Docker and private-network steps, warm toolchains, and multiple execution hosts. |
  164. ### Licensing
  165. - **Community self-host:** No license key required. You operate the server, storage, upgrades, networking, and workers. One server can be [monitored in Dagu Console](#monitor-every-server-from-one-place) for free. Start with the [installation guide](https://docs.dagu.sh/getting-started/installation/).
  166. - **Self-host license:** Adds SSO, RBAC, audit logging, incident SaaS integration, and [monitoring in Dagu Console](#monitor-every-server-from-one-place) to Dagu. See [self-host licensing](https://dagu.sh/pricing#self-host).
  167. ## Monitor Every Server from One Place
  168. Connect your Dagu servers to [Dagu Console](https://console.dagu.sh) and see all of them in one place: which servers report, what failed today, open alerts, and run and duration trends for each workflow. Alert emails go out when a workflow fails or recovers, and when a server stops reporting.
  169. <div align="center">
  170. <img src="./assets/images/readme-console-overview.png" width="720" alt="Dagu Console Overview showing two of three servers reporting, failures today, open alerts including a server that stopped reporting, and the 7-day success rate.">
  171. </div>
  172. - **Opt-in:** a server reports nothing until an administrator turns monitoring on in **Settings**, or sets `cloud.report: runs`.
  173. - **Outbound only:** the server reports over HTTPS every 60 seconds, or every 5 minutes on the free tier. No inbound port and no VPN.
  174. - **Metadata only:** run status, timing, and failed step names. Logs, outputs, parameters, and secrets are never sent, and the server shows the exact JSON it last sent.
  175. Workflows keep running on your servers whether or not Dagu Console can be reached. Monitoring is in beta. It is free for one server on Dagu v2.19.2 or later, with lower limits; a Team or Pro plan, or a trial, covers every licensed server. See [Monitoring](https://docs.dagu.sh/overview/console-monitoring).
  176. ## Key Features
  177. - **Observability:** Shared workflows and scheduling with clear visualizations, status tracking, and logs in the Web UI.
  178. - **Language-agnostic:** No framework required. Define workflow steps using shell commands, Docker containers, Kubernetes Jobs, SQL queries, HTTP requests, and any other tool via official and third-party Dagu Actions.
  179. - **Build workflows:** Reuse a step's result when its command and files have not changed. Dagu can also infer dependencies from matching file paths.
  180. - **Reproducibility:** Reproducible runs with pinned tools, plus automatic installation and caching on workers, eliminating the need to manually install dependencies on the server or workers.
  181. - **Human Tasks:** Pause a workflow for acknowledgement or typed operator input, then expose the response to downstream steps.
  182. - **Secret management:** Built-in secret management with secure log masking, preventing credentials from leaking into logs or the Web UI.
  183. - **Self-hosted:** A single binary that runs on Linux, macOS, and Windows. Execution scales out to a fleet of workers.
  184. - **Permission Control:** RBAC and SSO support for team environments, controlling who can view, run, and edit workflows through granular permissions and audit logging.
  185. - **MCP Server:** Authenticated MCP clients can inspect workflows and runs, maintain Wiki pages, apply changes, and control runs.
  186. ## Architecture
  187. One binary carries every role. Which roles you start, and where, is what the [deployment models](#how-you-run-dagu) differ on.
  188. - **Server** serves the Web UI and REST API.
  189. - **Scheduler** owns `schedule:` and drains the queue.
  190. - **Coordinator** is the gRPC endpoint workers poll. It also persists what they report: run status, streamed logs, and artifacts.
  191. - **Worker** polls a coordinator, executes dispatched runs locally, and reports back. Routed by labels.
  192. - `dagu start-all` runs the server, scheduler, and coordinator in one process.
  193. Set `DAGU_HEADLESS=true` to run without the Web UI, which applies to any of the topologies and suits CI or CLI-only environments.
  194. ```sh
  195. Single server:
  196. ┌─────────────────────────────────────────┐
  197. │ dagu start-all │
  198. │ ┌───────────┐ ┌───────────┐ ┌────────┐ │
  199. │ │ HTTP / UI │ │ Scheduler │ │Executor│ │
  200. │ └───────────┘ └───────────┘ └────────┘ │
  201. │ File-based storage (logs, state, queue)│
  202. └─────────────────────────────────────────┘
  203. Distributed workers:
  204. ┌────────────┐ ┌────────────┐
  205. │ Scheduler │ │ HTTP / UI │
  206. │ │ │ │
  207. │ ┌────────┐ │ └─────┬──────┘
  208. │ │ Queue │ │ Dispatch (gRPC) │ Dispatch / GetWorkers
  209. │ │(file) │ │─────────┐ │ (gRPC)
  210. │ └────────┘ │ │ │
  211. └────────────┘ ▼ ▼
  212. ┌─────────────────────────┐
  213. │ Coordinator │
  214. │ ┌───────────────────┐ │
  215. │ │ Dispatch Task │ │
  216. │ │ Store (pending/ │ │
  217. │ │ claimed) │ │
  218. │ └───────────────────┘ │
  219. └────────▲────────────────┘
  220. │
  221. Worker poll / task response
  222. Heartbeat / ReportStatus /
  223. StreamLogs (gRPC)
  224. │
  225. ┌─────────────┴─────────────┐
  226. │ │ │
  227. ┌────┴───┐ ┌────┴───┐ ┌────┴───┐
  228. │Worker 1│ │Worker 2│ │Worker N│ Sandbox execution of DAGs
  229. │ │ │ │ │ │
  230. └────────┘ └────────┘ └────────┘
  231. Temporary workers:
  232. ┌────────────┐ provisions ┌──────────────┐
  233. │ Launcher │───────────────▶│ dagu start │
  234. │ Cloud Run │ │ exits when │
  235. │ K8s Job/CI │ │ the run ends │
  236. └────────────┘ └──────┬───────┘
  237. │ writes
  238. ▼
  239. ┌────────────┐ reads ┌──────────────────┐
  240. │ Dagu server│◀───────────────│ Shared volume │
  241. │ UI / API │ │ dags/state/logs │
  242. └────────────┘ └──────────────────┘
  243. No coordinator, and no network path between the two.
  244. ```
  245. ## Parameter Definition
  246. Workflows can define parameters that render as typed input forms in the Web UI and can be referenced by steps.
  247. ```yaml
  248. params:
  249. - name: customer_id
  250. type: string
  251. description: Customer or account identifier
  252. - name: change_scope
  253. type: string
  254. description: What the repair is allowed to change
  255. enum:
  256. - metadata_only
  257. - permissions
  258. - full_account
  259. default: metadata_only
  260. - name: dry_run
  261. type: boolean
  262. default: true
  263. steps:
  264. - id: extract
  265. run: >-
  266. ./scripts/extract.sh
  267. --customer "${params.customer_id}"
  268. --scope "${params.change_scope}"
  269. --dry-run="${params.dry_run}"
  270. retry_policy:
  271. limit: 3
  272. interval_sec: 30
  273. ```
  274. <div align="center">
  275. <img src="./assets/images/ui-params.png" width="720" alt="Generated parameter input form in the Dagu Web UI">
  276. </div>
  277. ## Workflow Examples
  278. ### Docker step
  279. When Dagu itself runs in Docker, enable
  280. [Docker daemon access](https://docs.dagu.sh/getting-started/installation/docker#run-container-steps-when-dagu-runs-in-docker)
  281. before using container steps.
  282. Pass standard `docker run` options directly in YAML, including the image, pull policy, platform, volume mounts, working directory, and resource limits:
  283. ```yaml
  284. resources:
  285. limits:
  286. cpu: 500m
  287. memory: 512Mi
  288. steps:
  289. - id: report
  290. action: docker.run
  291. with:
  292. image: ghcr.io/acme/reporting:1.4.2
  293. pull: always
  294. platform: linux/amd64
  295. working_dir: /work
  296. volumes:
  297. - ~/orders:/work/data
  298. auto_remove: true
  299. command: python generate_report.py --input /work/data/orders.csv
  300. ```
  301. See the [Docker](https://docs.dagu.sh/step-types/docker) and [DAG Run Resource Limits](https://docs.dagu.sh/writing-workflows/dag-run-resource-limits) documentation for all configuration options.
  302. ### Parallel Sub-DAG execution
  303. The parent invokes the same child DAG for multiple targets and limits concurrent child runs:
  304. ```yaml
  305. steps:
  306. - id: patch
  307. action: dag.run
  308. with:
  309. dag: patch-host
  310. params:
  311. host: ${ITEM}
  312. parallel:
  313. items:
  314. - web-1.internal
  315. - web-2.internal
  316. - db-1.internal
  317. max_concurrent: 2
  318. ---
  319. name: patch-host
  320. params:
  321. - name: host
  322. type: string
  323. ssh:
  324. user: deploy
  325. host: ${params.host}
  326. steps:
  327. - id: apply
  328. run: apt-get update -q && apt-get upgrade -y
  329. ```
  330. See the [Sub-DAGs](https://docs.dagu.sh/writing-workflows/sub-dags) documentation for parameter passing and fan-out options.
  331. ### SSH remote execution
  332. ```yaml
  333. ssh:
  334. user: deploy
  335. host: web-1.internal
  336. key: ~/.ssh/deploy_key
  337. steps:
  338. - id: health
  339. run: curl -f http://localhost:8080/health
  340. retry_policy:
  341. limit: 3
  342. interval_sec: 10
  343. - id: restart
  344. run: systemctl restart myapp
  345. depends: health
  346. ```
  347. See the [SSH](https://docs.dagu.sh/step-types/ssh) documentation for authentication and connection options.
  348. ### Scheduling with overlap control and catch-up
  349. ```yaml
  350. schedule:
  351. - "0 */6 * * *" # Every 6 hours
  352. overlap_policy: skip # Skip if previous run is still active
  353. catchup_window: "5h" # Catch up missed runs when scheduler is down for up to 5 hours
  354. timeout_sec: 3600
  355. handler_on:
  356. failure:
  357. run: notify-team.sh
  358. exit:
  359. run: cleanup.sh
  360. ```
  361. See the [Scheduling](https://docs.dagu.sh/writing-workflows/scheduling) and [Lifecycle Handlers](https://docs.dagu.sh/writing-workflows/lifecycle-handlers) documentation for all options.
  362. ### Retry and error handling
  363. ```yaml
  364. steps:
  365. - name: flaky-api-call
  366. run: curl -f https://api.example.com/data
  367. retry_policy:
  368. limit: 3
  369. interval_sec: 10
  370. continue_on:
  371. failure: true
  372. ```
  373. See the [Durable Execution](https://docs.dagu.sh/writing-workflows/durable-execution) and [Continue On](https://docs.dagu.sh/writing-workflows/continue-on) documentation for retry policies and failure handling.
  374. ## More Workflow Examples
  375. ### Parallel executions
  376. ```yaml
  377. steps:
  378. - id: extract
  379. run: ./extract.sh
  380. - id: transform_a
  381. run: ./transform_a.sh
  382. depends: extract
  383. - id: transform_b
  384. run: ./transform_b.sh
  385. depends: extract
  386. - id: load
  387. run: ./load.sh
  388. depends: [transform_a, transform_b]
  389. ```
  390. ```mermaid
  391. %%{init: {'theme': 'base', 'themeVariables': {'background': '#18181B', 'primaryTextColor': '#fff', 'lineColor': '#888'}}}%%
  392. graph LR
  393. A[extract] --> B[transform_a]
  394. A --> C[transform_b]
  395. B --> D[load]
  396. C --> D
  397. style A fill:#18181B,stroke:#22C55E,stroke-width:1.6px,color:#fff
  398. style B fill:#18181B,stroke:#22C55E,stroke-width:1.6px,color:#fff
  399. style C fill:#18181B,stroke:#22C55E,stroke-width:1.6px,color:#fff
  400. style D fill:#18181B,stroke:#3B82F6,stroke-width:1.6px,color:#fff
  401. ```
  402. See the [Control Flow](https://docs.dagu.sh/writing-workflows/control-flow) documentation for dependencies, conditions, and repetition.
  403. ### Reuse unchanged results
  404. Save this as `workflow.yaml`:
  405. ```yaml
  406. type: build
  407. working_dir: .
  408. steps:
  409. - id: uppercase
  410. inputs:
  411. - name: source
  412. path: source.txt
  413. outputs:
  414. - name: result
  415. path: uppercase.txt
  416. run: |
  417. #!/bin/sh
  418. tr '[:lower:]' '[:upper:]' < "${inputs.source}" > "${outputs.result}"
  419. ```
  420. Run it:
  421. ```sh
  422. printf 'alpha\n' > source.txt
  423. dagu start workflow.yaml
  424. ```
  425. Run `dagu start workflow.yaml` again and Dagu reuses `uppercase.txt`. Change `source.txt` and the step runs again. `${outputs.result}` is a temporary path that Dagu publishes as `uppercase.txt` after the command succeeds.
  426. Build workflows currently run locally. See [Build Workflows](https://docs.dagu.sh/writing-workflows/incremental-workflows) for dependency inference and reuse rules.
  427. ### External tools with pinning and caching
  428. ```yaml
  429. tools:
  430. - jqlang/jq@jq-1.7.1
  431. steps:
  432. - id: inspect
  433. run: jq --version
  434. - id: summarize
  435. action: python-script@v1
  436. with:
  437. input:
  438. rows: [42, 8]
  439. script: |
  440. return {"total": sum(input["rows"])}
  441. ```
  442. Dagu installs declared portable CLIs before the DAG run, exposes them on `PATH` for host command steps, and caches them on each worker. Tool provisioning uses [aqua](https://aquaproj.github.io/) as the default provider; the standard registry resolves to the latest aqua-registry release automatically. Pin a specific artifact with `package@version#sha256:<hex>` when the release tag alone is not a strong enough guarantee. See the [Tools documentation](https://docs.dagu.sh/writing-workflows/tools) and [Dagu Actions](https://docs.dagu.sh/dagu-actions/) for more details.
  443. ### Third-party Dagu Actions
  444. ```yaml
  445. params:
  446. - BUILD_ID
  447. steps:
  448. - id: notify
  449. action: acme/dagu-action-notify@v1.2.0
  450. with:
  451. text: "Build ${params.BUILD_ID} finished"
  452. - id: audit
  453. depends: notify
  454. run: 'echo "Notification result: ${steps.notify.outputs.messageId}"'
  455. ```
  456. A third-party Dagu Action package contains a DAG, manifest, schemas, and helper files behind an `action:` reference. See the [Dagu Actions](https://docs.dagu.sh/dagu-actions/) and [Third-Party Actions](https://docs.dagu.sh/dagu-actions/third-party) documentation for details.
  457. ### Kubernetes Pod execution
  458. ```yaml
  459. steps:
  460. - name: batch-job
  461. action: kubernetes.run
  462. with:
  463. namespace: production
  464. image: my-registry/batch-processor:latest
  465. resources:
  466. requests:
  467. cpu: "2"
  468. memory: "4Gi"
  469. command: ./process.sh
  470. ```
  471. See the [Kubernetes](https://docs.dagu.sh/step-types/kubernetes) documentation for Job configuration options.
  472. For more examples, see the [Examples documentation](https://docs.dagu.sh/writing-workflows/examples).
  473. ## MCP
  474. Dagu includes a built-in MCP server at `http://localhost:8080/mcp`. MCP clients can inspect workflows and run state, maintain Dagu's built-in Wiki pages, preview and apply DAG or Wiki page changes, and control runs through the same authenticated server boundary as the REST API.
  475. See the [MCP overview](https://docs.dagu.sh/mcp/) and [quickstart](https://docs.dagu.sh/mcp/quickstart).
  476. ## Built-in Actions
  477. Dagu includes built-in actions that run within the Dagu process or on the selected worker. Local shell commands use the `run:` field; structured work uses `action:`.
  478. | Action | Purpose |
  479. |----------|---------|
  480. | `run:` field | Local shell commands and scripts (bash, sh, PowerShell, custom shells) |
  481. | `exec` | Direct process execution without shell parsing |
  482. | `noop` | Output-only or approval-only placeholder step |
  483. | `log.write` | Write structured log messages |
  484. | `docker.run` / `container.run` | Run containers with registry auth, volume mounts, and resource limits |
  485. | `kubernetes.run` / `k8s.run` | Execute Kubernetes Jobs with namespace, image, and resource settings |
  486. | `ssh.run` | Remote command execution over SSH |
  487. | `sftp.upload` / `sftp.download` | File transfer over SFTP |
  488. | `http.request` | HTTP requests with headers, auth, and request bodies |
  489. | `mcp.call` | Call a tool on a remote MCP server over Streamable HTTP |
  490. | `chat.completion` | Run an LLM chat completion step |
  491. | `harness.run` | Run external coding-agent CLIs such as Claude Code, Codex, Gemini CLI, Cursor, and DeepSeek Harness, or [managed OpenCode sessions](./skills/dagu/references/harnesses.md#managed-opencode-sessions) |
  492. | `browser.extract` / `browser.run` | Automate websites in a local Chrome with natural-language actions, structured extraction, and human input |
  493. | `computer.extract` / `computer.run` | Automate desktop applications on macOS and Windows workers with natural-language tasks, screen extraction, and human input |
  494. | `postgres.query` / `postgres.import` | PostgreSQL queries and imports |
  495. | `sqlite.query` / `sqlite.import` | SQLite queries and imports |
  496. | `clickhouse.query` | ClickHouse queries and statements over the HTTP interface |
  497. | `redis.<operation>` | Redis commands, pipelines, and Lua scripts |
  498. | `s3.upload` / `s3.download` / `s3.list` / `s3.delete` | Upload, download, list, and delete S3 objects |
  499. | `file.stat` / `file.read` / `file.write` / `file.copy` / `file.move` / `file.delete` / `file.mkdir` / `file.list` | Local file operations without shell commands |
  500. | `artifact.write` / `artifact.read` / `artifact.list` | Write, read, and list DAG-run artifacts |
  501. | `state.get` / `state.set` / `state.delete` / `state.list` / `state.diff` | Persistent JSON state across DAG runs |
  502. | `data.convert` / `data.pick` | Convert and select structured data |
  503. | `jq.filter` | JSON transformation using jq expressions |
  504. | `js.run` | JavaScript transforms in an embedded sandbox, no Node.js needed |
  505. | `archive.create` / `archive.extract` / `archive.list` | Create, extract, and list zip/tar archives |
  506. | `wait.duration` / `wait.until` / `wait.file` / `wait.http` | Wait for time, file state, or HTTP readiness |
  507. | `human.task` | Wait for acknowledgement or typed operator input before downstream steps continue |
  508. | `mail.send` | Send email via SMTP |
  509. | `mail.search` | Find email in an IMAP or Gmail mailbox |
  510. | `mail.organize` | Mark, move, archive, or trash email |
  511. | `xlsx.read` / `xlsx.info` / `xlsx.list_sheets` | Read typed rows and describe `.xlsx` workbooks without a spreadsheet application |
  512. | `xlsx.write` / `xlsx.append` / `xlsx.update_rows` | Write reports, append rows, and write per-row results back into a workbook |
  513. | `xlsx.validate` / `xlsx.write_cells` / `xlsx.sheet` / `xlsx.convert` | Check rows against rules, fill template cells and formulas, manage sheets, and export a sheet to CSV, JSON, or JSONL |
  514. | `xlsx.extract` | Read fields out of a form-like sheet: a model names the cells, the engine reads their typed values, and a layout seen before with the same instruction and fields is read from a cache without a model call |
  515. | `template.render` | Text generation with template rendering |
  516. | `router.route` | Conditional step routing based on values and patterns |
  517. | `dag.run` | Invoke another DAG as a sub-workflow with params and dependencies |
  518. | `dag.enqueue` | Queue another DAG asynchronously and continue after enqueue |
  519. | `git.checkout` | Clone or update Git repositories |
  520. | `outputs.write` | Publish DAG or Dagu Action outputs for callers |
  521. ## Custom Actions
  522. Custom Actions are inline reusable wrappers defined with the top-level `actions` field. They expand to built-in actions during DAG load, so you can wrap a common shell, HTTP, SQL, or other pattern behind a typed interface with validated input.
  523. ```yaml
  524. actions:
  525. webhook.send:
  526. input_schema:
  527. type: object
  528. additionalProperties: false
  529. required: [url, text]
  530. properties:
  531. url:
  532. type: string
  533. text:
  534. type: string
  535. template:
  536. action: http.request
  537. with:
  538. method: POST
  539. url: '{{ .input.url }}'
  540. headers:
  541. Content-Type: application/json
  542. body: |
  543. {"text": {{ json .input.text }}}
  544. steps:
  545. - action: webhook.send
  546. with:
  547. url: https://hooks.example.com/ops
  548. text: deploy complete
  549. ```
  550. See [Custom Actions](https://docs.dagu.sh/dagu-actions/custom) and the [YAML Specification](https://docs.dagu.sh/writing-workflows/yaml-specification) for the exact `actions`, `action`, and `run` field behavior.
  551. ## Official Dagu Actions
  552. Dagu Actions are official action packages maintained in the `dagucloud` GitHub organization. They use the same action package runtime as third-party action packages, but callers use the short form `action: name@version`.
  553. | Dagu Action | Purpose |
  554. |-------------|---------|
  555. | `node-script@v1` | Run small JavaScript transforms or glue code with action-owned Node.js |
  556. | `python-script@v1` | Run small Python transforms or glue code with action-owned Python and optional requirements |
  557. | `dbt@v1` | Run dbt Core commands with action-owned Python and adapter requirements |
  558. | `duckdb@v1` | Run DuckDB SQL through the DuckDB CLI without adding DuckDB to the core binary |
  559. | `ffmpeg@v1` | Run FFmpeg conversion, transcoding, probing, and stream-processing tasks |
  560. | `github-cli@v1` | Run GitHub issue, pull request, release, repository, and API automation through `gh` |
  561. | `rclone@v1` | Run portable copy, sync, check, list, and storage-management workflows through rclone |
  562. Versions are required. Pin production workflows to a version tag or commit SHA. See [Official Dagu Actions](https://docs.dagu.sh/dagu-actions/official) for the current Dagu Action list and exact input/output contracts.
  563. For non-official packages, use [Third-Party Actions](https://docs.dagu.sh/dagu-actions/third-party) such as `action: owner/repo@version`. They contain a `dagu-action.yaml` manifest and a DAG entrypoint, run as sub-DAGs, and are transferred to distributed workers as workspace bundles after the reference is resolved. See the documentation for package layout and reference formats.
  564. ## Security and Access Control
  565. ### Authentication
  566. Dagu supports three top-level authentication modes, configured via `DAGU_AUTH_MODE`:
  567. - **`none`**: No authentication
  568. - **`basic`**: HTTP Basic authentication
  569. - **`builtin`**: JWT-based authentication with user management, API keys, per-DAG webhook tokens, and optional OIDC/SSO integration
  570. ### Role-Based Access Control
  571. When using `builtin` auth, five roles control access:
  572. | Role | Capabilities |
  573. |------|-------------|
  574. | `admin` | Full access including user management |
  575. | `manager` | Create, edit, delete, run, stop DAGs; view audit logs |
  576. | `developer` | Create, edit, delete, run, stop DAGs |
  577. | `operator` | Run and stop DAGs only (no editing) |
  578. | `viewer` | Read-only access |
  579. API keys can be created with independent role assignments. Audit logging tracks all actions.
  580. ### TLS and Secrets
  581. - TLS for the HTTP server (`DAGU_CERT_FILE`, `DAGU_KEY_FILE`)
  582. - Mutual TLS for gRPC coordinator/worker communication (`DAGU_PEER_CERT_FILE`, `DAGU_PEER_KEY_FILE`, `DAGU_PEER_CLIENT_CA_FILE`)
  583. - Secret management with environment variables, files, Kubernetes Secrets, [HashiCorp Vault](https://www.vaultproject.io/), and cloud-provider secret stores
  584. ### Production Hardening
  585. For self-hosted production deployments, treat network exposure and execution boundaries as the primary controls:
  586. - Prefer `auth.mode: builtin` for any shared or network-exposed instance. Use `basic` only for simple private setups, and avoid `none` outside isolated local development.
  587. - Keep `metrics: private` unless the metrics endpoint is reachable only on a trusted private network.
  588. - Bind Dagu to loopback or a private interface when possible. If you must use `0.0.0.0`, place it behind a trusted reverse proxy, TLS, and network-level access controls.
  589. - Leave `terminal.enabled: false` unless the instance is admin-only and tightly scoped.
  590. - In distributed deployments, set `peer.insecure=false` and configure peer TLS when coordinator and workers communicate across host or network boundaries.
  591. - Treat Docker socket mounts, root containers, and host-level executors as privileged access to the underlying machine.
  592. See [Server Configuration](https://docs.dagu.sh/server-admin/configuration), [Docker deployment](https://docs.dagu.sh/server-admin/deployment/docker), and [Distributed execution](https://docs.dagu.sh/server-admin/distributed/) for operator-focused guidance.
  593. ## Observability
  594. ### Prometheus Metrics
  595. Dagu exposes Prometheus-compatible metrics:
  596. - `dagu_info`: Build information (version, Go version)
  597. - `dagu_uptime_seconds`: Server uptime
  598. - `dagu_dag_runs_total`: Total DAG runs by status
  599. - `dagu_dag_runs_total_by_dag`: Per-DAG run counts
  600. - `dagu_dag_run_duration_seconds`: Histogram of run durations
  601. - `dagu_dag_runs_currently_running`: Active DAG runs
  602. - `dagu_dag_runs_queued_total`: Queued runs
  603. - `dagu_workers_registered`: Registered distributed workers
  604. - `dagu_worker_info`: Worker heartbeat labels as key/value metadata
  605. - `dagu_worker_heartbeat_timestamp_seconds`: Last worker heartbeat timestamp
  606. - `dagu_worker_health_status`: Worker health by heartbeat freshness
  607. - `dagu_worker_pollers`: Worker poller capacity by state
  608. - `dagu_worker_running_tasks`: Running tasks per worker
  609. - `dagu_worker_oldest_running_task_age_seconds`: Age of the oldest running task per worker
  610. ### Dagu Console Monitoring
  611. Connected servers can report their health and run status to Dagu Console for one view across servers, trends, and alert emails. See [Monitor Every Server from One Place](#monitor-every-server-from-one-place).
  612. ### Structured Logging
  613. JSON or text format logging (`DAGU_LOG_FORMAT`). Logs are stored per-run with separate stdout/stderr capture per step.
  614. ### Notifications
  615. - Email notifications on DAG success, failure, or wait status via SMTP
  616. - Per-DAG webhook endpoints with token authentication
  617. - Delivery requires the event store, notification store, and writable notification state store
  618. SMTP inherited from global or workspace `base.yaml` is omitted from new run
  619. snapshots (`dag.json`). Retries, restarts, and queued runs use the current base
  620. SMTP configuration. Removing it disables email notifications unless the original
  621. DAG defines SMTP. Other base settings and the original DAG YAML remain captured.
  622. Runs without captured base configuration use the current base configuration in
  623. full; an explicitly empty captured configuration only reloads SMTP.
  624. SMTP written directly in DAG YAML, values stored in `env`, distributed task
  625. payloads, and workspace bundles are still retained. Existing history files and
  626. backups are not rewritten. Upgrade processes that read run snapshots (CLI, API,
  627. and scheduler) together so retries can reload base SMTP.
  628. Scoped notification routing is broken in v2.11.0-v2.11.2. Use v2.11.3 or later.
  629. ## Artifacts
  630. ![Artifact browser in dark mode](./assets/images/readme-artifacts-dark.png)
  631. Dagu runs can write arbitrary files under `${context.paths.artifacts_dir}` in value-resolved fields, with `DAG_RUN_ARTIFACTS_DIR` also exposed to step processes. Dagu stores those files per run as Artifacts. In the Web UI, operators can browse the file tree, preview Markdown, text, and image files inline, and download any artifact when they need the raw file.
  632. This is useful for generated reports, screenshots, charts, exported JSON or CSV files, and other outputs that do not fit simple key/value outputs.
  633. See the [Artifacts documentation](https://docs.dagu.sh/writing-workflows/artifacts) and the [Web UI guide](https://docs.dagu.sh/overview/web-ui) for the full artifact browser workflow and screenshots.
  634. ## Scheduling and Reliability
  635. - **Cron scheduling** with timezone support and multiple schedule entries per DAG
  636. - **Overlap policies**: `skip` (default: skip if previous run is still active), `all` (queue all), `latest` (keep only the most recent)
  637. - **Catch-up scheduling**: Automatically runs missed intervals when the scheduler was down
  638. - **Zombie detection**: Identifies and handles stalled DAG runs (configurable interval, default 45s)
  639. - **Retry policies**: Per-step retry with configurable limits, intervals, and exit code filtering
  640. - **Human tasks**: Pause root DAG runs for acknowledgement or schema-validated operator input, locally or on distributed workers, then expose form values to downstream steps
  641. - **Lifecycle hooks**: `onInit`, `onSuccess`, `onFailure`, `onAbort`, `onExit`, `onWait`
  642. - **Preconditions**: Gate DAG or step execution on shell command results
  643. - **High availability**: Scheduler lock with stale detection for failover
  644. ## Distributed Execution
  645. Operational detail for the [distributed workers](#how-you-run-dagu) topology:
  646. - **Coordinator**: gRPC server that manages task distribution, worker registry, and health monitoring
  647. - **Workers**: Poll the coordinator outbound, execute DAGs locally, and report status, logs, and artifacts back. No inbound port required
  648. - **Worker labels**: Route DAGs to specific workers based on labels (e.g., `gpu=true`, `region=us-east-1`)
  649. - **Health checks**: HTTP health endpoints on coordinator and workers for load balancer integration
  650. - **Queue system**: File-based persistent queue with configurable concurrency limits
  651. ```sh
  652. # Start coordinator
  653. dagu coordinator --coordinator.host=0.0.0.0 --coordinator.advertise=coordinator.internal
  654. # Start workers (on separate machines)
  655. DAGU_WORKER_COORDINATORS=coordinator.internal:50055 \
  656. DAGU_WORKER_LABELS=gpu=true,memory=64G dagu worker
  657. ```
  658. See the [distributed execution documentation](https://docs.dagu.sh/server-admin/distributed/) for setup details.
  659. ## CLI Reference
  660. | Command | Description |
  661. |---------|-------------|
  662. | `dagu start <dag>` | Execute a DAG |
  663. | `dagu start-all` | Start HTTP server + scheduler + coordinator |
  664. | `dagu server` | Start HTTP server only |
  665. | `dagu scheduler` | Start scheduler only |
  666. | `dagu coordinator` | Start coordinator (distributed mode) |
  667. | `dagu worker` | Start worker (distributed mode) |
  668. | `dagu stop <dag>` | Stop a running DAG |
  669. | `dagu restart <dag>` | Restart a DAG |
  670. | `dagu retry --run-id=<run-id> <dag>` | Retry a failed run |
  671. | `dagu human-task complete --run-id=<run-id> --step=<id> <dag>` | Complete a waiting human task |
  672. | `dagu dry <dag>` | Dry run: show what would execute |
  673. | `dagu status <dag>` | Show DAG run status |
  674. | `dagu history <dag>` | Show execution history |
  675. | `dagu validate <dag>` | Validate DAG YAML |
  676. | `dagu enqueue <dag>` | Add DAG to the execution queue |
  677. | `dagu dequeue <queue-name> [--dag-run=<dag>:<run-id>]` | Remove a DAG-run from the queue |
  678. | `dagu cleanup <dag>` | Clean up old run data |
  679. | `dagu version` | Show version |
  680. The table lists the most common commands. The binary ships 36 in total, including `exec`, `ls`, `ps`, `rm`, `prune-artifacts`, `sync`, `schema`, `example`, `config`, `profile`, `context`, `license`, `upgrade`, and `completion`; run `dagu --help` or see the [CLI reference](https://docs.dagu.sh/getting-started/cli) for all of them.
  681. ## Environment Variables
  682. **Precedence:** Command-line flags > Environment variables > Configuration file (`~/.config/dagu/config.yaml`)
  683. ### Server
  684. | Variable | Default | Description |
  685. |----------|---------|-------------|
  686. | `DAGU_HOST` | `127.0.0.1` | Bind address |
  687. | `DAGU_PORT` | `8080` | HTTP port |
  688. | `DAGU_BASE_PATH` | - | Base path for reverse proxy |
  689. | `DAGU_HEADLESS` | `false` | Run without web UI |
  690. | `DAGU_TZ` | - | Timezone (e.g., `Asia/Tokyo`) |
  691. | `DAGU_LOG_FORMAT` | `text` | `text` or `json` |
  692. | `DAGU_CERT_FILE` | - | TLS certificate |
  693. | `DAGU_KEY_FILE` | - | TLS private key |
  694. | `DAGU_CORS_ALLOWED_ORIGINS` | - | Comma-separated list of allowed CORS origins (e.g. `https://app.example.com`). When unset, cross-origin browser access is disabled. Exact origins enable credentials. An explicit `*` allows every origin without credentials and emits a security warning. |
  695. | `DAGU_IP_ACCESS_ALLOWED_IPS` | - | Comma-separated IPv4/IPv6 addresses and CIDR ranges allowed to access the HTTP server. Empty disables filtering. |
  696. | `DAGU_IP_ACCESS_TRUSTED_PROXIES` | - | Comma-separated proxy addresses and CIDR ranges permitted to supply forwarded client IP headers. |
  697. | `DAGU_PUBLIC_URL` | - | External Web UI URL used in generated links, including notification and incident DAG-run links |
  698. | `DAGU_SERVER_METRICS` | `private` | Metrics endpoint access: `private` or `public` |
  699. | `DAGU_TERMINAL_ENABLED` | `false` | Enable the web-based terminal |
  700. | `DAGU_DEFAULT_SHELL` | `$SHELL`, then `sh` | Default shell for command steps |
  701. | `DAGU_ENV_PASSTHROUGH_PREFIXES` | - | Comma-separated env var prefixes forwarded to step execution |
  702. | `DAGU_DEBUG` | - | Enable debug mode |
  703. The equivalent YAML protects every HTTP route, including health, metrics,
  704. webhooks, SSE, terminal, and MCP:
  705. ```yaml
  706. ip_access:
  707. allowed_ips:
  708. - 203.0.113.10
  709. - 10.0.0.0/8
  710. trusted_proxies:
  711. - 127.0.0.1
  712. - 10.42.0.0/16
  713. ```
  714. Forwarded addresses are used only when the direct peer matches
  715. `trusted_proxies`. The proxy must remove client-supplied forwarding headers and
  716. set or append the verified client address. Keep trusted proxy ranges narrow.
  717. Proxy authentication (`auth.proxy`) accepts identity headers only from
  718. `trusted_proxies`, or only from loopback peers when the list is empty.
  719. ### Paths
  720. | Variable | Default | Description |
  721. |----------|---------|-------------|
  722. | `DAGU_HOME` | - | Overrides all path defaults |
  723. | `DAGU_DAGS_DIR` | `~/.config/dagu/dags` | DAG definitions directory |
  724. | `DAGU_DAG_DISCOVERY_RECURSIVE` | `false` | Discover DAGs in subdirectories |
  725. | `DAGU_DAG_DISCOVERY_SYMLINKS` | `false` | Include recursive file symlinks and allow external targets |
  726. | `DAGU_DAG_DISCOVERY_SKIP_DIRECTORIES` | - | Comma-separated directories directly under the DAGs directory that recursive discovery skips |
  727. | `DAGU_LOG_DIR` | `~/.local/share/dagu/logs` | Log files |
  728. | `DAGU_DATA_DIR` | `~/.local/share/dagu/data` | Application state |
  729. | `DAGU_SUSPEND_FLAGS_DIR` | `{DAGU_DATA_DIR}/suspend` | DAG suspension flags |
  730. | `DAGU_TOOLS_DIR` | `{DAGU_DATA_DIR}/tools` | Managed DAG tool cache |
  731. | `DAGU_DAG_STATE_DIR` | `{DAGU_DATA_DIR}/dag-state` | Persistent DAG state files |
  732. | `DAGU_DAG_RUN_WORK_DIR` | `{DAGU_DATA_DIR}/dag-run-work` | Per-run working directories |
  733. | `DAGU_BASE_CONFIG` | - | Shared base configuration applied to all DAGs |
  734. Suspend flags default to `paths.data_dir/suspend`. Explicit
  735. `paths.suspend_flags_dir` settings, including environment variables and legacy
  736. configuration keys, retain their configured location and disable legacy fallback
  737. and migration. A path outside `data_dir` produces a warning because all server
  738. and scheduler processes must share the same suspension state.
  739. With the default path, flags in the previous default directory remain readable.
  740. After acquiring leadership, the scheduler copies them into `data_dir/suspend`
  741. before processing runs. Startup preserves the original flags. Suspending a DAG
  742. writes its new flag before removing its legacy flag; resuming removes both.
  743. Migration and state changes are serialized across processes.
  744. During upgrade, make the same legacy flags available to the server and scheduler;
  745. reconcile any private per-host flag directories first. Upgrade all processes
  746. together before changing suspension state. Older processes writing legacy flags
  747. are not supported alongside upgraded processes. Until scheduler migration has
  748. completed, include the legacy directory in backups. When rolling back, explicitly
  749. configure the older version to use the new shared suspend directory.
  750. Set the per-run work root in `config.yaml`, or use the corresponding environment variable above:
  751. ```yaml
  752. paths:
  753. dag_run_work_dir: /mnt/dagu/dag-run-work
  754. ```
  755. `DAGU_DAG_RUN_WORK_DIR` configures this root for Dagu processes. Workflow code should use the runtime `DAG_RUN_WORK_DIR` variable for its assigned per-run directory instead of constructing paths under DAG-run history.
  756. Processes sharing DAG runs must use the same work root.
  757. Backups that select individual data subdirectories must include both `paths.dag_runs_dir` and `paths.dag_run_work_dir`. A backup of the complete `paths.data_dir` includes both default locations. The Helm chart's default `/data/dag-run-work` path uses its existing `/data` volume and does not need an additional volume.
  758. When upgrading a deployment whose processes share a durable work root, do not let old and new Dagu versions execute the same run concurrently: drain or stop the processes, upgrade them together, and then resume execution. Mixed-version processes can otherwise choose the old nested directory and the new separate directory for one run.
  759. Recursive discovery can also be enabled in `config.yaml`:
  760. ```yaml
  761. dag_discovery:
  762. recursive: true
  763. ```
  764. It scans `paths.dags_dir`, excluding `workspaces/`, dot-directories, and symlinks. File stems and effective DAG names must each be unique, using case-sensitive comparison; conflicting files are excluded until the conflict is resolved. `paths.alt_dags_dir` remains lookup-only.
  765. Set `dag_discovery.symlinks: true` (or
  766. `DAGU_DAG_DISCOVERY_SYMLINKS=true`) to include YAML file symlinks in recursive
  767. discovery and to allow YAML file symlinks whose targets are outside
  768. `paths.dags_dir`. Symlinked directories are never traversed. External targets
  769. can be viewed, scheduled, and run, but cannot be updated, deleted, or renamed
  770. through Dagu. Without the opt-in, top-level YAML file symlinks whose targets
  771. remain inside `paths.dags_dir` continue to work. A symlink configured as
  772. `paths.dags_dir` itself is also supported.
  773. Set `dag_discovery.skip_directories` to keep directories directly under
  774. `paths.dags_dir` out of recursive discovery, such as a tree of child DAGs or
  775. build output. Their DAGs are not listed, scheduled, or watched, but a parent
  776. DAG can still run one by path. Entries are case-sensitive directory names, so
  777. `jobs` skips `jobs/` but not `team/jobs/`.
  778. ```yaml
  779. dag_discovery:
  780. recursive: true
  781. skip_directories: [jobs, build]
  782. ```
  783. ### Authentication
  784. | Variable | Default | Description |
  785. |----------|---------|-------------|
  786. | `DAGU_AUTH_MODE` | `builtin` | `none`, `basic`, or `builtin` |
  787. | `DAGU_AUTH_BASIC_USERNAME` | - | Basic auth username |
  788. | `DAGU_AUTH_BASIC_PASSWORD` | - | Basic auth password |
  789. | `DAGU_AUTH_TOKEN_SECRET` | (auto) | JWT signing secret |
  790. | `DAGU_AUTH_TOKEN_TTL` | `24h` | JWT token lifetime (maximum: `8760h` / 365 days) |
  791. | `DAGU_AUTH_BUILTIN_INITIAL_ADMIN_USERNAME` | - | Auto-provision the first admin on startup (requires the password variable) |
  792. | `DAGU_AUTH_BUILTIN_INITIAL_ADMIN_PASSWORD` | - | Password for the auto-provisioned admin (minimum 8 characters) |
  793. | `DAGU_LICENSE_KEY` | - | License key for licensed self-host features |
  794. OIDC variables: `DAGU_AUTH_OIDC_CLIENT_ID`, `DAGU_AUTH_OIDC_CLIENT_SECRET`, `DAGU_AUTH_OIDC_ISSUER`, `DAGU_AUTH_OIDC_SCOPES`, `DAGU_AUTH_OIDC_WHITELIST`, `DAGU_AUTH_OIDC_AUTO_SIGNUP`, `DAGU_AUTH_OIDC_DEFAULT_ROLE`, `DAGU_AUTH_OIDC_ALLOWED_DOMAINS`.
  795. The auth, remote node, Git sync and Vault secrets can also be read from files, such as mounted Kubernetes or Docker secrets. Each of those secret keys has a `_file` key, and each of those secret variables a `_FILE` variable, that names the file: `DAGU_AUTH_BASIC_PASSWORD_FILE`, `DAGU_AUTH_TOKEN_SECRET_FILE`, `DAGU_AUTH_BUILTIN_INITIAL_ADMIN_PASSWORD_FILE`, `DAGU_AUTH_OIDC_CLIENT_SECRET_FILE`, `DAGU_GITSYNC_AUTH_TOKEN_FILE`, `DAGU_GITSYNC_AUTH_SSH_PASSPHRASE_FILE`, and `DAGU_SECRETS_VAULT_TOKEN_FILE`; `remote_nodes` entries take `basic_auth_password_file` and `auth_token_file`. Dagu reads the file at startup, not when it uses the secret, and removes the trailing line ends. Setting a non-empty secret and its `_file` form together, or naming a file that cannot be read or is empty, stops the startup. `tunnel.tailscale.auth_key` and `license.key` have no `_file` form.
  796. ### Scheduler
  797. | Variable | Default | Description |
  798. |----------|---------|-------------|
  799. | `DAGU_SCHEDULER_PORT` | `8090` | Health check port |
  800. | `DAGU_SCHEDULER_ZOMBIE_DETECTION_INTERVAL` | `45s` | Zombie run detection interval (`0` to disable) |
  801. | `DAGU_SCHEDULER_LOCK_STALE_THRESHOLD` | `30s` | HA lock stale threshold |
  802. | `DAGU_QUEUE_ENABLED` | `true` | Enable queue system |
  803. | `DAGU_SIGNAL_PROPAGATION` | `false` | Forward shutdown signals (`SIGINT`/`SIGTERM`) received by the server or scheduler to the process groups of running DAG-run subprocesses it launched |
  804. ### Coordinator / Worker
  805. | Variable | Default | Description |
  806. |----------|---------|-------------|
  807. | `DAGU_COORDINATOR_HOST` | `127.0.0.1` | Coordinator bind address |
  808. | `DAGU_COORDINATOR_PORT` | `50055` | Coordinator gRPC port |
  809. | `DAGU_COORDINATOR_HEALTH_PORT` | `8091` | Coordinator health check port |
  810. | `DAGU_WORKER_ID` | - | Worker instance ID |
  811. | `DAGU_WORKER_MAX_ACTIVE_RUNS` | `100` | Max concurrent runs per worker |
  812. | `DAGU_WORKER_HEALTH_PORT` | `8092` | Worker health check port |
  813. | `DAGU_WORKER_SHUTDOWN_TIMEOUT` | `60s` | How long a worker stopping on `SIGINT`/`SIGTERM` drains running DAG runs before it cancels them (`0` waits without a bound). Set the supervisor's stop timeout at least `15s` above it: systemd `TimeoutStopSec` defaults to `90s`, `docker stop` to `10s`, Kubernetes `terminationGracePeriodSeconds` to `30s` |
  814. | `DAGU_WORKER_LABELS` | - | Worker labels (`key=value,key=value`); `os` and `arch` are built in and cannot be overridden |
  815. | `DAGU_COORDINATOR_ADVERTISE` | auto-detected hostname | Address advertised in the service registry |
  816. | `DAGU_WORKER_COORDINATORS` | - | Coordinator addresses required by `dagu worker` (`host:port,...`) |
  817. ### Peer TLS (gRPC)
  818. | Variable | Default | Description |
  819. |----------|---------|-------------|
  820. | `DAGU_PEER_CERT_FILE` | - | Peer TLS certificate |
  821. | `DAGU_PEER_KEY_FILE` | - | Peer TLS private key |
  822. | `DAGU_PEER_CLIENT_CA_FILE` | - | CA for client verification |
  823. | `DAGU_PEER_INSECURE` | `true` | Use h2c instead of TLS |
  824. | `DAGU_PEER_SKIP_TLS_VERIFY` | - | Skip TLS certificate verification |
  825. ### Git Sync
  826. Git Sync copies Git-tracked supporting files into the DAGs directory. Workflow and Wiki files retain their existing handling, and each `calendars/<name>.yaml` file syncs with `paths.calendars_dir` as a calendar, including one synced as a DAG before (deeper paths and other extensions keep their handling); supporting file IDs include their extension and preserve the executable bit. On Windows, Git Sync retains the repository mode when publishing because local execute-bit changes cannot be detected.
  827. | Variable | Default | Description |
  828. |----------|---------|-------------|
  829. | `DAGU_GITSYNC_ENABLED` | `false` | Enable Git sync |
  830. | `DAGU_GITSYNC_REPOSITORY` | - | Repository URL |
  831. | `DAGU_GITSYNC_BRANCH` | `main` | Branch to sync |
  832. | `DAGU_GITSYNC_AUTH_TYPE` | `token` | `token` or `ssh` |
  833. | `DAGU_GITSYNC_AUTH_TOKEN` | - | Personal access token for HTTPS auth |
  834. | `DAGU_GITSYNC_AUTH_SSH_KEY_PATH` | - | Path to the SSH private key |
  835. | `DAGU_GITSYNC_AUTOSYNC_ENABLED` | `false` | Enable periodic auto-pull |
  836. | `DAGU_GITSYNC_AUTOSYNC_INTERVAL` | `300` | Sync interval in seconds |
  837. These tables cover the variables most deployments touch. The full reference lists about 180 `DAGU_*` variables, including SSE, tunnel, UI, monitoring, audit, and secret-provider settings: see the [configuration reference](https://docs.dagu.sh/server-admin/reference).
  838. ## Embedded Go API (Experimental)
  839. Go applications can import Dagu and start DAG runs from the host process:
  840. ```go
  841. import "github.com/dagucloud/dagu/v2"
  842. ```
  843. ```go
  844. engine, err := dagu.New(ctx, dagu.Options{
  845. HomeDir: "/var/lib/myapp/dagu",
  846. })
  847. if err != nil {
  848. return err
  849. }
  850. defer engine.Close(context.Background())
  851. run, err := engine.RunYAML(ctx, []byte(`
  852. params:
  853. - MESSAGE
  854. steps:
  855. - name: hello
  856. run: echo "${params.MESSAGE}"
  857. `), dagu.WithParams(map[string]string{
  858. "MESSAGE": "hello from the host app",
  859. }))
  860. if err != nil {
  861. return err
  862. }
  863. status, err := run.Wait(ctx)
  864. if err != nil {
  865. return err
  866. }
  867. fmt.Println(status.Status)
  868. ```
  869. The embedded API is experimental and may change. See the [embedded API documentation](https://docs.dagu.sh/embedding/go-api) and [examples/embedded](./examples/embedded).
  870. ## Community
  871. - [Discord](https://discord.gg/gpahPUjGRk): Questions and discussion
  872. - [GitHub Issues](https://github.com/dagucloud/dagu/issues): Bug reports and feature requests
  873. - [Bluesky](https://bsky.app/profile/dagu-sh.bsky.social)
  874. ## Development
  875. **Prerequisites:** [Go (latest stable)](https://go.dev/doc/install), [Node.js (latest stable)](https://nodejs.org/en/download/), [pnpm](https://pnpm.io/installation)
  876. ```sh
  877. git clone https://github.com/dagucloud/dagu.git && cd dagu
  878. make build # Build frontend + Go binary
  879. make test # Run tests with race detection
  880. make lint # Run golangci-lint
  881. ```
  882. For the local UI development loop, run `make run-server` from the repository
  883. root, then follow [`ui/README.md`](./ui/README.md). See
  884. [`ARCHITECTURE.md`](./ARCHITECTURE.md) for a human-oriented repository map and
  885. [`CONTRIBUTING.md`](./CONTRIBUTING.md) for the focused first-PR workflow and
  886. code standards.
  887. ## Acknowledgements
  888. <div align="center">
  889. <h3>Premium Sponsors</h3>
  890. <a href="https://slashbinlabs.com/">
  891. <img src="https://wsrv.nl/?url=https%3A%2F%2Fslashbinlabs.com%2Flogo.png&w=150&h=150&fit=cover&mask=circle" width="100" height="100" alt="/bin labs">
  892. </a>
  893. <h3>Supporters</h3>
  894. <p align="center">
  895. <a href="https://github.com/gyger">
  896. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fgyger.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@gyger">
  897. </a>
  898. <a href="https://github.com/disizmj">
  899. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fdisizmj.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@disizmj">
  900. </a>
  901. <a href="https://github.com/Arvintian">
  902. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2FArvintian.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@Arvintian">
  903. </a>
  904. <a href="https://github.com/yurivish">
  905. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fyurivish.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@yurivish">
  906. </a>
  907. <a href="https://github.com/jayjoshi64">
  908. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fjayjoshi64.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@jayjoshi64">
  909. </a>
  910. <a href="https://github.com/alangrafu">
  911. <img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Falangrafu.png&w=128&h=128&fit=cover&mask=circle" width="50" alt="@alangrafu">
  912. </a>
  913. </p>
  914. <br/><br/>
  915. <a href="https://github.com/sponsors/dagucloud">
  916. <img src="https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86" width="150" alt="Sponsor">
  917. </a>
  918. </div>
  919. ## Contributing
  920. Contributions of all kinds are welcome. See the [Contribution Guide](./CONTRIBUTING.md) for details.
  921. <a href="https://github.com/dagucloud/dagu/graphs/contributors">
  922. <img src="https://contrib.rocks/image?repo=dagucloud/dagu" />
  923. </a>
  924. ## License
  925. GNU GPLv3: See [LICENSE](./LICENSE). See [LICENSING.md](./LICENSING.md) for embedded API and commercial embedding notes.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.