loki-mode
Autonomi marketplace: Loki Mode, the autonomous spec-to-product build system with a built-in trust layer.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 11 Nov 2026. Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/asklokesh/loki-mode with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/asklokesh/loki-mode'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/asklokesh--loki-mode)HTML
<a href="/listings/asklokesh--loki-mode"><img src="/api/listings/asklokesh--loki-mode/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
npm install -g loki-mode
The npm package installs the Bun runtime the engine needs as an optional dependency. Other ways in: `bun install -g loki-mode`, `brew tap asklokesh/tap && brew install loki-mode`, or `docker pull asklokesh/loki-mode:latest` ([Docker guide](docs/DOCKER.md)). Upgrade with `loki self-update`.
Want to see a result before you install or spend anything?
npx loki-mode tour # prints a real receipt from a past build: no key, no spend, no network
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 40 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- {
- "$schema": "https://json.schemastore.org/claude-code-marketplace.json",
- "name": "loki-mode",
- "owner": {
- "name": "Autonomi"
- },
- "description": "Autonomi marketplace: Loki Mode, the autonomous spec-to-product build system with a built-in trust layer.",
- "plugins": [
- {
- "name": "loki-mode",
- "source": "./plugins/loki-mode",
- "description": "Autonomous spec-to-product build system with a built-in trust layer. Ships Loki's spec-hardening (grill), living-spec drift detection, and deterministic PR verification commands, plus the Loki MCP server (memory, task queue, code search, build management). Requires the loki-mode CLI on PATH (npm install -g loki-mode or brew install).",
- "homepage": "https://github.com/asklokesh/loki-mode",
- "repository": "https://github.com/asklokesh/loki-mode",
- "license": "SEE LICENSE IN LICENSE",
- "category": "automation",
- "keywords": [
- "autonomous",
- "agent",
- "spec-driven",
- "verification",
- "code-review",
- "mcp",
- "loki"
- ]
- },
- {
- "name": "loki-seal",
- "source": "./packages/loki-seal",
- "description": "Stop hook that refuses 'done' while tests are red or were deleted, skipped, xfailed or weakened. Runs locally, no model calls.",
- "homepage": "https://github.com/asklokesh/loki-mode",
- "category": "testing",
- "keywords": [
- "verification",
- "tests",
- "hooks"
- ]
- }
- ]
- }
README.md · README · 190 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # Loki Mode
- Your agent says done. Loki proves it: an autonomous coding agent that turns a task or issue into a verified pull request with a signed receipt you can check yourself.
- [](https://www.npmjs.com/package/loki-mode)
- [](https://www.npmjs.com/package/loki-mode)
- [](https://hub.docker.com/r/asklokesh/loki-mode)
- [](LICENSE)
- [Website](https://www.autonomi.dev/) | [Quickstart](docs/QUICKSTART.md) | [Install guide](docs/INSTALLATION.md) | [User guide](docs/v10/GUIDE.md) | [CLI reference](docs/CLI-CORE-VERBS.md) | [Compare with opencode](docs/COMPARE-OPENCODE.md) | [Changelog](CHANGELOG.md)
- <!-- generated:facts -->
- The npm badge above tracks the current release; see [CHANGELOG.md](CHANGELOG.md) for release notes.
- CLI reference: [docs/CLI-REFERENCE.md](docs/CLI-REFERENCE.md) (generated from the command registry).
- <!-- /generated -->
- ## Install
- ```bash
- npm install -g loki-mode
- ```
- The npm package installs the Bun runtime the engine needs as an optional dependency. Other ways in: `bun install -g loki-mode`, `brew tap asklokesh/tap && brew install loki-mode`, or `docker pull asklokesh/loki-mode:latest` ([Docker guide](docs/DOCKER.md)). Upgrade with `loki self-update`.
- Want to see a result before you install or spend anything?
- ```bash
- npx loki-mode start # no git repo: a real receipt, no key, no spend
- npx loki-mode tour # the same receipt when you are already inside a repo
- ```
- `npx` downloads the package the first time, so that step needs the network. The receipt itself replays offline and checks with no provider and no key.
- ```bash
- npx loki-mode wow # a provider is ready: one tiny real task in a temp repo, its own receipt
- ```
- ## First run
- Loki needs a model. [Claude Code](https://docs.claude.com/en/docs/claude-code) (`claude`) is the default provider: install it and log in. The bundled Agent SDK with an `ANTHROPIC_API_KEY` can stand in for the CLI on the main path. Other coding CLIs are optional. See [Requirements](#requirements).
- ```bash
- cd your-repo
- loki setup # first-run wizard (or: loki doctor --fix)
- loki start "add rate limiting to /api/login" # the primary path: build, verify, seal a receipt, open a PR
- loki start prd.md # same, from a PRD file
- loki start owner/repo#123 # same, from a GitHub, GitLab or Jira issue
- loki "fix #123" # an issue of this checkout; a copied issue or PR URL works as is
- loki edit "rename fooBar to foo_bar" # one small change, no loop
- loki chat # interactive alternative: Plan and Build modes
- ```
- `loki start`, `loki "<task>"`, `loki owner/repo#N` and `loki quick "<task>"` all run on the Loki 10 engine by default, whatever the flag order or input form. <!-- loki10-default --> Add `--no-pr` to build and verify without opening a pull request, `--max-cost <usd>` to cap spend, and `--provider <name>` to pick another coding CLI. The spend cap defaults to $100 when `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` is set and to no dollar cap on a subscription login; `budgets.per_run` in `loki.yaml` also sets it. Inspect the route without running: `loki route start prd.md` or `--explain-route`. New here? The [5-minute quickstart](docs/QUICKSTART.md) walks through it.
- ## What you get
- Every run starts from a delivery contract derived from your task, writes checks from the task alone before touching code, implements, verifies, and ends in exactly one outcome: VERIFIED, ALREADY_SATISFIED, FAILED, BUDGET_STOP, BLOCKED or STALLED ([exit codes](docs/exit-codes.md)). If the contract cannot be satisfied as written, the run stops and asks one question instead of guessing.
- - **A pull request** written for a quick review: what the issue asked, what changed, how it was tested, and what was NOT PROVEN. A VERIFIED run opens a ready PR; a PARTIAL run opens a draft PR titled `[PARTIAL]` that leads with what is not proven; a FAILED run with committed work opens a draft PR titled `[FAILED: <check>]` that leads with the checks still failing. The exception is a security scan secret match: that branch is never pushed, and Loki prints the file and line it matched and the push command to run after review. Any other outcome keeps the branch local and prints where it is.
- - **A signed receipt** at `.loki/runs/<run-id>/receipt.json`, signed with a local Ed25519 key created on first run.
- - **A specialist review** when the review council runs: it selects reviewers from a specialist pool (`agents/types.json`, scored by `run.sh:FOCUS_KEYWORDS`) by matching the changed files and diff against each reviewer's keywords.
- ```bash
- loki verify # newest run; or: loki verify <run-id>
- ```
- `loki verify` re-hashes the receipt, checks its signature and checks it against the run's event log. It prints a verdict and exits 0 (VERIFIED), 1 (TAMPERED), 2 (UNCHECKED), 3 (UNSIGNED, refused unless `--allow-unsigned`), 4 (run outcome not verified) or 66 (no runs); `[ $rc -eq 0 ]` is the only safe test for "verified". It also checks that your checkout still matches the sealed code: a run that opened a PR returns you to your starting branch, so verify it from the run's branch (`git fetch origin loki/<run-id> && git checkout --detach FETCH_HEAD`), or it reads UNCHECKED and says so. `loki start` also requires fail-to-pass for a declared behavior change: a Wall test must fail on the old code and pass after, or the verdict stays PARTIAL (`LOKI_MUTATION_STRICT=0` opts out). Full table: [docs/exit-codes.md](docs/exit-codes.md). On another machine, export the public key with `loki keys export` and pass it with `loki verify --pubkey <file>`. A receipt claims only what its checks ran, and lists what they did not; signing proves who produced it, not that the code is bug-free. Details: [docs/SIGNED-RECEIPTS.md](docs/SIGNED-RECEIPTS.md).
- For a multi-repo change set, `loki start --workspace <name> "<brief>"` seals one signed `composite.json` over every child receipt, and `loki verify <run-dir>/composite.json` re-checks it offline ([docs/WORKSPACES.md](docs/WORKSPACES.md)). A composite never reads higher than its weakest child.
- Two vocabularies, not one. Receipt verdicts (sealed, 5): VERIFIED, PARTIAL, ALREADY_SATISFIED, SPEC_CONFLICT, FAILED. Run outcomes (printed, 8): VERIFIED, ALREADY_SATISFIED, PARTIAL, FAILED, BUDGET_STOP, BLOCKED, STALLED, CANCELLED. BUDGET_STOP, BLOCKED, STALLED and CANCELLED are run outcomes, never receipt verdicts.
- | Run outcome | Receipt verdict | `loki start` exit |
- |---|---|---|
- | VERIFIED, ALREADY_SATISFIED | same | 0 |
- | FAILED | same | 1 |
- | PARTIAL | same | 6 |
- | BUDGET_STOP | none (stopped at a cap) | 3 |
- | BLOCKED | SPEC_CONFLICT | 4 |
- | STALLED | none | 5 |
- | CANCELLED | none | 130 |
- Composite: every child VERIFIED or ALREADY_SATISFIED gives VERIFIED (all ALREADY_SATISFIED stays ALREADY_SATISFIED, any SPEC_CONFLICT child keeps SPEC_CONFLICT); one PARTIAL child caps it at PARTIAL; a FAILED child fails it. `loki verify` on a composite exits 0 VERIFIED, 1 TAMPERED, 2 UNCHECKED, 3 UNSIGNED.
- Autonomy and deploy: `.loki/policies.json` `autonomy: propose|execute|operate` governs what `loki start` may do (propose opens nothing, execute opens a PR, operate also allows deploy); no file keeps today's behavior. `loki trust` prints advisory graduation advice from your receipts and never writes the policy. Loki deploys only behind `loki deploy --execute` (or `loki start --deploy`) plus a VERIFIED, HEAD-anchored receipt and the policy permission; `--health-url` and `--auto-rollback` add a post-deploy probe and a documented provider rollback. A Sentry alert webhook can enqueue a receipt-gated fix PR ([docs/sentry-intake.md](docs/sentry-intake.md)).
- A finished run prints a short summary (every run on both engines also ends with a unified Result, Files, Cost, Elapsed, Receipt and `next:` block; `--quiet` prints only that block, `--json` adds a `summary` object):
- ```
- Outcome: VERIFIED
- PR: https://github.com/owner/repo/pull/1
- Receipt: sha256:2f9a41c7e0b3...
- NOT PROVEN: full suite, app boot, security scan
- Cost: $0.42 (claude, 212k tokens)
- Time: 4m12s (intake 11s, plan 20s, implement 3m10s, verify 31s)
- ```
- ## Core commands
- | Command | What it does |
- |---------|--------------|
- | `loki start "<task>"` | The command you run: same provider model as a raw session, then the harness re-runs the checks and seals a receipt (`--no-pr`, `--provider`, `--max-cost`, `--haiku` to run implement and fix on Haiku, climbing to Sonnet on a failed check). `loki "<task>"` and `loki owner/repo#N` are shorthand. No task and no git repo shows a receipt and spends nothing |
- | `loki edit "<change>"` | One targeted change in one pass, no PR, plan or council (`--verify`, `--receipt`, `--commit`, `--dry-run`) |
- | `loki quick "<task>"` | One small task with a verify loop, no PR |
- | `loki chat` | Interactive alternative: Plan and Build modes, slash commands, `/run` starts a `loki start` run, `--attach` joins one |
- | `loki route start <args>` | Show the engine, provider, model and reason without running (also `--explain-route`) |
- | `loki setup` | First-run wizard: tools, Claude login, provider, model route, spend cap |
- | `loki config list\|get\|set\|path` | Read and write settings |
- | `loki models [--local]` | List model routes (OpenRouter, Ollama, LM Studio, llama.cpp, vLLM, custom); discover local models |
- | `loki agents list\|new\|show\|export --claude` | Custom child agents in `.loki/agents/*.md` |
- | `loki status [run-id]` | Status of the latest run, or of a given run |
- | `loki verify [run-id]` | Check a receipt's hash, signature and event log |
- | `loki answer [run-id] "<text>"` | Answer a BLOCKED run; it continues under the same run id (also `--text "..."`) |
- | `loki doctor [--fix] [--json]` | Check prerequisites, providers and login; `--fix` applies safe fixes |
- | `loki control serve` | Start the Control Plane (see below) |
- | `loki dashboard` | Open the Control Plane UI, starting it if needed |
- | `loki tour` | Show a sample receipt offline. `loki start` does this when you are not in a git repo and did not pass a task |
- | `loki wow` | Same live mini-task as a normal start, in a temp repo. It says it makes model calls and what it can cost, and asks y/N first (`--yes` skips; with no terminal it spends nothing). Offline sample when no provider is ready |
- | `loki factory "<task>"` | Same as `loki start`. The policy line is printed by start |
- | `loki --help` | Everything else; `loki help <command>` for one command |
- Core verbs: [docs/CLI-CORE-VERBS.md](docs/CLI-CORE-VERBS.md). Every command (generated): [docs/CLI-REFERENCE.md](docs/CLI-REFERENCE.md). Engine guide, provider table, model routes, custom agents and chat: [docs/v10/GUIDE.md](docs/v10/GUIDE.md). Honest comparison with opencode: [docs/COMPARE-OPENCODE.md](docs/COMPARE-OPENCODE.md).
- ### Models, providers and local models
- Two providers are supported. Claude Code is the default Tier-1 provider (the `claude` CLI, or, with only `ANTHROPIC_API_KEY`, the Anthropic Agent SDK). `--provider opencode` (or `LOKI_PROVIDER=opencode`) runs opencode, which reaches any model, including OpenRouter: export `OPENROUTER_API_KEY` and pass `--model openrouter/<vendor>/<model>`; the Loki 10 engine enforces the wall-clock limit and a post-iteration guard scan there. codex, cline, aider and gemini were removed and exit 2 with a one-line migration message. To keep the Claude harness on OpenRouter or a local model, set `LOKI_MODEL_ROUTE=<kind>/<id>` (kinds: `openrouter`, `ollama`, `lmstudio`, `llamacpp`, `vllm`, `custom`) before `loki start`; `loki models --local` finds local runtimes and prints ready-to-paste values.
- ```bash
- LOKI_MODEL_ROUTE=ollama/qwen3-coder loki start prd.md
- OPENROUTER_API_KEY=sk-or-... loki start --provider opencode --model openrouter/deepseek/deepseek-v3.2 prd.md
- ```
- ## Control Plane
- A local UI and API for all your runs: outcomes, receipts, costs, and an answer box for BLOCKED runs. It is on by default (`LOKI_CONTROL=0` turns it off) and listens on 127.0.0.1 only.
- ```bash
- loki control serve # http://127.0.0.1:47821 (--port N, --db PATH)
- loki control backfill . # load existing ./.loki/runs
- loki control status # reachable? how many runs?
- ```
- While it is running, runs on this machine ship to it automatically. Guides: [container, Helm and ECS](docs/control-plane-container.md), [migrating from the old dashboard](docs/control-plane-migration.md).
- ## Requirements
- `loki doctor` enforces this list.
- Required:
- - Claude Code (`claude`), installed and logged in: the default provider and the only one with every feature. Or only `ANTHROPIC_API_KEY`: `loki "<goal>"` then runs on the bundled Agent SDK with no CLI and no extra setting, but `loki edit` and `loki chat` call the `claude` CLI.
- - Node.js 20+ (`node`; 22 LTS is the CI target)
- - Git 2.x (`git`)
- - `curl`
- Optional:
- - Python 3.8+ (`python3`): deep scans, Python repos and group runs use it. `loki "<goal>"` runs without it.
- - `jq`: `loki status` and the bash fallback CLI use it.
- - opencode (`--provider opencode`) for any model, including OpenRouter. Ollama or an OpenRouter key for model routes.
- - Bun 1.3+ (`bun`) on your PATH (the npm package also bundles it).
- - Docker, to run Loki from the published image.
- macOS and Linux are supported.
- ## More
- - [Installation](docs/INSTALLATION.md), [environment variables](docs/environment-variables.md), [exit codes](docs/exit-codes.md), [known issues](docs/KNOWN-ISSUES.md), [privacy and telemetry](docs/PRIVACY.md)
- - Agent Change Receipt format: [docs/AGENT-CHANGE-RECEIPT.md](docs/AGENT-CHANGE-RECEIPT.md)
- - Label an issue, get a pull request (GitHub Actions, no hosted service, about 15 lines of workflow): [setup](docs/v10/GUIDE.md#triggers-without-a-cloud), [full example](docs/examples/loki-issue-to-pr.yml.example)
- - Jira and Linear intake: [docs/trackers.md](docs/trackers.md). Two-way Slack: [docs/slack.md](docs/slack.md). Multi-repo runs and composite receipts: [docs/WORKSPACES.md](docs/WORKSPACES.md). Deploy: [docs/DEPLOY-PLAN.md](docs/DEPLOY-PLAN.md)
- - MCP: the bundled server exposes 48 tools over stdio (`loki mcp`), including starting a run, reading its status and verifying its receipt. One-click install: [Cursor](cursor://anysphere.cursor-deeplink/mcp/install?name=loki-mode&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsImxva2ktbW9kZSIsIm1jcCJdfQ==), [VS Code](vscode:mcp/install?%7B%22name%22%3A%22loki-mode%22%2C%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22loki-mode%22%2C%22mcp%22%5D%7D). Or run `loki mcp install --client claude|cursor|vscode|windsurf`.
- - Claude Code plugin: `claude plugin marketplace add asklokesh/loki-mode`, then `claude plugin install loki-mode@loki-mode`. The same marketplace ships [loki-seal](packages/loki-seal/README.md), a Stop hook that refuses "done" when tests newly fail or were weakened.
- - Architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). Upgrading: [UPGRADING.md](UPGRADING.md)
- ## Contributing
- ```bash
- git clone https://github.com/asklokesh/loki-mode.git && cd loki-mode
- npm install && npm test
- ```
- See [CONTRIBUTING.md](.github/CONTRIBUTING.md), [TESTING.md](docs/TESTING.md) and the [Code of Conduct](.github/CODE_OF_CONDUCT.md).
- ## License
- [Business Source License 1.1](LICENSE). Free for personal, internal, academic and non-commercial use; it converts to Apache 2.0 on March 19, 2030. Contact founder@autonomi.dev for other arrangements.
- Loki Mode is built by [Autonomi](https://www.autonomi.dev/).
autonomy/hooks/validate-bash.sh · Bundled script · 1,424 lines · 5 findings marked
- #!/usr/bin/env bash
- # Loki Mode PreToolUse Hook - Bash Command Validation
- # Blocks dangerous commands, logs all executions
- set -uo pipefail
- INPUT=$(cat)
- deny() {
- local reason="${1:-Blocked by Loki command policy}"
- if command -v python3 >/dev/null 2>&1; then
- printf '%s' "$reason" | python3 -c '
- import json, sys
- print(json.dumps({"hookSpecificOutput": {
- "hookEventName": "PreToolUse",
- "permissionDecision": "deny",
- "permissionDecisionReason": sys.stdin.read(),
- }}))
- '
- else
- printf '%s' '{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"Blocked by Loki command policy"}}'
- fi
- exit 2
- }
- HOST_GUARD=false
- case "${LOKI_HOST_GUARD:-0}" in
- 1|true|yes|on) HOST_GUARD=true ;;
- esac
- # P12-26d: when the resolved policy is compiled (explicit compiled, or unset with an
- # operator claude_permissions policy present) the compiled allow/deny list
- # and permission mode decide. An explicit hook "allow" skips the permission
- # prompt for every command that passes the blocklist, which would widen the
- # compiled policy, so the hook never emits one and fails closed on bad input.
- COMPILED_POLICY=false
- case "${LOKI_PERMISSION_POLICY:-}" in
- legacy) ;;
- "")
- _vb_helper="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)/src/policies/claude_permissions.js"
- if command -v node >/dev/null 2>&1 && [ -f "$_vb_helper" ] \
- && [ "$(node "$_vb_helper" --has-operator-policy 2>/dev/null)" = "1" ]; then
- COMPILED_POLICY=true
- fi
- ;;
- *) COMPILED_POLICY=true ;;
- esac
- if ! command -v python3 >/dev/null 2>&1; then
- "$HOST_GUARD" && deny "LOKI_HOST_GUARD: python3 is unavailable, so the Bash command cannot be inspected safely."
- "$COMPILED_POLICY" && deny "LOKI_PERMISSION_POLICY=compiled: python3 is unavailable, so the Bash command cannot be inspected safely."
- printf '%s' '{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"allow"}}'
- exit 0
- fi
- if ! COMMAND=$(printf '%s' "$INPUT" | python3 -c '
- import json, sys
- try:
- data = json.load(sys.stdin)
- command = (data.get("tool_input") or {}).get("command")
- except Exception:
- raise SystemExit(2)
- if not isinstance(command, str) or not command.strip():
- raise SystemExit(2)
- # Join backslash-newline continuations as bash does (an odd run of backslashes
- # before the newline), so every later split and pattern match sees one command.
- import re
- command = re.sub(r"(?<!\\)((?:\\\\)*)\\\r?\n", r"\1", command)
- sys.stdout.write(command)
- '); then
- "$HOST_GUARD" && deny "LOKI_HOST_GUARD: malformed or missing Bash command input was denied."
- "$COMPILED_POLICY" && deny "LOKI_PERMISSION_POLICY=compiled: malformed or missing Bash command input was denied."
- printf '%s' '{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"allow"}}'
- exit 0
- fi
- CWD=$(printf '%s' "$INPUT" | python3 -c '
- import json, sys
- try:
- cwd = json.load(sys.stdin).get("cwd", "")
- except Exception:
- cwd = ""
- sys.stdout.write(cwd if isinstance(cwd, str) else "")
- ' 2>/dev/null || true)
- if "$HOST_GUARD"; then
- GUARD_ROOT="${LOKI_TARGET_DIR:-$PWD}"
- if [ -z "$CWD" ] || ! _LOKI_GUARD_CWD="$CWD" _LOKI_GUARD_ROOT="$GUARD_ROOT" python3 -c '
- import os
- cwd = os.path.realpath(os.environ["_LOKI_GUARD_CWD"])
- root = os.path.realpath(os.environ["_LOKI_GUARD_ROOT"])
- raise SystemExit(0 if cwd == root or cwd.startswith(root + os.sep) else 1)
- '; then
- deny "LOKI_HOST_GUARD: Bash command cwd is outside the assigned workspace."
- fi
- fi
- CWD="${CWD:-${LOKI_TARGET_DIR:-$PWD}}"
- # Hosted builds run on a shared host. This guard prevents recognizable direct
- # command mistakes involving host containers, processes, and listening ports.
- # It is not process isolation: an allowed package script or interpreter can
- # perform the same operations internally. Production builds still require a
- # separate container or microVM with no host Docker socket, restricted
- # credentials, and network policy. This parser only inspects text.
- if "$HOST_GUARD"; then
- HOST_GUARD_REASON=$(_LOKI_GUARD_COMMAND="$COMMAND" _LOKI_GUARD_INPUT="$INPUT" python3 - <<'PY'
- import os
- import re
- import shlex
- command = os.environ.get("_LOKI_GUARD_COMMAND", "")
- blocked = {
- "docker": "host Docker access",
- "docker-compose": "host Docker access",
- "kill": "host process control",
- "killall": "host process control",
- "pkill": "host process control",
- "sudo": "host privilege escalation",
- "doas": "host privilege escalation",
- "launchctl": "host service control",
- "service": "host service control",
- "systemctl": "host service control",
- }
- wrappers = {"command", "env", "exec", "nohup", "time"}
- wrapper_value_options = {
- "env": {"-u", "--unset", "-C", "--chdir"},
- "exec": {"-a"},
- "time": {"-f", "--format", "-o", "--output"},
- }
- shells = {"bash", "dash", "ksh", "sh", "zsh"}
- separators = {";", "&", "&&", "|", "||", "(", ")"}
- assignment = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=.*$", re.S)
- def basename(value):
- return value.rstrip("/").rsplit("/", 1)[-1]
- def words(value):
- lexer = shlex.shlex(value.replace("\n", " ; "), posix=True,
- punctuation_chars=";&|()")
- lexer.whitespace_split = True
- lexer.commenters = ""
- return list(lexer)
- def first_command(segment):
- index = 0
- while index < len(segment) and assignment.match(segment[index]):
- index += 1
- while index < len(segment):
- name = basename(segment[index])
- if name not in wrappers:
- return name, index
- index += 1
- while index < len(segment):
- token = segment[index]
- if assignment.match(token):
- index += 1
- continue
- if token == "--":
- index += 1
- break
- if name == "env" and token in {"-S", "--split-string"}:
- return "__uninspectable_wrapper__", index
- if token in wrapper_value_options.get(name, set()):
- index += 2
- continue
- if token.startswith("-"):
- index += 1
- continue
- break
- return "", index
- def classify(value, depth=0):
- if depth > 2:
- return "command nesting exceeds the safe inspection limit"
- try:
- tokens = words(value)
- except ValueError:
- return "command text could not be parsed safely"
- segment = []
- segments = []
- for token in tokens:
- if token in separators:
- if segment:
- segments.append(segment)
- segment = []
- else:
- segment.append(token)
- if segment:
- segments.append(segment)
- for part in segments:
- name, index = first_command(part)
- if name == "__uninspectable_wrapper__":
- return "command wrapper cannot be inspected safely"
- if name in blocked:
- return blocked[name]
- args = part[index + 1:]
- if name in shells:
- for pos, arg in enumerate(args[:-1]):
- if arg.startswith("-") and "c" in arg:
- nested = classify(args[pos + 1], depth + 1)
- if nested:
- return nested
- if name == "xargs":
- for arg in args:
- if arg.startswith("-"):
- continue
- target = basename(arg)
- if target in blocked:
- return blocked[target]
- break
- if name == "find":
- for pos, arg in enumerate(args[:-1]):
- if arg in {"-exec", "-execdir", "-ok", "-okdir"}:
- target = basename(args[pos + 1])
- if target in blocked:
- return blocked[target]
- if name in {"bunx", "npx"} and args:
- target = basename(next((arg for arg in args if not arg.startswith("-")), ""))
- if target in blocked:
- return blocked[target]
- if name in {"npm", "pnpm", "yarn"} and args and args[0] in {"exec", "x"}:
- target = basename(next((arg for arg in args[1:] if not arg.startswith("-")), ""))
- if target in blocked:
- return blocked[target]
- if name == "brew" and args and args[0] == "services":
- return "host service control"
- if name in {"nc", "ncat", "netcat"} and any(arg == "-l" or arg.startswith("-l") for arg in args):
- return "host port listener"
- if name == "socat" and any("LISTEN:" in arg.upper() for arg in args):
- return "host port listener"
- port_assignment = re.search(
- r"(?<![A-Za-z0-9_])(?:PORT|HOST_PORT|LISTEN_PORT)\s*=\s*['\"]?([0-9]+)",
- command,
- )
- if port_assignment and int(port_assignment.group(1)) != 0:
- return "explicit host port claim"
- port_flag = re.search(
- r"(?:^|\s)(?:--port|--publish)(?:=|\s+)([0-9]+)",
- command,
- )
- if port_flag and int(port_flag.group(1)) != 0:
- return "explicit host port claim"
- http_server = re.search(r"\bhttp\.server\s+([0-9]+)", command)
- if http_server and int(http_server.group(1)) != 0:
- return "host port listener"
- return ""
- # Q1 HEREDOC-MASK: the body of a heredoc whose delimiter is quoted (<<'EOF', <<"EOF", <<-'EOF') is data: bash
- # expands nothing in it. Tokenizing it made a file write whose body holds one apostrophe or one double quote
- # unparseable, and the guard refused it ("command text could not be parsed safely"). The masker drops such a body
- # (and its terminator line) only when the body cannot become a command: the command reading it is cat or tee, its
- # output is not piped on, and no command line names a shell, eval, source, ., exec, xargs, ssh or coproc. Every
- # command line and every unquoted body (which expands $(...)) is kept and checked as before. It works on the
- # command as the tool sent it, before continuations are joined: inside a quoted body a backslash-newline is
- # literal, so a joined copy would end the body at a different line than bash does. On anything it does not model
- # (an unclosed quote, a backtick, $(, ((, <( or >( on a command line, another delimiter form, a body with no
- # terminator line) it gives up and the command is classified exactly as before.
- QUOTED_HEREDOC = re.compile(r"<<(-?)[ \t]*(['\"])([A-Za-z_][A-Za-z0-9_]*)\2(?=[\s;&|()<>]|$)")
- UNQUOTED_HEREDOC = re.compile(r"<<(-?)[ \t]*([A-Za-z_][A-Za-z0-9_]*)(?=[\s;&|()<>]|$)")
- DATA_READERS = {"cat", "tee"}
- RUNS_TEXT = re.compile(r"(?:^|[\s;&|()/])(?:bash|sh|zsh|dash|ksh|eval|source|exec|xargs|ssh|coproc)(?=$|[\s;&|()])|(?:^|[\s;&|()])\.(?=[ \t])")
- def mask_quoted_heredocs(raw):
- n, i, out, pending = len(raw), 0, [], []
- seg_start, no_pipe = 0, False
- while i < n:
- c = raw[i]
- if c == "\\":
- out.append(raw[i:i + 2])
- i += 2
- continue
- if c == "`" or raw.startswith(("$(", "((", "<(", ">("), i):
- return None
- if c == "'":
- j = raw.find("'", i + 1)
- if j < 0:
- return None
- out.append(raw[i:j + 1])
- i = j + 1
- continue
- if c == '"':
- j = i + 1
- while j < n and raw[j] != '"':
- if raw[j] == "`" or raw.startswith("$(", j):
- return None
- j += 2 if raw[j] == "\\" else 1
- if j >= n:
- return None
- out.append(raw[i:j + 1])
- i = j + 1
- continue
- if c == "#" and (i == 0 or raw[i - 1] in " \t\n;&|()<>"):
- j = raw.find("\n", i)
- j = n if j < 0 else j
- out.append(raw[i:j])
- i = j
- continue
- if raw.startswith("<<<", i):
- out.append("<<<")
- i += 3
- continue
- if raw.startswith("<<", i):
- m = QUOTED_HEREDOC.match(raw, i)
- if m:
- try:
- head = words(raw[seg_start:i])
- except ValueError:
- return None
- while head and assignment.match(head[0]):
- head = head[1:]
- if not head or basename(head[0]) not in DATA_READERS:
- return None
- pending.append((m.group(3), m.group(1) == "-", False))
- no_pipe = True
- else:
- m = UNQUOTED_HEREDOC.match(raw, i)
- if not m:
- return None
- pending.append((m.group(2), m.group(1) == "-", True))
- out.append(m.group(0))
- i = m.end()
- continue
- if c in ";&|()\n":
- if c == "|" and no_pipe and not raw.startswith("||", i):
- return None
- if c == "|" and raw.startswith("||", i):
- out.append("||")
- i += 2
- seg_start, no_pipe = i, False
- continue
- seg_start, no_pipe = i + 1, False
- out.append(c)
- i += 1
- if c == "\n" and pending:
- for delim, strip, keep in pending:
- while True:
- if i >= n:
- return None
- nl = raw.find("\n", i)
- stop = n if nl < 0 else nl + 1
- line = raw[i:stop].rstrip("\n")
- if keep:
- out.append(raw[i:stop])
- i = stop
- if (line.lstrip("\t") if strip else line) == delim:
- break
- pending = []
- seg_start = i
- if pending:
- return None
- masked = "".join(out)
- return None if RUNS_TEXT.search(masked) else masked
- def to_classify():
- try:
- import json
- raw = (json.loads(os.environ.get("_LOKI_GUARD_INPUT", "")).get("tool_input") or {}).get("command")
- except Exception:
- return command
- masked = mask_quoted_heredocs(raw.replace("\0", "")) if isinstance(raw, str) else None # bash drops NUL bytes from $COMMAND; so does this copy
- if masked is None:
- return command
- return re.sub(r"(?<!\\)((?:\\\\)*)\\\r?\n", r"\1", masked) # the continuation join the input step applies
- reason = classify(to_classify())
- if reason:
- print("LOKI_HOST_GUARD: refusing " + reason + ". Use the platform sandbox and lifecycle APIs instead.")
- PY
- )
- if [ -n "$HOST_GUARD_REASON" ]; then
- deny "$HOST_GUARD_REASON"
- fi
- fi
- # The supervised simple-web worker prepares dependencies and owns preview,
- # browser, proof, and process lifecycle. Model-launched installs, watchers, dev
- # servers, and Git loops only add latency or escape the bounded execution plan.
- # Enforce that contract at the command boundary so it applies to every model,
- # not only to models that follow the prompt perfectly.
- if [ "${LOKI_SUPERVISED_BUILD:-0}" = "1" ] \
- && [ "${LOKI_BUILD_PROFILE:-}" = "simple-web" ]; then
- SIMPLE_WEB_GUARD_REASON=$(_LOKI_SIMPLE_COMMAND="$COMMAND" \
- _LOKI_SIMPLE_CWD="$CWD" python3 - <<'PY'
- import json
- import os
- import re
- command = os.environ.get("_LOKI_SIMPLE_COMMAND", "")
- cwd = os.environ.get("_LOKI_SIMPLE_CWD", "")
- rules = (
- (
- re.compile(r"(?:^|[;&|()]|&&|\|\|)\s*(?:env\s+[^;&|()]*\s+)?(?:npm|pnpm|yarn|bun)\s+(?:i|install|add|update|upgrade)\b", re.I),
- "dependency changes are owned by the prepared scaffold",
- ),
- (
- re.compile(r"(?:^|[;&|()]|&&|\|\|)\s*(?:npm\s+run\s+dev|pnpm\s+(?:run\s+)?dev|yarn\s+dev|bun\s+run\s+dev|next\s+dev|vite(?:\s|$))", re.I),
- "development servers are owned by the preview harness",
- ),
- (
- re.compile(r"(?:^|[;&|()]|&&|\|\|)\s*git(?:\s|$)", re.I),
- "Git operations are outside the hosted simple-web implementation step",
- ),
- (
- re.compile(r"(?:^|\s)--watch(?:All)?(?:=\S+)?(?:\s|$)", re.I),
- "watch-mode tests do not terminate",
- ),
- )
- for pattern, reason in rules:
- if pattern.search(command):
- print(reason)
- raise SystemExit(0)
- # `npm test` can hide a watcher in package.json. Inspect only the assigned
- # workspace manifest and direct the model to a one-shot script when available.
- if re.search(r"(?:^|[;&|()]|&&|\|\|)\s*npm\s+(?:run\s+)?test(?:\s|$)", command, re.I):
- try:
- with open(os.path.join(cwd, "package.json"), encoding="utf-8") as handle:
- script = str((json.load(handle).get("scripts") or {}).get("test") or "")
- except (OSError, ValueError, TypeError):
- script = ""
- if re.search(r"(?:^|\s)--watch(?:All)?(?:=\S+)?(?:\s|$)", script, re.I):
- print("the package test script starts a watcher; use test:ci or a one-shot runner")
- raise SystemExit(0)
- print("")
- PY
- )
- if [ -n "$SIMPLE_WEB_GUARD_REASON" ]; then
- deny "LOKI_SIMPLE_WEB_GUARD: ${SIMPLE_WEB_GUARD_REASON}."
- fi
- fi
- # Dangerous command patterns (matched anywhere in the command string)
- # Safe paths like /tmp/ and relative paths (./) are excluded below
- # NOTE: This is defense-in-depth, not a security boundary. Motivated attackers
- # can bypass with advanced techniques (heredocs, printf, arbitrary string building).
- # This hook catches common mistakes and simple bypass attempts.
- BLOCKED_PATTERNS=(
- "rm -rf /"
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 459 on GitHub
- "rm -rf ~"
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 460 on GitHub
- "rm -rf \\\$HOME"
- "> /dev/sd"
- "mkfs[. ]"
- "dd if=/dev/zero"
- "chmod -R 777 /"
- Note: Quoted as data in code: Sets world-writable permissions on a root path (chmod 777 /)
- Line 465 on GitHub
- "eval.*base64"
- "base64.*\|.*sh"
- "base64.*\|.*bash"
- "\$\(base64"
- "eval.*\\\$\("
- "curl.*\|.*sh"
- "wget.*\|.*sh"
- "curl.*\|.*bash"
- "wget.*\|.*bash"
- # Config self-protection: prevent agents from corrupting internal state
- "rm -rf \.loki"
- "rm -rf \./\.loki"
- "rm .*\.loki/council/"
- "rm .*\.loki/config\.yaml"
- "rm .*\.loki/logs/bash-audit"
- "rm .*\.loki/session\.lock"
- "> \.loki/council/"
- "> \.loki/config\.yaml"
- # Fork bomb patterns
- ":\(\)\{.*\|.*&"
- ":\(\) *\{.*\|.*&"
- )
- # Safe path patterns that override blocked pattern matches. They are matched
- # per SEGMENT (the command split on ; & | newline CR), never against the whole
- # string: an unanchored or per-line match let `rm -rf /tmp/x; rm -rf /` and
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 491 on GitHub
- # `rm -rf /tmp/x<newline>rm -rf /` ride the override. Only the segment that hit
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 492 on GitHub
- # a blocked pattern must be safe; an optional trailing /dev/null redirection is
- # allowed, and `*` and `$$` in /tmp names. A `..` anywhere in the segment is
- # never safe.
- _SAFE_REDIR='([[:space:]]+[0-9]*>>?[[:space:]]*/dev/null)*[[:space:]]*'
- _SAFE_ENV='([ABCDEFGHIJKLMNOPQRSTUVWXYZ_][ABCDEFGHIJKLMNOPQRSTUVWXYZ0-9_]*=[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0-9._/:-]*[[:space:]]+)*'
- # A dot-leading path component with a glob char (`/tmp/.*` expands to `/tmp/..`).
- _DOT_GLOB='/\.[^/[:space:]]*[*?[]'
- SAFE_PATTERNS=(
- '^[[:space:]]*'"$_SAFE_ENV"'rm -rf (/tmp/([abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0-9._/*-]|\$\$)+[[:space:]]*)+'"$_SAFE_REDIR"'$'
- '^[[:space:]]*'"$_SAFE_ENV"'rm -rf \.loki/queue/dead-letter[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0-9._/-]*'"$_SAFE_REDIR"'$'
- )
- # Env names that change what the allowed command executes or loads; a prefix
- # naming one never rides the safe-path override.
- _env_prefix_is_risky() {
- local rest="$1" name
- while [[ "$rest" =~ ^[[:space:]]*([abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_][abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0-9_]*)=[^[:space:]]*(.*)$ ]]; do
- name="${BASH_REMATCH[1]}"
- rest="${BASH_REMATCH[2]}"
- case "$name" in
- PATH|LD_*|DYLD_*|BASH_ENV|ENV|IFS|PS4|SHELLOPTS|BASHOPTS|PROMPT_COMMAND|GIT_SSH_COMMAND|GIT_EXEC_PATH|NODE_OPTIONS|PYTHONPATH|PERL5OPT|GCONV_PATH|LOCPATH|EXECIGNORE) return 0 ;;
- esac
- done
- return 1
- }
- # Allowlist: the safe-path override applies only when EVERY segment of the
- # command is blank, an accepted safe rm, or a bare `NAME=value` assignment
- # (uppercase name not on the risky list, value free of quoting, expansion,
- # grouping and redirection characters). Anything else (export, declare, unset,
- # eval, source, hash, alias, function, subshell, echo, cd, ...) can change how a
- # later rm resolves, so it disables the override. Over-denial is intended.
- _ASSIGN_ONLY='^[[:space:]]*([ABCDEFGHIJKLMNOPQRSTUVWXYZ_][ABCDEFGHIJKLMNOPQRSTUVWXYZ_0-9]*)=([^][\"'"'"'\\$`(){}<>[:space:]]*)[[:space:]]*$'
- _segment_is_inert() {
- local seg="$1"
- [[ "$seg" =~ ^[[:space:]]*(true|false|:)?[[:space:]]*$ ]] && return 0
- if [[ "$seg" =~ $_ASSIGN_ONLY ]]; then
- _env_prefix_is_risky "$seg" && return 1
- return 0
- fi
- return 1
- }
- _all_segments_allowlisted() {
- local seg
- while IFS= read -r seg; do
- _segment_is_inert "$seg" && continue
- _segment_is_safe "$seg" || return 1
- done <<< "$1"
- return 0
- }
- _segment_is_safe() {
- local seg="$1" safe
- [[ "$seg" == *".."* ]] && return 1
- _env_prefix_is_risky "$seg" && return 1
- [[ "$seg" =~ $_DOT_GLOB ]] && return 1
- for safe in "${SAFE_PATTERNS[@]}"; do
- [[ "$seg" =~ $safe ]] && return 0
- done
- return 1
- }
- # Succeeds when every segment hitting $1 is safe and at least one segment hits
- # (a pattern that only matches across segments, like `curl x | sh`, is unsafe).
- _blocked_hits_all_safe() {
- local pat="$1" flat seg hit=false
- # Strip fd duplication (2>&1) first: its `&` is not a command separator, and removing it hides no command.
- flat=$(printf '%s' "$COMMAND" | sed -E 's/[0-9]*>&[0-9]+//g' | tr ';&|\r' '\n')
- _all_segments_allowlisted "$flat" || return 1
- while IFS= read -r seg; do
- printf '%s' "$seg" | grep -qE "$pat" || continue
- hit=true
- _segment_is_safe "$seg" || return 1
- done <<< "$flat"
- "$hit"
- }
- # A recursive rm whose target IS a temp root, or a bare glob directly under
- # one, wipes every tenant of the temp dir. The target is normalized first
- # (quotes and backslashes dropped, a glued trailing redirect cut, repeated and
- # trailing slashes and `/./` collapsed, `..` resolved lexically) so `//tmp`,
- # `/tmp/x/../*`, `/t\mp` and `"$TMPDIR"/*` cannot hide. Single-level brace
- # alternatives are expanded and every alternative is checked. A named child
- # (`/tmp/loki-run.abc123`, `/tmp/foo*`) is not a bare glob and stays ok.
- _normalize_rm_target() {
- local t="${1%%[<>]*}" prev="" comp joined="" sc
- local -a parts=() stack=()
- t="${t//[\"\'\\]/}"
- while [ "$t" != "$prev" ]; do
- prev="$t"
- t="${t//\/\//\/}"
- t="${t//\/.\//\/}"
- done
- if [[ "$t" == */* ]]; then
- IFS=/ read -r -a parts <<< "$t"
- for comp in ${parts[@]+"${parts[@]}"}; do
- case "$comp" in
- ''|.) ;;
- ..)
- # A dotdot that would pop a temp root component is
- # unresolvable lexically ($TMPDIR/.. is the root's parent
- # on some hosts): report the temp root itself (deny).
- joined=""
- for sc in ${stack[@]+"${stack[@]}"}; do joined="${joined}/${sc}"; done
- case "$joined" in
- /tmp|/var/tmp|/private/tmp|/private/var/tmp|'/$TMPDIR'|'/${TMPDIR}')
- printf '%s' "/tmp"
- return 0
- ;;
- esac
- joined=""
- if [ "${#stack[@]}" -gt 1 ] || { [ "${#stack[@]}" -eq 1 ] && [[ "$t" == /* ]]; }; then
- unset 'stack[${#stack[@]}-1]'
- fi
- ;;
- *) stack+=("$comp") ;;
- esac
- done
- for comp in ${stack[@]+"${stack[@]}"}; do joined="${joined}/${comp}"; done
- if [[ "$t" == /* ]]; then
- t="${joined:-/}"
- else
- t="${joined#/}"
- fi
- fi
- while [[ "$t" == */. && "$t" != "/." ]]; do t="${t%/.}"; done
- while [[ "$t" == */ && "$t" != "/" ]]; do t="${t%/}"; done
- printf '%s' "$t"
- }
- # Expand single-level comma brace groups into the global _ALTS array without
- # a subshell. Fails closed (returns 1) on nested braces, `{x..y}` ranges, or
- # more than 64 alternatives; the caller denies.
- _BRACE_CAP=64
- _expand_braces() {
- local s="$1" cur pre body post b alt
- local -a work=("$s")
- local temp=false
- _ALTS=()
- # Fail-closed rules (nested braces, ranges, cap, residual groups) apply to
- # every target except a plain relative path (no leading / ~ $ {, no `..`
- # component). The run-owned LOKI_RUN_TMP variable is also exempt.
- local q="${s//[\"\'\\]/}" stripped
- temp=true
- case "$q" in
- '$LOKI_RUN_TMP'|'$LOKI_RUN_TMP/'*|'${LOKI_RUN_TMP}'|'${LOKI_RUN_TMP}/'*) temp=false ;;
- /*|'~'*|'$'*|'{'*) ;;
- *)
- if [[ "/${q}/" != *"/../"* ]]; then temp=false; fi
- ;;
- esac
- if $temp; then
- # Drop parameter expansions (innermost first) so `${A:-${B}}` is not nesting.
- stripped="$s"
- while [[ "$stripped" =~ \$\{[^\{\}]*\} ]]; do
- stripped="${stripped/"${BASH_REMATCH[0]}"/}"
- done
- # A numeric range under a non-TMPDIR variable only appends digits, so it
- # cannot build a temp root (`$BUILD_DIR/shard{1..4}`).
- if [[ "$q" =~ ^\$\{?([A-Za-z_][A-Za-z0-9_]*) && "${BASH_REMATCH[1]}" != TMPDIR ]]; then
- while [[ "$stripped" =~ \{-?[0-9]+\.\.-?[0-9]+\} ]]; do
- stripped="${stripped/"${BASH_REMATCH[0]}"/}"
- done
- fi
- if [[ "$stripped" =~ \{[^\}]*\{ ]] || [[ "$stripped" =~ \{[^\}]*\.\.[^\}]*\} ]]; then
- return 1
- fi
- fi
- while [ "${#work[@]}" -gt 0 ]; do
- cur="${work[0]}"
- work=("${work[@]:1}")
- if [[ "$cur" =~ ^((\$\{[^\}]*\}|[^\{])*)\{([^\}]*,[^\}]*)\}(.*)$ ]]; then
- pre="${BASH_REMATCH[1]}"; body="${BASH_REMATCH[3]}"; post="${BASH_REMATCH[4]}"
- b="$body"
- while :; do
- alt="${b%%,*}"
- work+=("${pre}${alt}${post}")
- [[ "$b" == *,* ]] || break
- b="${b#*,}"
- done
- [ $(( ${#work[@]} + ${#_ALTS[@]} )) -le "$_BRACE_CAP" ] || { $temp && return 1; return 2; }
- else
- # A comma group the regex could not peel off stays unexpanded.
- if $temp && [[ "$cur" =~ \{[^\}]*,[^\}]*\} ]]; then
- return 1
- fi
- _ALTS+=("$cur")
- [ $(( ${#work[@]} + ${#_ALTS[@]} )) -le "$_BRACE_CAP" ] || { $temp && return 1; return 2; }
- fi
- done
- return 0
- }
- # A component is glob-only when it is only `*`, `?`, `.`, or when it holds a
- # `[` with no literal prefix before it (bracket parsing is ambiguous: `[!]]`,
- # `[]a]`, `[=a=]`, so fail closed).
- _glob_only() {
- local c="$1" pre
- [[ "$c" =~ ^[*?.]+$ ]] && return 0
- if [[ "$c" == *"["* ]]; then
- pre="${c%%\[*}"
- [[ "$pre" =~ ^[*?.]*$ ]] && return 0
- fi
- return 1
- }
- _temp_target_is_root_or_glob() {
- local t rest
- t=$(_normalize_rm_target "$1")
- case "$t" in
- /tmp|/var/tmp|/private/tmp|/private/var/tmp|'$TMPDIR'|'${TMPDIR}') return 0 ;;
- /private/var/tmp/*) rest="${t#/private/var/tmp/}" ;;
- /private/tmp/*) rest="${t#/private/tmp/}" ;;
- /var/tmp/*) rest="${t#/var/tmp/}" ;;
- /tmp/*) rest="${t#/tmp/}" ;;
- '$TMPDIR'/*) rest="${t#\$TMPDIR/}" ;;
- '${TMPDIR}'/*) rest="${t#\$\{TMPDIR\}/}" ;;
- /private/var/tmp*) rest="${t#/private/var/tmp}"; _glob_only "$rest"; return ;;
- /private/tmp*) rest="${t#/private/tmp}"; _glob_only "$rest"; return ;;
- /var/tmp*) rest="${t#/var/tmp}"; _glob_only "$rest"; return ;;
- /tmp*) rest="${t#/tmp}"; _glob_only "$rest"; return ;;
- '$TMPDIR'*) rest="${t#\$TMPDIR}"; _glob_only "$rest"; return ;;
- '${TMPDIR}'*) rest="${t#\$\{TMPDIR\}}"; _glob_only "$rest"; return ;;
- *) return 1 ;;
- esac
- # first component made only of glob characters, dots and bracket expressions
- _glob_only "${rest%%/*}" && return 0
- return 1
- }
- _rm_target_is_temp_root_or_glob() {
- local alt
- if [[ "$1" == *"{"* ]]; then
- _expand_braces "$1" || {
- # 1 = fail closed (temp target); 2 = over cap on a non-temp target
- [ "$?" -eq 2 ] && return 1
- return 0
- }
- for alt in ${_ALTS[@]+"${_ALTS[@]}"}; do
- _temp_target_is_root_or_glob "$alt" && return 0
- done
- return 1
- fi
- _temp_target_is_root_or_glob "$1"
- }
- # P12-26g-c: residual shapes that reach a temp root or the filesystem root
- # without naming it. Each helper returns 0 to DENY (fail closed).
- _RUN_TMP_RE='^/([^/]+/)*loki-run\.[A-Za-z0-9]+$'
- _run_tmp_valid() {
- local v="${LOKI_RUN_TMP:-}"
- [[ "$v" =~ $_RUN_TMP_RE ]] || return 1
- [[ "/${v}/" != *"/../"* && "/${v}/" != *"/./"* ]]
- }
- # A component that can expand to `..`: a literal `..`, one built by gluing
- # variables onto dots (`..$X`, `.$X.`; X may be empty or `.`), or a dot-leading
- # glob (`.?`, `.*`, `..*`, `.[.]`; bash < 5.2 has no globskipdots, so these
- # match `..`). Worst case is assumed everywhere.
- _is_dotdotish() {
- local c="$1" s="$1"
- [[ "$c" == *.* ]] || return 1
- while [[ "$s" =~ (\$\{[^\}]*\}|\$[[:alpha:]_][[:alnum:]_]*) ]]; do
- s="${s/"${BASH_REMATCH[0]}"/}"
- done
- [ "$s" = "." ] && [ "$s" = "$c" ] && return 1
- [[ "$s" =~ ^\.[.*?]*$ ]] || [[ "$s" =~ ^\.[.*?]*\[ ]]
- }
- _has_dotdot_component() {
- local c
- local -a p=()
- [[ "$1" == *.* ]] || return 1
- IFS=/ read -r -a p <<< "$1"
- for c in ${p[@]+"${p[@]}"}; do _is_dotdotish "$c" && return 0; done
- return 1
- }
- # Lexical resolve of an absolute path; `..` (or a possible `..`) at the root
- # stays at the root.
- _lex_resolve() {
- local comp out=""
- local -a parts=() stack=()
- IFS=/ read -r -a parts <<< "$1"
- for comp in ${parts[@]+"${parts[@]}"}; do
- case "$comp" in
- ''|.) ;;
- *)
- if _is_dotdotish "$comp"; then
- [ "${#stack[@]}" -gt 0 ] && unset 'stack[${#stack[@]}-1]'
- else
- stack+=("$comp")
- fi
- ;;
- esac
- done
- for comp in ${stack[@]+"${stack[@]}"}; do out="${out}/${comp}"; done
- printf '%s' "${out:-/}"
- }
- # Resolved path is a temp root, a bare temp glob, or an ancestor of a temp root.
- _resolved_reaches_temp() {
- case "$1" in
- /|/tmp|/var|/var/tmp|/private|/private/tmp|/private/var|/private/var/tmp) return 0 ;;
- esac
- _temp_target_is_root_or_glob "$1"
- }
- # `..` climbing from a known base: deny when the result leaves the base or
- # reaches a temp root; an unknown or relative base is unresolvable (deny).
- _dotdot_escapes_base() {
- local base="$1" rel="$2" res
- [[ "$base" == /* ]] || return 0
- base=$(_lex_resolve "$base")
- res=$(_lex_resolve "${base}/${rel}")
- _resolved_reaches_temp "$res" && return 0
- [[ "$res" == "$base" || "$res" == "$base"/* ]] || return 0
- return 1
- }
- # Absolute path an empty or defaulted variable can build: deny when it is, or
- # climbs to, a temp root or the filesystem root, or holds a glob root.
- _abs_path_deny() {
- _resolved_reaches_temp "$(_lex_resolve "$1")" && return 0
- _alt_extra_deny "$1"
- }
- _alt_extra_deny() {
- local a="$1" name="" mod="" rest="" rest0="" r n c1 c2 c3 w combined
- local multi=false
- if [[ "$a" =~ ^\$\{([A-Za-z_][A-Za-z0-9_]*)([^\}]*)\}(.*)$ ]]; then
- name="${BASH_REMATCH[1]}"; mod="${BASH_REMATCH[2]}"; rest="${BASH_REMATCH[3]}"
- elif [[ "$a" =~ ^\$([A-Za-z_][A-Za-z0-9_]*)(.*)$ ]]; then
- name="${BASH_REMATCH[1]}"; rest="${BASH_REMATCH[2]}"
- fi
- rest0="$rest"
- if [ -n "$name" ]; then
- # Strip every further leading expansion ($X$Y/..., "$X""$Y"/...).
- while [[ "$rest" =~ ^(\$\{[[:alpha:]_][[:alnum:]_]*[^\}]*\}|\$[[:alpha:]_][[:alnum:]_]*)(.*)$ ]]; do
- multi=true
- rest="${BASH_REMATCH[2]}"
- done
- # A default or alternative word can supply the root: ${X:-/}tmp/*,
- # ${X:-/tmp}. Check the word plus the rest as the path it can build.
- if [ -n "$mod" ] && [ "$name" != "LOKI_RUN_TMP" ]; then
- case "$mod" in
- :-*|:=*|:+*) w="${mod:2}" ;;
- -*|=*|+*) w="${mod:1}" ;;
- *) w="" ;;
- esac
- combined="${w}${rest0}"
- if [[ "$combined" == /* ]]; then
- _abs_path_deny "$combined" && return 0
- fi
- fi
- case "$name" in
- LOKI_RUN_TMP*)
- [ "$name" = "LOKI_RUN_TMP" ] || return 0
- [ -z "$mod" ] || return 0
- "$multi" && return 0
- _run_tmp_valid || return 0
- [ -z "$rest" ] && return 1
- [[ "$rest" == /* ]] || return 0
- _has_dotdot_component "$rest" && return 0
- [[ "$rest" =~ $_DOT_GLOB ]] && return 0
- return 1
- ;;
- TMPDIR) "$multi" || return 1 ;;
- HOME)
- if ! "$multi"; then
- _has_dotdot_component "$rest" || return 1
- _dotdot_escapes_base "${HOME:-}" "$rest"
- return
- fi
- ;;
- esac
- _has_dotdot_component "$rest" && return 0
- if [[ "$rest" == /* ]]; then
- r="$rest"
- while [[ "$r" == /* || "$r" == ./* ]]; do r="${r#/}"; r="${r#./}"; done
- case "$r" in ''|'*'*|'?'*|'['*|'{'*) return 0 ;; esac
- # An unset variable makes the rest an absolute path: deny when it
- # is, or climbs to, a temp root, or any component holds a glob.
- # The only brace left after expansion that stays ok is a numeric
- # range glued to a literal prefix in the last component.
- local lastc="${rest##*/}" headp="${rest%/*}"
- [[ "$headp" == *[\*\?\[\{]* ]] && return 0
- [[ "$lastc" == *[\*\?\[]* ]] && return 0
- if [[ "$lastc" == *"{"* ]]; then
- [[ "$lastc" =~ ^[^\{]+\{-?[0-9]+\.\.-?[0-9]+\}$ ]] || return 0
- fi
- _resolved_reaches_temp "$(_lex_resolve "$rest")" && return 0
- fi
- return 1
- fi
- # shellcheck disable=SC2088 # literal tilde in a string being inspected
- case "$a" in
- '~'|'~/'*)
- rest="${a#\~}"
- _has_dotdot_component "$rest" || return 1
- _dotdot_escapes_base "${HOME:-}" "$rest"
- return
- ;;
- '~+'*|'~-'*)
- # PWD / OLDPWD: OLDPWD is unknown, so a glob, dotdot or bare form denies.
- rest="${a#\~?}"
- [ -z "$rest" ] && return 0
- [[ "$rest" == /* ]] || return 1
- r="${rest#/}"
- case "$r" in ''|'*'*|'?'*|'['*|'{'*) return 0 ;; esac
- _has_dotdot_component "$rest" && return 0
- return 1
- ;;
- '$'*|'{'*) return 1 ;;
- /*)
- n="$a"
- r=""
- while [ "$n" != "$r" ]; do r="$n"; n="${n//\/\//\/}"; n="${n//\/.\//\/}"; done
- if _has_dotdot_component "$n"; then
- n=$(_lex_resolve "$n")
- _resolved_reaches_temp "$n" && return 0
- fi
- n="${n#/}"
- c1="${n%%/*}"
- c2=""
- [[ "$n" == */* ]] && { c2="${n#*/}"; c2="${c2%%/*}"; }
- case "$c1" in *[\*\?\[]*) return 0 ;; esac
- case "$c2" in *[\*\?\[]*) [ "$c1" = "tmp" ] || return 0 ;; esac
- if [ "$c1" = "private" ] && [ "$c2" = "var" ]; then
- c3="${n#*/}"; c3="${c3#*/}"; c3="${c3%%/*}"
- case "$c3" in *[\*\?\[]*) return 0 ;; esac
- fi
- return 1
- ;;
- esac
- _has_dotdot_component "$a" || return 1
- _dotdot_escapes_base "$CWD" "$a"
- }
- # Over-cap groups cannot be enumerated: deny when any dot touches a brace,
- # comma, glob, bracket or variable (a possible `..`), or a `*` alternative.
- _OVERCAP_RE='\.\.|[{,]\.|\.[{},]|\.[^/,{}]*[*?[$]|[{,]\*[,}]'
- _rm_target_extra_deny() {
- local q="${1%%[<>]*}" alt rc=0
- q="${q//[\"\'\\]/}"
- local -a alts=("$q")
- if [[ "$q" == *"{"* ]]; then
- _expand_braces "$q" || rc=$?
- [ "$rc" -eq 1 ] && return 0
- # Over cap (rc 2): a `.` or `..` alternative or a dot glued to a group
- # can build a dotdot component bash will expand; fail closed.
- if [ "$rc" -eq 2 ] && [[ "$q" =~ $_OVERCAP_RE ]]; then
- return 0
- fi
- [ "$rc" -eq 0 ] && alts=(${_ALTS[@]+"${_ALTS[@]}"})
- fi
- local stripped
- for alt in ${alts[@]+"${alts[@]}"}; do
- # A comma group the expander could not peel (a parameter expansion
- # inside it) is unenumerable: fail closed.
- if [ "$rc" -eq 0 ] && [[ "$alt" == *"{"* ]]; then
- stripped="$alt"
- while [[ "$stripped" =~ \$\{[^\{\}]*\} ]]; do
- stripped="${stripped/"${BASH_REMATCH[0]}"/}"
- done
- [[ "$stripped" =~ \{[^\}]*,[^\}]*\} ]] && return 0
- fi
- _alt_extra_deny "$alt" && return 0
- done
- return 1
- }
- # P12-26g-d: allow-grammar for recursive rm targets. A target is allowed only
- # when, after quotes and braces are removed, it matches one of the shapes
- # below; anything else denies (fail closed). It runs BEFORE the older
- # temp-root checks, which still run on every target the grammar lets through.
- # (a) plain relative path: first component not a glob, never climbs above start
- # (b) absolute path outside every temp root and its ancestors, no glob in the
- # first two components (except /tmp/<name>*), no expansion inside it
- # (c) "$LOKI_RUN_TMP"[/...] with a valid LOKI_RUN_TMP
- # (d) $TMPDIR or ${TMPDIR} with no modifier, then /<non-glob name>
- # (e) $HOME or ~ then /..., staying inside HOME and off every temp root
- # (f) another named variable, then /<non-glob, non-dotdot name>
- # Denied outright: positional or special parameters, ${!X}, ${#X}, $'..',
- # $"..", command substitution, nested ${..${..}}, any ${X<modifier>} other than
- # a whole-target :- or - default, and any expansion inside an absolute path.
- # Every function below returns 0 to DENY.
- _G_CAP=64
- _g_expand() {
- local cur pre body post lo hi i span b alt
- local -a work=("$1")
- _G_ALTS=()
- while [ "${#work[@]}" -gt 0 ]; do
- cur="${work[0]}"
- work=("${work[@]:1}")
- # A leading sentinel keeps the regex free of an empty alternation.
- if [[ "@${cur}" =~ ^(.*[^$])\{([^\{\}]*)\}(.*)$ ]]; then
- pre="${BASH_REMATCH[1]#@}"; body="${BASH_REMATCH[2]}"; post="${BASH_REMATCH[3]}"
- if [[ "$body" =~ ^(-?[0-9]+)\.\.(-?[0-9]+)$ ]]; then
- lo="${BASH_REMATCH[1]}"; hi="${BASH_REMATCH[2]}"
- [ "${#lo}" -le 6 ] && [ "${#hi}" -le 6 ] || return 2
- case "$lo" in -*) lo=$(( -(10#${lo#-}) )) ;; *) lo=$(( 10#$lo )) ;; esac
- case "$hi" in -*) hi=$(( -(10#${hi#-}) )) ;; *) hi=$(( 10#$hi )) ;; esac
- span=$(( hi - lo )); [ "$span" -lt 0 ] && span=$(( -span ))
- [ "$span" -lt "$_G_CAP" ] || return 2
- i="$lo"
- while :; do
- work+=("${pre}${i}${post}")
- [ "$i" -eq "$hi" ] && break
- if [ "$lo" -le "$hi" ]; then i=$(( i + 1 )); else i=$(( i - 1 )); fi
- done
- elif [[ "$body" == *,* ]]; then
- b="$body"
- while :; do
- alt="${b%%,*}"
- work+=("${pre}${alt}${post}")
- [[ "$b" == *,* ]] || break
- b="${b#*,}"
- done
- else
- return 1
- fi
- else
- [[ "$cur" =~ (^|[^$])\{ ]] && return 1
- _G_ALTS+=("$cur")
- fi
- [ $(( ${#work[@]} + ${#_G_ALTS[@]} )) -le "$_G_CAP" ] || return 2
- done
- return 0
- }
- # After the first variable, a component that is exactly $name or ${name}
- # (not TMPDIR or HOME) is folded to a plain word; the glob and dotdot checks
- # then judge the folded path.
- _g_fold_vars() {
- local rest="$1" c out="" n
- local -a p=()
- [[ "$rest" == /* ]] || { printf '%s' "$rest"; return; }
- IFS=/ read -r -a p <<< "${rest#/}"
- for c in ${p[@]+"${p[@]}"}; do
- if [[ "$c" =~ ^\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?$ ]]; then
- n="${BASH_REMATCH[1]}"
- case "$n" in TMPDIR|HOME) ;; *) c="v" ;; esac
- fi
- out="${out}/${c}"
- done
- [[ "$rest" == */ ]] && out="${out}/"
- printf '%s' "$out"
- }
- # Rest after a variable: "/" then a non-glob first component, and no
- # dotdot-capable component anywhere.
- _g_named_rest_deny() {
- local rest="$1" c first=true
- local -a p=()
- [[ "$rest" == /* ]] || return 0
- IFS=/ read -r -a p <<< "${rest#/}"
- [ "${#p[@]}" -gt 0 ] || return 0
- for c in "${p[@]}"; do
- _is_dotdotish "$c" && return 0
- if $first; then
- [ -n "$c" ] || return 0
- [[ "$c" == *[\*\?\[]* ]] && return 0
- first=false
- fi
- done
- return 1
- }
- _g_home_deny() {
- local rest="$1" home="${HOME:-}" hr res
- [[ "$home" == /* ]] || return 0
- [ -n "$rest" ] || return 0
- [[ "$rest" == /* ]] || return 0
- hr=$(_lex_resolve "$home")
- [ "$hr" != "/" ] || return 0
- _resolved_reaches_temp "$hr" && return 0
- res=$(_lex_resolve "${hr}${rest}")
- _resolved_reaches_temp "$res" && return 0
- [[ "$res" == "$hr"/* ]] || return 0
- return 1
- }
- _g_abs_deny() {
- local a="$1" r n c1 c2 c3
- [[ "$a" == *'$'* ]] && return 0
- [[ "$a" =~ $_DOT_GLOB ]] && return 0
- r=$(_lex_resolve "$a")
- _resolved_reaches_temp "$r" && return 0
- n="${r#/}"
- c1="${n%%/*}"
- c2=""
- [[ "$n" == */* ]] && { c2="${n#*/}"; c2="${c2%%/*}"; }
- case "$c1" in *[\*\?\[]*) return 0 ;; esac
- case "$c2" in *[\*\?\[]*) [ "$c1" = "tmp" ] || return 0 ;; esac
- if [ "$c1" = "private" ] && [ "$c2" = "var" ]; then
- c3="${n#*/}"; c3="${c3#*/}"; c3="${c3%%/*}"
- case "$c3" in *[\*\?\[]*) return 0 ;; esac
- fi
- return 1
- }
- _g_rel_deny() {
- local a="$1" c depth=0 seen=false
- local -a p=()
- [[ "$a" == *'$'* ]] && return 0
- IFS=/ read -r -a p <<< "$a"
- for c in ${p[@]+"${p[@]}"}; do
- case "$c" in
- '') continue ;;
- .) seen=true; continue ;;
- ..) depth=$(( depth - 1 )); [ "$depth" -ge 0 ] || return 0; continue ;;
- esac
- _is_dotdotish "$c" && return 0
- if ! $seen; then
- seen=true
- case "$c" in [\*\?\[]*) return 0 ;; esac
- fi
- depth=$(( depth + 1 ))
- done
- [ "$depth" -gt 0 ] && return 1
- return 0
- }
- _g_alt_deny() {
- local a="$1" name rest w
- [ -n "$a" ] || return 0
- # Whole-target default: ${N:-W} or ${N-W}, N not TMPDIR, W a variable, a
- # relative word with no leading glob, or an allowed absolute path.
- if [[ "$a" =~ ^\$\{([A-Za-z_][A-Za-z0-9_]*)(:-|-)(.*)\}$ ]]; then
- name="${BASH_REMATCH[1]}"; w="${BASH_REMATCH[3]}"
- [ "$name" != "TMPDIR" ] && [ "$name" != "HOME" ] || return 0
- [ -n "$w" ] || return 0
- if [[ "$w" =~ ^\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?$ ]]; then
- [ "${BASH_REMATCH[1]}" != "TMPDIR" ] && [ "${BASH_REMATCH[1]}" != "HOME" ] || return 0
- return 1
- fi
- [[ "$w" == *'$'* || "$w" == *'{'* || "$w" == *'}'* ]] && return 0
- _g_alt_deny "$w"
- return
- fi
- # ${TMPDIR:-<temp root>}/<named child>
- if [[ "$a" =~ ^\$\{TMPDIR:-(/[^\}\$]*)\}(/.*)$ ]]; then
- w="${BASH_REMATCH[1]}"; rest="${BASH_REMATCH[2]}"
- case "$(_lex_resolve "$w")" in
- /tmp|/var/tmp|/private/tmp|/private/var/tmp) ;;
- *) return 0 ;;
- esac
- [[ "$rest" == *'$'* ]] && return 0
- _g_named_rest_deny "$rest"
- return
- fi
- # ${N:?} and ${N:?msg} guards behave exactly like a bare ${N}.
- if [[ "$a" =~ ^\$\{([A-Za-z_][A-Za-z0-9_]*):\?[^\}\$\{]*\}(.*)$ ]]; then
- a="\${${BASH_REMATCH[1]}}${BASH_REMATCH[2]}"
- fi
- # shellcheck disable=SC2088 # literal tilde in a string being inspected
- case "$a" in
- '$'*)
- if [[ "$a" =~ ^\$\{([A-Za-z_][A-Za-z0-9_]*)\}(.*)$ ]] || [[ "$a" =~ ^\$([A-Za-z_][A-Za-z0-9_]*)(.*)$ ]]; then
- name="${BASH_REMATCH[1]}"; rest="${BASH_REMATCH[2]}"
- else
- return 0
- fi
- case "$name" in
- LOKI_RUN_TMP|HOME|TMPDIR) ;;
- *) rest=$(_g_fold_vars "$rest") ;;
- esac
- [[ "$rest" == *'$'* || "$rest" == *'{'* || "$rest" == *'}'* ]] && return 0
- [ -z "$rest" ] || [[ "$rest" == /* ]] || return 0
- case "$name" in
- LOKI_RUN_TMP)
- _run_tmp_valid || return 0
- [ -z "$rest" ] && return 1
- _has_dotdot_component "$rest" && return 0
- [[ "$rest" =~ $_DOT_GLOB ]] && return 0
- return 1
- ;;
- HOME) _g_home_deny "$rest"; return ;;
- *)
- # A whole-target named variable (rm -r "$dir") is allowed;
- # TMPDIR bare names the temp root itself.
- if [ -z "$rest" ]; then
- [ "$name" != "TMPDIR" ] || return 0
- return 1
- fi
- _g_named_rest_deny "$rest"
- return
- ;;
- esac
- ;;
- '~'|'~/'*)
- [[ "$a" == *'$'* ]] && return 0
- _g_home_deny "${a#\~}"
- return
- ;;
- '~'*|'{'*|*'{'*|*'}'*) return 0 ;;
- /*) _g_abs_deny "$a"; return ;;
- esac
- _g_rel_deny "$a"
- }
- _rm_grammar_deny() {
- local t="${1%%[<>]*}" q alt rc=0
- [ -n "$t" ] || return 1
- # $' and $" open ANSI-C / locale quoting; a $$ before the quote is a PID.
- [[ "$t" =~ (^|[^$])\$[\'\"] ]] && return 0
- q="${t//[\"\'\\]/}"
- # find -exec rm -r {} + / \; : a bare {} is find's placeholder, not a brace.
- [ "$q" = "{}" ] && return 1
- [[ "$q" == *'`'* || "$q" == *'$('* ]] && return 0
- # $$ is a PID only mid-component (/tmp/test-$$); a leading $$ denies below.
- while [[ "$q" =~ ^(.*[^/\$])\$\$(.*)$ ]]; do q="${BASH_REMATCH[1]}0${BASH_REMATCH[2]}"; done
- local -a alts=("$q")
- if [[ "$q" == *"{"* ]]; then
- _g_expand "$q" || rc=$?
- [ "$rc" -eq 1 ] && return 0
- if [ "$rc" -eq 2 ]; then
- # Over the cap: only a named relative first component, no
- # expansion and no dot trick, can be left unexpanded.
- [[ "$q" =~ ^[A-Za-z0-9_-][A-Za-z0-9._-]*/ ]] || return 0
- [[ "$q" == *'$'* ]] && return 0
- [[ "$q" =~ $_OVERCAP_RE ]] && return 0
- return 1
- fi
- alts=(${_G_ALTS[@]+"${_G_ALTS[@]}"})
- fi
- for alt in ${alts[@]+"${alts[@]}"}; do
- _g_alt_deny "$alt" && return 0
- done
- return 1
- }
- # `rm`, `/bin/rm`, `\rm`, `"rm"` all name rm.
- _is_rm_token() {
- local x="${1//[\"\'\\]/}"
- [ "${x##*/}" = "rm" ]
- }
- _segment_rm_hits_temp_root() {
- local seg="$1" tok recursive=false seen_rm=false
- local -a words
- # Command substitution and grouping open a new command word: split on them.
- [[ "$seg" == *'`'* ]] && seg="${seg//\`/ \$( }"
- seg="${seg//[()\`]/ }"
- read -r -a words <<< "$seg"
- for tok in "${words[@]}"; do
- if ! $seen_rm; then
- _is_rm_token "$tok" && seen_rm=true
- continue
- fi
- case "$tok" in
- --recursive) recursive=true ;;
- -*) [[ "$tok" =~ ^-[[:alpha:]]*[rR] ]] && recursive=true ;;
- esac
- done
- $seen_rm && $recursive || return 1
- seen_rm=false
- for tok in "${words[@]}"; do
- if ! $seen_rm; then
- _is_rm_token "$tok" && seen_rm=true
- continue
- fi
- case "$tok" in -*) continue ;; esac
- [ -n "${tok//[\"\'\\]/}" ] || continue
- # Fail closed before the hook timeout (Claude Code treats it as allow).
- [ $((SECONDS - _RM_T0)) -ge "$_RM_BUDGET" ] && return 0
- _rm_grammar_deny "$tok" && return 0
- _rm_target_is_temp_root_or_glob "$tok" && return 0
- _rm_target_extra_deny "$tok" && return 0
- done
- return 1
- }
- _RM_T0=$SECONDS
- _RM_BUDGET="${LOKI_RM_GUARD_BUDGET_SECS:-20}"
- case "$_RM_BUDGET" in ""|*[!0-9]*) _RM_BUDGET=20 ;; esac
- # P12-26g-d: inspect only real command words. Quoted data of a final-in-pipeline
- # printf/echo/cat (and a cat heredoc body) is masked; any construct that can run
- # data as code (eval, shells, $( ), backticks, pipes into them) disables masking.
- _INSPECT=$(_LOKI_MASK_CMD="$COMMAND" python3 - <<'PY'
- import os
- import re
- import sys
- cmd = os.environ.get("_LOKI_MASK_CMD", "")
- DATA = {"printf", "echo", "cat"}
- RISKY_WORD = re.compile(
- r"(^|[^\w./-])(eval|source|sh|bash|zsh|dash|ksh|xargs|exec|python3?|perl|ruby|node|ssh|find|parallel)(?![\w-])"
- )
- DOT_CMD = re.compile(r"(^|[;&|\n(]\s*)\.\s")
- def emit(text):
- sys.stdout.write(text)
- raise SystemExit(0)
- # Any construct that can turn data into code disables masking entirely.
- # ANSI-C / locale quoting ($' $") hides quote boundaries from this walker.
- if re.search(r"\x60|\$\(|<\(|>\(|\$['\"]", cmd) or RISKY_WORD.search(cmd) or DOT_CMD.search(cmd):
- emit(cmd)
- n = len(cmd)
- out = list(cmd)
- HEREDOC = re.compile(r"(?<!<)<<(?!<)-?\s*([\"']?)([A-Za-z_]\w*)\1")
- ASSIGN = re.compile(r"^\s*[A-Za-z_]\w*=\S*\s+")
- def is_data(text):
- while ASSIGN.match(text):
- text = ASSIGN.sub("", text, count=1)
- m = re.match(r"\s*(\S+)", text)
- if not m or m.group(1) not in DATA:
- return False
- if m.group(1) == "printf" and re.search(r"(^|\s)-v", text):
- return False
- return True
- def mask(s, e):
- for k in range(s, e):
- if out[k] != "\n":
- out[k] = "x"
- i = 0
- unit_start = 0
- spans = []
- pending = [] # (delimiter, strip_tabs, mask_body)
- unit_hd = [] # heredocs opened by an unquoted << in the current unit
- def end_unit(end, piped):
- data = is_data(cmd[unit_start:end]) and not piped
- if data:
- for s, e in spans:
- mask(s, e)
- for delim, strip in unit_hd:
- pending.append((delim, strip, data))
- while i < n:
- c = cmd[i]
- if c == "\\":
- i += 2
- continue
- if c == "#" and (i == 0 or cmd[i - 1] in " \t;&|()\n"):
- j = cmd.find("\n", i)
- i = n if j < 0 else j
- continue
- if c == "<" and cmd.startswith("<<", i):
- m = HEREDOC.match(cmd, i)
- if m:
- unit_hd.append((m.group(2), cmd.startswith("<<-", i)))
- i = m.end()
- continue
- if c == "'":
- j = cmd.find("'", i + 1)
- if j < 0:
- emit(cmd)
- spans.append((i + 1, j))
- i = j + 1
- continue
- if c == '"':
- j = i + 1
- while j < n and cmd[j] != '"':
- j += 2 if cmd[j] == "\\" else 1
- if j >= n:
- emit(cmd)
- spans.append((i + 1, j))
- i = j + 1
- continue
- if c in ";&|\n":
- piped = c == "|" and cmd[i + 1:i + 2] != "|"
- end_unit(i, piped)
- spans = []
- unit_hd = []
- if c == "\n" and pending:
- pos = i + 1
- for delim, strip, do_mask in pending:
- while pos < n:
- nl = cmd.find("\n", pos)
- stop = n if nl < 0 else nl
- line = cmd[pos:stop]
- if (line.lstrip("\t") if strip else line) == delim:
- pos = stop
- break
- if do_mask:
- mask(pos, stop)
- pos = stop + 1
- pending = []
- i = pos
- unit_start = i
- continue
- i += 1
- unit_start = i
- continue
- i += 1
- end_unit(n, False)
- emit("".join(out))
- PY
- ) || _INSPECT="$COMMAND"
- [ -n "$_INSPECT" ] || _INSPECT="$COMMAND"
- _flat_for_temp_root=$(printf '%s' "$_INSPECT" | sed -E 's/[0-9]*>&[0-9]+//g' | tr ';&|\r' '\n')
- while IFS= read -r _seg; do
- if _segment_rm_hits_temp_root "$_seg"; then
- deny "Blocked: recursive delete of a temp root or a bare glob under it"
- fi
- done <<< "$_flat_for_temp_root"
- # Check for blocked patterns
- for pattern in "${BLOCKED_PATTERNS[@]}"; do
- if echo "$COMMAND" | grep -qE "$pattern"; then
- _blocked_hits_all_safe "$pattern" && continue
- deny "Blocked: potentially dangerous command pattern detected"
- fi
- done
- # Log command to audit trail
- LOG_DIR="$CWD/.loki/logs"
- mkdir -p "$LOG_DIR"
- printf '%s' "{\"timestamp\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"command\":$(echo "$COMMAND" | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))')}" >> "$LOG_DIR/bash-audit.jsonl"
- echo >> "$LOG_DIR/bash-audit.jsonl"
- # Allow command. Compiled policy: no decision, the permission flow decides.
- if "$COMPILED_POLICY"; then
- exit 0
- fi
- printf '%s' '{"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "allow"}}'
- exit 0
autonomy/lib/voter-agents.sh · Bundled script · 584 lines · 1 finding marked
- #!/usr/bin/env bash
- # autonomy/lib/voter-agents.sh -- Phase C (v7.5.20) bash side.
- #
- # Builds the JSON declaration of the council voter agents and orchestrates a
- # single `claude --agents <json> --json-schema <inline-content>` dispatch per iteration.
- # Replaces the ad-hoc per-voter heuristic loop in council_aggregate_votes when
- # the locally installed Claude CLI supports both flags.
- #
- # Vote enum (per architect, BINDING):
- # APPROVE | REJECT | CANNOT_VALIDATE
- #
- # Translation to the legacy aggregator shape used by council_evaluate and
- # council_write_transcript:
- # APPROVE -> COMPLETE
- # REJECT -> CONTINUE
- # CANNOT_VALIDATE -> CONTINUE (mirrors managed council _vote_to_legacy)
- #
- # Public API (all functions are pure: read env + filesystem, write stdout).
- # loki_voter_agents_json -- emits the 3-voter agents JSON
- # loki_devils_advocate_json <summary> -- emits a one-key JSON for the DA
- # loki_finding_schema_path -- echoes absolute path to schema
- # loki_council_dispatch_agents <iter> <prd_path>
- # -- runs the single claude call,
- # writes verdict files and the
- # aggregator round file. Returns 0
- # on success, 1 on any fallback
- # condition (caller falls through
- # to existing heuristic dispatch).
- #
- # Inline Python: the dispatch runs with the cwd inside the agent's repo, so
- # every python3 here drops '' and '.' from sys.path before any other import; a
- # committed json.py or sitecustomize.py cannot read the votes.
- # Pinned by tests/test-voter-agents-json.sh (Case D).
- #
- # S-202 (BACKLOG 54): -E alone still loads the user site-packages, whose .pth
- # "import" lines run before that scrub and can forge json.loads (3 REJECT
- # findings read as a COMPLETE round). Every call site resolves its interpreter
- # through _loki_snapshot_py_tool and runs it -I -S; when none resolves, the
- # dispatch returns 1 and completion-council's fail-closed path applies. run.sh
- # is the source of truth for the helper; the guarded copy below only serves
- # callers that source this file on its own. Keep it byte-identical to run.sh's.
- # Pinned by tests/test-voter-agents-no-user-site-pth.sh.
- # Guard against double-source.
- if [ "${__LOKI_VOTER_AGENTS_SH_LOADED:-0}" = "1" ]; then
- return 0 2>/dev/null || true
- fi
- __LOKI_VOTER_AGENTS_SH_LOADED=1
- declare -F _loki_snapshot_py_tool >/dev/null 2>&1 || \
- _loki_snapshot_py_tool() {
- local c
- for c in /usr/bin/python3 /bin/python3; do
- [ -x "$c" ] && [ ! -d "$c" ] && "$c" -I -S -c '' >/dev/null 2>&1 && { printf '%s\n' "$c"; return 0; }
- done
- local dir
- local IFS=:
- for dir in $PATH; do
- case "$dir" in
- /*) ;;
- *) continue ;;
- esac
- if [ -x "$dir/python3" ] && [ ! -d "$dir/python3" ] \
- && "$dir/python3" -I -S -c '' >/dev/null 2>&1; then
- printf '%s\n' "$dir/python3"
- return 0
- fi
- done
- return 1
- }
- # Resolve repo root once. This file lives at autonomy/lib/voter-agents.sh.
- __LOKI_VA_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" 2>/dev/null && pwd)"
- __LOKI_VA_REPO_ROOT="$(cd "${__LOKI_VA_LIB_DIR}/../.." 2>/dev/null && pwd)"
- # ---------- Voter agents JSON ----------
- # Emits a top-level JSON object with exactly 3 keys (the 3 base voters).
- # Required env:
- # LOKI_ITER -- iteration number (int-ish)
- # LOKI_PRD_PATH -- PRD path (may be empty)
- # Optional env:
- # LOKI_RARV_TIER -- planning|development|fast (informational only)
- #
- # Returns 0 always. Emits `{}` when required env missing.
- loki_voter_agents_json() {
- local iter="${LOKI_ITER:-}"
- if [ -z "$iter" ]; then
- printf '%s' '{}'
- return 0
- fi
- local prd="${LOKI_PRD_PATH:-}"
- local tier="${LOKI_RARV_TIER:-development}"
- local _va_py
- _va_py="$(_loki_snapshot_py_tool)" || { printf '%s' '{}'; return 0; }
- _VA_ITER="$iter" \
- _VA_PRD="$prd" \
- _VA_TIER="$tier" \
- _VA_COMPLEXITY="${LOKI_COMPLEXITY:-standard}" \
- "$_va_py" -I -S -c 'import sys; sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- import json, os
- iter_n = os.environ.get("_VA_ITER", "0")
- prd = os.environ.get("_VA_PRD", "")
- tier = os.environ.get("_VA_TIER", "development")
- complexity = os.environ.get("_VA_COMPLEXITY", "standard")
- # Effort selection mirrors Bun route loki-ts/src/providers/claude_flags.ts::effortForTier
- # so bash and Bun emit the same effort levels under all LOKI_COMPLEXITY values.
- # Architect roster (standard complexity): requirements-verifier=high, test-auditor=high,
- # convergence-voter=medium. Under complex, each shifts up one notch (medium->high, high->xhigh).
- _effort_bump = {"low": "medium", "medium": "high", "high": "xhigh"}
- def _effort(base):
- return _effort_bump.get(base, base) if complexity == "complex" else base
- # Cost S7: the three base voters run on haiku for small/standard work; the
- # complex tier keeps the original opus/sonnet roster. haiku takes no effort knob.
- _lean = complexity != "complex"
- def _model(default):
- return "haiku" if _lean else default
- prd_clause = (
- f"PRD at {prd}. Read it for ground-truth requirements."
- if prd else
- "No PRD path provided; rely on session context."
- )
- agents = {
- "requirements-verifier": {
- "description": "Verifies the iteration delivered the PRD-required behavior.",
- "model": _model("opus"),
- "tools": ["Read", "Grep", "Bash"],
- "prompt": (
- f"You are the requirements-verifier for iteration {iter_n}. "
- f"{prd_clause} Compare current code+tests against requirements. "
- "Emit a single finding object with role=requirements-verifier, "
- "vote in {APPROVE,REJECT,CANNOT_VALIDATE}, terse reason, "
- "confidence float in [0,1]."
- ),
- },
- "test-auditor": {
- "description": "Audits test coverage and pass status for the iteration.",
- "model": _model("sonnet"),
- "tools": ["Read", "Grep", "Bash"],
- "prompt": (
- f"You are the test-auditor for iteration {iter_n}. "
- "Inspect test logs, coverage, and pass/fail counts. "
- "Emit a single finding object with role=test-auditor, "
- "vote in {APPROVE,REJECT,CANNOT_VALIDATE}, terse reason, "
- "confidence float in [0,1]."
- ),
- },
- "convergence-voter": {
- "description": "Checks code churn and progress signals to judge convergence.",
- "model": _model("sonnet"),
- "tools": ["Read", "Grep", "Bash"],
- "prompt": (
- f"You are the convergence-voter for iteration {iter_n} on tier {tier}. "
- "Examine git diff trends, queue depth, and stagnation signals. "
- "Emit a single finding object with role=convergence-voter, "
- "vote in {APPROVE,REJECT,CANNOT_VALIDATE}, terse reason, "
- "confidence float in [0,1]."
- ),
- },
- }
- if not _lean:
- for _k, _b in (("requirements-verifier", "high"), ("test-auditor", "high"), ("convergence-voter", "medium")):
- agents[_k]["effort"] = _effort(_b)
- print(json.dumps(agents, separators=(",", ":")))
- '
- return 0
- }
- # ---------- Devil's advocate agent JSON ----------
- # Emits a single-key JSON object describing the conditional 4th voter.
- # Arg $1: terse base findings summary used to brief the DA prompt.
- loki_devils_advocate_json() {
- local summary="${1:-}"
- local _va_py
- _va_py="$(_loki_snapshot_py_tool)" || { printf '%s' '{}'; return 0; }
- _VA_SUMMARY="$summary" "$_va_py" -I -S -c 'import sys; sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- import json, os
- summary = os.environ.get("_VA_SUMMARY", "")[:1000]
- agents = {
- "devils-advocate": {
- "description": "Skeptical re-review when the first three voters unanimously approve.",
- "model": "opus",
- "effort": "xhigh",
- "tools": ["Read", "Grep", "Bash"],
- "prompt": (
- "You are the devils-advocate. The first three voters unanimously "
- "voted APPROVE. Find the strongest reason this is wrong. "
- "Base voter summary:\n"
- f"{summary}\n"
- "Emit one finding with role=devils-advocate, "
- "vote in {APPROVE,REJECT,CANNOT_VALIDATE}, terse reason, "
- "confidence float in [0,1]."
- ),
- },
- }
- print(json.dumps(agents, separators=(",", ":")))
- '
- return 0
- }
- # ---------- Finding schema path ----------
- # Echo the absolute path to the schema file Dev-A maintains.
- # Returns 1 if the file is missing; the path string is still printed so callers
- # and tests can assert on the resolved location.
- loki_finding_schema_path() {
- local schema="${__LOKI_VA_REPO_ROOT}/loki-ts/data/finding-schema.json"
- printf '%s' "$schema"
- if [ -f "$schema" ]; then
- return 0
- fi
- return 1
- }
- # ---------- Council dispatch via --agents + --json-schema ----------
- # Orchestrator. Args:
- # $1 -- iteration number (required, int-ish)
- # $2 -- prd path (may be empty)
- #
- # Behavior:
- # 1. Source autonomy/lib/claude-flags.sh; require both --agents and
- # --json-schema in `claude --help`. Either missing -> return 1 (fallback).
- # 2. Build the agents JSON via loki_voter_agents_json.
- # 3. Invoke `claude --dangerously-skip-permissions -p <prompt>
- Note: Starts a coding agent with its permission prompts off
- Line 223 on GitHub
- # --agents <json> --json-schema <inline-content>` and capture stdout.
- # Any non-zero exit or empty stdout -> return 1 (fallback).
- # 4. Parse the response via python3 (no jq dependency): extract
- # findings[].{role, vote, reason, confidence}. On parse failure -> 1.
- # 5. Write per-voter verdicts/<role>-iter-<iter>.json AND the aggregator
- # round file votes/round-<iter>.json in the existing shape consumed by
- # council_evaluate + council_write_transcript.
- # 6. Return 0 on success.
- loki_council_dispatch_agents() {
- local iteration="${1:-}"
- local prd_path="${2:-}"
- if [ -z "$iteration" ]; then
- return 1
- fi
- if [ -z "${COUNCIL_STATE_DIR:-}" ]; then
- return 1
- fi
- # 1a. Custom council size gate (added per v7.5.21 Phase C reviewer council
- # Opus #2 HIGH finding): the Phase C dispatch hardcodes 3 voters, so when a
- # user sets LOKI_COUNCIL_SIZE != 3 (documented public env in Docker README,
- # wiki Environment-Variables, certification lesson), the dispatch helper
- # would silently emit total_members=3 + threshold=2, breaking the unanimous
- # devil's-advocate check at completion-council.sh that compares
- # complete_count -eq COUNCIL_SIZE. Cheapest correct fix: fall back to the
- # heuristic path which respects COUNCIL_SIZE, until a multi-voter dispatch
- # ships in a later phase. Default COUNCIL_SIZE=3 -> Phase C dispatch active.
- if [ "${COUNCIL_SIZE:-3}" != "3" ]; then
- return 1
- fi
- # 1b. Managed-council bypass gate (Opus #2 MEDIUM finding): if the
- # experimental managed council is enabled, defer to it -- otherwise its
- # member output would be shadowed by the round file the dispatch writes.
- if [ "${LOKI_EXPERIMENTAL_MANAGED_COUNCIL:-false}" = "true" ]; then
- return 1
- fi
- # 1c. Flag support gate.
- # shellcheck disable=SC1091
- . "${__LOKI_VA_LIB_DIR}/claude-flags.sh" 2>/dev/null || return 1
- if ! loki_claude_flag_supported "--agents"; then
- return 1
- fi
- if ! loki_claude_flag_supported "--json-schema"; then
- return 1
- fi
- # 2. Build agents JSON. Empty result means we cannot proceed.
- local agents_json
- agents_json=$(LOKI_ITER="$iteration" LOKI_PRD_PATH="$prd_path" \
- LOKI_RARV_TIER="${LOKI_RARV_TIER:-development}" \
- loki_voter_agents_json)
- if [ -z "$agents_json" ] || [ "$agents_json" = "{}" ]; then
- return 1
- fi
- local schema_path
- schema_path=$(loki_finding_schema_path) || return 1
- # v8.x FIX: `claude --json-schema` takes the schema JSON as INLINE CONTENT,
- # not a file path. Live-verified on CLI 2.1.207: passing a path errors with
- # "--json-schema is not valid JSON: Unrecognized token '/'" -> rc!=0 ->
- # fail-closed to the heuristic path, so this structured dispatch was SILENTLY
- # DEAD (never actually schema-constrained the council). Read the file and pass
- # its content. Fail-closed if unreadable/empty (return 1 -> heuristic path).
- local schema_content
- schema_content=$(cat "$schema_path" 2>/dev/null) || return 1
- [ -n "$schema_content" ] || return 1
- local prompt
- prompt=$(printf 'Loki council iteration %s. Run each declared agent against the current workspace, return one finding per agent matching the provided JSON Schema. Be terse, be honest.' "$iteration")
- # Cost S7: hand the council the PRD path so voters stop reporting "No PRD was
- # provided". A missing/unreadable PRD adds nothing; voters then vote
- # CANNOT_VALIDATE, which never approves.
- if [ -n "$prd_path" ] && [ -r "$prd_path" ]; then
- prompt="${prompt} PRD file: ${prd_path} (read it for ground-truth requirements)."
- fi
- # Shared dispatch state (declared once, before either the SDK or claude arm).
- local response=""
- local rc=0
- local stderr_log="$COUNCIL_STATE_DIR/votes/dispatch-stderr-${iteration}.log"
- mkdir -p "$(dirname "$stderr_log")" 2>/dev/null || true
- local _va_sdk_done=0
- # v8 RAW-SDK COUNCIL PATH (opt-in LOKI_SDK_VOTER_AGENTS=1). The bash route
- # uses `claude --agents <json> --json-schema` (one dispatch fanning out to N
- # named agents). The raw @anthropic-ai/sdk has no --agents primitive, so we
- # inline the agent roster into the prompt and constrain the SAME
- # finding-schema.json via the sdk-judge bridge -- the model returns the same
- # {findings:[...]} object the downstream parser (line 302+) consumes, one
- # finding per declared agent. Runs BEFORE the claude-binary guard so the
- # no-binary deploy win holds. Fail-closed: on ANY miss (flag off, no key,
- # bun/entrypoint absent, non-zero, empty) fall through to the claude path
- # below, then to the heuristic council on a claude miss -- a hung/failed
- # council can never become a false COMPLETE (same guarantee as the claude arm).
- if [ "${LOKI_SDK_VOTER_AGENTS:-0}" = "1" ]; then
- local _va_root _va_loki _va_schema_f _va_pf _va_out _va_rc
- _va_root="$(cd "${__LOKI_VA_REPO_ROOT}" 2>/dev/null && pwd)" || _va_root="${__LOKI_VA_REPO_ROOT}"
- _va_loki="${_va_root}/bin/loki"
- _va_schema_f="${_va_root}/loki-ts/data/finding-schema.json"
- if [ -x "$_va_loki" ] && [ -f "$_va_schema_f" ] && command -v bun >/dev/null 2>&1; then
- _va_pf="$(mktemp 2>/dev/null)" || _va_pf=""
- if [ -n "$_va_pf" ]; then
- # inline the agent roster so a single schema-constrained call
- # produces one finding per declared council agent.
- printf '%s\n\nDeclared council agents (produce exactly one finding per agent, in this order):\n%s\n' \
- "$prompt" "$agents_json" > "$_va_pf"
- _va_rc=0
- local _va_to_s="${LOKI_COUNCIL_REVIEW_TIMEOUT:-600}"
- local _va_wrap
- if command -v timeout >/dev/null 2>&1; then _va_wrap="timeout $(( _va_to_s + 15 ))"
- elif command -v gtimeout >/dev/null 2>&1; then _va_wrap="gtimeout $(( _va_to_s + 15 ))"
- else _va_wrap=""; fi
- _va_out="$($_va_wrap "$_va_loki" internal sdk-judge \
- --prompt-file "$_va_pf" --schema-file "$_va_schema_f" \
- --model "${LOKI_SDK_COUNCIL_MODEL:-sonnet}" --effort high \
- --timeout-ms "$(( _va_to_s * 1000 ))" 2>"$COUNCIL_STATE_DIR/votes/dispatch-stderr-${iteration}.log")" || _va_rc=$?
- rm -f "$_va_pf" 2>/dev/null || true
- if [ "$_va_rc" -eq 0 ] && [ -n "$_va_out" ]; then
- # feed the SDK object into the SAME parse+materialize path
- response="$_va_out"
- rc=0
- _va_sdk_done=1 # skip the claude dispatch below
- fi
- fi
- fi
- # fall through to the claude path (fail-closed) if not _va_sdk_done
- fi
- # 3. Invoke claude (only when the SDK path did not already produce a response).
- # Guard against absent binary or non-zero exit.
- if [ "$_va_sdk_done" != "1" ]; then
- # Only the provider the operator chose may see a prompt (V10 P5): with any
- # other provider this falls back to the heuristic council. Policy (and the
- # LOKI_ALLOW_CLAUDE_SIDECALLS=1 opt-in) lives in providers/loader.sh.
- # ponytail: upgrade path is a provider-neutral structured dispatch.
- type loki_claude_sidecall_allowed >/dev/null 2>&1 \
- || . "${__LOKI_VA_REPO_ROOT}/providers/loader.sh" 2>/dev/null
- loki_claude_sidecall_allowed 2>/dev/null || return 1
- command -v claude >/dev/null 2>&1 || return 1
- # caveman HARD-SUPPRESS (parsed output, v7.41.0): the response is parsed for
- # findings[].vote against the JSON Schema. A globally-active caveman would
- # compress/reword it and break the schema match or flip a vote. The tree-wide
- # default-off export in claude-flags.sh (sourced above) already covers this;
- # the inline prefix is belt-and-suspenders, self-documenting, and a no-op when
- # caveman is absent.
- #
- # timeout-guard the dispatch (parity with the heuristic council path in
- # completion-council.sh:2074 and :2200, same LOKI_COUNCIL_REVIEW_TIMEOUT:-600
- # knob). The heuristic loop this dispatch replaced wrapped every claude
- # subcall in `timeout`; this helper did not, so a hung `claude --agents`
- # would stall the entire run indefinitely. `timeout` precedes the env-var
- # assignment, so `env` is used to set CAVEMAN_DEFAULT_MODE for the child.
- # On timeout, `timeout` exits 124; that exit is the command-substitution exit
- # (no pipe here), captured into rc via `|| rc=$?`. The existing rc check below
- # then routes any non-zero exit (timeout 124 or any other claude failure) to
- # `return 1` -- the heuristic fallback the caller falls through to
- # (completion-council.sh:2792). Fail-closed: a hung or timed-out council can
- # never become a false COMPLETE; it always degrades to the heuristic path
- # (which has its own timeout + conservative defaults).
- _LOKI_PERM_ARGV=(--dangerously-skip-permissions)
- ! type _loki_claude_permission_argv >/dev/null 2>&1 || _loki_claude_permission_argv
- # Cost S7 lean child: council role flags (no hooks/MCP/tools on the main
- # child) and --output-format json so usage lands in the child ledger. The
- # envelope is unwrapped back to the {findings:[...]} object below.
- local _va_lean=() _va_lf
- if type loki_lean_child_flags >/dev/null 2>&1; then
- while IFS= read -r _va_lf; do
- [ -n "$_va_lf" ] && _va_lean+=("$_va_lf")
- done < <(loki_lean_child_flags council 2>/dev/null)
- fi
- local _va_fmt=()
- if loki_claude_flag_supported "--output-format"; then
- _va_fmt=(--output-format json)
- fi
- response=$(timeout "${LOKI_COUNCIL_REVIEW_TIMEOUT:-600}" \
- env "SHELL=$(_loki_claude_shell_value 2>/dev/null || printf '%s' "${SHELL-}")" CAVEMAN_DEFAULT_MODE=off claude "${_LOKI_PERM_ARGV[@]}" \
- ${_va_lean[@]+"${_va_lean[@]}"} \
- ${_va_fmt[@]+"${_va_fmt[@]}"} \
- -p "$prompt" \
- --agents "$agents_json" \
- --json-schema "$schema_content" 2>"$stderr_log") || rc=$?
- if [ "$rc" -eq 0 ] && [ -n "$response" ] && [ "${#_va_fmt[@]}" -gt 0 ]; then
- if type loki_child_ledger_append >/dev/null 2>&1; then
- loki_child_ledger_append council "$response" >/dev/null 2>&1 || true
- fi
- local _va_unwrapped _va_upy
- if _va_upy="$(_loki_snapshot_py_tool)"; then
- # Unwrap the CLI envelope: structured_output, else a JSON string in
- # result. A bare {findings:[...]} object passes through unchanged. Any
- # other shape leaves the response as-is, so the parser fails closed.
- _va_unwrapped=$(printf '%s' "$response" | "$_va_upy" -I -S -c 'import sys; sys.path[:] = [p for p in sys.path if p not in ("", ".")]; import json
- try:
- e = json.load(sys.stdin)
- if isinstance(e, dict) and "findings" in e:
- p = e
- else:
- p = e.get("structured_output") if isinstance(e, dict) else None
- if not isinstance(p, dict):
- r = e.get("result") if isinstance(e, dict) else None
- p = json.loads(r) if isinstance(r, str) else None
- if isinstance(p, dict):
- sys.stdout.write(json.dumps(p))
- else:
- sys.exit(1)
- except Exception:
- sys.exit(1)' 2>/dev/null) || _va_unwrapped=""
- if [ -n "$_va_unwrapped" ]; then response="$_va_unwrapped"; fi
- fi
- fi
- fi # end claude-dispatch guard (SDK path skips it, already has $response)
- # Shared fail-closed check for BOTH paths: any non-zero exit (claude/SDK
- # timeout 124 or failure) or empty response -> return 1 -> heuristic fallback.
- if [ "$rc" -ne 0 ] || [ -z "$response" ]; then
- return 1
- fi
- # 4 + 5. Parse + materialize state files.
- local verdicts_dir="$COUNCIL_STATE_DIR/verdicts"
- local votes_dir="$COUNCIL_STATE_DIR/votes"
- mkdir -p "$verdicts_dir" "$votes_dir" 2>/dev/null || return 1
- local _va_py
- _va_py="$(_loki_snapshot_py_tool)" || return 1
- _VA_RESP="$response" \
- _VA_ITER="$iteration" \
- _VA_VDIR="$verdicts_dir" \
- _VA_RFILE="$votes_dir/round-${iteration}.json" \
- _VA_EXPECTED="${COUNCIL_SIZE:-3}" \
- "$_va_py" -I -S -c 'import sys; sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- import json, os, sys
- from datetime import datetime, timezone
- try:
- resp = json.loads(os.environ["_VA_RESP"])
- except Exception:
- sys.exit(2)
- findings = resp.get("findings") if isinstance(resp, dict) else None
- if not isinstance(findings, list) or not findings:
- sys.exit(3)
- it = int(os.environ.get("_VA_ITER", "0") or 0)
- vdir = os.environ["_VA_VDIR"]
- rfile = os.environ["_VA_RFILE"]
- # WAVE13 CRITICAL quorum fix: the quorum denominator MUST be the EXPECTED
- # council size (COUNCIL_SIZE), never the number of findings the model happened
- # to return. Pre-fix this parser computed threshold = (returned*2+2)//3, so a
- # degraded response with a single APPROVE finding (returned=1) yielded
- # threshold=1 and a COMPLETE verdict from a SINGLE voter, with the missing
- # voters silently dropped. That fails OPEN on the completion-detection trust
- # core. We now fail CLOSED: any undercount (returned < expected) forces a
- # CONTINUE verdict so a partial/degraded model response can never reach
- # COMPLETE on the returned subset. Design choice (Option 2): compute the
- # verdict in-path (rather than sys.exit -> heuristic fallback) so the round
- # file always records the actual returned count in total_members, making the
- # downstream quorum assertion in completion-council.sh meaningful and locally
- # testable without depending on the heuristic-path disk-state behavior.
- try:
- expected_count = int(os.environ.get("_VA_EXPECTED", "3") or 3)
- except (TypeError, ValueError):
- expected_count = 3
- if expected_count < 1:
- expected_count = 1
- def to_legacy(vote: str) -> str:
- v = (vote or "").upper()
- if v == "APPROVE":
- return "COMPLETE"
- return "CONTINUE"
- votes = []
- complete = 0
- total = 0
- for idx, f in enumerate(findings, start=1):
- if not isinstance(f, dict):
- continue
- role = str(f.get("role") or f.get("name") or f"voter-{idx}")
- raw_vote = str(f.get("vote") or "CANNOT_VALIDATE").upper()
- if raw_vote not in ("APPROVE", "REJECT", "CANNOT_VALIDATE"):
- raw_vote = "CANNOT_VALIDATE"
- reason = (f.get("reason") or "").strip()
- try:
- confidence = float(f.get("confidence", 0.0))
- except Exception:
- confidence = 0.0
- legacy = to_legacy(raw_vote)
- total += 1
- if legacy == "COMPLETE":
- complete += 1
- # Per-voter file (architect-specified shape).
- per = {
- "role": role,
- "vote": raw_vote,
- "reason": reason,
- "confidence": confidence,
- "iteration": it,
- }
- try:
- with open(os.path.join(vdir, f"{role}-iter-{it}.json"), "w") as fp:
- json.dump(per, fp, indent=2)
- except OSError:
- sys.exit(4)
- votes.append({
- "member": idx,
- "role": role,
- "vote": legacy,
- "reason": reason,
- })
- if total == 0:
- sys.exit(5)
- # Quorum-aware threshold (WAVE13). threshold is computed against the EXPECTED
- # council size so it can never shrink to 1 on a degraded response. Absent
- # voters (total < expected) are treated as non-approval: the round is forced
- # to CONTINUE and can never reach COMPLETE on the returned subset. With
- # expected=3, threshold = ceil(2/3 * 3) = 2, so 1-of-3 (or any single voter)
- # is structurally incapable of producing COMPLETE.
- threshold = (expected_count * 2 + 2) // 3
- if total != expected_count:
- # Fail closed on ANY quorum mismatch:
- # - undercount (total < expected): missing voters count as non-approval.
- # - overcount (total > expected): extra/unprompted findings (e.g. a model
- # adding a 4th finding) would otherwise let a low-approval-ratio response
- # clear the fixed threshold. A degraded response in either direction must
- # never reach COMPLETE on the returned subset.
- verdict = "CONTINUE"
- else:
- verdict = "COMPLETE" if complete >= threshold else "CONTINUE"
- round_data = {
- "round": it,
- "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
- "complete_votes": complete,
- "continue_votes": total - complete,
- # total_members records the ACTUAL number of voters that responded (not the
- # expected size) so the completion-council quorum assertion can detect an
- # undercount. expected_members records the size the verdict was judged
- # against.
- "total_members": total,
- "expected_members": expected_count,
- "threshold": threshold,
- "verdict": verdict,
- "votes": votes,
- "source": "voter-agents-dispatch",
- }
- try:
- with open(rfile, "w") as fp:
- json.dump(round_data, fp, indent=2)
- except OSError:
- sys.exit(6)
- ' || return 1
- return 0
- }
autonomy/run.sh · Bundled script · 11,065 lines · 2 findings marked
- #!/usr/bin/env bash
- # shellcheck disable=SC2034 # Many variables are used by sourced scripts
- # shellcheck disable=SC2155 # Declare and assign separately (acceptable in this codebase)
- # shellcheck disable=SC2329 # Functions may be invoked indirectly or via dynamic dispatch
- # shellcheck disable=SC2086 # Word splitting is intentional in some contexts
- #===============================================================================
- # Loki Mode - Autonomous Runner
- # Single script that handles prerequisites, setup, and autonomous execution
- #
- # Usage:
- # ./autonomy/run.sh [OPTIONS] [PRD_PATH]
- # ./autonomy/run.sh ./docs/requirements.md
- # ./autonomy/run.sh # Interactive mode
- # ./autonomy/run.sh --parallel # Parallel mode with git worktrees
- # ./autonomy/run.sh --parallel ./prd.md # Parallel mode with PRD
- #
- # Environment Variables:
- # LOKI_PROVIDER - AI provider: claude (default) or opencode
- # LOKI_MAX_RETRIES - Max retry attempts (default: 50)
- # LOKI_BASE_WAIT - Base wait time in seconds (default: 60)
- # LOKI_MAX_WAIT - Max wait time in seconds (default: 3600)
- # LOKI_SKIP_PREREQS - Skip prerequisite checks (default: false)
- # LOKI_DASHBOARD - Enable web dashboard (default: true)
- # LOKI_DASHBOARD_PORT - Dashboard port (default: 57374)
- # LOKI_TLS_CERT - Path to PEM certificate (enables HTTPS for dashboard)
- # LOKI_TLS_KEY - Path to PEM private key (enables HTTPS for dashboard)
- #
- # Resource Monitoring (prevents system overload):
- # LOKI_RESOURCE_CHECK_INTERVAL - Check resources every N seconds (default: 300 = 5min)
- # LOKI_RESOURCE_CPU_THRESHOLD - CPU % threshold to warn (default: 80)
- # LOKI_RESOURCE_MEM_THRESHOLD - Memory % threshold to warn (default: 80)
- #
- # Budget / Cost Limits (opt-in):
- # LOKI_BUDGET_LIMIT - Max USD spend before auto-pause (default: loki-ts/data/model-pricing.json default_max_cost_usd, 100 USD; empty = unlimited)
- # Example: "50.00" pauses session when estimated cost >= $50
- #
- # Security & Autonomy Controls (Enterprise):
- # LOKI_STAGED_AUTONOMY - Require approval before execution (default: false)
- # LOKI_AUDIT_LOG - Enable audit logging (default: true)
- # LOKI_AUDIT_DISABLED - Disable audit logging (default: false)
- # LOKI_MAX_PARALLEL_AGENTS - Limit concurrent agent spawning (default: 10)
- # LOKI_SANDBOX_MODE - Run in sandboxed container (default: false, requires Docker)
- # LOKI_ALLOWED_PATHS - Comma-separated path allowlist. PARTIAL enforcement
- # (honest scope): when set, run.sh rejects a sandbox
- # custom --mount whose host path is outside the
- # allowlist (a write surface run.sh itself controls).
- # It does NOT and CANNOT restrict provider-driven
- # agent file writes: claude/opencode write
- # files directly, not through run.sh, so run.sh never
- # sees those writes. The containment for agent writes
- # is the OS user + the LOKI_SANDBOX_MODE container
- # (cap-drop, seccomp, read-only mounts), not this var.
- # Treat ALLOWED_PATHS as defense-in-depth on the
- # mount surface, not a complete write sandbox.
- # LOKI_BLOCKED_COMMANDS - Comma-separated blocked shell commands (default: rm -rf /)
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 55 on GitHub
- #
- # OIDC / SSO Authentication (optional, works alongside token auth):
- # LOKI_OIDC_ISSUER - OIDC issuer URL (e.g., https://accounts.google.com)
- # LOKI_OIDC_CLIENT_ID - OIDC client/application ID
- # LOKI_OIDC_AUDIENCE - Expected JWT audience (default: same as client_id)
- #
- # SDLC Phase Controls (all enabled by default, set to 'false' to skip):
- # LOKI_PHASE_UNIT_TESTS - Run unit tests (default: true)
- # LOKI_PHASE_API_TESTS - Functional API testing (default: true)
- # LOKI_PHASE_E2E_TESTS - E2E/UI testing with Playwright (default: true)
- # LOKI_PHASE_SECURITY - Security scanning OWASP/auth (default: true)
- # LOKI_PHASE_INTEGRATION - Integration tests SAML/OIDC/SSO (default: true)
- # LOKI_PHASE_CODE_REVIEW - 3-reviewer parallel code review (default: true)
- # LOKI_PHASE_WEB_RESEARCH - Competitor/feature gap research (default: true)
- # LOKI_PHASE_PERFORMANCE - Load/performance testing (default: true)
- # LOKI_PHASE_ACCESSIBILITY - WCAG compliance testing (default: true)
- # LOKI_PHASE_REGRESSION - Regression testing (default: true)
- # LOKI_PHASE_UAT - UAT simulation (default: true)
- #
- # Autonomous Loop Controls (Ralph Wiggum Mode):
- # LOKI_COMPLETION_PROMISE - EXPLICIT stop condition text (default: none - runs forever)
- # Example: "ALL TESTS PASSING 100%"
- # Only stops when the AI provider outputs this EXACT text
- # LOKI_MAX_ITERATIONS - Max loop iterations before exit (default: 1000)
- # LOKI_PERPETUAL_MODE - Ignore ALL completion signals (default: false)
- # Set to 'true' for truly infinite operation
- #
- # Completion Council (v5.25.0) - Multi-agent completion verification:
- # LOKI_COUNCIL_ENABLED - Enable completion council (default: true)
- # LOKI_COUNCIL_SIZE - Number of council members (default: 3)
- # LOKI_COUNCIL_THRESHOLD - Votes needed for completion (default: 2)
- # LOKI_COUNCIL_CHECK_INTERVAL - Check every N iterations (default: 5)
- # LOKI_COUNCIL_MIN_ITERATIONS - Min iterations before council runs (default: 3)
- # LOKI_COUNCIL_STAGNATION_LIMIT - Max iterations with no git changes (default: 5)
- #
- # Model Selection:
- # LOKI_ALLOW_HAIKU - Enable Haiku model for fast tier (default: false)
- # When false: Opus for dev/bugfix, Sonnet for tests/docs
- # When true: Sonnet for dev, Haiku for tests/docs (original)
- # Use --allow-haiku flag or set to 'true'
- #
- # 2026 Research Enhancements:
- # LOKI_PROMPT_REPETITION - Enable prompt repetition for Haiku agents (default: true)
- # arXiv 2512.14982v1: Improves accuracy 4-5x on structured tasks
- # LOKI_CONFIDENCE_ROUTING - Enable confidence-based routing (default: true)
- # HN Production: 4-tier routing (auto-approve, direct, supervisor, escalate)
- # LOKI_AUTONOMY_MODE - Autonomy level (default: perpetual)
- # Options: perpetual, checkpoint, supervised
- # Tim Dettmers: "Shorter bursts of autonomy with feedback loops"
- #
- # Parallel Workflows (Git Worktrees):
- # LOKI_PARALLEL_MODE - Enable git worktree-based parallelism (default: false)
- # Use --parallel flag or set to 'true'
- # LOKI_MAX_WORKTREES - Maximum parallel worktrees (default: 5)
- # LOKI_MAX_PARALLEL_SESSIONS - Maximum concurrent AI sessions (default: 3)
- # LOKI_PARALLEL_TESTING - Run testing stream in parallel (default: true)
- # LOKI_PARALLEL_DOCS - Run documentation stream in parallel (default: true)
- # LOKI_PARALLEL_BLOG - Run blog stream if site has blog (default: false)
- # LOKI_AUTO_MERGE - Auto-merge completed features (default: true)
- #
- # Complexity Tiers (Auto-Claude pattern):
- # LOKI_COMPLEXITY - Force complexity tier (default: auto)
- # Options: auto, simple, standard, complex
- # Simple (3 phases): 1-2 files, single service, UI fixes, text changes
- # Standard (6 phases): 3-10 files, 1-2 services, features, bug fixes
- # Complex (8 phases): 10+ files, multiple services, external integrations
- #
- # GitHub Integration (v4.1.0):
- # LOKI_GITHUB_IMPORT - Import open issues as tasks (default: false)
- # LOKI_GITHUB_PR - Create PR when feature complete (default: false)
- # LOKI_GITHUB_SYNC - Sync status back to issues (default: false)
- # LOKI_GITHUB_REPO - Override repo detection (default: from git remote)
- # LOKI_GITHUB_LABELS - Filter by labels (comma-separated)
- # LOKI_GITHUB_MILESTONE - Filter by milestone
- # LOKI_GITHUB_ASSIGNEE - Filter by assignee
- # LOKI_GITHUB_LIMIT - Max issues to import (default: 100)
- # LOKI_GITHUB_PR_LABEL - Label for PRs (default: none, avoids error if label missing)
- #
- # Desktop Notifications (v4.1.0):
- # LOKI_NOTIFICATIONS - Enable desktop notifications (default: true)
- # LOKI_NOTIFICATION_SOUND - Play sound with notifications (default: true)
- #
- # Uncertainty-Gated Escalation (v7.19.2, default-on):
- # LOKI_UNCERTAINTY_ESCALATION - Master on/off for proactive stuck-escalation (default: 1; set 0 to
- # disable; byte-identical when off). Decision lives in
- # completion-council.sh (uncertainty_should_escalate); action in run.sh.
- # NOTE: AUTONOMY_MODE defaults to "perpetual"; in perpetual mode PAUSE
- # is auto-cleared by check_human_intervention, so escalation degrades
- # to notify-only (notification fires, run does NOT halt).
- # LOKI_UNCERTAINTY_ROUNDS - Consecutive rounds where >=2 of 3 proxies must co-occur before
- # escalating (default: 2; recommended range 2-3). Debounces transient
- # noise: a single hot proxy never escalates alone.
- # LOKI_UNCERTAINTY_NOCHANGE_MIN - Proxy 1 threshold: consecutive_no_change value that marks p1 hot.
- # (default: COUNCIL_STAGNATION_LIMIT - 1, i.e. one below the circuit-
- # breaker limit so escalation fires before the breaker ends the run).
- # Floored at 1 at runtime.
- # LOKI_UNCERTAINTY_SPLIT_ROUNDS - Proxy 3 threshold: number of consecutive trailing council verdicts
- # that must be REJECTED-with-approver (split) to mark p3 hot
- # (default: 2). Between council votes p3 may be stale; it is always
- # fresh when proxy 1 is hot because proxy 1 hot forces a circuit-
- # breaker vote that refreshes verdicts.
- #
- # Human Intervention (Auto-Claude pattern):
- # PAUSE file: touch .loki/PAUSE - pauses after current session
- # HUMAN_INPUT.md: echo "instructions" > .loki/HUMAN_INPUT.md
- # STOP file: touch .loki/STOP - graceful; read at the TOP of each
- # iteration, so a STOP written mid-dispatch waits for that
- # provider call to return (bounded by
- # LOKI_PROVIDER_CALL_TIMEOUT, default 7200s). It said
- # "stops immediately", which was false. For an immediate
- # stop use `loki stop` (process-group SIGTERM, 1s grace,
- # then SIGKILL). See docs/stop-latency.md.
- # Ctrl+C (once): Pauses execution, shows options
- # Ctrl+C (twice): Exits immediately
- #
- # Security (Enterprise):
- # LOKI_PROMPT_INJECTION - Enable HUMAN_INPUT.md processing (default: false)
- # Set to "true" only in trusted environments
- #
- # Branch Protection (agent isolation):
- # LOKI_BRANCH_PROTECTION - Create feature branch for agent changes (default: true)
- # Agent works on loki/session-<timestamp>-<pid> branch
- # Set to "false" to opt out and work on the current branch
- # LOKI_AUTO_PR - Auto push + open a PR on session end (default: off)
- # Default behavior PRINTS the PR command (advisory, no push)
- #
- # Process Supervision (opt-in):
- # LOKI_WATCHDOG - Enable process health monitoring (default: false)
- # LOKI_WATCHDOG_INTERVAL - Check interval in seconds (default: 30)
- #===============================================================================
- #
- # Compatibility: bash 3.2+ (macOS default), bash 4+ (Linux), WSL
- # Parallel mode (--parallel) requires bash 4.0+ for associative arrays
- #===============================================================================
- set -uo pipefail
- # Compatibility check: Ensure we're running in bash (not sh, dash, zsh)
- if [ -z "${BASH_VERSION:-}" ]; then
- echo "[ERROR] This script requires bash. Please run with: bash $0" >&2
- exit 1
- fi
- # Extract major version for feature checks
- BASH_VERSION_MAJOR="${BASH_VERSION%%.*}"
- BASH_VERSION_MINOR="${BASH_VERSION#*.}"
- BASH_VERSION_MINOR="${BASH_VERSION_MINOR%%.*}"
- # Warn if bash version is very old (< 3.2)
- if [ "$BASH_VERSION_MAJOR" -lt 3 ] || { [ "$BASH_VERSION_MAJOR" -eq 3 ] && [ "$BASH_VERSION_MINOR" -lt 2 ]; }; then
- echo "[WARN] Bash version $BASH_VERSION is old. Recommend bash 3.2+ for full compatibility." >&2
- echo "[WARN] Some features may not work correctly." >&2
- fi
- SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
- PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
- #===============================================================================
- # Self-Copy Protection
- # Bash reads scripts incrementally, so editing a running script corrupts execution.
- # Solution: Copy ourselves to /tmp and run from there. The original can be safely edited.
- #===============================================================================
- if [[ -z "${LOKI_RUNNING_FROM_TEMP:-}" ]] && [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
- TEMP_SCRIPT=$(mktemp /tmp/loki-run-XXXXXX)
- mv "$TEMP_SCRIPT" "${TEMP_SCRIPT}.sh"
- TEMP_SCRIPT="${TEMP_SCRIPT}.sh"
- cp "${BASH_SOURCE[0]}" "$TEMP_SCRIPT"
- chmod 700 "$TEMP_SCRIPT"
- # BUG-XC-011: Set trap BEFORE exec so the temp file gets cleaned up
- trap 'rm -f "$TEMP_SCRIPT"' EXIT
- export LOKI_RUNNING_FROM_TEMP=1
- # Record the EXACT temp-copy path so the post-exec cleanup trap deletes THIS
- # temp file and NEVER the canonical source (root cause of the recurring
- # "run.sh self-deleted on build spawn" bug: an inherited LOKI_RUNNING_FROM_TEMP
- # skips the self-copy block, leaving BASH_SOURCE[0]=the real run.sh).
- export LOKI_TEMP_SCRIPT_PATH="$TEMP_SCRIPT"
- export LOKI_ORIGINAL_SCRIPT_DIR="$SCRIPT_DIR"
- export LOKI_ORIGINAL_PROJECT_DIR="$PROJECT_DIR"
- exec "$TEMP_SCRIPT" "$@"
- fi
- # Restore original paths when running from temp
- SCRIPT_DIR="${LOKI_ORIGINAL_SCRIPT_DIR:-$SCRIPT_DIR}"
- PROJECT_DIR="${LOKI_ORIGINAL_PROJECT_DIR:-$PROJECT_DIR}"
- # A-134: `loki quick` is quiet unless LOKI_VERBOSE=1. The run re-executes itself with
- # stdout in .loki/quick-run.log; this shell prints the inner run's compact receipt and
- # a warning count. Errors go to stderr.
- if [ -z "${LOKI_QUICK_INNER:-}" ] && [ "${LOKI_VERBOSE:-0}" != "1" ] && [ "${BASH_SOURCE[0]}" = "${0}" ]; then
- case "${1:-}" in */quick-prd-*.md | quick-prd-*.md)
- _qd="$(cd "$(dirname "$1")" && pwd -P)"; rm -f "$_qd/quick-receipt.txt"
- # Async with default INT/QUIT (a bare `&` child ignores them), signals forwarded.
- # A Node/Bun parent starts us with SIGPIPE ignored, and bash cannot reset a
- # signal ignored on entry; every `echo big | head -1` then prints "echo: write
- # error: Broken pipe" onto the user's terminal. Re-exec the inner run through
- # python3 with SIGPIPE back at default (SIG_DFL survives exec).
- _qexec=("$BASH" "$0" "$@")
- # Env differences under python (not bash): LC_CTYPE may be set when the locale is
- # C, macOS adds __CF_USER_TEXT_ENCODING, and SIGXFSZ stays ignored.
- # The import probe guards a python3 that exists but is broken (CLT stub, pyenv shim).
- if command -v python3 >/dev/null 2>&1 && python3 -c 'import os,signal' >/dev/null 2>&1; then
- _qexec=(python3 -c 'import os,signal,sys;signal.signal(signal.SIGPIPE,signal.SIG_DFL);os.execv(sys.argv[1],sys.argv[1:])' "${_qexec[@]}")
- fi
- # A background job gets stdin=/dev/null. Only a TTY needs it back (pause-mode
- # keypress resume); a pipe would block the provider, which reads inherited stdin.
- if [ -t 0 ]; then exec 3<&0; else exec 3</dev/null; fi
- ( trap - INT QUIT; LOKI_QUICK_DIR="$_qd" LOKI_QUICK_OUTER_PID=$$ LOKI_QUICK_INNER=1 exec "${_qexec[@]}" >"$_qd/quick-run.log" <&3 3<&- ) &
- _qpid=$!
- trap 'kill -INT "$_qpid" 2>/dev/null' INT
- trap 'kill -TERM "$_qpid" 2>/dev/null' TERM HUP
- wait "$_qpid"; _qrc=$?
- kill -0 "$_qpid" 2>/dev/null && { wait "$_qpid"; _qrc=$?; }
- trap - INT TERM HUP
- _qw=$(grep -cF '[WARN]' "$_qd/quick-run.log" 2>/dev/null); _qw="${_qw:-0}"
- if [ "$_qrc" -ne 0 ] || [ ! -s "$_qd/quick-receipt.txt" ]; then
- # Anchored to log_error/log_warn lines (optional ANSI prefix) and capped at 200
- # chars so a long prompt line mentioning "failed" is never dumped.
- _qr=$(grep -E $'^(\033\\[[0-9;]*m)?\\[(ERROR|WARN)\\]' "$_qd/quick-run.log" 2>/dev/null | tail -n 3 | cut -c1-200)
- echo "${_qr:-$(tail -n 3 "$_qd/quick-run.log" 2>/dev/null | cut -c1-200)}"
- fi
- [ "$_qw" -gt 0 ] && echo "Warnings: $_qw (see $_qd/quick-run.log)"
- cat "$_qd/quick-receipt.txt" 2>/dev/null
- exit "$_qrc" ;;
- esac
- fi
- # Clean up ONLY the recorded temp copy, and ONLY if it is a real temp file.
- # Deleting BASH_SOURCE[0] here was the bug: an inherited LOKI_RUNNING_FROM_TEMP
- # made BASH_SOURCE[0] the canonical source, so run.sh deleted itself on spawned
- # builds. Guard on the recorded path being a real /tmp/loki-run-* file.
- if [[ "${LOKI_RUNNING_FROM_TEMP:-}" == "1" ]] \
- && [[ -n "${LOKI_TEMP_SCRIPT_PATH:-}" ]] \
- && [[ "${LOKI_TEMP_SCRIPT_PATH}" == /tmp/loki-run-* || "${LOKI_TEMP_SCRIPT_PATH}" == "${TMPDIR:-/tmp}"loki-run-* ]]; then
- trap 'rm -f "${LOKI_TEMP_SCRIPT_PATH}" 2>/dev/null' EXIT
- fi
- #===============================================================================
- # Configuration File Support (v4.1.0)
- # Loads settings from config file, environment variables take precedence
- #===============================================================================
- # #691: source the canonical config mapping + shared per-key export helper. This
- # is the SAME lib the loki CLI pre-pass uses, so the YAML/config-file mapping
- # cannot drift between the CLI and the runner. Side-effect-free on source. The
- # parsers below delegate to loki_config_export_key with override=0, preserving
- # the shipped env-wins auto-discovery contract byte-for-byte.
- _LOKI_CONFIG_MAP_LIB="$SCRIPT_DIR/lib/config-map.sh"
- if [ -f "$_LOKI_CONFIG_MAP_LIB" ]; then
- # shellcheck source=lib/config-map.sh
- source "$_LOKI_CONFIG_MAP_LIB"
- fi
- # v8.1: one-switch SDK activation. LOKI_SDK_MODE=off|judges|full (default off)
- # sets the default for the 8 per-site LOKI_SDK_* flags via a write-once resolver;
- # per-site flags still win. Sourced + resolved here, before any judge lib or the
- # main loop reads a flag. No-op (byte-identical) when the mode is unset.
- _LOKI_SDK_MODE_LIB="$SCRIPT_DIR/lib/sdk-mode.sh"
- if [ -f "$_LOKI_SDK_MODE_LIB" ]; then
- # shellcheck source=lib/sdk-mode.sh
- source "$_LOKI_SDK_MODE_LIB"
- loki_sdk_resolve_mode
- fi
- load_config_file() {
- local config_file=""
- # Search for config file in order of priority
- # Security: Reject symlinks to prevent path traversal attacks
- # 1. Project-local config
- if [ -f ".loki/config.yaml" ] && [ ! -L ".loki/config.yaml" ]; then
- config_file=".loki/config.yaml"
- elif [ -f ".loki/config.yml" ] && [ ! -L ".loki/config.yml" ]; then
- config_file=".loki/config.yml"
- # 2. User-global config (symlinks allowed in home dir - user controls it)
- elif [ -f "${HOME}/.config/loki-mode/config.yaml" ]; then
- config_file="${HOME}/.config/loki-mode/config.yaml"
- elif [ -f "${HOME}/.config/loki-mode/config.yml" ]; then
- config_file="${HOME}/.config/loki-mode/config.yml"
- fi
- # If no config file found, return silently
- if [ -z "$config_file" ]; then
- return 0
- fi
- # Check for yq (YAML parser)
- if ! command -v yq &> /dev/null; then
- # Fallback: parse simple YAML with sed/grep
- parse_simple_yaml "$config_file"
- return 0
- fi
- # Use yq for proper YAML parsing
- parse_yaml_with_yq "$config_file"
- }
- # Fallback YAML parser for simple key: value format.
- # #691: iterates the canonical LOKI_CONFIG_MAP (single source of truth) instead
- # of 64 hand-maintained set_from_yaml calls. override=0 -> ambient env still
- # wins (auto-discovery contract unchanged). Falls back to nothing if the lib is
- # absent (defensive; the lib ships alongside run.sh).
- parse_simple_yaml() {
- local file="$1"
- if ! declare -p LOKI_CONFIG_MAP >/dev/null 2>&1; then
- return 0
- fi
- local mapping yaml_path env_var
- for mapping in "${LOKI_CONFIG_MAP[@]}"; do
- yaml_path="${mapping%%:*}"
- env_var="${mapping##*:}"
- set_from_yaml "$file" "$yaml_path" "$env_var"
- done
- }
- # Validate YAML value to prevent injection attacks
- validate_yaml_value() {
- local value="$1"
- local max_length="${2:-1000}"
- # Reject empty values
- if [ -z "$value" ]; then
- return 1
- fi
- # Reject values with dangerous shell metacharacters.
- # SECURITY FIX (#691 wave): the previous [[ "$value" =~ [\$\`...] ]]
- # bracket-regex matched NOTHING -- backslash escapes inside a bash regex
- # bracket class are taken literally, so the guard ACCEPTED $(...), backticks,
- # pipes, etc. (verified behaviorally: it returned 0 for "$(touch /tmp/x)").
- # A `case` glob class has no such ambiguity and actually rejects the metachars.
- # Allow alphanumeric, spaces, dots, dashes, underscores, slashes, colons,
- # commas, @.
- case "$value" in
- *'$'* | *'`'* | *'|'* | *';'* | *'&'* | *'>'* | *'<'* | *'('* | *')'* \
- | *'{'* | *'}'* | *'['* | *']'* | *'\'* )
- return 1 ;;
- esac
- # Reject values that are too long (DoS protection)
- if [ "${#value}" -gt "$max_length" ]; then
- return 1
- fi
- # Reject values with newlines (could corrupt variables)
- if [[ "$value" == *$'\n'* ]]; then
- return 1
- fi
- return 0
- }
- # Escape regex metacharacters for safe grep usage
- escape_regex() {
- local input="$1"
- # Escape: . * ? + [ ] ^ $ { } | ( ) \
- printf '%s' "$input" | sed 's/[.[\*?+^${}|()\\]/\\&/g'
- }
- # Helper: Extract value from YAML and set env var if not already set
- set_from_yaml() {
- local file="$1"
- local yaml_path="$2"
- local env_var="$3"
- # Skip if env var is already set (env-wins guard; the shared helper also
- # enforces this with override=0, but checking here avoids the extraction).
- if [ -n "${!env_var:-}" ]; then
- return 0
- fi
- # Extract value using grep and sed (handles simple YAML)
- # Convert yaml path like "core.max_retries" to search pattern
- local value=""
- local key="${yaml_path##*.}" # Get last part of path
- # Escape regex metacharacters in key for safe grep
- local escaped_key
- escaped_key=$(escape_regex "$key")
- # Simple grep for the key (works for flat or indented YAML)
- # Use read to avoid xargs command execution risks
- value=$(grep -E "^\s*${escaped_key}:" "$file" 2>/dev/null | head -1 | sed -E 's/.*:\s*//' | sed 's/#.*//' | sed 's/^["\x27]//;s/["\x27]$//' | tr -d '\n' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
- # #691: delegate the env-wins guard + ${VAR}-expand + validate + export to the
- # single shared helper (override=0 -> ambient env wins). Falls back to the
- # original inline export if the lib is unavailable, so behavior is identical
- # either way.
- if declare -f loki_config_export_key >/dev/null 2>&1; then
- loki_config_export_key "$env_var" "$value" 0 || true
- elif [ -n "$value" ] && [ "$value" != "null" ] && validate_yaml_value "$value"; then
- export "$env_var=$value"
- fi
- }
- # Parse YAML using yq (proper parser).
- # #691: iterates the canonical LOKI_CONFIG_MAP and delegates the env-wins guard +
- # expand + validate + export to the single shared helper (override=0 -> ambient
- # env wins; auto-discovery contract unchanged). The inline 61-entry table that
- # used to drift from parse_simple_yaml's 64 is gone.
- parse_yaml_with_yq() {
- local file="$1"
- if ! declare -p LOKI_CONFIG_MAP >/dev/null 2>&1; then
- return 0
- fi
- local mapping yaml_path env_var value
- for mapping in "${LOKI_CONFIG_MAP[@]}"; do
- yaml_path="${mapping%%:*}"
- env_var="${mapping##*:}"
- # Extract value using yq (env-wins guard is enforced by the shared helper).
- value=$(yq eval ".$yaml_path // \"\"" "$file" 2>/dev/null)
- if declare -f loki_config_export_key >/dev/null 2>&1; then
- loki_config_export_key "$env_var" "$value" 0 || true
- elif [ -n "$value" ] && [ "$value" != "null" ] && [ "$value" != "" ] \
- && [ -z "${!env_var:-}" ] && validate_yaml_value "$value"; then
- export "$env_var=$value"
- fi
- done
- }
- # Load config file before setting defaults
- load_config_file
- # Load JSON settings from loki config set (v6.0.0)
- #
- # SECURITY NOTE (v7.5.10, L12#2 audit): The eval below is intentional and safe.
- # The Python script's output is constrained to a fixed template:
- # [ -z "${VAR:-}" ] && export VAR=<value>
- # where:
- # - VAR is a hardcoded env var name from the `mapping` dict (NOT user-controlled).
- # - <value> is produced by shlex.quote(), which emits POSIX-shell-safe single-
- # quoted strings even for adversarial input (e.g. quotes, semicolons, $()).
- # - Non-string values from settings.json are skipped (isinstance check).
- # Therefore no user-controlled bytes can break out of the quoted value or alter
- # the surrounding shell syntax. Do NOT remove the shlex.quote() call or relax
- # the isinstance(val, str) guard without re-auditing this eval.
- _load_json_settings() {
- local settings_file="${TARGET_DIR:-.}/.loki/config/settings.json"
- [ -f "$settings_file" ] || return 0
- eval "$(_LOKI_SETTINGS_FILE="$settings_file" python3 -c "
- import json, sys, os, shlex
- def get_nested(d, key):
- \"\"\"Resolve dotted keys through nested dicts (model.planning -> data['model']['planning'])\"\"\"
- parts = key.split('.')
- cur = d
- for p in parts:
- if isinstance(cur, dict):
- cur = cur.get(p)
- else:
- return None
- return cur
- try:
- with open(os.environ['_LOKI_SETTINGS_FILE']) as f:
- data = json.load(f)
- except Exception:
- sys.exit(0)
- mapping = {
- 'maxTier': 'LOKI_MAX_TIER',
- 'model.planning': 'LOKI_MODEL_PLANNING',
- 'model.development': 'LOKI_MODEL_DEVELOPMENT',
- 'model.fast': 'LOKI_MODEL_FAST',
- 'notify.slack': 'LOKI_SLACK_WEBHOOK',
- 'notify.discord': 'LOKI_DISCORD_WEBHOOK',
- 'provider': 'LOKI_PROVIDER',
- 'issue.provider': 'LOKI_ISSUE_PROVIDER',
- 'blind_validation': 'LOKI_BLIND_VALIDATION',
- 'adversarial_testing': 'LOKI_ADVERSARIAL_TESTING',
- 'spawn_timeout': 'LOKI_SPAWN_TIMEOUT',
- 'spawn_retries': 'LOKI_SPAWN_RETRIES',
- 'budget': 'LOKI_BUDGET_LIMIT',
- }
- for key, env_var in mapping.items():
- # Try nested dict lookup first, then flat key, then underscore variant
- val = get_nested(data, key) or data.get(key) or data.get(key.replace('.', '_'))
- if val and isinstance(val, str):
- safe_val = shlex.quote(val)
- print(f'[ -z \"\${{{env_var}:-}}\" ] && export {env_var}={safe_val}' + (\" && export LOKI_BUDGET_SOURCE=config\" if env_var == 'LOKI_BUDGET_LIMIT' else ''))
- " 2>/dev/null)" 2>/dev/null || true
- }
- _LOKI_SETTINGS_FILE="${TARGET_DIR:-.}/.loki/config/settings.json" _load_json_settings
- # Configuration
- MAX_RETRIES=${LOKI_MAX_RETRIES:-50}
- BASE_WAIT=${LOKI_BASE_WAIT:-60}
- MAX_WAIT=${LOKI_MAX_WAIT:-3600}
- SKIP_PREREQS=${LOKI_SKIP_PREREQS:-false}
- ENABLE_DASHBOARD=${LOKI_DASHBOARD:-true}
- DASHBOARD_PORT=${LOKI_DASHBOARD_PORT:-57374}
- RESOURCE_CHECK_INTERVAL=${LOKI_RESOURCE_CHECK_INTERVAL:-300} # Check every 5 minutes
- RESOURCE_CPU_THRESHOLD=${LOKI_RESOURCE_CPU_THRESHOLD:-80} # CPU % threshold
- RESOURCE_MEM_THRESHOLD=${LOKI_RESOURCE_MEM_THRESHOLD:-80} # Memory % threshold
- # Budget / Cost Limit.
- #
- # DEFAULT-ON as of v9.42.0. This shipped as `""` (unlimited), and run.sh's own
- # F4 analysis names it as one of three runaway valves that all shipped
- # DISABLED: LOKI_BUDGET_LIMIT="" returns immediately, LOKI_MAX_DURATION=0 never
- # stops, leaving LOKI_MAX_ITERATIONS=1000 (a measured 8.3-DAY ceiling) as the
- # only backstop. A user who mistypes a spec, or hits a loop, had no cost
- # backstop at all.
- #
- # 100 USD, not a tight number. Measured envelope from 79 recorded benchmark
- # trials: median 0.48 USD/trial, max 3.06. A real build is far under this, so
- # the cap catches runaways, not ordinary work.
- #
- # SAFE BY CONSTRUCTION: breaching PAUSES (writes .loki/PAUSE and saves state);
- # it never kills work or discards a deliverable. The user removes .loki/PAUSE
- # to resume, or raises LOKI_BUDGET_LIMIT. Set LOKI_BUDGET_LIMIT="" to restore
- # the old unlimited behavior explicitly.
- # LM-002: the default value is read from default_max_cost_usd in loki-ts/data/model-pricing.json, the same
- # file the Loki 10 engine reads (docs/cost-controls.md). "100.00" is only the
- # fallback when that file is absent, and budget_cap.test.ts pins it to the file.
- _loki_budget_default_from() {
- local f="$1" v=""
- if [ ! -e "$f" ]; then
- printf '%s\n' "100.00"
- return 0
- fi
- v="$(sed -n 's/.*"default_max_cost_usd"[[:space:]]*:[[:space:]]*\([0-9][0-9.]*\).*/\1/p' "$f" 2>/dev/null | head -n 1)"
- case "$v" in
- '' | *[!0-9.]*)
- printf '%s\n' "loki: warning: cannot read default_max_cost_usd from $f (unreadable or no parsable key); using fallback 100.00" >&2
- printf '%s\n' "100.00"
- ;;
- *) awk -v v="$v" 'BEGIN{printf "%.2f\n", v}' ;;
- esac
- }
- _loki_budget_default="$(_loki_budget_default_from "${SCRIPT_DIR}/../loki-ts/data/model-pricing.json")"
- # Where the active cap came from: flag (loki start --budget), config (settings.json),
- # env (LOKI_BUDGET_LIMIT set by the caller), or default. Printed once at run start.
- if [ "${LOKI_BUDGET_LIMIT+set}" = "set" ]; then
- BUDGET_SOURCE="${LOKI_BUDGET_SOURCE:-env}"
- else
- BUDGET_SOURCE="default"
- fi
- BUDGET_LIMIT=${LOKI_BUDGET_LIMIT-"$_loki_budget_default"} # USD. Note `-` not `:-`: an
- # explicit empty value means the
- # operator chose unlimited, and is
- # honored.
- # Background Mode
- BACKGROUND_MODE=${LOKI_BACKGROUND:-false} # Run in background
- # Security & Autonomy Controls
- STAGED_AUTONOMY=${LOKI_STAGED_AUTONOMY:-false} # Require plan approval
- AUDIT_LOG_ENABLED=${LOKI_AUDIT_LOG:-true} # Enable audit logging (on by default)
- MAX_PARALLEL_AGENTS=${LOKI_MAX_PARALLEL_AGENTS:-10} # Limit concurrent agents
- SANDBOX_MODE=${LOKI_SANDBOX_MODE:-false} # Docker sandbox mode (informational; the real dispatch reads LOKI_SANDBOX_MODE at autonomy/loki:1965 and execs sandbox.sh -- this var is not consumed in run.sh)
- ALLOWED_PATHS=${LOKI_ALLOWED_PATHS:-""} # PARTIAL enforcement. When set, a sandbox custom --mount host path outside the allowlist is rejected in sandbox.sh (_sandbox_path_within_allowed), NOT in run.sh. Does NOT restrict provider-driven agent writes (run.sh never sees them); the sandbox container is the containment for those. Default empty = no restriction.
- BLOCKED_COMMANDS=${LOKI_BLOCKED_COMMANDS:-"rm -rf /,dd if=,mkfs,:(){ :|:& };:"}
- Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
- Line 608 on GitHub
- # LOKI_SESSION_ID is woven into many filesystem paths (.loki/sessions/<id>/...,
- # session locks, PID files, per-session state). It is operator/issue-derived
- # (trusted-ish), but a value like "../../tmp/x" would escape the .loki/ tree.
- # Sanitize ONCE here at intake: any char outside [A-Za-z0-9._-] is replaced with
- # "_" and a leading run of dots/dashes is neutralized, so every downstream
- # session path is contained. Unset stays unset (default behavior unchanged).
- if [ -n "${LOKI_SESSION_ID:-}" ]; then
- _loki_sid_raw="$LOKI_SESSION_ID"
- _loki_sid_safe="${_loki_sid_raw//[^A-Za-z0-9._-]/_}"
- # Strip leading dots/dashes so "../x" -> "x" cannot reference a parent dir
- # or look like an option; collapse a now-empty result to a safe literal.
- _loki_sid_safe="${_loki_sid_safe#"${_loki_sid_safe%%[!.-]*}"}"
- [ -n "$_loki_sid_safe" ] || _loki_sid_safe="session"
- if [ "$_loki_sid_safe" != "$_loki_sid_raw" ]; then
- printf 'loki: WARNING sanitized unsafe LOKI_SESSION_ID %s -> %s\n' "$_loki_sid_raw" "$_loki_sid_safe" >&2
- fi
- export LOKI_SESSION_ID="$_loki_sid_safe"
- unset _loki_sid_raw _loki_sid_safe
- fi
- # GENERIC TIER VOCABULARY (small|medium|high). A user should be able to ask for
- # a capability class without naming a vendor model, and get that provider's
- # latest model in the class. LOKI_SESSION_MODEL is the knob that already does
- # this -- it accepts the raw tier names planning|development|fast alongside the
- # Claude aliases -- so the generic words are normalized ONTO it here rather than
- # becoming a fourth spelling. LOKI_MAX_TIER (a cost CEILING) and LOKI_TIER (the
- # OSS/enterprise licensing seam) mean different things and are left alone.
- #
- # WHY NORMALIZE AT THE ENTRY POINT: the session-pin case block is byte-mirrored
- # in the estimator (autonomy/loki) and the dashboard (dashboard/server.py), and
- # every one of those mirrors is locked by a parity test. Translating here means
- # they keep seeing only the three canonical tier names and none of them change.
- #
- # The mapping is loki_tier_alias() in providers/models.sh -- the single source
- # of truth, not a second copy. Inlined as a case because run.sh must not source
- # a provider file this early in startup. Kept in lockstep by
- # tests/test-generic-tiers.sh.
- #
- # ONLY the three new words are translated. sonnet/haiku/opus/fable and the raw
- # tier names pass through untouched, so an unset LOKI_SESSION_MODEL still
- # defaults to sonnet and "medium" resolves to the same development-tier model
- # today's builds already use. This changes no existing run's model.
- # NORMALIZATION IS TRIM-ONLY + LOWERCASE, matching the estimator and dashboard
- # mirrors exactly. Interior whitespace is deliberately PRESERVED, so " med ium "
- # stays junk here just as it does there. Stripping interior spaces would make
- # this reader accept a value the other two reject, which is the precise kind of
- # divergence the session-pin parity tests exist to catch.
- _loki_generic_tier="${LOKI_SESSION_MODEL:-}"
- _loki_generic_tier="${_loki_generic_tier#"${_loki_generic_tier%%[![:space:]]*}"}"
- _loki_generic_tier="${_loki_generic_tier%"${_loki_generic_tier##*[![:space:]]}"}"
- case "$(printf '%s' "$_loki_generic_tier" | tr '[:upper:]' '[:lower:]')" in
- small) LOKI_SESSION_MODEL="fast" ; export LOKI_SESSION_MODEL ;;
- medium) LOKI_SESSION_MODEL="development" ; export LOKI_SESSION_MODEL ;;
- high) LOKI_SESSION_MODEL="planning" ; export LOKI_SESSION_MODEL ;;
- esac
- unset _loki_generic_tier
- # Process Supervision (opt-in)
- WATCHDOG_ENABLED=${LOKI_WATCHDOG:-"false"} # Enable process health monitoring
- WATCHDOG_INTERVAL=${LOKI_WATCHDOG_INTERVAL:-30} # Check interval in seconds
- LAST_WATCHDOG_CHECK=0
- STATUS_MONITOR_PID=""
- DASHBOARD_PID=""
- DASHBOARD_LAST_ALIVE=0
- _DASHBOARD_RESTARTING=false
- RESOURCE_MONITOR_PID=""
- _LOKI_NODE_PROBE_STATUS="not_checked"
- _LOKI_NODE_PROBE_VERSION=""
- _LOKI_NPM_PROBE_STATUS="not_checked"
- _LOKI_NPM_PROBE_VERSION=""
- # Build-profile fast-path (v7.129+). LOKI_BUILD_PROFILE selects a gate profile.
- # Default unset = current full behavior (every phase runs). "simple-web" is a
- # fast profile for static marketing/landing builds: it drops the phases that
- # are irrelevant to a static frontend (API tests, SAML/OIDC/SSO integration,
- # load/performance, regression, UAT, competitor web research) while KEEPING the
- # gates that catch real defects on such a page: E2E/Playwright (catches the
- # broken-reveal / blank-page class of bug), CODE_REVIEW, SECURITY, and
- # ACCESSIBILITY (a landing page must be a11y-clean). The completion council and
- # the evidence/proof gate are NOT phases and are untouched -- the moat stays.
- # Pure env-defaulting: it only fills a LOKI_PHASE_* that the operator has NOT
- # already set, so an explicit operator LOKI_PHASE_* override always wins.
- loki_is_supervised_simple_web() {
- [ "${LOKI_SUPERVISED_BUILD:-0}" = "1" ] \
- && [ "${LOKI_BUILD_PROFILE:-}" = "simple-web" ]
- }
- _loki_bind_supervised_spec_sha() {
- local spec_path="${1:-}"
- loki_is_supervised_simple_web || return 0
- [ -n "$spec_path" ] && [ -f "$spec_path" ] && [ -r "$spec_path" ] || {
- log_error "Supervised build specification is missing or unreadable."
- return 1
- }
- local computed_sha supplied_sha="${LOKI_SPEC_SHA256:-}"
- computed_sha=$(python3 - "$spec_path" <<'PY'
- import hashlib
- import sys
- with open(sys.argv[1], "rb") as handle:
- print(hashlib.sha256(handle.read()).hexdigest())
- PY
- ) || return 1
- [[ "$computed_sha" =~ ^[0-9a-f]{64}$ ]] || return 1
- if [ -n "$supplied_sha" ] && [ "$supplied_sha" != "$computed_sha" ]; then
- log_error "Supervised build specification changed before execution."
- return 1
- fi
- LOKI_SPEC_SHA256="$computed_sha"
- export LOKI_SPEC_SHA256
- }
- # Scoped-change profile: a bug fix or a single feature in an EXISTING repo.
- #
- # Measured on a real user's run: a 322-word GitHub issue against an existing
- # codebase spent 25+ minutes still inside iteration 1. The work itself was
- # genuine, but the build was also running competitor web research, load and
- # performance testing, regression simulation and UAT -- all of which default to
- # true, and none of which a scoped issue fix needs. That is the difference
- # between a 5-minute fix and a 30-minute one.
- #
- # What this NEVER touches: code review, security, tests, E2E, and the
- # completion council all stay on. Speed here comes from not running phases that
- # are irrelevant to the change, never from skipping verification. A greenfield
- # build or a whole-repo refactor does not match this profile and keeps the full
- # suite.
- #
- # Auto-detected rather than another flag the user has to know: an existing git
- # repo with real history, plus a spec that reads as a scoped change. Set
- # LOKI_SCOPED_CHANGE=0 to force the full suite, or =1 to force this profile.
- loki_detect_scoped_change() {
- # Explicit operator intent always wins, in both directions.
- case "${LOKI_SCOPED_CHANGE:-}" in
- 0|false) return 1 ;;
- 1|true) return 0 ;;
- esac
- # Greenfield is not a scoped change: no repo, or a repo with almost no
- # history, means we are building something new.
- #
- # Ask git whether this is a work tree rather than testing for a .git
- # DIRECTORY: in a git worktree (and in a submodule) .git is a FILE, so the
- # old -d test rejected every worktree-based run -- including the parallel
- # workflow streams this project runs by default. rev-parse is true for a
- # plain clone, a worktree, and a submodule alike, and replaces two
- # subprocesses' worth of checking with one.
- local target="${TARGET_DIR:-.}"
- git -C "$target" rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 1
- local commits
- commits="$(git -C "$target" rev-list --count HEAD 2>/dev/null || echo 0)"
- [ "${commits:-0}" -ge 5 ] || return 1
- # An issue-sourced spec is the canonical scoped change: someone filed a
- # discrete request against code that already exists.
- #
- # The spec path is passed in by the caller ($1). It used to be read only
- # from LOKI_PRD_FILE / LOKI_ISSUE_REF, but `loki start <issue>` writes
- # .loki/prd-issue-N.md and hands run.sh that path as a POSITIONAL argument,
- # so neither variable was ever set and this check could not fire.
- local spec="${1:-${LOKI_PRD_FILE:-}}"
- [ -n "${LOKI_ISSUE_REF:-}" ] && return 0
- case "$spec" in
- *prd-issue-*) return 0 ;;
- esac
- return 1
- }
- loki_apply_scoped_change_profile() {
- loki_detect_scoped_change "${1:-}" || return 0
- # Off: cannot affect the correctness of a scoped change to existing code.
- : "${LOKI_PHASE_WEB_RESEARCH:=false}"
- : "${LOKI_PHASE_PERFORMANCE:=false}"
- : "${LOKI_PHASE_REGRESSION:=false}"
- : "${LOKI_PHASE_UAT:=false}"
- # On: every trust gate, unchanged. These are the moat.
- : "${LOKI_PHASE_CODE_REVIEW:=true}"
- : "${LOKI_PHASE_SECURITY:=true}"
- : "${LOKI_PHASE_UNIT_TESTS:=true}"
- : "${LOKI_PHASE_E2E_TESTS:=true}"
- export LOKI_PHASE_WEB_RESEARCH LOKI_PHASE_PERFORMANCE LOKI_PHASE_REGRESSION
- export LOKI_PHASE_UAT LOKI_PHASE_CODE_REVIEW LOKI_PHASE_SECURITY
- export LOKI_PHASE_UNIT_TESTS LOKI_PHASE_E2E_TESTS
- export LOKI_SCOPED_CHANGE_ACTIVE=1
- }
- # Small-task fast path (cost S5). Applied AFTER detect_complexity, because the
- # tier is not known when this file is sourced. Trigger: DETECTED_COMPLEXITY is
- # simple and, for a scoped change in an existing repo, the working-tree diff
- # fits LOKI_SMALL_DIFF_BUDGET changed lines (default 400). LOKI_SMALL_PROFILE=0
- # opts out. It never overrides another explicit profile.
- #
- # What this NEVER touches: the deterministic gates, holdout and Seal. It only
- # drops model-bearing orchestration that has no reader on a small task. Every
- # knob uses ":=" so explicit user env always beats the profile.
- loki_small_profile_eligible() {
- [ "${LOKI_SMALL_PROFILE:-1}" != "0" ] || return 1
- [ "${DETECTED_COMPLEXITY:-}" = "simple" ] || return 1
- case "${LOKI_BUILD_PROFILE:-}" in
- ""|small) ;;
- *) return 1 ;;
- esac
- if [ "${LOKI_SCOPED_CHANGE_ACTIVE:-0}" = "1" ]; then
- local changed budget="${LOKI_SMALL_DIFF_BUDGET:-400}"
- changed="$(git -C "${TARGET_DIR:-.}" diff --numstat HEAD 2>/dev/null \
- | awk '{ if ($1 ~ /^[0-9]+$/) n += $1; if ($2 ~ /^[0-9]+$/) n += $2 } END { print n + 0 }')"
- [ "${changed:-0}" -le "$budget" ] 2>/dev/null || return 1
- fi
- return 0
- }
- loki_apply_small_profile() {
- : "${LOKI_SPEC_GRILL:=0}"
- : "${LOKI_PRD_ENRICH:=0}"
- : "${LOKI_WIKI_AUTO:=0}"
- : "${LOKI_AUTO_DOCS:=false}"
- : "${LOKI_GATE_DEVILS_ADVOCATE:=false}"
- : "${LOKI_COUNCIL_MODEL_VOTERS:=0}"
- : "${LOKI_ADVISOR_MODEL:=haiku}"
- : "${LOKI_SIMPLE:=1}"
- : "${LOKI_DOCS_TIER:=small}"
- export LOKI_SPEC_GRILL LOKI_PRD_ENRICH LOKI_WIKI_AUTO LOKI_AUTO_DOCS
- export LOKI_GATE_DEVILS_ADVOCATE
- export LOKI_COUNCIL_MODEL_VOTERS LOKI_ADVISOR_MODEL LOKI_SIMPLE LOKI_DOCS_TIER
- LOKI_SMALL_PROFILE_ACTIVE=1
- export LOKI_SMALL_PROFILE_ACTIVE
- }
- # Main-session lean flags, small profile only. Fills _LOKI_SMALL_MAIN_ARGV from
- # loki_lean_child_flags (one argument per line) and widens the OAuth-host
- # "--setting-sources project" to "project,local" so project-local settings
- # still apply. Returns 1 (array empty) on any other profile, so the caller
- # keeps its existing "user,project,local" pin. LOKI_SMALL_LEAN_MAIN=0 opts out.
- # "--bare" is never passed to the main session: it skips hooks and the host
- # command guard. Lean flags apply to child sessions only.
- loki_small_main_session_flags() {
- _LOKI_SMALL_MAIN_ARGV=()
- [ "${LOKI_SMALL_PROFILE_ACTIVE:-0}" = "1" ] || return 1
- [ "${LOKI_SMALL_LEAN_MAIN:-1}" != "0" ] || return 1
- type loki_lean_child_flags >/dev/null 2>&1 || return 1
- local _line _prev=""
- while IFS= read -r _line; do
- [ -n "$_line" ] || continue
- [ "$_line" != "--bare" ] || continue
- if [ "$_prev" = "--setting-sources" ] && [ "$_line" = "project" ]; then
- _line="project,local"
- fi
- _LOKI_SMALL_MAIN_ARGV+=("$_line")
- _prev="$_line"
- done <<<"$(loki_lean_child_flags main)"
- [ "${#_LOKI_SMALL_MAIN_ARGV[@]}" -gt 0 ]
- }
- loki_maybe_apply_small_profile() {
- loki_small_profile_eligible || return 0
- LOKI_BUILD_PROFILE=small
- export LOKI_BUILD_PROFILE
- loki_apply_build_profile
- }
- loki_apply_build_profile() {
- if [ "${LOKI_BUILD_PROFILE:-}" = "small" ]; then
- loki_apply_small_profile
- return 0
- fi
- [ "${LOKI_BUILD_PROFILE:-}" = "simple-web" ] || return 0
- : "${LOKI_PHASE_API_TESTS:=false}"
- : "${LOKI_PHASE_INTEGRATION:=false}"
- : "${LOKI_PHASE_PERFORMANCE:=false}"
- : "${LOKI_PHASE_REGRESSION:=false}"
- : "${LOKI_PHASE_UAT:=false}"
- : "${LOKI_PHASE_WEB_RESEARCH:=false}"
- : "${LOKI_PHASE_E2E_TESTS:=true}"
- : "${LOKI_PHASE_CODE_REVIEW:=true}"
- : "${LOKI_PHASE_SECURITY:=true}"
- : "${LOKI_PHASE_ACCESSIBILITY:=true}"
- # The hosted simple-web path is a measured execution policy, not an
- # operator preset. It removes only orchestration that cannot affect the
- # generated app, pins every model-bearing subcall to the selected model,
- # and keeps all applicable verification and proof gates fail closed.
- loki_is_supervised_simple_web || return 0
- local selected_model sdk_model
- selected_model="$(printf '%s' "${LOKI_SESSION_MODEL:-sonnet}" | tr '[:upper:]' '[:lower:]')"
- case "$selected_model" in
- haiku|sonnet|opus) ;;
- *) selected_model="sonnet" ;;
- esac
- sdk_model="$selected_model"
- if command -v python3 >/dev/null 2>&1 && [ -r "$PROJECT_DIR/providers/model_catalog.json" ]; then
- sdk_model="$(_LOKI_SELECTED_MODEL="$selected_model" _LOKI_MODEL_CATALOG="$PROJECT_DIR/providers/model_catalog.json" python3 -c '
- import json, os
- with open(os.environ["_LOKI_MODEL_CATALOG"], encoding="utf-8") as f:
- catalog = json.load(f)
- print(catalog["providers"]["claude"]["cli_aliases"].get(os.environ["_LOKI_SELECTED_MODEL"], ""))
- ' 2>/dev/null)" || sdk_model=""
- [ -n "$sdk_model" ] || sdk_model="$selected_model"
- fi
- LOKI_PHASE_UNIT_TESTS=true
- LOKI_PHASE_E2E_TESTS=true
- LOKI_PHASE_CODE_REVIEW=true
- LOKI_PHASE_SECURITY=true
- LOKI_PHASE_ACCESSIBILITY=true
- LOKI_COUNCIL_ENABLED=true
- LOKI_EVIDENCE_GATE=1
- LOKI_PROOF_GATE=1
- LOKI_PROOF=1
- LOKI_DASHBOARD=false
- LOKI_PARALLEL_MODE=false
- LOKI_MAX_PARALLEL_AGENTS=1
- LOKI_MAX_ITERATIONS=2
- # MAX_RETRIES is the loop attempt ceiling, so 2 means an initial attempt
- # plus one targeted repair attempt.
- LOKI_MAX_RETRIES=2
- LOKI_BASE_WAIT=2
- LOKI_MAX_WAIT=10
- LOKI_PROVIDER_CALL_TIMEOUT=240
- LOKI_PROVIDER_IDLE_TIMEOUT=60
- LOKI_DRAFT_EFFORT=medium
- # One low-effort requirements review gets a hard 90 second ceiling. The
- # prior 75 second ceiling clipped valid Haiku responses, while a second
- # identical full review doubled latency without adding evidence.
- LOKI_REVIEW_CALL_TIMEOUT=90
- LOKI_SDK_REVIEW_TIMEOUT=90
- LOKI_COUNCIL_REVIEW_TIMEOUT=90
- LOKI_COUNCIL_TIMEOUT_MS=90000
- LOKI_REVIEW_EFFORT=low
- LOKI_GATE_TIMEOUT=60
- LOKI_DEPENDENCY_SETUP_TIMEOUT=60
- LOKI_COUNCIL_MIN_ITERATIONS=1
- LOKI_COUNCIL_CHECK_INTERVAL=1
- LOKI_SPEC_GRILL=0
- LOKI_DONE_RECOGNITION=0
- LOKI_INTELLIGENT_USAGE=0
- LOKI_PRD_ENRICH=0
- LOKI_AUTO_DOCS=false
- LOKI_GATE_DOC_COVERAGE=false
- LOKI_GATE_MAGIC_DEBATE=false
- LOKI_WIKI_AUTO=0
- LOKI_AUTO_FALLBACK=off
- LOKI_CAVEMAN=0
- LOKI_CAVEMAN_AUTO_BOOTSTRAP=0
- CAVEMAN_DEFAULT_MODE=off
- LOKI_FABLE_ARCHITECT=0
- LOKI_LEGACY_TIER_SWITCHING=false
- LOKI_REVIEW_RETRY=0
- LOKI_REVIEW_JSON_SCHEMA=off
- LOKI_REVIEW_REQUIREMENTS_ONLY=1
- LOKI_GATE_DEVILS_ADVOCATE=false
- LOKI_REVIEW_INCONCLUSIVE_BLOCK=1
- LOKI_EXPERIMENTAL_MANAGED_AGENTS=false
- LOKI_EXPERIMENTAL_MANAGED_REVIEW=false
- LOKI_EXPERIMENTAL_MANAGED_COUNCIL=false
- LOKI_COUNCIL_MODEL_VOTERS=0
- LOKI_OVERRIDE_COUNCIL=0
- LOKI_BUILD_CHECK_BLOCK=1
- LOKI_SESSION_MODEL="$selected_model"
- LOKI_MAX_TIER="$selected_model"
- LOKI_MODEL_OVERRIDE="$selected_model"
- [ "$selected_model" = "haiku" ] && LOKI_ALLOW_HAIKU=true
- LOKI_ADVISOR_MODEL="$selected_model"
- LOKI_GRILL_MODEL="$selected_model"
- LOKI_CLAUDE_MODEL_PLANNING="$selected_model"
- LOKI_CLAUDE_MODEL_DEVELOPMENT="$selected_model"
- LOKI_CLAUDE_MODEL_FAST="$selected_model"
- LOKI_SDK_COUNCIL_MODEL="$sdk_model"
- LOKI_SDK_GRILL_MODEL="$sdk_model"
- LOKI_SDK_JUDGE_MODEL="$sdk_model"
- LOKI_SDK_PRD_ENRICH_MODEL="$sdk_model"
- LOKI_SDK_REVIEW_MODEL="$sdk_model"
- MAX_RETRIES="$LOKI_MAX_RETRIES"
- BASE_WAIT="$LOKI_BASE_WAIT"
- MAX_WAIT="$LOKI_MAX_WAIT"
- ENABLE_DASHBOARD="$LOKI_DASHBOARD"
- MAX_PARALLEL_AGENTS="$LOKI_MAX_PARALLEL_AGENTS"
- export LOKI_PHASE_UNIT_TESTS LOKI_PHASE_E2E_TESTS
- export LOKI_PHASE_CODE_REVIEW LOKI_PHASE_SECURITY LOKI_PHASE_ACCESSIBILITY
- # EXPORT THE PHASES THIS PROFILE TURNS OFF, not only the ones it leaves on.
- # The receipt derives quality_gates.disabled_phases by scanning the
- # environment for LOKI_PHASE_* set to false (proof-generator.py:470-478), so
- # a phase that is disabled but never exported is INVISIBLE to it: the receipt
- # reported disabled_phases [] and all_phases_enabled true while six phases
- # were switched off. That defeats the field's stated purpose -- "a receipt
- # must be able to say what was NOT checked" -- and makes a narrowed run look
- # identical to a full one. The sibling loki_apply_scoped_change_profile
- # already exports the phases it changes; this one exported only its enables.
- export LOKI_PHASE_API_TESTS LOKI_PHASE_INTEGRATION LOKI_PHASE_PERFORMANCE
- export LOKI_PHASE_REGRESSION LOKI_PHASE_UAT LOKI_PHASE_WEB_RESEARCH
- export LOKI_COUNCIL_ENABLED LOKI_EVIDENCE_GATE LOKI_PROOF_GATE LOKI_PROOF
- export LOKI_DASHBOARD LOKI_PARALLEL_MODE LOKI_MAX_PARALLEL_AGENTS
- export LOKI_MAX_ITERATIONS LOKI_MAX_RETRIES LOKI_BASE_WAIT LOKI_MAX_WAIT
- export LOKI_PROVIDER_CALL_TIMEOUT LOKI_PROVIDER_IDLE_TIMEOUT LOKI_DRAFT_EFFORT
- export LOKI_REVIEW_CALL_TIMEOUT LOKI_SDK_REVIEW_TIMEOUT LOKI_REVIEW_EFFORT
- export LOKI_COUNCIL_REVIEW_TIMEOUT LOKI_COUNCIL_TIMEOUT_MS LOKI_GATE_TIMEOUT
- export LOKI_DEPENDENCY_SETUP_TIMEOUT
- export LOKI_COUNCIL_MIN_ITERATIONS LOKI_COUNCIL_CHECK_INTERVAL
- export LOKI_SPEC_GRILL LOKI_DONE_RECOGNITION LOKI_INTELLIGENT_USAGE LOKI_PRD_ENRICH
- export LOKI_AUTO_DOCS LOKI_GATE_DOC_COVERAGE LOKI_GATE_MAGIC_DEBATE
- export LOKI_WIKI_AUTO LOKI_AUTO_FALLBACK LOKI_CAVEMAN
- export LOKI_CAVEMAN_AUTO_BOOTSTRAP CAVEMAN_DEFAULT_MODE
- export LOKI_FABLE_ARCHITECT LOKI_LEGACY_TIER_SWITCHING LOKI_REVIEW_RETRY
- export LOKI_REVIEW_JSON_SCHEMA LOKI_REVIEW_REQUIREMENTS_ONLY LOKI_GATE_DEVILS_ADVOCATE
- export LOKI_REVIEW_INCONCLUSIVE_BLOCK LOKI_EXPERIMENTAL_MANAGED_AGENTS
- export LOKI_EXPERIMENTAL_MANAGED_REVIEW LOKI_EXPERIMENTAL_MANAGED_COUNCIL
- export LOKI_COUNCIL_MODEL_VOTERS LOKI_OVERRIDE_COUNCIL LOKI_BUILD_CHECK_BLOCK
- export LOKI_SESSION_MODEL LOKI_MAX_TIER LOKI_MODEL_OVERRIDE LOKI_ALLOW_HAIKU
- export LOKI_ADVISOR_MODEL LOKI_GRILL_MODEL
- export LOKI_CLAUDE_MODEL_PLANNING LOKI_CLAUDE_MODEL_DEVELOPMENT
- export LOKI_CLAUDE_MODEL_FAST LOKI_SDK_COUNCIL_MODEL LOKI_SDK_GRILL_MODEL
- export LOKI_SDK_JUDGE_MODEL LOKI_SDK_PRD_ENRICH_MODEL LOKI_SDK_REVIEW_MODEL
- }
- loki_apply_build_profile
- loki_apply_scoped_change_profile
- # Default hang guard for EVERY build, not just simple-web.
- #
- # The two timeouts above are set inside loki_apply_build_profile(), which
- # returns immediately unless LOKI_BUILD_PROFILE=simple-web. So on a normal
- # build both resolved to 0, and 0 means no guard at all -- verified by running
- # the deadline helper directly: `deadline.py 0 0 3 -- sleep 5` runs to
- # completion unkilled. A provider that hung had nothing to stop it.
- #
- # IDLE only, and no retry. That is what keeps this compatible with the standing
- # objection recorded above (search: "former invoke_with_timeout"), whose two
- # reasons remain correct:
- #
- # 1. "No safe generous default" applies to a fixed TOTAL timeout, which
- # cannot tell a long legitimate iteration from a hang. An idle timeout
- # can: it measures silence, not duration. Verified both directions --
- # `sleep 600` under a 120s idle cap dies, while a process emitting output
- # every second survives indefinitely. A coding agent streams constantly;
- # one silent for two minutes is not working.
- # 2. "Wrong retry semantics" stands, so nothing here retries. The call is
- # killed, and the existing failure path handles it. Re-running an agent
- # that may have already edited files remains off the table.
- #
- # 7200s hard ceiling is a backstop against a process that streams forever
- # without converging; the idle cap is the load-bearing guard. Both are
- # overridable, and setting either to 0 restores the old unguarded behaviour.
- : "${LOKI_PROVIDER_IDLE_TIMEOUT:=120}"
- : "${LOKI_PROVIDER_CALL_TIMEOUT:=7200}"
- export LOKI_PROVIDER_IDLE_TIMEOUT LOKI_PROVIDER_CALL_TIMEOUT
- loki_background_services_enabled() {
- ! loki_is_supervised_simple_web
- }
- _loki_with_deadline() {
- local seconds="${1:-0}"
- local kill_grace="${LOKI_DEADLINE_KILL_GRACE:-2}"
- local idle_seconds="${LOKI_DEADLINE_IDLE_TIMEOUT:-0}"
- shift
- case "$seconds" in
- ''|*[!0-9]*) seconds=0 ;;
- esac
- case "$kill_grace" in
- ''|*[!0-9]*) kill_grace=2 ;;
- esac
- case "$idle_seconds" in
- ''|*[!0-9]*) idle_seconds=0 ;;
- esac
- [ "$kill_grace" -gt 0 ] 2>/dev/null || kill_grace=2
- [ "$idle_seconds" -ge 0 ] 2>/dev/null || idle_seconds=0
- if [ "$seconds" -le 0 ] 2>/dev/null; then
- "$@"
- elif command -v python3 >/dev/null 2>&1 \
- && [ -r "${SCRIPT_DIR}/lib/deadline.py" ]; then
- python3 "${SCRIPT_DIR}/lib/deadline.py" \
- "$seconds" "$kill_grace" "$idle_seconds" -- "$@"
- else
- log_error "Required command deadline cannot be enforced: deadline helper is unavailable"
- return 125
- fi
- }
- _loki_with_deadline_stdin_text() {
- local seconds="${1:-0}"
- local input_text="${2:-}"
- local input_file rc=0
- shift 2
- input_file="$(mktemp "${TMPDIR:-/tmp}/loki-review-input.XXXXXX")" || return 125
- chmod 600 "$input_file" 2>/dev/null || {
- rm -f "$input_file" 2>/dev/null || true
- return 125
- }
- if ! printf '%s' "$input_text" > "$input_file"; then
- rm -f "$input_file" 2>/dev/null || true
- return 125
- fi
- _loki_with_deadline "$seconds" "$@" < "$input_file" || rc=$?
- rm -f "$input_file" 2>/dev/null || true
- return "$rc"
- }
- _loki_snapshot_workspace_tree() {
- local tree="${1:-${TARGET_DIR:-.}}"
- local snapshot_dir snapshot_index index_tree snapshot_tree rc=0
- snapshot_dir="$(mktemp -d "${TMPDIR:-/tmp}/loki-iter-index.XXXXXX")" || return 1
- snapshot_index="$snapshot_dir/index"
- index_tree=$(git -C "$tree" write-tree 2>/dev/null) || rc=$?
- if [ "$rc" -eq 0 ]; then
- GIT_INDEX_FILE="$snapshot_index" git -C "$tree" read-tree "$index_tree" \
- >/dev/null 2>&1 || rc=$?
- fi
- if [ "$rc" -eq 0 ]; then
- GIT_INDEX_FILE="$snapshot_index" git -C "$tree" add -A -- . ':(exclude).loki/' \
- >/dev/null 2>&1 || rc=$?
- fi
- if [ "$rc" -eq 0 ]; then
- snapshot_tree=$(GIT_INDEX_FILE="$snapshot_index" git -C "$tree" write-tree 2>/dev/null) || rc=$?
- fi
- rm -f "$snapshot_index" "$snapshot_index.lock" 2>/dev/null || true
- rmdir "$snapshot_dir" 2>/dev/null || true
- [ "$rc" -eq 0 ] && [ -n "${snapshot_tree:-}" ] || return 1
- printf '%s\n' "$snapshot_tree"
- }
- _loki_provider_pipeline_exit_code() {
- local provider_rc="${1:-125}"
- local tee_rc="${2:-125}"
- local parser_rc="${3:-0}"
- if [ "$provider_rc" -ne 0 ]; then
- printf '%s\n' "$provider_rc"
- elif [ "$tee_rc" -ne 0 ]; then
- printf '%s\n' "$tee_rc"
- else
- printf '%s\n' "$parser_rc"
- fi
- }
- # Invoke through a provider's argv-builder seam while preserving the main
- # loop's deadline, fan-out logging, and provider-stage exit status.
- _loki_invoke_argv_provider() {
- local tier="${1:-development}"
- local prompt="${2:-}"
- local log_file="${3:-}"
- local agent_log="${4:-}"
- local iter_output="${5:-}"
- local -a _loki_argv_pipe_status=()
- if ! type provider_invoke_argv >/dev/null 2>&1; then
- log_error "Provider ${PROVIDER_NAME:-unknown} does not implement provider_invoke_argv"
- return 125
- fi
- if ! provider_invoke_argv "$tier" "$prompt" \
- || [ "${#_LOKI_INVOKE_ARGV[@]}" -eq 0 ]; then
- log_error "Provider ${PROVIDER_NAME:-unknown} failed to build invocation argv"
- return 125
- fi
- LOKI_DEADLINE_IDLE_TIMEOUT="${LOKI_PROVIDER_IDLE_TIMEOUT:-0}" \
- _loki_with_deadline "${LOKI_PROVIDER_CALL_TIMEOUT:-0}" \
- "${_LOKI_INVOKE_ARGV[@]}" < /dev/null 2>&1 \
- | tee -a "$log_file" "$agent_log" "$iter_output"
- _loki_argv_pipe_status=("${PIPESTATUS[@]}")
- return "$(_loki_provider_pipeline_exit_code \
- "${_loki_argv_pipe_status[0]:-125}" \
- "${_loki_argv_pipe_status[1]:-125}" 0)"
- }
- _LOKI_DEPENDENCY_SETUP_LIB="${SCRIPT_DIR}/lib/dependency-setup.sh"
- if [ -r "$_LOKI_DEPENDENCY_SETUP_LIB" ]; then
- # shellcheck source=lib/dependency-setup.sh
- source "$_LOKI_DEPENDENCY_SETUP_LIB"
- fi
- _loki_workspace_changed_since_iteration() {
- local start_sha="$1"
- local tree="${TARGET_DIR:-.}"
- [ -n "$start_sha" ] || return 0
- git -C "$tree" rev-parse --verify "$start_sha" >/dev/null 2>&1 || return 0
- git -C "$tree" diff --quiet "$start_sha" -- >/dev/null 2>&1
- case "$?" in
- 1) return 0 ;;
- 0) ;;
- *) return 0 ;;
- esac
- local untracked path
- untracked=$(git -C "$tree" ls-files --others --exclude-standard 2>/dev/null) || return 0
- while IFS= read -r path; do
- [ -n "$path" ] || continue
- case "$path" in
- .loki|.loki/*) continue ;;
- *) return 0 ;;
- esac
- done << UNTRACKED_EOF
- $untracked
- UNTRACKED_EOF
- return 1
- }
- _loki_supervised_claude_isolation_ready() {
- loki_is_supervised_simple_web || return 1
- type loki_claude_flag_supported >/dev/null 2>&1 || return 1
- loki_claude_flag_supported "--setting-sources" || return 1
- loki_claude_flag_supported "--disallowedTools" || return 1
- loki_claude_flag_supported "--tools" || return 1
- loki_claude_flag_supported "--mcp-config" || return 1
- loki_claude_flag_supported "--strict-mcp-config" || return 1
- loki_claude_flag_supported "--disable-slash-commands" || return 1
- }
- _loki_write_supervised_simple_web_policy() {
- loki_is_supervised_simple_web || return 0
- local loki_dir="${LOKI_DIR:-${TARGET_DIR:-.}/.loki}"
- local policy_file="$loki_dir/state/execution-policy.json"
- mkdir -p "$(dirname "$policy_file")" || return 1
- _LOKI_POLICY_FILE="$policy_file" \
- _LOKI_POLICY_MODEL="${LOKI_SESSION_MODEL:-}" \
- _LOKI_POLICY_MODEL_ID="${LOKI_SDK_REVIEW_MODEL:-}" \
- _LOKI_POLICY_LEARNINGS="$loki_dir" \
- _LOKI_POLICY_GIT_CONFIG="${GIT_CONFIG_GLOBAL:-}" \
- _LOKI_POLICY_NODE_STATUS="$_LOKI_NODE_PROBE_STATUS" \
- _LOKI_POLICY_NODE_VERSION="$_LOKI_NODE_PROBE_VERSION" \
- _LOKI_POLICY_NPM_STATUS="$_LOKI_NPM_PROBE_STATUS" \
- _LOKI_POLICY_NPM_VERSION="$_LOKI_NPM_PROBE_VERSION" \
- python3 -c '
- import json, os
- def number(name):
- try:
- return int(os.environ.get(name, "0"))
- except ValueError:
- return 0
- target = os.environ["_LOKI_POLICY_FILE"]
- policy = {
- "schema_version": 1,
- "policy": "supervised-simple-web",
- "active": True,
- "fail_closed": True,
- "build_profile": "simple-web",
- "verification_status": "required_not_yet_proven",
- "model": {
- "alias": os.environ.get("_LOKI_POLICY_MODEL", ""),
- "sdk_id": os.environ.get("_LOKI_POLICY_MODEL_ID", ""),
- "max_tier": os.environ.get("LOKI_MAX_TIER", ""),
- "nested_task_tool": False,
- "fallback": False,
- },
- "limits": {
- "iterations": number("LOKI_MAX_ITERATIONS"),
- "attempts": number("LOKI_MAX_RETRIES"),
- "provider_hard_seconds": number("LOKI_PROVIDER_CALL_TIMEOUT"),
- "provider_idle_seconds": number("LOKI_PROVIDER_IDLE_TIMEOUT"),
- "review_seconds": number("LOKI_REVIEW_CALL_TIMEOUT"),
- "council_seconds": number("LOKI_COUNCIL_REVIEW_TIMEOUT"),
- "gate_seconds": number("LOKI_GATE_TIMEOUT"),
- },
- "skipped": [
- "dashboard", "status_monitor", "resource_monitor", "enterprise_services",
- "spec_grill", "done_recognition", "prd_model_enrichment", "auto_docs",
- "auto_wiki", "caveman", "completion_model_voters",
- ],
- "required_gates": [
- "production_build", "unit_tests", "browser_e2e", "accessibility",
- "security", "code_review", "completion_checklist", "completion_heldout",
- "completion_evidence", "completion_assumptions", "proof",
- ],
- "state": {
- "learnings_root": os.environ.get("_LOKI_POLICY_LEARNINGS", ""),
- "git_config_global": os.environ.get("_LOKI_POLICY_GIT_CONFIG", ""),
- "git_baseline_required": True,
- },
- "runtime_probes": {
- "node": {"status": os.environ.get("_LOKI_POLICY_NODE_STATUS", ""), "version": os.environ.get("_LOKI_POLICY_NODE_VERSION", "")},
- "npm": {"status": os.environ.get("_LOKI_POLICY_NPM_STATUS", ""), "version": os.environ.get("_LOKI_POLICY_NPM_VERSION", "")},
- },
- }
- tmp = target + ".tmp." + str(os.getpid())
- with open(tmp, "w", encoding="utf-8") as f:
- json.dump(policy, f, indent=2, sort_keys=True)
- f.write("\n")
- os.replace(tmp, target)
- ' || return 1
- log_info "Execution policy: $policy_file"
- }
- # SDLC Phase Controls (all enabled by default)
- PHASE_UNIT_TESTS=${LOKI_PHASE_UNIT_TESTS:-true}
- PHASE_API_TESTS=${LOKI_PHASE_API_TESTS:-true}
- PHASE_E2E_TESTS=${LOKI_PHASE_E2E_TESTS:-true}
- PHASE_SECURITY=${LOKI_PHASE_SECURITY:-true}
- PHASE_INTEGRATION=${LOKI_PHASE_INTEGRATION:-true}
- PHASE_CODE_REVIEW=${LOKI_PHASE_CODE_REVIEW:-true}
- PHASE_WEB_RESEARCH=${LOKI_PHASE_WEB_RESEARCH:-true}
- PHASE_PERFORMANCE=${LOKI_PHASE_PERFORMANCE:-true}
- PHASE_ACCESSIBILITY=${LOKI_PHASE_ACCESSIBILITY:-true}
- PHASE_REGRESSION=${LOKI_PHASE_REGRESSION:-true}
- PHASE_UAT=${LOKI_PHASE_UAT:-true}
- # Autonomous Loop Controls (Ralph Wiggum Mode)
- # Default: No auto-completion - runs until max iterations or explicit promise
- COMPLETION_PROMISE=${LOKI_COMPLETION_PROMISE:-""}
- MAX_ITERATIONS=${LOKI_MAX_ITERATIONS:-1000}
- ITERATION_COUNT=0
- # If this is an auto-fix task, allow more iterations
- if [[ "${LOKI_AUTO_FIX:-}" == "true" ]]; then
- MAX_ITERATIONS="${LOKI_MAX_ITERATIONS:-5}"
- fi
- # Perpetual mode: never stop unless max iterations (ignores all completion signals)
- PERPETUAL_MODE=${LOKI_PERPETUAL_MODE:-false}
- # F4: bound the runaway ceiling for ORDINARY runs.
- #
- # MEASURED, not chosen. Real per-iteration wall clock across every recorded run
- # on this machine: median 718s, max 1746s. So the 1000 default is an 8.3-DAY
- # ceiling. When this analysis was written it was also the ONLY backstop,
- # because the other two valves shipped disabled. That is no longer true of the
- # budget: since v9.42.0 LOKI_BUDGET_LIMIT defaults to "100.00" (see :527 --
- # note `-` not `:-`, so only an EXPLICIT "" means unlimited), and breaching it
- # pauses the run. LOKI_MAX_DURATION does still default to 0 (check_max_duration
- # returns "never stop" at :18278).
- #
- # It also contradicts our own documentation. SETUP.md tells users to RAISE the
- # budget for large work with LOKI_MAX_ITERATIONS=40, and the demo uses 10 -- so
- # the shipped default is 25x the documented "large" setting.
- #
- # What real runs actually use: 1, 1, 3, 4. Every one terminated `completed` via
- # council approval or a completion promise; NONE hit a cap. Those are the
- # evidence-driven terminals, and they are unaffected by this -- the cap is a
- # backstop, not the mechanism.
- #
- # 25 is deliberately generous against that evidence (6x the observed maximum),
- # because the research is explicit that a too-small cap fails runs whose
- # approach was sound: 1-2 caps fail even when the agent was on track, and the
- # recommended range is 5-10. This is not a first-pass target; F0 (v8.45.0)
- # already stops a doomed run at its CAUSE, which is the better instrument.
- #
- # TWO GUARDS, both load-bearing:
- # - an explicit LOKI_MAX_ITERATIONS always wins, so nobody's setting changes
- # - PERPETUAL_MODE is untouched: it deliberately ignores every completion
- # signal and relies on max-iterations as its ONLY stop, so lowering the cap
- # there would silently truncate exactly the runs that opted out of stopping
- if [ -z "${LOKI_MAX_ITERATIONS:-}" ] \
- && [ "$PERPETUAL_MODE" != "true" ] \
- && [ "${LOKI_AUTO_FIX:-}" != "true" ]; then
- MAX_ITERATIONS="${LOKI_MAX_ITERATIONS_DEFAULT:-25}"
- fi
- # Enterprise background service PIDs (OTEL bridge, audit subscriber, integration sync)
- ENTERPRISE_PIDS=()
- # Portable lock helper (v7.5.12) -- mkdir-mutex replacement for flock(1).
- # Provides safe_acquire_lock / safe_release_lock / safe_with_lock so bash
- # callers no longer need a Linux-only flock binary. Macs do not ship
- # flock; pre-7.5.12 the fallback was a non-atomic PID check that emitted
- # "[WARN] flock not available - using non-atomic PID check ...".
- LOCK_LIB="$SCRIPT_DIR/lib/lock.sh"
- if [ -f "$LOCK_LIB" ]; then
- # shellcheck source=lib/lock.sh
- source "$LOCK_LIB"
- fi
- # Guarded browser opener (loki_open_url): the only way run.sh opens a browser.
- if [ -f "$SCRIPT_DIR/lib/browser-open.sh" ]; then
- # shellcheck source=lib/browser-open.sh
- source "$SCRIPT_DIR/lib/browser-open.sh"
- fi
- # Git PR advisory (shared print-only helper for create_session_pr and loki deploy)
- GIT_PR_ADVISORY_LIB="$SCRIPT_DIR/lib/git-pr-advisory.sh"
- if [ -f "$GIT_PR_ADVISORY_LIB" ]; then
- # shellcheck source=lib/git-pr-advisory.sh
- source "$GIT_PR_ADVISORY_LIB"
- fi
- # Proven PR (Loop 6): shared print-only Evidence Receipt renderer for PR bodies.
- # render_evidence_receipt_md prints the run's honest headline + facts +
- # verify-yourself block into the PR body. Pure, never pushes/PRs/mutates.
- PROOF_PR_LIB="$SCRIPT_DIR/lib/proof-pr.sh"
- if [ -f "$PROOF_PR_LIB" ]; then
- # shellcheck source=lib/proof-pr.sh
- source "$PROOF_PR_LIB"
- fi
- # Proven PR (Loop 6 / Slice B): optional advisory verified-completion check-run.
- # Owned by Slice B (autonomy/lib/proof-check.sh); sourced guarded so this slice
- # is correct whether or not the file is present in the tree, and the single call
- # site is itself guarded on declare -f + LOKI_PROVEN_PR_CHECK.
- PROOF_CHECK_LIB="$SCRIPT_DIR/lib/proof-check.sh"
- if [ -f "$PROOF_CHECK_LIB" ]; then
- # shellcheck source=lib/proof-check.sh
- source "$PROOF_CHECK_LIB"
- fi
- # Completion Council (v5.25.0) - Multi-agent completion verification
- # Source completion council module
- COUNCIL_SCRIPT="$SCRIPT_DIR/completion-council.sh"
- if [ -f "$COUNCIL_SCRIPT" ]; then
- # shellcheck source=completion-council.sh
- source "$COUNCIL_SCRIPT"
- fi
- # PRD Checklist module (v5.44.0)
- if [ -f "${SCRIPT_DIR}/prd-checklist.sh" ]; then
- # shellcheck source=prd-checklist.sh
- source "${SCRIPT_DIR}/prd-checklist.sh"
- fi
- # App Runner module (v5.45.0)
- if [ -f "${SCRIPT_DIR}/app-runner.sh" ]; then
- # shellcheck source=app-runner.sh
- source "${SCRIPT_DIR}/app-runner.sh"
- fi
- # Build-time HOME isolation (F49).
- #
- # When Loki executes/tests the GENERATED app during a build (the app-runner
- # server launch, restarts, and the project's own test suite in
- # enforce_test_coverage), the child process inherits Loki's environment --
- # including the user's REAL $HOME. A generated app that defaults its state file
- # to a HOME-relative path (e.g. a todo CLI writing ~/.todo.json) would then
- # litter the user's home directory with Loki's in-build test data. These two
- # helpers give those in-build executions an isolated HOME/XDG/TMPDIR rooted
- # under .loki/ so the app can run normally but cannot write into the real home.
- #
- # Scope: ONLY Loki's own in-build test executions are sandboxed. The user's
- # later manual `npm start`/run of the app is unaffected -- they invoke it
- # themselves in their own shell with their own HOME.
- # Lazily create and echo a persistent isolated HOME directory under .loki.
- # Persistent (not a per-call mktemp) so an app launched in one iteration and
- # restarted in a later one keeps a stable home, and so its state survives
- # across the build the same way a real run would.
- _loki_app_sandbox_dir() {
- local _base="${TARGET_DIR:-.}/.loki/app-sandbox"
- if [ ! -d "$_base/home" ]; then
- mkdir -p "$_base/home" "$_base/config" "$_base/data" \
- "$_base/cache" "$_base/state" "$_base/tmp" 2>/dev/null || return 1
- fi
- printf '%s\n' "$_base"
- }
- # Run a command with the build-time app sandbox env applied, then restore the
- # previous environment. Works for both shell functions (app_runner_start) and
- # external commands because it only mutates env vars around the call. The
- # background app launched by app_runner_start captures the sandbox HOME at fork
- # time and keeps it for its lifetime; restoring here ensures the orchestrator
- # (and the next iteration's provider invocation, which needs the REAL HOME for
- # OAuth/credentials) is never left pointing at the sandbox.
- _loki_with_app_sandbox() {
- local _sb
- _sb=$(_loki_app_sandbox_dir 2>/dev/null) || _sb=""
- # If the sandbox could not be created, run unsandboxed rather than failing
- # the build -- correctness of the app run takes priority over isolation.
- if [ -z "$_sb" ]; then
- "$@"
- return $?
- fi
- # Resolve to an absolute path so child `cd` into the target dir cannot make
- # a relative HOME point somewhere unexpected.
- local _abs
- _abs=$(cd "$_sb" 2>/dev/null && pwd) || _abs="$_sb"
- # Save current values plus their set/unset state so restore is exact.
- local _had_home="${HOME+x}" _old_home="${HOME:-}"
- local _had_xch="${XDG_CONFIG_HOME+x}" _old_xch="${XDG_CONFIG_HOME:-}"
- local _had_xdh="${XDG_DATA_HOME+x}" _old_xdh="${XDG_DATA_HOME:-}"
- local _had_xcache="${XDG_CACHE_HOME+x}" _old_xcache="${XDG_CACHE_HOME:-}"
- local _had_xsh="${XDG_STATE_HOME+x}" _old_xsh="${XDG_STATE_HOME:-}"
- local _had_tmp="${TMPDIR+x}" _old_tmp="${TMPDIR:-}"
- export HOME="$_abs/home"
- export XDG_CONFIG_HOME="$_abs/config"
- export XDG_DATA_HOME="$_abs/data"
- export XDG_CACHE_HOME="$_abs/cache"
- export XDG_STATE_HOME="$_abs/state"
- export TMPDIR="$_abs/tmp"
- local _rc=0
- "$@" || _rc=$?
- # Restore exactly (re-export prior value, or unset if it was unset before).
- if [ -n "$_had_home" ]; then export HOME="$_old_home"; else unset HOME; fi
- if [ -n "$_had_xch" ]; then export XDG_CONFIG_HOME="$_old_xch"; else unset XDG_CONFIG_HOME; fi
- if [ -n "$_had_xdh" ]; then export XDG_DATA_HOME="$_old_xdh"; else unset XDG_DATA_HOME; fi
- if [ -n "$_had_xcache" ]; then export XDG_CACHE_HOME="$_old_xcache"; else unset XDG_CACHE_HOME; fi
- if [ -n "$_had_xsh" ]; then export XDG_STATE_HOME="$_old_xsh"; else unset XDG_STATE_HOME; fi
- if [ -n "$_had_tmp" ]; then export TMPDIR="$_old_tmp"; else unset TMPDIR; fi
- return "$_rc"
- }
- # Playwright Smoke Test module (v5.46.0)
- if [ -f "${SCRIPT_DIR}/playwright-verify.sh" ]; then
- # shellcheck source=playwright-verify.sh
- source "${SCRIPT_DIR}/playwright-verify.sh"
- fi
- # Anonymous usage telemetry (opt-out: LOKI_TELEMETRY_DISABLED=true or DO_NOT_TRACK=1)
- # Also check persistent opt-out from ~/.loki/config (#77)
- if [ -f "${HOME}/.loki/config" ] && grep -q "^TELEMETRY_DISABLED=true" "${HOME}/.loki/config" 2>/dev/null; then
- LOKI_TELEMETRY_DISABLED=true
- export LOKI_TELEMETRY_DISABLED
- unset LOKI_OTEL_ENDPOINT
- fi
- TELEMETRY_SCRIPT="$SCRIPT_DIR/telemetry.sh"
- if [ -f "$TELEMETRY_SCRIPT" ]; then
- # shellcheck source=telemetry.sh
- source "$TELEMETRY_SCRIPT"
- fi
- # Crash-reporting helpers (Phase 0: local-only, zero egress).
- # Provides loki_collection_enabled (unified opt-out), loki_crash_capture,
- # loki_crash_friction, loki_show_disclosure_once.
- CRASH_SCRIPT="$SCRIPT_DIR/crash.sh"
- if [ -f "$CRASH_SCRIPT" ]; then
- # shellcheck source=crash.sh
- source "$CRASH_SCRIPT"
- fi
- # 2026 Research Enhancements (minimal additions)
- PROMPT_REPETITION=${LOKI_PROMPT_REPETITION:-true}
- CONFIDENCE_ROUTING=${LOKI_CONFIDENCE_ROUTING:-true}
- AUTONOMY_MODE=${LOKI_AUTONOMY_MODE:-perpetual} # perpetual|checkpoint|supervised
- # Session-pinned model (S0.1): pin a single tier for the whole main loop.
- # Default is "sonnet" -> resolved via the "development" tier in the abstract
- # tier map. Set LOKI_LEGACY_TIER_SWITCHING=true to restore the old
- # per-iteration RARV-driven tier rotation in the main loop. The
- # get_rarv_tier function is preserved for subagent dispatch regardless.
- LOKI_SESSION_MODEL="${LOKI_SESSION_MODEL:-sonnet}"
- LOKI_LEGACY_TIER_SWITCHING="${LOKI_LEGACY_TIER_SWITCHING:-false}"
- export LOKI_SESSION_MODEL LOKI_LEGACY_TIER_SWITCHING
- # Managed Agents (v6.83.0 Phase 1): opt-in integration with Claude Managed Agents
- # Memory store backend. Parent flag gates everything; child flags gate features.
- # Both off (default) => zero behavior change from v6.82.0.
- LOKI_MANAGED_AGENTS="${LOKI_MANAGED_AGENTS:-false}"
- LOKI_MANAGED_MEMORY="${LOKI_MANAGED_MEMORY:-false}"
- # v7.0.0 Phase 2: remote->local hydrate on session boot (grandchild of MEMORY).
- # Pulls semantic patterns + procedural skills once at init_loki_dir time.
- LOKI_MANAGED_MEMORY_HYDRATE="${LOKI_MANAGED_MEMORY_HYDRATE:-false}"
- # v7.0.0 Phase 3+4 foundation: umbrella flag for the multiagent-session path.
- # Gates every providers/managed.py entry point (run_council,
- # run_completion_council). Off by default because the Managed Agents
- # multiagent surface is a research preview.
- LOKI_EXPERIMENTAL_MANAGED_AGENTS="${LOKI_EXPERIMENTAL_MANAGED_AGENTS:-false}"
- # v7.0.0 Phase 3 (T5): managed code-review council. Routes run_code_review
- # through providers/managed.py::run_council when true. Requires parent +
- # umbrella.
- LOKI_EXPERIMENTAL_MANAGED_REVIEW="${LOKI_EXPERIMENTAL_MANAGED_REVIEW:-false}"
- # v7.0.0 Phase 4 (T6): managed completion council. Routes council_should_stop
- # through providers/managed.py::run_completion_council when true. Requires
- # parent + umbrella.
- LOKI_EXPERIMENTAL_MANAGED_COUNCIL="${LOKI_EXPERIMENTAL_MANAGED_COUNCIL:-false}"
- export LOKI_MANAGED_AGENTS LOKI_MANAGED_MEMORY LOKI_MANAGED_MEMORY_HYDRATE LOKI_EXPERIMENTAL_MANAGED_AGENTS LOKI_EXPERIMENTAL_MANAGED_REVIEW LOKI_EXPERIMENTAL_MANAGED_COUNCIL
- # Fail-fast: child on with parent off is a misconfiguration.
- if [ "$LOKI_MANAGED_MEMORY" = "true" ] && [ "$LOKI_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_MANAGED_MEMORY=true requires LOKI_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- # Phase 2 fail-fast: HYDRATE is a grandchild of MEMORY.
- if [ "$LOKI_MANAGED_MEMORY_HYDRATE" = "true" ] && [ "$LOKI_MANAGED_MEMORY" != "true" ]; then
- echo "ERROR: LOKI_MANAGED_MEMORY_HYDRATE=true requires LOKI_MANAGED_MEMORY=true" >&2
- exit 2
- fi
- # Same fail-fast for the experimental multiagent session path.
- if [ "$LOKI_EXPERIMENTAL_MANAGED_AGENTS" = "true" ] && [ "$LOKI_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_EXPERIMENTAL_MANAGED_AGENTS=true requires LOKI_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- # Phase 3 fail-fast: REVIEW requires parent AND umbrella.
- if [ "$LOKI_EXPERIMENTAL_MANAGED_REVIEW" = "true" ]; then
- if [ "$LOKI_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_EXPERIMENTAL_MANAGED_REVIEW=true requires LOKI_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- if [ "$LOKI_EXPERIMENTAL_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_EXPERIMENTAL_MANAGED_REVIEW=true requires LOKI_EXPERIMENTAL_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- fi
- # Phase 4 fail-fast: COUNCIL requires parent AND umbrella.
- if [ "$LOKI_EXPERIMENTAL_MANAGED_COUNCIL" = "true" ]; then
- if [ "$LOKI_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_EXPERIMENTAL_MANAGED_COUNCIL=true requires LOKI_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- if [ "$LOKI_EXPERIMENTAL_MANAGED_AGENTS" != "true" ]; then
- echo "ERROR: LOKI_EXPERIMENTAL_MANAGED_COUNCIL=true requires LOKI_EXPERIMENTAL_MANAGED_AGENTS=true" >&2
- exit 2
- fi
- fi
- # Research-preview warning banner.
- if [ "$LOKI_EXPERIMENTAL_MANAGED_AGENTS" = "true" ]; then
- echo "WARN: LOKI_EXPERIMENTAL_MANAGED_AGENTS uses Managed Agents research preview; expect beta churn." >&2
- fi
- # Parallel Workflows (Git Worktrees)
- PARALLEL_MODE=${LOKI_PARALLEL_MODE:-false}
- MAX_WORKTREES=${LOKI_MAX_WORKTREES:-5}
- MAX_PARALLEL_SESSIONS=${LOKI_MAX_PARALLEL_SESSIONS:-3}
- # Nested-agent guard: the fixed testing/docs sub-streams each spawn a FURTHER
- # nested `claude` session in a NON-namespaced worktree (`<project>-testing`,
- # `<project>-docs`). When Loki itself runs inside another agent session (Claude
- # Code sets CLAUDECODE; the Loki Mode skill, or a user driving the CLI as a
- # background process from an agent), those nested spawns fight over the shared
- # checkout/worktree paths and flood the log -- the reported "N issues in one dir"
- # failure. So default the sub-streams OFF when nested; the core issue work still
- # runs. Explicit LOKI_PARALLEL_TESTING/DOCS=true overrides (opt back in). This
- # does NOT disable the user's own --parallel/--pr issue work, only the auxiliary
- # testing/docs fan-out that is unsafe to nest.
- _loki_nested_agent=false
- if [ -n "${CLAUDECODE:-}" ] || [ -n "${LOKI_NESTED_AGENT:-}" ]; then
- _loki_nested_agent=true
- fi
- if [ "$_loki_nested_agent" = "true" ]; then
- PARALLEL_TESTING=${LOKI_PARALLEL_TESTING:-false}
- PARALLEL_DOCS=${LOKI_PARALLEL_DOCS:-false}
- else
- PARALLEL_TESTING=${LOKI_PARALLEL_TESTING:-true}
- PARALLEL_DOCS=${LOKI_PARALLEL_DOCS:-true}
- fi
- # Dynamic resource-aware session concurrency (Release 3, slice 3).
- # DEFAULT OFF: when LOKI_DYNAMIC_CONCURRENCY is unset, effective_session_cap()
- # returns exactly MAX_PARALLEL_SESSIONS, so behavior is identical to before.
- # Opt in with LOKI_DYNAMIC_CONCURRENCY=1 to scale the session cap down when
- # system CPU or memory is under pressure (read from .loki/state/resources.json).
- DYNAMIC_CONCURRENCY=${LOKI_DYNAMIC_CONCURRENCY:-0}
- # Optional higher ceiling on capable machines. Only takes effect with dynamic
- # concurrency enabled; still resource-gated. Defaults to MAX_PARALLEL_SESSIONS.
- MAX_PARALLEL_SESSIONS_CEILING=${LOKI_MAX_PARALLEL_SESSIONS_CEILING:-$MAX_PARALLEL_SESSIONS}
- # Usage thresholds (percent). At/above CPU or MEM threshold the cap is halved.
- CONCURRENCY_CPU_THRESHOLD=${LOKI_CONCURRENCY_CPU_THRESHOLD:-85}
- CONCURRENCY_MEM_THRESHOLD=${LOKI_CONCURRENCY_MEM_THRESHOLD:-85}
- # Critical threshold (percent). At/above this the cap is forced to 1.
- CONCURRENCY_CRITICAL_THRESHOLD=${LOKI_CONCURRENCY_CRITICAL_THRESHOLD:-95}
- # Gate Escalation Ladder (v6.10.0). LOKI_GATE_CLEAR_LIMIT is retained for
- # compatibility, but it is now the repeated-blocker escalation threshold.
- # A blocking gate is never cleared into a pass merely because it repeated.
- GATE_CLEAR_LIMIT=${LOKI_GATE_CLEAR_LIMIT:-3}
- GATE_ESCALATE_LIMIT=${LOKI_GATE_ESCALATE_LIMIT:-5}
- GATE_PAUSE_LIMIT=${LOKI_GATE_PAUSE_LIMIT:-10}
- # Workspace resolution: the build runs against TARGET_DIR, defaulting to the
- # launch cwd. A caller can pin a per-build workspace by exporting
- # LOKI_TARGET_DIR (and the matching LOKI_DIR=<dir>/.loki); the dashboard
- # /api/control/start `workspace` param (Track-1 S1) does exactly this so a
- # hosted build runs in its own dir instead of the engine's repo. Every
- # $TARGET_DIR/.loki reference below then resolves under that workspace.
- TARGET_DIR="${LOKI_TARGET_DIR:-$(pwd)}"
- PARALLEL_BLOG=${LOKI_PARALLEL_BLOG:-false}
- AUTO_MERGE=${LOKI_AUTO_MERGE:-true}
- # Tier-aware harness policy (model-equivalence experiment, Planned/Stage-3).
- # Given the resolved tier, export the steering-lever env vars UNLESS the operator
- # already set them (operator override ALWAYS wins). Gated on
- # LOKI_TIER_HARNESS_POLICY (default 0 = off) until Stage-2 ablation data justifies
- # the numbers. Pure env-export, no control flow. The table below is a STUB --
- # replace with Stage-2 ablation-derived numbers (see
- # docs/MODEL-EQUIVALENCE-HARNESS-PLAN.md section 3).
- loki_apply_tier_harness_policy() {
- [ "${LOKI_TIER_HARNESS_POLICY:-0}" = "1" ] || return 0
- local tier="$1" _iters _council _heal
- # STUB - replace with Stage-2 ablation-derived numbers.
- case "$tier" in
- haiku|fast) _iters=8; _council=true; _heal=1 ;;
- sonnet|development) _iters=5; _council=true; _heal=1 ;;
- opus|planning) _iters=3; _council=true; _heal=0 ;;
- *) return 0 ;;
- esac
- [ -z "${LOKI_MAX_ITERATIONS:-}" ] && export LOKI_MAX_ITERATIONS="$_iters"
- [ -z "${LOKI_COUNCIL_ENABLED:-}" ] && export LOKI_COUNCIL_ENABLED="$_council"
- [ -z "${LOKI_SELF_HEAL:-}" ] && export LOKI_SELF_HEAL="$_heal"
- }
- # Multi-project registry (v7.7.29): register this running project in the
- # machine-global registry (~/.loki/dashboard/projects.json) so the dashboard
- # can list and switch between projects running in different folders. Records
- # the absolute path, pid, port, and status. Fully non-blocking and
- # failure-swallowed: registry problems must never affect a build. Marked
- # inactive again on exit via the trap below.
- loki_register_running_project() {
- local _status="${1:-running}"
- [ -n "${LOKI_SKIP_PROJECT_REGISTRY:-}" ] && return 0
- command -v python3 >/dev/null 2>&1 || return 0
- local _skill="${LOKI_SKILL_DIR:-${PROJECT_DIR:-$SCRIPT_DIR/..}}"
- LOKI_REG_TARGET="$TARGET_DIR" LOKI_REG_SKILL="$_skill" \
- LOKI_REG_PID="$$" LOKI_REG_PORT="${LOKI_DASHBOARD_PORT:-57374}" \
- LOKI_REG_STATUS="$_status" \
- python3 - <<'PYREG' >/dev/null 2>&1 || true
- import os, sys
- sys.path.insert(0, os.environ.get("LOKI_REG_SKILL", "."))
- try:
- from dashboard import registry
- target = os.path.abspath(os.environ["LOKI_REG_TARGET"])
- entry = registry.register_project(target)
- # Enrich with runtime fields the dashboard switcher uses.
- reg = registry._load_registry()
- pid = entry.get("id") or registry._generate_project_id(target)
- if pid in reg.get("projects", {}):
- reg["projects"][pid]["pid"] = int(os.environ.get("LOKI_REG_PID", "0") or 0)
- reg["projects"][pid]["port"] = int(os.environ.get("LOKI_REG_PORT", "57374") or 57374)
- reg["projects"][pid]["status"] = os.environ.get("LOKI_REG_STATUS", "running")
- registry._save_registry(reg)
- except Exception:
- pass
- PYREG
- }
- # v7.7.30: deliberate-exit teardown for the shared dashboard + registry.
- # Marks THIS project (abspath of TARGET_DIR) stopped in the machine-global
- # registry, then decides whether the shared standalone dashboard at
- # ~/.loki/dashboard/dashboard.pid should be killed. The shared dashboard is
- # killed ONLY when no other registered project still has a live pid (CLEAR);
- # if any other project is still running (KEEP) it is left up. NEVER uses a
- # blanket pkill and NEVER touches another folder's pids. Best-effort and
- # failure-swallowed: teardown bookkeeping must never block a clean exit.
- # Is this pid positively identified as our dashboard, rather than a recycled
- # number now naming something else? Ownership-sensitive cleanup must fail closed:
- # an absent `ps` result is not evidence, and a generic python/loki substring is
- # not specific enough to authorize a signal.
- _loki_pid_looks_like_dashboard() {
- local _p="$1" _cmd
- case "$_p" in ''|*[!0-9]*) return 1 ;; esac
- [ "$_p" -gt 1 ] 2>/dev/null || return 1 # never signal pid 0/1
- _cmd="$(ps -o command= -p "$_p" 2>/dev/null)"
- [ -n "$_cmd" ] || return 1
- # run.sh and `loki dashboard start` both launch this exact module form.
- # Anchor the interpreter and module tokens so an unrelated command merely
- # mentioning "dashboard.server" in an argument cannot forge identity.
- [[ "$_cmd" =~ ^[[:space:]]*([^[:space:]]*/)?[Pp]ython([0-9]+([.][0-9]+)*)?[[:space:]]+-m[[:space:]]+dashboard[.]server([[:space:]]|$) ]]
- }
- loki_mark_project_stopped_and_maybe_kill_shared_dashboard() {
- local _skill="${LOKI_SKILL_DIR:-${PROJECT_DIR:-$SCRIPT_DIR/..}}"
- local _shared_pidfile="${HOME}/.loki/dashboard/dashboard.pid"
- local _decision="CLEAR"
- if [ -z "${LOKI_SKIP_PROJECT_REGISTRY:-}" ] && command -v python3 >/dev/null 2>&1; then
- # (a) Mark this project stopped in the shared registry.
- LOKI_REG_TARGET="$TARGET_DIR" LOKI_REG_SKILL="$_skill" \
- python3 - <<'PYSTOP' >/dev/null 2>&1 || true
- import os, sys
- sys.path.insert(0, os.environ.get("LOKI_REG_SKILL", "."))
- try:
- from dashboard import registry
- registry.mark_project_stopped(os.path.abspath(os.environ["LOKI_REG_TARGET"]))
- except Exception:
- pass
- PYSTOP
- # (b) CLEAR/KEEP check: any OTHER project still alive keeps the
- # shared dashboard up (this project is already marked stopped above).
- _decision="$(LOKI_REG_SKILL="$_skill" python3 - <<'PYCHECK' 2>/dev/null || echo CLEAR
- import os, sys
- sys.path.insert(0, os.environ.get("LOKI_REG_SKILL", "."))
- try:
- from dashboard import registry
- alive = 0
- for p in registry.list_projects(include_inactive=True):
- pid = p.get("pid")
- if isinstance(pid, int) and pid > 0:
- try:
- os.kill(pid, 0)
- alive += 1
- except OSError:
- pass
- print("CLEAR" if alive == 0 else "KEEP")
- except Exception:
- print("CLEAR")
- PYCHECK
- )"
- fi
- # (c) Only tear down the SHARED dashboard when no other project remains
- # (CLEAR), or when python3 was unavailable (legacy fallback: avoid leaking
- # the shared dashboard on minimal systems).
- if [ "$_decision" = "CLEAR" ]; then
- if [ -f "$_shared_pidfile" ]; then
- local _shared_pid
- _shared_pid=$(cat "$_shared_pidfile" 2>/dev/null)
- # Identity check before signalling. dashboard.pid is removed only on
- # the explicit stop paths (:16608, :16658, :17399) -- nothing covers
- # a crash, so the file outlives its dashboard. Measured in this very
- # checkout: .loki/dashboard/dashboard.pid held a DEAD 87992 with an
- # 8-day-old mtime. PIDs recycle, so a stale number eventually names
- # an unrelated LIVE process and this `kill -9` hits it.
- #
- # kill -0 is NOT sufficient: a recycled pid is alive by definition,
- # which is exactly the insufficiency the loki.pgid self-check had.
- # This guard fails CLOSED: if `ps` says nothing, or says something
- # that is not the dashboard's own command, we do not signal. The
- # cost of failing closed is a leaked dashboard the next explicit
- # stop reaps; the cost of failing open was signalling a stranger.
- if [ -n "$_shared_pid" ] && _loki_pid_looks_like_dashboard "$_shared_pid"; then
- kill "$_shared_pid" 2>/dev/null || true
- sleep 0.5
- kill -9 "$_shared_pid" 2>/dev/null || true
- fi
- rm -f "$_shared_pidfile" 2>/dev/null || true
- fi
- # A port number and an HTTP response are not ownership evidence. In
- # particular, a 404 can be an unrelated healthy service. The shared PID
- # file plus positive command identity above is the only authority to
- # signal a process; absent or forged ownership leaves every port holder
- # untouched.
- fi
- }
- # Register as running now. We deliberately do NOT install an EXIT trap to
- # flip it to idle: a top-level EXIT trap here would be clobbered by the
- # lock-release EXIT trap installed later in the main path (and could
- # interfere with it). Instead the dashboard determines live vs stale by
- # checking whether the recorded pid is still alive (registry stores pid),
- # which is robust even on hard kills where a trap would never fire.
- loki_register_running_project running
- # Complexity Tiers (Auto-Claude pattern)
- # auto = detect from PRD/codebase, simple = 3 phases, standard = 6 phases, complex = 8 phases
- COMPLEXITY_TIER=${LOKI_COMPLEXITY:-auto}
- DETECTED_COMPLEXITY=""
- # Multi-Provider Support (v5.0.0)
- # Provider: auto-detected when unset; claude > opencode.
- #
- # WHY NOT `:-claude`. That default was a hard failure for anyone who has opencode
- # but not Claude: run.sh would select a provider that is not installed and die,
- # even though auto_detect_provider() has existed in providers/loader.sh -- with
- # the right priority order and its own passing test -- since v5.0.0. Nothing in
- # production ever called it. Detection was built, tested, and never wired.
- #
- # An EXPLICIT choice still wins: this only fills an unset value, so
- # LOKI_PROVIDER=opencode and --provider opencode are untouched.
- _LOKI_PROVIDER_WAS_EXPLICIT=1
- [ -z "${LOKI_PROVIDER:-}" ] && _LOKI_PROVIDER_WAS_EXPLICIT=0
- # Source provider configuration
- PROVIDERS_DIR="$PROJECT_DIR/providers"
- if [ -f "$PROVIDERS_DIR/loader.sh" ]; then
- # shellcheck source=/dev/null
- source "$PROVIDERS_DIR/loader.sh"
- # Detect only when the operator expressed no preference. Sourcing the
- # loader first is required -- auto_detect_provider() is defined there.
- if [ "$_LOKI_PROVIDER_WAS_EXPLICIT" -eq 0 ]; then
- _detected="$(auto_detect_provider 2>/dev/null || true)"
- if [ -n "$_detected" ]; then
- LOKI_PROVIDER="$_detected"
- [ -n "${LOKI_QUICK_INNER:-}" ] || echo "[loki] provider: $LOKI_PROVIDER (auto-detected)" >&2
- else
- # Nothing installed. Keep the historical default so the existing
- # "not installed" error path reports claude, which is the actionable
- # message -- rather than an empty provider name.
- LOKI_PROVIDER=claude
- fi
- fi
- # Validate provider. A removed provider (REMOVED_PROVIDERS, providers/loader.sh) gets
- # the one migration line from providers/loader.sh and exit 2.
- if loki_is_removed_provider "$LOKI_PROVIDER"; then
- loki_removed_provider_message "$LOKI_PROVIDER" >&2
- exit 2
- fi
- if ! validate_provider "$LOKI_PROVIDER"; then
- echo "ERROR: Unknown provider: $LOKI_PROVIDER" >&2
- echo "Supported providers: ${SUPPORTED_PROVIDERS[*]}" >&2
- exit 1
- fi
- # Load provider config
- if ! load_provider "$LOKI_PROVIDER"; then
- echo "ERROR: Failed to load provider config: $LOKI_PROVIDER" >&2
- exit 1
- fi
- # Save provider for future runs (if .loki dir exists or will be created)
- if [ -d ".loki/state" ] || mkdir -p ".loki/state" 2>/dev/null; then
- echo "$LOKI_PROVIDER" > ".loki/state/provider"
- fi
- else
- # Fallback: Claude-only mode (backwards compatibility)
- PROVIDER_NAME="claude"
- PROVIDER_CLI="claude"
- PROVIDER_AUTONOMOUS_FLAG="--dangerously-skip-permissions"
- PROVIDER_PROMPT_FLAG="-p"
- PROVIDER_DEGRADED=false
- PROVIDER_DISPLAY_NAME="Claude Code"
- PROVIDER_HAS_PARALLEL=true
- PROVIDER_HAS_SUBAGENTS=true
- PROVIDER_HAS_TASK_TOOL=true
- PROVIDER_HAS_MCP=true
- PROVIDER_PROMPT_POSITIONAL=false
- fi
- # Track worktree PIDs for cleanup (requires bash 4+ for associative arrays)
- # BASH_VERSION_MAJOR is defined at script startup
- if [ "$BASH_VERSION_MAJOR" -ge 4 ] 2>/dev/null; then
- declare -A WORKTREE_PIDS=()
- declare -A WORKTREE_PATHS=()
- declare -A WORKTREE_BASE_SHAS=()
- else
- # Fallback: parallel mode will check and warn
- # shellcheck disable=SC2178
- WORKTREE_PIDS=""
- # shellcheck disable=SC2178
- WORKTREE_PATHS=""
- # shellcheck disable=SC2178
- WORKTREE_BASE_SHAS=""
- fi
- # Track background install PIDs for cleanup (indexed array, works on all bash versions)
- WORKTREE_INSTALL_PIDS=()
- # Colors
- RED='\033[0;31m'
- GREEN='\033[0;32m'
- YELLOW='\033[1;33m'
- BLUE='\033[0;34m'
- CYAN='\033[0;36m'
- BOLD='\033[1m'
- DIM='\033[2m'
- NC='\033[0m'
- #===============================================================================
- # Logging Functions
- #===============================================================================
- log_header() {
- echo ""
- echo -e "${BLUE}╔════════════════════════════════════════════════════════════════╗${NC}"
- echo -e "${BLUE}║${NC} ${BOLD}$1${NC}"
- echo -e "${BLUE}╚════════════════════════════════════════════════════════════════╝${NC}"
- }
- # VERBOSITY. LOKI_LOG_LEVEL is one of debug|info|warn|error (default info);
- # LOKI_QUIET=1 is shorthand for warn.
- #
- # Implemented at these five functions rather than at 527 call sites, which is
- # both the smaller diff and the only version that cannot drift -- a new
- # log_info added next year is covered without anyone remembering to gate it.
- #
- # WHY THIS IS NEEDED DESPITE GOOD TTY DETECTION. The heavy decoration (HUD,
- # completion card, start headline) is already `[ -t 1 ]`-guarded and vanishes
- # off a TTY, so CI output was never the wall of banners it might have been.
- # But 527 log_info/log_step calls are unguarded and print regardless, and a
- # pipeline that wants only warnings had no way to ask. log_debug already
- # honored LOKI_DEBUG; the other levels honored nothing.
- #
- # ERRORS ARE NEVER SUPPRESSED. `error` is the floor: the quietest setting still
- # prints failures. A verbosity flag that can hide the reason a build failed is
- # a footgun, not a feature.
- _loki_log_threshold() {
- case "${LOKI_LOG_LEVEL:-$([ "${LOKI_QUIET:-0}" = "1" ] && echo warn || echo info)}" in
- debug) echo 0 ;;
- info) echo 1 ;;
- warn) echo 2 ;;
- error) echo 3 ;;
- *) echo 1 ;; # unrecognized value behaves as the default, never silences
- esac
- }
- _loki_log_enabled() { [ "$1" -ge "$(_loki_log_threshold)" ]; }
- log_info() { _loki_log_enabled 1 && echo -e "${GREEN}[INFO]${NC} $*" || true; }
- log_warn() { _loki_log_enabled 2 && echo -e "${YELLOW}[WARN]${NC} $*" || true; }
- log_warning() { log_warn "$@"; } # Alias for backwards compatibility
- log_error() { local _fd=1; [ -z "${LOKI_QUICK_INNER:-}" ] || _fd=2; echo -e "${RED}[ERROR]${NC} $*" >&"$_fd"; }
- log_step() { _loki_log_enabled 1 && echo -e "${CYAN}[STEP]${NC} $*" || true; }
- log_debug() { [[ "${LOKI_DEBUG:-}" == "true" ]] && echo -e "${CYAN}[DEBUG]${NC} $*" >&2 || true; }
- #===============================================================================
- # Failure diagnosis helpers (T2.4 / T2.5 / T2.6)
- #
- # These make a failing or crashed build self-explanatory: a copy-pasteable
- # "loki why" hint on any non-zero exit, a durable CLASSIFIED LAST_ERROR record
- # on a failed iteration, and a best-effort terminal record on an untrapped
- # death. Every one is best-effort and NEVER alters the build's exit code.
- #===============================================================================
- # _loki_surface_why_hint (T2.4): print the "loki why" hint to stderr and write
- # it to .loki/NEXT_STEPS.txt. Called once from main() finalization when the run
- # failed (result != 0). Best-effort: never crashes, never changes exit code.
- _loki_surface_why_hint() {
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local hint="For a plain-language diagnosis of what happened, run: loki why"
- printf '%s\n' "$hint" >&2 || true
- mkdir -p "$loki_dir" 2>/dev/null || true
- printf '%s\n' "$hint" > "$loki_dir/NEXT_STEPS.txt" 2>/dev/null || true
- return 0
- }
- # _loki_restore_base_branch (FC-EMPTY-RUN-BRANCH-LEFT-CHECKED-OUT, legacy sibling
- # of engine10 finishRunBranch): when a session ended non-zero and its branch
- # tip is still the recorded start commit, nothing was committed. Put the checkout
- # back on the starting branch and delete that empty session branch. Narrower than
- # the engine10 side, which also bundles a branch with commits into .loki/runs/
- # before removing it: here a branch with any commit is never touched, a dirty tracked
- # tree keeps the branch and prints the way back, and a stop signal (130, 143) is
- # left alone because the cleanup trap owns it. Best-effort, never changes the
- # exit code.
- _loki_restore_base_branch() {
- local result="${1:-0}" td="${TARGET_DIR:-.}"
- case "$result" in 0|130|143) return 0 ;; esac
- local base_file="$td/.loki/state/base-branch.txt" sha_file="$td/.loki/state/session-start-sha" br_file="$td/.loki/state/agent-branch.txt"
- [ -s "$base_file" ] && [ -s "$sha_file" ] && [ -s "$br_file" ] || return 0
- local base branch start tip cur
- base="$(cat "$base_file" 2>/dev/null)"; branch="$(cat "$br_file" 2>/dev/null)"; start="$(cat "$sha_file" 2>/dev/null)"
- case "$branch" in loki/session-*) ;; *) return 0 ;; esac
- [ -n "$base" ] && [ "$base" != "$branch" ] || return 0
- tip="$(git -C "$td" rev-parse --verify -q "refs/heads/$branch" 2>/dev/null)" || return 0
- [ "$tip" = "$start" ] || return 0
- cur="$(git -C "$td" symbolic-ref --short -q HEAD 2>/dev/null || true)"
- if [ "$cur" = "$branch" ]; then
- if [ -n "$(git -C "$td" status --porcelain --untracked-files=no 2>/dev/null)" ]; then
- printf 'loki: tree left on %s, return with: git checkout %s\n' "$branch" "$base" >&2
- return 0
- fi
- git -C "$td" checkout -q "$base" 2>/dev/null || { printf 'loki: tree left on %s, return with: git checkout %s\n' "$branch" "$base" >&2; return 0; }
- cur="$base"
- fi
- [ "$cur" = "$base" ] || return 0
- git -C "$td" branch -d "$branch" >/dev/null 2>&1 && rm -f "$br_file" 2>/dev/null
- return 0
- }
- # _loki_write_last_error (T2.5): write a durable, classified failure record to
- # .loki/state/LAST_ERROR.json. Schema:
- # {
- # "iteration": <int>,
- # "error_class": "provider_empty_output"|"build_timeout"|"rate_limited"
- # |"auth_error"|"unknown",
- # "brief": "<one honest sentence>",
- # "timestamp": "<UTC ISO-8601>"
- # }
- # This is what `loki why` (in autonomy/loki, NOT owned here) can later read.
- # Built via python3 so the free-text brief can never break the JSON. Entirely
- # best-effort: any failure is swallowed and the build is never crashed.
- # Usage: _loki_write_last_error <iteration> <error_class> <brief>
- _loki_write_last_error() {
- local iteration="${1:-0}"
- local error_class="${2:-unknown}"
- local brief="${3:-An iteration failed.}"
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local state_dir="$loki_dir/state"
- mkdir -p "$state_dir" 2>/dev/null || true
- LOKI_LE_ITER="$iteration" \
- LOKI_LE_CLASS="$error_class" \
- LOKI_LE_BRIEF="$brief" \
- LOKI_LE_FILE="$state_dir/LAST_ERROR.json" \
- python3 -c "
- import json, os, tempfile
- try:
- rec = {
- 'iteration': int(os.environ.get('LOKI_LE_ITER', '0') or 0),
- 'error_class': os.environ.get('LOKI_LE_CLASS', 'unknown'),
- 'brief': os.environ.get('LOKI_LE_BRIEF', ''),
- 'timestamp': __import__('datetime').datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ'),
- }
- target = os.environ['LOKI_LE_FILE']
- d = os.path.dirname(target)
- fd, tmp = tempfile.mkstemp(dir=d, suffix='.json')
- with os.fdopen(fd, 'w') as f:
- json.dump(rec, f)
- os.replace(tmp, target)
- except Exception:
- pass
- " 2>/dev/null || true
- return 0
- }
- # _loki_archive_last_error <last_error_json> <history_jsonl>
- # LEARN-FORWARD: append a prior run's LAST_ERROR record to an append-only,
- # bounded failure-history so the next run can detect a REPEATED failure signature
- # (same error_class recurring) even though the single LAST_ERROR.json is cleared
- # each run. No-op when there is no prior error. Bounded to the most recent 50
- # entries so it never grows unbounded. Best-effort; never fails the run.
- _loki_archive_last_error() {
- local src="${1:-}"
- local hist="${2:-}"
- [ -n "$src" ] && [ -n "$hist" ] || return 0
- [ -f "$src" ] || return 0
- LOKI_AE_SRC="$src" LOKI_AE_HIST="$hist" python3 -c "
- import json, os, tempfile
- src = os.environ['LOKI_AE_SRC']
- hist = os.environ['LOKI_AE_HIST']
- try:
- with open(src) as f:
- rec = json.load(f)
- if not isinstance(rec, dict):
- raise ValueError('bad record')
- lines = []
- if os.path.exists(hist):
- try:
- with open(hist) as hf:
- lines = [ln for ln in hf.read().splitlines() if ln.strip()]
- except Exception:
- lines = []
- lines.append(json.dumps(rec, separators=(',', ':')))
- lines = lines[-50:]
- d = os.path.dirname(hist) or '.'
- fd, tmp = tempfile.mkstemp(dir=d, suffix='.jsonl')
- with os.fdopen(fd, 'w') as f:
- f.write('\n'.join(lines) + '\n')
- os.replace(tmp, hist)
- except Exception:
- pass
- " 2>/dev/null || true
- return 0
- }
- # _loki_classify_iteration_error (T2.5 helper): map an iteration's signals to one
- # of the LAST_ERROR error_class values. Conservative: only returns a specific
- # class when a signal confidently supports it, else "unknown". Never fabricates
- # build_timeout (no timeout signal is detected here, so it is reserved for a
- # caller that has one). Echoes the class on stdout.
- # Usage: _loki_classify_iteration_error <iter_output_file> <empty_output_flag 0|1>
- _loki_classify_iteration_error() {
- local iter_output="${1:-}"
- local empty_flag="${2:-0}"
- if [ "$empty_flag" = "1" ]; then
- echo "provider_empty_output"
- return 0
- fi
- # A failed provider start (for example claude -p exiting 1) leaves a 0-byte log.
- if [ -n "$iter_output" ] && [ -f "$iter_output" ] && [ ! -s "$iter_output" ]; then
- echo "provider_empty_output"
- return 0
- fi
- # Rate limit: reuse the same detector the retry path uses.
- if [ -n "$iter_output" ] && [ -f "$iter_output" ] && detect_rate_limit "$iter_output" 2>/dev/null | grep -qE '^[1-9]'; then
- echo "rate_limited"
- return 0
- fi
- # Auth error: a clear 401/403/unauthorized in the output tail.
- if [ -n "$iter_output" ] && [ -f "$iter_output" ]; then
- local _tail
- _tail="$(tail -n 40 "$iter_output" 2>/dev/null || true)"
- if printf '%s\n' "$_tail" | grep -qiE '(http[ /]?40[13]|status[: ]+40[13]|unauthorized|invalid api key|authentication[_ ]error|401 )' 2>/dev/null; then
- echo "auth_error"
- return 0
- fi
- fi
- echo "unknown"
- return 0
- }
- # _loki_build_self_heal_hint (SLICE 6c): route the PRIOR iteration's classified
- # error signature into the NEXT iteration's prompt so the loop fixes forward.
- # Reads .loki/state/LAST_ERROR.json (written by _loki_write_last_error on a
- # failed iteration), emits a concise structured heal hint naming the error_class
- # + brief on stdout, then ARCHIVES-then-CLEARS the record so the hint injects
- # exactly ONCE (reuses the existing archive-then-clear pattern; it does not
- # repeat forever). Best-effort: any failure emits nothing and never crashes the
- # build.
- #
- # Gated by the caller on LOKI_SELF_HEAL (default 0 -- opt-in, stock runs
- # unaffected). "unknown" is treated as NOT actionable (no specific class to
- # target), so an unclassified failure produces no hint.
- # Usage: _loki_build_self_heal_hint (echoes hint or nothing)
- _loki_build_self_heal_hint() {
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local src="$loki_dir/state/LAST_ERROR.json"
- [ -f "$src" ] || return 0
- local hint
- hint="$(_LOKI_SH_SRC="$src" python3 -c "
- import json, os
- try:
- with open(os.environ['_LOKI_SH_SRC']) as f:
- rec = json.load(f)
- if not isinstance(rec, dict):
- raise SystemExit
- ec = str(rec.get('error_class', '') or '').strip()
- if ec in ('', 'unknown'):
- raise SystemExit
- brief = str(rec.get('brief', '') or '').strip()
- it = rec.get('iteration', '?')
- print('SELF_HEAL_HINT: the previous iteration (#%s) failed with error_class=%s. %s Address this specific failure FIRST before any new work.' % (it, ec, brief))
- except SystemExit:
- pass
- except Exception:
- pass
- " 2>/dev/null || true)"
- [ -n "$hint" ] || return 0
- # Archive-then-clear so it injects once (LEARN-FORWARD: the lesson still
- # lands in failure-history.jsonl before the single record is removed).
- _loki_archive_last_error "$src" "$loki_dir/state/failure-history.jsonl" 2>/dev/null || true
- rm -f "$src" 2>/dev/null || true
- printf '%s' "$hint"
- return 0
- }
- # _loki_terminal_record (T2.6, SAFE SUBSET): on an untrapped exit where the
- # persisted run status is still "running" (a true mid-provider-call crash on a
- # trappable signal -- SIGTERM/SIGINT/SIGHUP via the lock-release trap), leave a
- # best-effort, classified LAST_ERROR record so a post-crash `loki why` is not
- # stale. Piggybacks the existing lock-release EXIT trap (see main()) -- it does
- # NOT install a new broad EXIT trap.
- #
- # IMPORTANT (why this does NOT rewrite autonomy-state.json): the resume-detection
- # block in this file (search "Durable resume") keys the ENT-2 pod-loss resume on
- # prev_status == "running". Flipping the status to "exited" here would make a
- # crashed-but-resumable build (LOKI_DURABLE_STATE=1) reset to iteration 0 on the
- # next start, destroying durable progress. So this writes ONLY the LAST_ERROR
- # side-record (which is what `loki why` reads) and deliberately leaves the
- # status untouched. SIGKILL / power-loss are uncatchable (no trap fires); the
- # ENT-2 durable-resume path covers those. Never alters the exit code; best-effort.
- _loki_terminal_record() {
- local state_file
- state_file="$(_loki_state_file 2>/dev/null)" || return 0
- [ -n "$state_file" ] || return 0
- [ -f "$state_file" ] || return 0
- local _status
- _status="$(LOKI_TR_FILE="$state_file" python3 -c "
- import json, os
- try:
- print(json.load(open(os.environ['LOKI_TR_FILE'])).get('status','unknown'))
- except Exception:
- print('unknown')
- " 2>/dev/null || echo "unknown")"
- # Only act on a genuinely mid-flight "running" status. Any settled status
- # (council_approved, failed, exited, paused, ...) is left untouched.
- [ "$_status" = "running" ] || return 0
- # Leave a classified LAST_ERROR so `loki why` has something honest -- WITHOUT
- # touching autonomy-state.json (preserving the ENT-2 "running" resume signal).
- _loki_write_last_error "${ITERATION_COUNT:-0}" "unknown" \
- "The build process exited unexpectedly before finishing (possible crash or kill)." 2>/dev/null || true
- return 0
- }
- # _loki_write_rate_limit_signal (T2.7): write a best-effort
- # .loki/signals/RATE_LIMITED file. This is FORWARD-LAID infrastructure: no
- # consumer reads it yet (a future dashboard / external watcher could, to tell a
- # normal provider rate-limit wait apart from a hang). The user-visible signal
- # today is the log_info line at the wait site; this file is the durable record. Schema:
- # {"rate_limited": true, "wait_seconds": <int>, "reset_time": "<string>"}
- # Built via python3 so the reset-time string can never break the JSON. Entirely
- # best-effort: never crashes, never alters the build.
- # Usage: _loki_write_rate_limit_signal <wait_seconds> <reset_time>
- _loki_write_rate_limit_signal() {
- local wait_seconds="${1:-0}"
- local reset_time="${2:-}"
- local signals_dir="${TARGET_DIR:-.}/.loki/signals"
- mkdir -p "$signals_dir" 2>/dev/null || true
- LOKI_RL_WAIT="$wait_seconds" \
- LOKI_RL_RESET="$reset_time" \
- LOKI_RL_FILE="$signals_dir/RATE_LIMITED" \
- python3 -c "
- import json, os, tempfile
- try:
- w = os.environ.get('LOKI_RL_WAIT', '0')
- try:
- w = int(w)
- except Exception:
- w = 0
- rec = {'rate_limited': True, 'wait_seconds': w, 'reset_time': os.environ.get('LOKI_RL_RESET', '')}
- target = os.environ['LOKI_RL_FILE']
- d = os.path.dirname(target)
- fd, tmp = tempfile.mkstemp(dir=d, suffix='.json')
- with os.fdopen(fd, 'w') as f:
- json.dump(rec, f)
- os.replace(tmp, target)
- except Exception:
- pass
- " 2>/dev/null || true
- return 0
- }
- # Live Build HUD (v7.71.0): a single append-only per-iteration status line on the
- # interactive TTY path. Pure additive stdout decoration -- never piped into any
- # tee, so the dashboard agent.log and the stream-json parser are untouched. The
- # whole function is structured so any internal failure still returns 0; it can
- # never abort the iteration loop. Gated: TTY + not background + LOKI_HUD != 0.
- # Usage: render_build_hud <iter> <phase> <duration_secs>
- render_build_hud() {
- # Gate first: emit nothing unless interactive TTY, not --bg, and not opted out.
- # Inverted into an early return so the off-TTY path is byte-identical (no output).
- if ! { [ -t 1 ] && [ "${BACKGROUND_MODE:-false}" != "true" ] && [ "${LOKI_HUD:-1}" != "0" ]; }; then
- return 0
- fi
- local _iter="${1:-0}" _phase="${2:-?}" _dur="${3:-0}"
- [ -z "$_phase" ] && _phase="?"
- # Sanitize numerics so set -u arithmetic/format below can never error out.
- case "$_dur" in (*[!0-9]*|'') _dur=0 ;; esac
- local _max="${MAX_ITERATIONS:-0}"
- case "$_max" in (*[!0-9]*|'') _max=0 ;; esac
- # Field list: each field is appended only when its data is present. Missing
- # data => the field is omitted entirely (never a fabricated value).
- local _fields="$_phase"
- _fields="${_fields} | iter ${_iter}/${_max}"
- # Cost from the context tracker totals (the same field the dashboard shows).
- # NOTE: tracking.json totals accumulate across runs in the same dir (not reset
- # on a fresh `loki start`), so this is cumulative cost for the project dir, not
- # strictly this single run. Labeled plainly "$" / "cost" to avoid overclaiming.
- # OMIT entirely when empty/missing/non-Claude.
- local _cost=""
- if command -v python3 >/dev/null 2>&1; then
- _cost="$(python3 -c "
- import json
- try:
- t = json.load(open('.loki/context/tracking.json'))
- c = t.get('totals', {}).get('total_cost_usd', 0)
- c = float(c)
- if c > 0:
- print('%.2f' % c)
- except Exception:
- pass
- " 2>/dev/null || true)"
- fi
- [ -n "$_cost" ] && _fields="${_fields} | \$${_cost}"
- # Files changed (+ins/-del and file count) vs the run start SHA. Reuse the
- # build_completion_summary diff approach incl. the .loki/.git exclude pathspec.
- # OMIT when no start sha, not a git repo, or no diff data.
- local _start_sha="${_LOKI_RUN_START_SHA:-}"
- if [ -n "$_start_sha" ]; then
- local _shortstat _ins _del _files
- _shortstat="$( (cd "${TARGET_DIR:-.}" && git diff --shortstat "${_start_sha}..HEAD" -- . ':(exclude).loki/' ':(exclude).git/' ':(exclude)**/.loki/**') 2>/dev/null || true )"
- if [ -n "$_shortstat" ]; then
- _files="$(printf '%s\n' "$_shortstat" | grep -oE '[0-9]+ file' | grep -oE '[0-9]+' | head -1 2>/dev/null || true)"
- _ins="$(printf '%s\n' "$_shortstat" | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' | head -1 2>/dev/null || true)"
- _del="$(printf '%s\n' "$_shortstat" | grep -oE '[0-9]+ deletion' | grep -oE '[0-9]+' | head -1 2>/dev/null || true)"
- [ -z "$_files" ] && _files=0
- [ -z "$_ins" ] && _ins=0
- [ -z "$_del" ] && _del=0
- _fields="${_fields} | +${_ins}/-${_del} (${_files} files)"
- fi
- fi
- # Per-iteration time (the duration arg). Labeled "took" (not "iter") so it
- # does not collide with the "iter N/max" iteration-count field above.
- _fields="${_fields} | took $(_hud_fmt_secs "$_dur")"
- # Elapsed time for the whole run, from the once-captured run-start epoch.
- local _start_epoch="${_LOKI_RUN_START_EPOCH:-}"
- case "$_start_epoch" in (*[!0-9]*|'') _start_epoch="" ;; esac
- if [ -n "$_start_epoch" ]; then
- local _now _elapsed
- _now="$(date +%s 2>/dev/null || true)"
- case "$_now" in (*[!0-9]*|'') _now="" ;; esac
- if [ -n "$_now" ] && [ "$_now" -ge "$_start_epoch" ] 2>/dev/null; then
- _elapsed=$(( _now - _start_epoch ))
- _fields="${_fields} | elapsed $(_hud_fmt_secs "$_elapsed")"
- fi
- fi
- # ETA (PO lock #1): omit by default. Only a SMALL user-set LOKI_MAX_ITERATIONS
- # (not the 1000 default) yields a meaningful target, so that is the single ETA
- # trigger here. A LOKI_BUDGET_LIMIT cap is also an allowed trigger per the
- # plan, but a budget-derived ETA needs cost-rate math that is easy to get
- # wrong; per "keep it simple and safe, when unsure omit", budget ETA is left
- # out rather than rendered approximately.
- local _lmi="${LOKI_MAX_ITERATIONS:-}"
- case "$_lmi" in (*[!0-9]*|'') _lmi="" ;; esac
- if [ -n "$_lmi" ] && [ "$_lmi" -gt 0 ] 2>/dev/null && [ "$_lmi" -lt 1000 ] 2>/dev/null \
- && [ "$_iter" -gt 0 ] 2>/dev/null && [ "$_iter" -lt "$_lmi" ] 2>/dev/null \
- && [ -n "$_start_epoch" ]; then
- local _now2 _el2 _per _remain_iters _eta
- _now2="$(date +%s 2>/dev/null || true)"
- case "$_now2" in (*[!0-9]*|'') _now2="" ;; esac
- if [ -n "$_now2" ] && [ "$_now2" -ge "$_start_epoch" ] 2>/dev/null; then
- _el2=$(( _now2 - _start_epoch ))
- _per=$(( _el2 / _iter ))
- _remain_iters=$(( _lmi - _iter ))
- _eta=$(( _per * _remain_iters ))
- [ "$_eta" -gt 0 ] 2>/dev/null && _fields="${_fields} | eta ~$(_hud_fmt_secs "$_eta")"
- fi
- fi
- echo -e "${CYAN}[HUD]${NC} ${_fields}"
- return 0
- }
- # Format a whole-seconds integer as a compact duration: 37s, 2m11s, 1h03m.
- # Best-effort and set -u safe; any odd input degrades to "0s".
- _hud_fmt_secs() {
- local _s="${1:-0}"
- case "$_s" in (*[!0-9]*|'') _s=0 ;; esac
- if [ "$_s" -lt 60 ] 2>/dev/null; then
- printf '%ds' "$_s"
- elif [ "$_s" -lt 3600 ] 2>/dev/null; then
- printf '%dm%02ds' "$(( _s / 60 ))" "$(( _s % 60 ))"
- else
- printf '%dh%02dm' "$(( _s / 3600 ))" "$(( (_s % 3600) / 60 ))"
- fi
- return 0
- }
- #===============================================================================
- # Process Registry (PID Supervisor)
- # Central registry of all spawned child processes for reliable cleanup
- #===============================================================================
- PID_REGISTRY_DIR=""
- # Initialize the PID registry directory
- init_pid_registry() {
- PID_REGISTRY_DIR="${TARGET_DIR:-.}/.loki/pids"
- mkdir -p "$PID_REGISTRY_DIR"
- }
- # Parse a field from a JSON registry entry (python3 with shell fallback)
- # Usage: _parse_json_field <file> <field>
- _parse_json_field() {
- local file="$1" field="$2"
- if command -v python3 >/dev/null 2>&1; then
- python3 -c "import json,sys; print(json.load(open(sys.argv[1])).get(sys.argv[2],''))" "$file" "$field" 2>/dev/null
- else
- # Shell fallback (no python3): extract value for simple flat JSON. Handles
- # BOTH quoted string values ("kind":"wrapper") and bare numeric values
- # ("ppid":206). The prior impl stripped the leading quote's content to empty
- # for string fields -- broke "kind" parsing on python3-less hosts, which
- # would drop wrapper entries into the legacy child path (loki-mode #92).
- sed 's/.*"'"$field"'":[[:space:]]*//' "$file" 2>/dev/null \
- | sed 's/^"//' \
- | sed 's/[",}].*//' | head -1
- fi
- }
- # Register a spawned process in the central registry
- # Usage: register_pid <pid> <label> [<extra_info>]
- # Example: register_pid $! "dashboard" "port=57374"
- register_pid() {
- local pid="$1"
- # Sanitize label and extra for JSON safety (escape backslash first, then double-quote, strip newlines)
- local label="${2//\\/\\\\}"
- label="${label//\"/\\\"}"
- label="$(printf '%s' "$label" | tr -d '\n\r')"
- local extra="${3:-}"
- extra="${extra//\\/\\\\}"
- extra="${extra//\"/\\\"}"
- extra="$(printf '%s' "$extra" | tr -d '\n\r')"
- [ -z "$PID_REGISTRY_DIR" ] && init_pid_registry
- local entry_file="$PID_REGISTRY_DIR/${pid}.json"
- cat > "$entry_file" << EOF
- {"pid":$pid,"label":"$label","started":"$(date -u +%Y-%m-%dT%H:%M:%SZ)","ppid":$$,"extra":"$extra"}
- EOF
- }
- # Unregister a process from the registry (called on clean shutdown)
- # Usage: unregister_pid <pid>
- unregister_pid() {
- local pid="$1"
- [ -z "$PID_REGISTRY_DIR" ] && init_pid_registry
- rm -f "$PID_REGISTRY_DIR/${pid}.json" 2>/dev/null
- }
- # Kill a registered process with SIGTERM -> wait -> SIGKILL escalation
- # Usage: kill_registered_pid <pid>
- kill_registered_pid() {
- local pid="$1"
- if kill -0 "$pid" 2>/dev/null; then
- kill "$pid" 2>/dev/null || true
- # Wait up to 2 seconds for graceful exit
- local waited=0
- while [ $waited -lt 4 ] && kill -0 "$pid" 2>/dev/null; do
- sleep 0.5
- waited=$((waited + 1))
- done
- # Escalate to SIGKILL if still alive
- if kill -0 "$pid" 2>/dev/null; then
- kill -9 "$pid" 2>/dev/null || true
- fi
- fi
- unregister_pid "$pid"
- }
- # Register the CURRENTLY-RUNNING loki-run wrapper itself in the registry.
- # The wrapper registers its children but historically never itself, so a wrapper
- # whose launching session dies is never reaped (loki-mode #92). We record the
- # LAUNCHER's mortal pid (NOT $$ -- $$ is the wrapper's own pid, which would make
- # the parent-death check always succeed and the reaper INERT; NOT $PPID either --
- # in the detached setsid/nohup path bash caches getppid()==1 at exec, and kill -0 1
- # is always true -> also inert). The launcher exports LOKI_LAUNCHER_PID=$$ at each
- # backgrounding site; that launcher shell exits immediately after backgrounding, so
- # the recorded ppid genuinely dies and the parent-death precondition CAN fire. For
- # the foreground path LOKI_LAUNCHER_PID is unset -> $PPID (the user's shell) is the
- # correct mortal fallback. Reap-vs-spare is then decided by the LIVENESS predicate,
- # not by parentage alone. Uses a "kind":"wrapper" tag so cleanup_orphan_pids routes
- # only wrapper entries through the predicate and keeps child entries byte-identical.
- register_self_wrapper() {
- [ -z "$PID_REGISTRY_DIR" ] && init_pid_registry
- local launcher_ppid="${LOKI_LAUNCHER_PID:-$PPID}"
- case "$launcher_ppid" in ''|*[!0-9]*) launcher_ppid="$PPID" ;; esac
- local entry_file="$PID_REGISTRY_DIR/$$.json"
- cat > "$entry_file" << EOF
- {"pid":$$,"label":"loki-wrapper","started":"$(date -u +%Y-%m-%dT%H:%M:%SZ)","ppid":$launcher_ppid,"kind":"wrapper","extra":""}
- EOF
- }
- # Pure, side-effect-free reap predicate (loki-mode #92). ALL inputs are
- # pre-computed scalars (0/1 booleans or integer ms); NO stat/kill/pgrep inside so
- # it is hermetically unit-testable. Prints "1" (reap) or "0" (spare); returns 0.
- # Reap ONLY when: parent dead AND no live engine child AND idle past the budget.
- # Mirrors the SaaS BFF sweepStuckBuilds and the #91 worker "no-activity != dead"
- # symmetry: a genuinely-detached run that keeps writing .loki activity is spared.
- # parent_alive: 1 if recorded ppid is still alive, else 0
- # last_activity_ms: epoch-ms of most recent .loki activity (0 if none found)
- # now_ms: current epoch-ms
- # idle_budget_ms: generous idle budget (default 900000 = 15 min)
- # has_live_child: 1 if wrapper has a live ENGINE child (claude/node), else 0
- shouldReapOrphan() {
- local parent_alive="$1" last_activity_ms="$2" now_ms="$3" idle_budget_ms="$4" has_live_child="$5"
- if [ "$parent_alive" = "0" ] && [ "$has_live_child" = "0" ] \
- && [ $(( now_ms - last_activity_ms )) -ge "$idle_budget_ms" ]; then
- echo 1
- else
- echo 0
- fi
- }
- # Most-recent .loki activity as epoch-ms across events.jsonl, signals/*,
- # autonomy-state.json. Cross-platform stat (macOS -f %m / Linux -c %Y). On a stat
- # PARSE FAILURE we default to now_ms (SPARE), never 0 -- a wrong flag must not make a
- # live run look maximally idle and get reaped. "No activity files exist" legitimately
- # yields 0 (reap-eligible); that is distinct from a stat error.
- _loki_last_activity_ms() {
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local newest=0 f mt stat_ok=0
- local candidates=()
- [ -f "$loki_dir/events.jsonl" ] && candidates+=("$loki_dir/events.jsonl")
- [ -f "$loki_dir/autonomy-state.json" ] && candidates+=("$loki_dir/autonomy-state.json")
- if [ -d "$loki_dir/signals" ]; then
- for f in "$loki_dir/signals"/*; do [ -e "$f" ] && candidates+=("$f"); done
- fi
- for f in "${candidates[@]:-}"; do
- [ -e "$f" ] || continue
- mt=$(stat -f %m "$f" 2>/dev/null) || mt=""
- [ -z "$mt" ] && { mt=$(stat -c %Y "$f" 2>/dev/null) || mt=""; }
- case "$mt" in ''|*[!0-9]*) continue ;; esac
- stat_ok=1
- [ "$mt" -gt "$newest" ] && newest="$mt"
- done
- # candidates present but every stat failed -> parse failure -> spare (now_ms)
- if [ "${#candidates[@]}" -gt 0 ] && [ "$stat_ok" = "0" ]; then
- echo "$(( $(date +%s) * 1000 ))"
- return 0
- fi
- echo "$(( newest * 1000 ))"
- }
- # Count LIVE ENGINE children of a wrapper (claude/node), EXCLUDING sleep and the
- # status/resource monitors. The observed 3h orphan had only sleep children, so a
- # naive `pgrep -P` (any child) would falsely spare it; the comm filter is what makes
- # the reaper actually fire on the real orphan shape. Prints 1 (has engine child) or 0.
- _loki_has_live_engine_child() {
- local wrapper_pid="$1" c cc
- for c in $(pgrep -P "$wrapper_pid" 2>/dev/null); do
- cc=$(ps -o comm= -p "$c" 2>/dev/null)
- case "$cc" in *claude*|*node*) echo 1; return 0 ;; esac
- done
- echo 0
- }
- # Scan registry for orphaned processes and kill them
- # Called on startup and by `loki cleanup`
- # Returns: number of orphans killed
- cleanup_orphan_pids() {
- [ -z "$PID_REGISTRY_DIR" ] && init_pid_registry
- local orphan_count=0
- if [ ! -d "$PID_REGISTRY_DIR" ]; then
- echo "0"
- return 0
- fi
- for entry_file in "$PID_REGISTRY_DIR"/*.json; do
- [ -f "$entry_file" ] || continue
- local pid
- pid=$(basename "$entry_file" .json)
- # Skip non-numeric filenames
- case "$pid" in
- ''|*[!0-9]*) continue ;;
- esac
- # Never reap the currently-running wrapper itself (loki-mode #92 self-skip).
- [ "$pid" = "$$" ] && continue
- if kill -0 "$pid" 2>/dev/null; then
- # Process is alive -- check if its parent session is dead
- local ppid_val=""
- ppid_val=$(_parse_json_field "$entry_file" "ppid") || true
- # Validate ppid_val is numeric before using with kill
- case "$ppid_val" in ''|*[!0-9]*) ppid_val="" ;; esac
- # Route WRAPPER entries through the liveness predicate; CHILD entries
- # (no "kind" field) keep the exact parent-death-only behavior (#92).
- local kind=""
- kind=$(_parse_json_field "$entry_file" "kind") || true
- if [ "$kind" = "wrapper" ]; then
- # Wrapper: reap only if orphaned AND idle-past-budget AND no live
- # engine child. Compute the impure inputs here (predicate stays pure).
- if [ -n "$ppid_val" ]; then
- local parent_alive=0
- kill -0 "$ppid_val" 2>/dev/null && parent_alive=1
- local last_activity_ms now_ms has_live_child idle_budget_ms
- last_activity_ms=$(_loki_last_activity_ms)
- now_ms=$(( $(date +%s) * 1000 ))
- has_live_child=$(_loki_has_live_engine_child "$pid")
- idle_budget_ms="${LOKI_WRAPPER_IDLE_BUDGET_MS:-900000}"
- if [ "$(shouldReapOrphan "$parent_alive" "$last_activity_ms" "$now_ms" "$idle_budget_ms" "$has_live_child")" = "1" ]; then
- log_warn "Reaping idle orphaned loki wrapper PID=$pid (parent $ppid_val dead, idle >=$((idle_budget_ms/60000))m, no engine child)" >&2
- kill_registered_pid "$pid"
- orphan_count=$((orphan_count + 1))
- fi
- fi
- elif [ -n "$ppid_val" ] && [ "$ppid_val" != "$$" ]; then
- if ! kill -0 "$ppid_val" 2>/dev/null; then
- # Parent is dead -- this is an orphan
- local label=""
- label=$(_parse_json_field "$entry_file" "label") || label="unknown"
- log_warn "Killing orphaned process: PID=$pid label=$label (parent $ppid_val is dead)" >&2
- kill_registered_pid "$pid"
- orphan_count=$((orphan_count + 1))
- fi
- fi
- else
- # Process is dead -- clean up stale registry entry
- rm -f "$entry_file" 2>/dev/null
- fi
- done
- echo "$orphan_count"
- }
- # Kill ALL registered processes (used during full shutdown)
- kill_all_registered() {
- [ -z "$PID_REGISTRY_DIR" ] && init_pid_registry
- if [ ! -d "$PID_REGISTRY_DIR" ]; then
- return 0
- fi
- for entry_file in "$PID_REGISTRY_DIR"/*.json; do
- [ -f "$entry_file" ] || continue
- local pid
- pid=$(basename "$entry_file" .json)
- case "$pid" in
- ''|*[!0-9]*) continue ;;
- esac
- # Never SIGKILL the running wrapper itself mid-shutdown (loki-mode #92):
- # it now self-registers, so it would otherwise appear in this sweep.
- [ "$pid" = "$$" ] && continue
- kill_registered_pid "$pid"
- done
- }
- #===============================================================================
- # Event Emission (Dashboard Integration)
- # Writes events to .loki/events.jsonl for dashboard consumption
- #===============================================================================
- emit_event() {
- local event_type="$1"
- shift
- local event_data="$*"
- local events_file=".loki/events.jsonl"
- local timestamp
- timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)
- mkdir -p .loki
- # Build JSON event with proper escaping
- local json_event
- json_event=$(python3 -c "
- import json, sys
- event = {
- 'timestamp': sys.argv[1],
- 'type': sys.argv[2],
- 'data': sys.argv[3]
- }
- print(json.dumps(event))
- " "$timestamp" "$event_type" "$event_data" 2>/dev/null)
- # Fallback to simple JSON if python fails
- if [ -z "$json_event" ]; then
- # Escape quotes and special chars for JSON
- local escaped_data
- escaped_data=$(printf '%s' "$event_data" | sed 's/\\/\\\\/g; s/"/\\"/g; s/ /\\t/g' | tr -d '\n')
- json_event="{\"timestamp\":\"$timestamp\",\"type\":\"$event_type\",\"data\":\"$escaped_data\"}"
- fi
- echo "$json_event" >> "$events_file"
- # Also log for debugging
- log_debug "Event: $event_type - $event_data"
- }
- # Emit structured event with key-value pairs
- # Record that the operator's explicitly-pinned model was NOT the model dispatched.
- #
- # WHY THIS EXISTS: a tier resolution that quietly returns something other than the
- # pin is indistinguishable, in the receipt, from a run that got what it asked for.
- # The substitution itself is often legitimate; the SILENCE is the defect. This
- # emits both an event (machine-readable) and an audit line (the hash-chained
- # chain), so the swap and its stated reason can be audited and, if wrong,
- # refuted by whoever has the evidence to refute it.
- #
- # Never changes the dispatched model and never fails the caller: an observability
- # record that can break a run would be worse than the silence it replaces.
- emit_model_substituted() {
- local pinned="$1" dispatched="$2" reason="$3" site="$4"
- # Nothing to report when the pin was honored.
- [ "$pinned" = "$dispatched" ] && return 0
- emit_event_json "model_substituted" \
- "pinned=$pinned" \
- "dispatched=$dispatched" \
- "reason=$reason" \
- "site=$site" 2>/dev/null || true
- if type audit_agent_action >/dev/null 2>&1; then
- audit_agent_action "model_substituted" \
- "Pinned model '$pinned' dispatched as '$dispatched' ($reason)" \
- "site=$site" 2>/dev/null || true
- fi
- log_warn "Model pin '$pinned' dispatched as '$dispatched' ($reason)" 2>/dev/null || true
- return 0
- }
- emit_event_json() {
- local event_type="$1"
- shift
- local events_file=".loki/events.jsonl"
- local timestamp
- timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)
- mkdir -p .loki
- # Build JSON from remaining args as key=value pairs
- local json_data="{"
- local first=true
- while [ $# -gt 0 ]; do
- local key="${1%%=*}"
- local value="${1#*=}"
- if [ "$first" = true ]; then
- first=false
- else
- json_data+=","
- fi
- # Quote string values, leave numbers/booleans/floats as-is
- # BUG-NEW-004: Also match floats (e.g., cost=3.14) not just integers
- if [[ "$value" =~ ^[0-9]+\.?[0-9]*$ ]] || [[ "$value" =~ ^(true|false|null)$ ]]; then
- json_data+="\"$key\":$value"
- else
- # Escape backslashes, quotes, and special chars in value
- value=$(printf '%s' "$value" | sed 's/\\/\\\\/g; s/"/\\"/g; s/ /\\t/g')
- json_data+="\"$key\":\"$value\""
- fi
- shift
- done
- json_data+="}"
- local json_event="{\"timestamp\":\"$timestamp\",\"type\":\"$event_type\",\"data\":$json_data}"
- echo "$json_event" >> "$events_file"
- log_debug "Event: $event_type - $json_data"
- }
- # Per-stage timeline event (v7.91.x). Emits one stage_complete record per
- # quality-gate / build stage so the SaaS timeline can show where build time
- # goes within an iteration (the provider call itself is already bracketed by
- # iteration_start/iteration_complete). This is purely ADDITIVE: it appends one
- # event line via emit_event_json and never changes a gate verdict, gate exit
- # code, or control flow. Duration is computed by the caller (whole-second
- # resolution, sufficient for multi-second gates) and carried on the event so
- # the SaaS does not have to infer it from coarse ISO timestamps.
- # emit_stage_complete <stage_name> <status: pass|fail|not_run> <start_epoch_seconds>
- # Event shape:
- # {type:"stage_complete", timestamp, data:{stage, status, duration_s, iteration}}
- # Best-effort: any failure is swallowed so it can never block the build.
- emit_stage_complete() {
- local stage="$1"
- local status="$2"
- local t0="$3"
- local now dur
- now=$(date +%s 2>/dev/null) || return 0
- [ -n "$t0" ] || return 0
- dur=$(( now - t0 ))
- [ "$dur" -ge 0 ] 2>/dev/null || dur=0
- emit_event_json "stage_complete" \
- "stage=$stage" \
- "status=$status" \
- "duration_s=$dur" \
- "iteration=${ITERATION_COUNT:-0}" 2>/dev/null || true
- }
- # Trust-layer metrics event writer (benchmark program section 3). Appends one
- # durable record per trust event to .loki/metrics/trust-events.jsonl via the
- # Python writer (single source of truth for the JSONL schema). This is ADDITIVE
- # and purely a side effect: it writes nothing to stdout, ignores all errors, and
- # never alters control flow or any caller's return value. The single-state
- # control files (evidence-block.json, gate-failure-count.json) are untouched;
- # this log exists because those files are erased on the successful-run path,
- # losing exactly the self-correction events the trust metrics publish.
- # Resolve a stable, UNIQUE-PER-RUN id for the trust event log. The cross-run
- # denominators (block rate, gate distribution) require ids that are distinct per
- # run. A persisted per-run file is the source of truth, NOT LOKI_SESSION_ID:
- # - On `loki start ./prd.md`, LOKI_SESSION_ID is unset entirely.
- # - On `loki run <issue>`, LOKI_SESSION_ID is the issue NUMBER, which is stable
- # across re-runs by design (so `loki stop <n>` works); using it would merge
- # every re-run of the same issue into one bucket and skew the rates.
- # So a fresh run always MINTS a new unique id into .loki/state/trust-run-id, and
- # every later event in that run reads it back. LOKI_SESSION_ID is only a
- # last-resort fallback when no minted file exists (e.g. an event fired before
- # any run_start, which the aggregator then treats as un-instrumented anyway).
- # Events never join to proof.json (Metrics 1-3 are events-only, Metric 4 is
- # proofs-only), so intra-log uniqueness is the only requirement.
- # Usage: _loki_trust_run_id [--new]
- _loki_trust_run_id() {
- local loki_dir="${LOKI_DIR:-${TARGET_DIR:-.}/.loki}"
- local id_file="$loki_dir/state/trust-run-id"
- if [ "${1:-}" = "--new" ]; then
- # Fresh run: mint a new unique id (epoch + pid + short random) and
- # persist it as the source of truth for this run's events.
- local new_id
- new_id="run-$(date -u +%Y%m%d%H%M%S)-$$-${RANDOM:-0}"
- mkdir -p "$loki_dir/state" 2>/dev/null || true
- printf '%s' "$new_id" > "$id_file" 2>/dev/null || true
- printf '%s' "$new_id"
- return 0
- fi
- # Read path: the minted per-run file wins over LOKI_SESSION_ID so a resume
- # in a separate process (no exported LOKI_TRUST_RUN_ID) still resolves to
- # the same run, and a stable issue-number session id never collapses re-runs.
- if [ -s "$id_file" ]; then
- cat "$id_file" 2>/dev/null || true
- return 0
- fi
- if [ -n "${LOKI_SESSION_ID:-}" ]; then
- printf '%s' "$LOKI_SESSION_ID"
- return 0
- fi
- # No persisted id and no session id: empty -> writer records "unknown".
- printf '%s' ""
- }
- # _advance_current_phase: atomically advance currentPhase in orchestrator.json.
- # This is the single source of truth for the build dashboard and the BFF
- # reconciliation gate (isTerminalPhase). The engine initialises it to "BOOTSTRAP"
- # at session start; call this to advance through the SDLC lifecycle phases.
- # Args: $1 = the new phase value (REASONING, BUILDING, VERIFYING, COMPLETED, etc.)
- _advance_current_phase() {
- local new_phase="${1:?phase required}"
- local loki_dir="${LOKI_DIR:-${TARGET_DIR:-.}/.loki}"
- local orch="$loki_dir/state/orchestrator.json"
- [ -f "$orch" ] || return 0
- # Values are passed via argv (not interpolated into the source) so a phase or
- # path containing quotes can never break or inject into the python.
- python3 -E -c "import sys; sys.path[:] = [p for p in sys.path if p not in ('', '.')]
- import json, sys
- f, phase = sys.argv[1], sys.argv[2]
- try:
- d = json.load(open(f))
- d['currentPhase'] = phase
- json.dump(d, open(f, 'w'))
- except (json.JSONDecodeError, OSError):
- pass
- " "$orch" "$new_phase" 2>/dev/null || true
- }
- # Usage: record_trust_event_bash <event_type> [key=value ...]
- # Pass LOKI_TRUST_RUN_ID in the environment to override the resolved id (the
- # run_start site sets it to the freshly minted id so the first event matches).
- record_trust_event_bash() {
- local event_type="$1"
- shift || true
- local tm_mod="$SCRIPT_DIR/lib/trust_metrics.py"
- [ -f "$tm_mod" ] || return 0
- command -v python3 >/dev/null 2>&1 || return 0
- local loki_dir="${LOKI_DIR:-${TARGET_DIR:-.}/.loki}"
- local run_id="${LOKI_TRUST_RUN_ID:-$(_loki_trust_run_id)}"
- # Pass kv pairs as argv so Python parses (no shell JSON building). All
- # values stay strings except where the reader coerces (iteration -> int).
- _TM_LOKI_DIR="$loki_dir" \
- _TM_MOD_PATH="$tm_mod" \
- _TM_EVENT_TYPE="$event_type" \
- _TM_RUN_ID="$run_id" \
- _TM_ITERATION="${ITERATION_COUNT:-0}" \
- python3 - "$@" <<'TRUST_EVENT_PY' >/dev/null 2>&1 || true
- import os, sys, importlib.util
- spec = importlib.util.spec_from_file_location("trust_metrics", os.environ["_TM_MOD_PATH"])
- tm = importlib.util.module_from_spec(spec)
- spec.loader.exec_module(tm)
- fields = {}
- for arg in sys.argv[1:]:
- if "=" in arg:
- k, v = arg.split("=", 1)
- fields[k] = v
- tm.record_trust_event(
- os.environ["_TM_LOKI_DIR"],
- os.environ["_TM_EVENT_TYPE"],
- run_id=os.environ.get("_TM_RUN_ID", "") or None,
- iteration=os.environ.get("_TM_ITERATION", "0"),
- **fields,
- )
- TRUST_EVENT_PY
- }
- # v7.0.2: Bash helper to emit a managed-agents event to the dashboard's
- # managed event log (.loki/managed/events.ndjson). Mirrors the Python
- # emit_managed_event helper so bash callers can land events in the same
- # stream the dashboard reads. Schema: {ts, type, payload}.
- emit_managed_event_bash() {
- local event_type="$1"
- shift
- local target_dir="${TARGET_DIR:-.}"
- local events_file="$target_dir/.loki/managed/events.ndjson"
- mkdir -p "$target_dir/.loki/managed"
- local timestamp
- timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)
- # Build payload JSON from key=value args (same convention as emit_event_json)
- local payload="{"
- local first=true
- while [ $# -gt 0 ]; do
- local key="${1%%=*}"
- local value="${1#*=}"
- if [ "$first" = true ]; then first=false; else payload+=","; fi
- if [[ "$value" =~ ^[0-9]+\.?[0-9]*$ ]] || [[ "$value" =~ ^(true|false|null)$ ]]; then
- payload+="\"$key\":$value"
- else
- value=$(printf '%s' "$value" | sed 's/\\/\\\\/g; s/"/\\"/g; s/ /\\t/g')
- payload+="\"$key\":\"$value\""
- fi
- shift
- done
- payload+="}"
- local json_event="{\"ts\":\"$timestamp\",\"type\":\"$event_type\",\"payload\":$payload}"
- echo "$json_event" >> "$events_file"
- }
- # Emit event to .loki/events/pending/ directory (for event bus subscribers)
- # Used by OTEL bridge and other enterprise services that watch the pending dir.
- # Usage: emit_event_pending <type> [key=value ...]
- emit_event_pending() {
- local event_type="$1"
- shift
- local events_dir=".loki/events/pending"
- mkdir -p "$events_dir"
- local timestamp
- timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)
- local event_id
- event_id=$(head -c 4 /dev/urandom | od -An -tx1 | tr -d ' \n')
- # Build payload JSON from key=value args
- local payload="{"
- local first=true
- while [ $# -gt 0 ]; do
- local key="${1%%=*}"
- local value="${1#*=}"
- if [ "$first" = true ]; then
- first=false
- else
- payload+=","
- fi
- value=$(printf '%s' "$value" | sed 's/\\/\\\\/g; s/"/\\"/g; s/ /\\t/g')
- payload+="\"$key\":\"$value\""
- shift
- done
- payload+="}"
- local event_file="$events_dir/${timestamp//:/-}_${event_id}.json"
- printf '{"id":"%s","type":"%s","timestamp":"%s","payload":%s,"version":"1.0"}\n' \
- "$event_id" "$event_type" "$timestamp" "$payload" > "${event_file}.tmp" && mv "${event_file}.tmp" "$event_file"
- }
- #===============================================================================
- # Enterprise Process Manager
- # Manages background services: OTEL bridge, audit subscriber, integration sync
- #===============================================================================
- # Start enterprise background services
- start_enterprise_services() {
- loki_background_services_enabled || return 0
- log_info "Starting enterprise services..."
- # OTEL Bridge (requires LOKI_OTEL_ENDPOINT and node)
- if [ -n "${LOKI_OTEL_ENDPOINT:-}" ] && command -v node >/dev/null 2>&1; then
- LOKI_TRACE_ID=$(node -e "console.log(require('crypto').randomBytes(16).toString('hex'))")
- export LOKI_TRACE_ID
- local bridge_script="${SCRIPT_DIR}/../src/observability/otel-bridge.js"
- if [ -f "$bridge_script" ]; then
- LOKI_OTEL_ENDPOINT="$LOKI_OTEL_ENDPOINT" \
- LOKI_TRACE_ID="$LOKI_TRACE_ID" \
- LOKI_DIR=".loki" \
- node "$bridge_script" &
- ENTERPRISE_PIDS+=($!)
- log_info "Started OTEL bridge (PID: ${ENTERPRISE_PIDS[-1]})"
- else
- log_warn "OTEL bridge script not found: $bridge_script"
- fi
- fi
- # Audit subscriber (P0.5-3)
- if [ "${LOKI_AUDIT_ENABLED:-false}" = "true" ]; then
- if command -v node >/dev/null 2>&1; then
- node "${SCRIPT_DIR}/../src/audit/subscriber.js" &
- ENTERPRISE_PIDS+=($!)
- log_info "Started audit subscriber (PID: ${ENTERPRISE_PIDS[-1]})"
- fi
- fi
- }
- # Stop all enterprise background services
- stop_enterprise_services() {
- if [ ${#ENTERPRISE_PIDS[@]} -eq 0 ]; then
- return
- fi
- log_info "Stopping enterprise services (${#ENTERPRISE_PIDS[@]} processes)..."
- for pid in "${ENTERPRISE_PIDS[@]}"; do
- if kill -0 "$pid" 2>/dev/null; then
- kill -TERM "$pid" 2>/dev/null || true
- # Wait briefly for graceful shutdown
- local wait_count=0
- while kill -0 "$pid" 2>/dev/null && [ $wait_count -lt 10 ]; do
- sleep 0.1
- ((wait_count++))
- done
- # Force kill if still running
- if kill -0 "$pid" 2>/dev/null; then
- kill -9 "$pid" 2>/dev/null || true
- log_warn "Force-killed enterprise service PID $pid"
- else
- log_info "Enterprise service PID $pid stopped gracefully"
- fi
- fi
- done
- ENTERPRISE_PIDS=()
- }
- # Policy engine check wrapper (P0.5-2)
- # Evaluates policies via Node.js CLI and returns appropriate exit codes.
- # Exit 0 = ALLOW, Exit 1 = DENY, Exit 2 = REQUIRE_APPROVAL (logged but allowed for now)
- check_policy() {
- local enforcement_point="$1"
- # Default to a valid empty-object JSON. Do NOT inline `${2:-{}}`: the
- # closing brace of the parameter expansion eats the first `}` of the
- # `{}` default, so a non-empty $2 like {"a":1} would pass through as
- # {"a":1}} (invalid JSON -> check.js JSON.parse fails -> exit 1 DENY
- # every iteration). Split the default assignment to avoid the footgun.
- local context_json="${2:-}"
- [ -z "$context_json" ] && context_json='{}'
- # Only check if a policy file exists. This gate MUST agree with what
- # src/policies/engine.js:_init actually loads, or the two disagree silently:
- # engine.js falls back to the global ~/.loki policy (which is the only place
- # the dashboard writes, api_v2.py:75,105), and a project-local-only test
- # here would short-circuit before that fallback was ever consulted.
- local _pol_global_dir="${LOKI_DATA_DIR:-$HOME/.loki}"
- if [ ! -f ".loki/policies.json" ] && [ ! -f ".loki/policies.yaml" ] \
- && [ ! -f "$_pol_global_dir/policies.json" ] \
- && [ ! -f "$_pol_global_dir/policies.yaml" ]; then
- return 0
- fi
- # Requires Node.js. Reaching here means a policy file EXISTS, so the
- # operator has expressed intent to enforce. Returning 0 for a missing
- # runtime silently disables that enforcement and reports nothing: the run
- # proceeds exactly as if every action were permitted. That is the worst
- # shape a security control can take, because the operator believes it is on.
- #
- # Fail CLOSED instead, matching what check.js already does for a corrupt
- # policy file (tests/test-policy-failclosed.sh). LOKI_POLICY_REQUIRE_NODE=0
- # restores the old fail-open for a host that genuinely cannot install node
- # and accepts running unenforced.
- if ! command -v node >/dev/null 2>&1; then
- if [ "${LOKI_POLICY_REQUIRE_NODE:-1}" = "0" ]; then
- log_warn "Policy file present but node is unavailable; enforcement SKIPPED (LOKI_POLICY_REQUIRE_NODE=0)"
- return 0
- fi
- log_error "Policy file present but node is unavailable: cannot evaluate policy, refusing the action (fail-closed). Set LOKI_POLICY_REQUIRE_NODE=0 to run unenforced."
- # Recorded like any other block. A refusal that leaves no trace is
- # indistinguishable from a run that was never gated, and the receipt
- # must be able to say WHY the action did not happen. The reason is
- # distinct from a DENY: the policy was never evaluated.
- audit_agent_action "policy_unevaluable" "Policy present but node unavailable" "enforcement=$enforcement_point"
- emit_event_json "policy_unevaluable" \
- "enforcement=$enforcement_point" \
- "reason=node_unavailable"
- return 1
- fi
- local result
- result=$(LOKI_PROJECT_DIR="$(pwd)" node "${SCRIPT_DIR}/../src/policies/check.js" "$enforcement_point" "$context_json" 2>/dev/null)
- local exit_code=$?
- if [ $exit_code -eq 1 ]; then
- log_error "Policy DENIED: $result"
- audit_agent_action "policy_denied" "Policy denied execution" "enforcement=$enforcement_point"
- emit_event_json "policy_denied" \
- "enforcement=$enforcement_point" \
- "result=$result"
- return 1
- elif [ $exit_code -eq 2 ]; then
- log_warn "Policy requires APPROVAL: $result"
- audit_agent_action "policy_approval_required" "Policy requires approval" "enforcement=$enforcement_point"
- # P3-3 (v7.51.0): honor the approval requirement when the operator has
- # opted into enforcement. This is OPT-IN and changes NOTHING for existing
- # users: the wait fires only when staged autonomy is on
- # (LOKI_STAGED_AUTONOMY=true) or the explicit
- # LOKI_POLICY_APPROVAL_ENFORCE=1 knob is set. Otherwise it stays advisory
- # (log + proceed), preserving the historical default behavior. The wait
- # reuses the same .loki/signals/ file-signal mechanism as staged-autonomy
- # plan approval (check_staged_autonomy), extended with a reject arm so an
- # operator can deny (deny == policy DENIED == return 1).
- if [ "$STAGED_AUTONOMY" = "true" ] || [ "${LOKI_POLICY_APPROVAL_ENFORCE:-0}" = "1" ]; then
- local _approve_sig=".loki/signals/POLICY_APPROVED"
- local _reject_sig=".loki/signals/POLICY_REJECTED"
- log_warn "Policy enforcement: waiting for approval at enforcement point '$enforcement_point'."
- log_warn " Approve: create $_approve_sig | Reject: create $_reject_sig"
- audit_agent_action "policy_approval_wait" "Waiting for policy approval signal" "enforcement=$enforcement_point"
- while [ ! -f "$_approve_sig" ] && [ ! -f "$_reject_sig" ]; do
- sleep 5
- done
- if [ -f "$_reject_sig" ]; then
- rm -f "$_reject_sig" "$_approve_sig" 2>/dev/null || true
- log_error "Policy REJECTED by operator at enforcement point '$enforcement_point'"
- audit_agent_action "policy_approval_rejected" "Operator rejected policy approval" "enforcement=$enforcement_point"
- emit_event_json "policy_denied" \
- "enforcement=$enforcement_point" \
- "result=operator_rejected"
- return 1
- fi
- rm -f "$_approve_sig" 2>/dev/null || true
- log_info "Policy approved by operator at enforcement point '$enforcement_point'; continuing."
- audit_agent_action "policy_approval_granted" "Operator approved policy" "enforcement=$enforcement_point"
- return 0
- fi
- # Default (no staged autonomy, no enforce knob): advisory only -- log and
- # proceed. This preserves the historical behavior for existing users.
- return 0
- fi
- return 0
- }
- #===============================================================================
- # Learning Signal Emission (SYN-018)
- # Emits learning signals for cross-tool learning system
- #===============================================================================
- # Path to learning signal emitter
- LEARNING_EMIT_SH="$SCRIPT_DIR/../learning/emit.sh"
- # Emit learning signal (non-blocking)
- # Usage: emit_learning_signal <signal_type> [options]
- emit_learning_signal() {
- if [ -f "$LEARNING_EMIT_SH" ]; then
- # Run in background to be non-blocking
- (LOKI_DIR=".loki" LOKI_SKILL_DIR="$PROJECT_DIR" "$LEARNING_EMIT_SH" "$@" >/dev/null 2>&1 &)
- fi
- }
- # Track iteration timing for efficiency signals
- ITERATION_START_MS=""
- # Get current time in milliseconds (portable: works on macOS BSD date and GNU date)
- _now_ms() {
- local ms
- ms=$(date +%s%3N 2>/dev/null)
- # macOS BSD date doesn't support %N -- outputs literal "N" or "%3N"
- # Detect non-numeric output and fall back to seconds * 1000
- case "$ms" in
- *[!0-9]*) echo $(( $(date +%s) * 1000 )) ;;
- *) echo "$ms" ;;
- esac
- }
- record_iteration_start() {
- ITERATION_START_MS=$(_now_ms)
- }
- # Get iteration duration in milliseconds
- get_iteration_duration_ms() {
- if [ -n "$ITERATION_START_MS" ]; then
- local end_ms
- end_ms=$(_now_ms)
- echo $((end_ms - ITERATION_START_MS))
- else
- echo "0"
- fi
- }
- #===============================================================================
- # API Key Validation
- # Validates required API key is set for the selected provider.
- # Supports Docker/K8s secret file mounts as fallback.
- #===============================================================================
- # Zero-friction preflight helpers (T1.1). These run for ALL environments
- # (not just Docker/K8s) BEFORE the build starts, via validate_api_keys. git is a
- # genuine hard requirement (the build inits a repo) so a missing git BLOCKS with a
- # copy-pasteable fix; node-version and network reachability are ADVISORY (warn and
- # continue, fail-open) so a probabilistic or optional signal never blocks a working
- # user. The goal: surface real problems early without ever wrongly refusing to start.
- #
- # _loki_check_node_version: ADVISORY only. If node is present and its major
- # version is < 18, log a warning (node only matters for node-based builds) and
- # continue. If node is absent entirely this is a NO-OP. Always returns 0 -- it
- # never blocks the build (fail-open); the real node call, if any, is the test.
- _loki_check_node_version() {
- command -v node >/dev/null 2>&1 || return 0
- local node_version major
- node_version="$(node --version 2>/dev/null || echo '')"
- # node --version -> "v20.11.0"; extract the leading major integer.
- major="$(printf '%s' "$node_version" | sed -E 's/^v?([0-9]+).*/\1/')"
- # Advisory only: node is OPTIONAL (absence is a no-op above, and many builds -
- # Python/Go/Rust - never touch node). A present-but-old node only matters for
- # node-based builds, so WARN and continue (fail-open); never hard-block a
- # working user. The actual node call (only for JS/TS work) is the real test.
- if [ -n "$major" ] && [[ "$major" =~ ^[0-9]+$ ]] && [ "$major" -lt 18 ]; then
- log_warn "Node.js >= 18 recommended for node-based builds; found ${node_version:-unknown}. Upgrade if your project uses node: https://nodejs.org"
- fi
- return 0
- }
- # _loki_check_git_present: the build initializes a git repo, so git is required.
- _loki_check_git_present() {
- if ! command -v git >/dev/null 2>&1; then
- log_error "Git is required (the build initializes a repo). Install: https://git-scm.com/downloads"
- return 1
- fi
- return 0
- }
- # _loki_claude_login_state: report the Claude Code login state WITHOUT a network
- # call, printing exactly one of: loggedin | expired | loggedout | unknown.
- #
- # v7.104.2: the previous preflight assumed OAuth credentials always live in
- # ~/.claude/.credentials.json. That is false for the native install (Claude Code
- # 2.x on macOS), which stores the login in the macOS Keychain under the service
- # "Claude Code-credentials" and creates NO .credentials.json. A genuinely logged
- # in subscription user was therefore falsely told "installed but not logged in"
- # and blocked from every build -- the exact opposite of the zero-friction intent.
- #
- # Design: prefer the durable, CLI-owned signal (`claude auth status`, local +
- # fast + non-interactive, JSON with "loggedIn"), because the credential STORE
- # location has now moved twice (file -> keychain) and will move again. Fall back
- # to the file, then the keychain, for older CLIs that lack `auth status`. Crucial
- # rule (inconclusive-never-false-fails applied to auth): only ever return a hard
- # "loggedout" on POSITIVE evidence of no login; when we cannot tell, return
- # "unknown" and let the caller fail OPEN (warn, proceed, let the real call
- # decide) -- never hard-block a paying user on uncertainty.
- _loki_claude_login_state() {
- # 1) Primary: the CLI's own local auth-status (zero-network, ~0.2s). Newer
- # claude CLIs print JSON with a boolean "loggedIn"; parse defensively.
- if command -v claude >/dev/null 2>&1; then
- local _status_json
- _status_json="$(claude auth status 2>/dev/null)"
- if [[ -n "$_status_json" ]]; then
- local _parsed
- _parsed="$(printf '%s' "$_status_json" | python3 -c "
- import json, sys
- try:
- d = json.load(sys.stdin)
- v = d.get('loggedIn')
- # ONLY the two explicit booleans are trusted. Anything else -- a missing
- # field, a renamed schema, a JSON error object -- is inconclusive and MUST
- # fall through to the file/keychain checks, never a hard 'loggedout' (that
- # would block a genuinely logged-in user with a valid creds file, violating
- # fail-open). Mirrors the bun doctor's === true / === false / else shape.
- if v is True:
- print('loggedin')
- elif v is False:
- print('loggedout')
- # else: print nothing -> fall through
- except Exception:
- pass # not JSON (older CLI / different output) -> stay silent, fall through
- " 2>/dev/null)"
- if [[ "$_parsed" == "loggedin" ]]; then
- echo "loggedin"; return 0
- elif [[ "$_parsed" == "loggedout" ]]; then
- echo "loggedout"; return 0
- fi
- # auth status ran but was inconclusive (unparseable, or valid JSON
- # without an explicit loggedIn boolean) -> do NOT trust it as a
- # negative; fall through to the credential-store checks below.
- fi
- fi
- # 2) Fallback: a VALID (non-expired) OAuth credentials file (older CLIs write
- # it here). Compute the expiry once; a valid file is a positive login.
- # IMPORTANT: we do NOT conclude "expired" here yet -- a stale expired file
- # can sit next to a live macOS-Keychain login (the native install writes
- # the file once, then rotates only the Keychain). Concluding "expired" off
- # the file before consulting the Keychain would wrongly block a genuinely
- # logged-in user -- the exact class of bug this whole change fixes. So the
- # Keychain (step 3, a POSITIVE signal) is checked BEFORE we ever return
- # "expired" (step 4). Unknown schema -> treat as valid (fail open).
- local _creds="${CLAUDE_CONFIG_DIR:-$HOME/.claude}/.credentials.json"
- local _file_state="" # ""=absent, "valid"=present+fresh, "expired"=present+stale
- if [[ -s "$_creds" ]]; then
- _file_state="$(python3 -c "
- import json, sys, time
- try:
- d = json.load(open(sys.argv[1]))
- exp = d.get('claudeAiOauth', {}).get('expiresAt')
- if isinstance(exp, (int, float)) and exp > 0 and (exp / 1000.0) <= (time.time() + 60):
- print('expired')
- else:
- print('valid')
- except Exception:
- print('valid') # unreadable/unknown -> fail open (treat as valid)
- " "$_creds" 2>/dev/null || echo valid)"
- fi
- if [[ "$_file_state" == "valid" ]]; then
- echo "loggedin"; return 0
- fi
- # 3) macOS Keychain (native install stores the login here, often with NO file
- # at all, or alongside a now-stale file). A present Keychain entry is a
- # POSITIVE login signal and OVERRIDES an expired file. Existence check
- # ONLY -- never `-w` (reading the secret can pop a GUI keychain prompt that
- # would hang a headless/CI build). Guarded to Darwin.
- if [[ "$(uname 2>/dev/null)" == "Darwin" ]] && command -v security >/dev/null 2>&1; then
- if security find-generic-password -s "Claude Code-credentials" >/dev/null 2>&1; then
- echo "loggedin"; return 0
- fi
- fi
- # 4) Only now, with no valid file and no Keychain login, is an expired file a
- # real "expired" state (genuine expiry that would 401 mid-build).
- if [[ "$_file_state" == "expired" ]]; then
- echo "expired"; return 0
- fi
- # 5) No positive login evidence at all. On macOS (where the file AND the
- # Keychain are the two real stores, both absent) with the CLI present, that
- # is confident "logged out". Anywhere else we cannot prove absence of a
- # login (no Keychain to consult), so stay "unknown" and let the caller fail
- # open.
- if [[ "$(uname 2>/dev/null)" == "Darwin" ]] && command -v claude >/dev/null 2>&1; then
- echo "loggedout"; return 0
- fi
- echo "unknown"; return 0
- }
- # _loki_check_workspace_writable: the build writes state, proofs, and source into
- # the working directory and its .loki/ subtree on every iteration. A read-only
- # volume, a permission-denied directory, or a full disk would otherwise fail
- # silently deep inside the loop (every state write swallows errors with
- # `2>/dev/null || true`), leaving the user with a confusing half-built run. Catch
- # it once, up front, with one actionable message instead. HARD requirement.
- _loki_check_workspace_writable() {
- local dir="${TARGET_DIR:-$(pwd)}"
- local probe="${dir}/.loki/.write-probe.$$"
- # Ensure the .loki subtree can be created and written to.
- if ! mkdir -p "${dir}/.loki" 2>/dev/null; then
- if [ ! -w "$dir" ]; then
- log_error "Cannot create ${dir}/.loki -- the working directory is not writable. Fix permissions (e.g. chmod u+w '${dir}') or run from a directory you own, then retry."
- else
- log_error "Cannot create ${dir}/.loki. If the disk is full, free space (df -h '${dir}'); if it is a read-only mount, run from a writable location, then retry."
- fi
- return 1
- fi
- # Confirm we can actually write a byte (catches full disk / read-only mount
- # where the directory exists but writes fail).
- if ! (: > "$probe") 2>/dev/null; then
- rm -f "$probe" 2>/dev/null || true
- log_error "Cannot write to ${dir}/.loki -- check free disk space (df -h '${dir}') and that the volume is not read-only, then retry."
- return 1
- fi
- rm -f "$probe" 2>/dev/null || true
- return 0
- }
- # _loki_check_network_reachable: ADVISORY only. A fast (3s) reachability probe to
- # the active provider endpoint that WARNS and continues if it cannot reach it -- a
- # curl failure does not prove the provider CLI cannot connect (3s timeout under
- # load, transient DNS, or a proxy set for the CLI but not the shell all curl-fail
- # while the real build succeeds). Always returns 0 (fail-open); the actual
- # provider call is the authoritative connectivity test. Skipped entirely when curl
- # is missing, when LOKI_SKIP_NET_PREFLIGHT=1, when ANTHROPIC_BASE_URL is set (alt
- # provider endpoint we cannot assume), or for any provider whose endpoint we do
- # not know. It NEVER blocks the build.
- # Alt-provider guard: ANTHROPIC_BASE_URL routes Loki at a non-Anthropic endpoint
- # (OpenRouter, Ollama, LiteLLM, vLLM, self-hosted). The tier aliases Loki resolves
- # by default -- opus / sonnet / haiku -- are Anthropic-only names, and most
- # alt-providers reject them outright, so the run dies at the FIRST model call with
- # a provider-side error that is hard to trace back to the missing override.
- #
- # LOKI_MODEL_OVERRIDE is what makes that path work (providers/claude.sh:520 and
- # loki-ts/src/runner/providers.ts:328 both apply it only when BOTH vars are set).
- # `loki doctor` already warns on this, but doctor is opt-in and the run is not.
- #
- # NON-FATAL by design: a mapping proxy (LiteLLM can do this) legitimately resolves
- # the aliases server-side, so this is a warning and never a block. Emitted once per
- # run, and silent unless an alt endpoint is actually configured.
- _loki_warn_alt_provider_model_alias() {
- [ -n "${ANTHROPIC_BASE_URL:-}" ] || return 0
- [ -z "${LOKI_MODEL_OVERRIDE:-}" ] || return 0
- log_warn "ANTHROPIC_BASE_URL is set (${ANTHROPIC_BASE_URL}) but LOKI_MODEL_OVERRIDE is not."
- log_warn " Loki will ask for the Anthropic tier aliases (opus/sonnet/haiku), which most"
- log_warn " alt-providers do not serve. Set the exact model id your endpoint expects:"
- log_warn " export LOKI_MODEL_OVERRIDE=<model-id> # e.g. from 'ollama list' or your provider's model page"
- log_warn " Ignore this if your gateway maps those aliases for you (LiteLLM can)."
- return 0
- }
- _loki_check_network_reachable() {
- local provider="${1:-claude}"
- [ "${LOKI_SKIP_NET_PREFLIGHT:-}" = "1" ] && return 0
- command -v curl >/dev/null 2>&1 || return 0
- # Alternate provider base URL set -> do not assume the default endpoint.
- [ -n "${ANTHROPIC_BASE_URL:-}" ] && return 0
- local endpoint=""
- case "$provider" in
- claude) endpoint="https://api.anthropic.com" ;;
- *) return 0 ;; # unknown endpoint -> fail open, never guess
- esac
- # Advisory only: a fast curl probe failing does NOT prove the provider CLI
- # cannot connect (a 3s timeout under load, transient DNS, or a proxy set for
- # the CLI but not the shell all curl-fail while the real build succeeds). WARN
- # and continue - the actual provider call is the authoritative connectivity
- # test. Silence this with LOKI_SKIP_NET_PREFLIGHT=1. Never hard-block here.
- if ! curl -sS -m 3 -o /dev/null "$endpoint" 2>/dev/null; then
- log_warn "Could not verify network reachability to the AI provider (firewall/VPN/transient?). Continuing; the provider call will be the real test. Silence with LOKI_SKIP_NET_PREFLIGHT=1."
- fi
- return 0
- }
- validate_api_keys() {
- local provider="${LOKI_PROVIDER:-claude}"
- # Zero-friction preflight (T1.1): toolchain + reachability checks that apply
- # to EVERY environment, run BEFORE the Docker/K8s early-return below so they
- # are not silently skipped in the common local case. Node/git are genuinely
- # required (node only when present-but-too-old); the network probe is
- # fail-open and opt-out (LOKI_SKIP_NET_PREFLIGHT=1).
- if ! _loki_check_node_version; then
- return 1
- fi
- if ! _loki_check_git_present; then
- return 1
- fi
- if ! _loki_check_workspace_writable; then
- return 1
- fi
- if ! _loki_check_network_reachable "$provider"; then
- return 1
- fi
- _loki_warn_alt_provider_model_alias
- # Zero-friction auth preflight for LOCAL runs (must run BEFORE the early
- # return below, which exits for non-Docker/K8s envs). When claude is the
- # provider and there is no ANTHROPIC_API_KEY, ask the login-state helper
- # (auth-status -> file -> keychain, all zero-network) whether the user is
- # actually logged in. A never-logged-in user would otherwise enter the build,
- # make a failing call, and 401 -- the worst first impression -- so we fail
- # fast with the one-step fix. But we ONLY hard-block on a confident
- # "loggedout"; an "expired" login gets its own message; "unknown" (we cannot
- # prove the login state, e.g. an older CLI on a non-macOS box with no file)
- # fails OPEN so a genuinely logged-in user is never blocked. This is the
- # v7.104.2 fix for the native/Keychain login being misread as logged out.
- # Opt out entirely with LOKI_SKIP_AUTH_PREFLIGHT=1.
- if [[ "$provider" == "claude" && "${LOKI_SKIP_AUTH_PREFLIGHT:-}" != "1" && -z "${ANTHROPIC_API_KEY:-}" ]]; then
- local _login_state
- _login_state="$(_loki_claude_login_state)"
- # Both branches report the blocker before returning. This is the wall a
- # user hits AFTER answering every quickstart prompt and confirming the
- # spend, and until now it emitted nothing -- so the funnel showed a first
- # run attempted, then silence, indistinguishable from a successful build.
- # Bounded enum only (`not_logged_in`), never the login state, path or
- # credential; backgrounded and non-fatal so a diagnostic can never break
- # the refusal it is describing.
- if [[ "$_login_state" == "loggedout" ]]; then
- if declare -f loki_emit_first_run_blocked >/dev/null 2>&1; then
- ( loki_emit_first_run_blocked "not_logged_in" >/dev/null 2>&1 </dev/null & ) 2>/dev/null || true
- fi
- log_error "Claude Code is installed but not logged in -- the build would stall instead of running."
- log_error "Log in once, then retry:"
- log_error " claude auth login"
- log_error "(or set ANTHROPIC_API_KEY, or LOKI_SKIP_AUTH_PREFLIGHT=1 to bypass this check)"
- return 1
- elif [[ "$_login_state" == "expired" ]]; then
- if declare -f loki_emit_first_run_blocked >/dev/null 2>&1; then
- ( loki_emit_first_run_blocked "not_logged_in" >/dev/null 2>&1 </dev/null & ) 2>/dev/null || true
- fi
- log_error "Your Claude Code login has expired -- the build would stall instead of running."
- log_error "Fix it in one step, then retry:"
- log_error " claude auth login"
- log_error "(or set ANTHROPIC_API_KEY, or LOKI_SKIP_AUTH_PREFLIGHT=1 to bypass this check)"
- return 1
- fi
- # "loggedin" or "unknown" -> proceed (fail open on uncertainty).
- fi
- # CLI tools (claude, opencode) use their own login sessions.
- # Only require API keys inside Docker/K8s where CLI login isn't available.
- if [[ ! -f "/.dockerenv" ]] && [[ -z "${KUBERNETES_SERVICE_HOST:-}" ]]; then
- return 0
- fi
- local key_var=""
- case "$provider" in
- claude)
- # Inside Docker, the Claude Code CLI can authenticate either via
- # ANTHROPIC_API_KEY OR via a mounted OAuth credentials file (the
- # zero-friction `loki docker` wrapper mounts the host login at
- # ~/.claude/.credentials.json). If that file is present, the CLI
- # has a valid login and we must NOT block on the env var -- doing
- # so was the bug that made OAuth-based Docker runs exit at
- # pre-flight with "Required API key ... is not set". Mirror the
- # OAuth-aware doctor check (run.sh:9268).
- if [[ -z "${ANTHROPIC_API_KEY:-}" ]]; then
- local _claude_creds="${CLAUDE_CONFIG_DIR:-$HOME/.claude}/.credentials.json"
- if [[ -s "$_claude_creds" ]]; then
- log_info "Claude Code OAuth credentials detected ($_claude_creds); using CLI login."
- return 0
- fi
- fi
- key_var="ANTHROPIC_API_KEY"
- ;;
- esac
- if [[ -z "$key_var" ]]; then
- return 0
- fi
- local key_value="${!key_var:-}"
- # Try loading from secret file mounts (Docker/K8s)
- if [[ -z "$key_value" ]]; then
- local lower_name
- lower_name=$(echo "$key_var" | tr '[:upper:]' '[:lower:]')
- for mount_path in /run/secrets /var/run/secrets; do
- if [[ -f "$mount_path/$lower_name" ]]; then
- key_value=$(cat "$mount_path/$lower_name" 2>/dev/null | tr -d '[:space:]')
- if [[ -n "$key_value" ]]; then
- export "$key_var=$key_value"
- log_info "Loaded $key_var from secret file: $mount_path/$lower_name"
- break
- fi
- fi
- done
- fi
- if [[ -z "$key_value" ]]; then
- log_error "Required API key $key_var is not set for provider $provider"
- log_error "Set via environment variable or Docker/K8s secret mount"
- return 1
- fi
- # Log masked key for debugging
- local masked="${key_value:0:8}...${key_value: -4}"
- log_info "API key $key_var: $masked (${#key_value} chars)"
- # Fail-fast auth preflight (v7.91): for Claude OAuth logins, a present-but-
- # EXPIRED token passes the presence check above but then 401s on the first
- # provider call, leaving the build stalled at BOOTSTRAP with no clear cause.
- # Catch that here with a zero-network, zero-token local expiry check so the
- # user gets an instant, copy-pasteable fix instead of a silent stall.
- # Opt out with LOKI_SKIP_AUTH_PREFLIGHT=1 (offline/odd setups).
- if [[ "$provider" == "claude" && "${LOKI_SKIP_AUTH_PREFLIGHT:-}" != "1" && -z "${ANTHROPIC_API_KEY:-}" ]]; then
- local _creds="${CLAUDE_CONFIG_DIR:-$HOME/.claude}/.credentials.json"
- if [[ -s "$_creds" ]]; then
- local _expired
- _expired=$(python3 -c "
- import json, sys, time
- try:
- d = json.load(open(sys.argv[1]))
- exp = d.get('claudeAiOauth', {}).get('expiresAt')
- # expiresAt is epoch milliseconds; compare with a 60s safety margin.
- if isinstance(exp, (int, float)) and exp > 0 and (exp / 1000.0) <= (time.time() + 60):
- print('expired')
- except Exception:
- pass # unreadable/unknown schema -> do not block (fail open, let the call decide)
- " "$_creds" 2>/dev/null || true)
- if [[ "$_expired" == "expired" ]]; then
- log_error "Your Claude Code login has expired -- the build would stall instead of running."
- log_error "Fix it in one step, then retry:"
- log_error " claude auth login"
- log_error "(or set ANTHROPIC_API_KEY, or LOKI_SKIP_AUTH_PREFLIGHT=1 to bypass this check)"
- return 1
- fi
- fi
- # NOTE: the never-logged-in (no credentials file) case is handled earlier,
- # BEFORE the local early-return, so it covers local runs too (this Docker/
- # K8s-only block would otherwise be unreachable for that case).
- fi
- return 0
- }
- #===============================================================================
- # Complexity Tier Detection (Auto-Claude pattern)
- #===============================================================================
- # Detect project complexity from PRD and codebase
- detect_complexity() {
- local prd_path="${1:-}"
- local target_dir="${TARGET_DIR:-.}"
- # If forced, use that
- if [ "$COMPLEXITY_TIER" != "auto" ]; then
- DETECTED_COMPLEXITY="$COMPLEXITY_TIER"
- return 0
- fi
- # Count files in project (excluding common non-source dirs)
- local file_count=0
- file_count=$(find "$target_dir" -type f \
- \( -name "*.ts" -o -name "*.tsx" -o -name "*.js" -o -name "*.jsx" \
- -o -name "*.py" -o -name "*.go" -o -name "*.rs" -o -name "*.java" \
- -o -name "*.rb" -o -name "*.php" -o -name "*.swift" -o -name "*.kt" \) \
- ! -path "*/node_modules/*" ! -path "*/.git/*" ! -path "*/vendor/*" \
- ! -path "*/dist/*" ! -path "*/build/*" ! -path "*/__pycache__/*" \
- 2>/dev/null | wc -l | tr -d ' ')
- # Validate file_count is numeric (guard against empty/malformed pipeline output)
- file_count="${file_count:-0}"
- file_count="${file_count//[^0-9]/}"
- # Check for external integrations
- local has_external=false
- if grep -rq "oauth\|SAML\|OIDC\|stripe\|twilio\|aws-sdk\|@google-cloud\|azure" \
- "$target_dir" --include="*.json" --include="*.ts" --include="*.js" 2>/dev/null; then
- has_external=true
- fi
- # Check for multiple services (docker-compose, k8s)
- local has_microservices=false
- if [ -f "$target_dir/docker-compose.yml" ] || [ -d "$target_dir/k8s" ] || \
- [ -f "$target_dir/docker-compose.yaml" ]; then
- has_microservices=true
- fi
- # Analyze PRD if provided
- local prd_complexity="standard"
- if [ -n "$prd_path" ] && [ -f "$prd_path" ]; then
- local prd_words=$(wc -w < "$prd_path" | tr -d ' ')
- local feature_count=0
- local prd_lines=$(wc -l < "$prd_path" | tr -d ' ')
- # Detect PRD format and count features accordingly
- if [[ "$prd_path" == *.json ]]; then
- # JSON PRD: count features, requirements, tasks arrays
- if command -v jq &>/dev/null; then
- feature_count=$(jq '
- [.features, .requirements, .tasks, .user_stories, .epics] |
- map(select(. != null) | if type == "array" then length else 0 end) |
- add // 0
- ' "$prd_path" 2>/dev/null || echo "0")
- else
- # Fallback: count array elements by pattern
- feature_count=$(grep -c '"title"\|"name"\|"feature"\|"requirement"' "$prd_path" 2>/dev/null || echo "0")
- fi
- else
- # Markdown PRD: count headers and checkboxes
- feature_count=$(grep -c "^##\|^- \[" "$prd_path" 2>/dev/null || echo "0")
- fi
- # WAVE8 FIX run.sh-provider-F1 (HIGH): grep -c prints "0" AND exits 1 on
- # zero matches; with the '|| echo "0"' fallback that yields "0\n0", which
- # crashes the integer tests below ([: 0\n0: integer expression expected)
- # and silently drops complexity from simple->standard. Strip to digits
- # after every assignment path (jq, both greps), mirroring file_count:1688.
- # "0\n0" -> "00" -> arithmetically 0.
- feature_count="${feature_count:-0}"
- feature_count="${feature_count//[^0-9]/}"
- feature_count="${feature_count:-0}"
- # Count distinct sections (h2/h3 headers) for structural complexity (#74)
- local section_count=0
- if [[ "$prd_path" != *.json ]]; then
- section_count=$(grep -c "^##\|^###" "$prd_path" 2>/dev/null || echo "0")
- fi
- # WAVE8 FIX run.sh-provider-F1: same grep -c double-output guard.
- section_count="${section_count:-0}"
- section_count="${section_count//[^0-9]/}"
- section_count="${section_count:-0}"
- # PRD complexity uses content length, feature count, AND structural depth (#74)
- # A PRD with multiple sections or substantial content is not "simple" even with few project files
- if [ "$prd_words" -lt 200 ] && [ "$feature_count" -lt 5 ] && [ "$section_count" -lt 3 ]; then
- prd_complexity="simple"
- elif [ "$prd_words" -gt 1000 ] || [ "$feature_count" -gt 15 ] || [ "$section_count" -gt 10 ]; then
- prd_complexity="complex"
- fi
- fi
- # Determine final complexity
- # A non-simple PRD always prevents "simple" classification regardless of file count (#74)
- if [ "$file_count" -le 5 ] && [ "$prd_complexity" = "simple" ] && \
- [ "$has_external" = "false" ] && [ "$has_microservices" = "false" ]; then
- DETECTED_COMPLEXITY="simple"
- elif [ "$file_count" -gt 50 ] || [ "$has_microservices" = "true" ] || \
- [ "$has_external" = "true" ] || [ "$prd_complexity" = "complex" ]; then
- DETECTED_COMPLEXITY="complex"
- else
- DETECTED_COMPLEXITY="standard"
- fi
- log_info "Detected complexity: $DETECTED_COMPLEXITY (files: $file_count, prd: $prd_complexity, external: $has_external, microservices: $has_microservices)"
- }
- # Get phases based on complexity tier
- # Get phase names based on complexity tier
- #===============================================================================
- # Dynamic Tier Selection (RARV-aware model routing)
- #===============================================================================
- # Maps RARV cycle phases to model tiers. IMPORTANT (post-v7.104.0): this rotation
- # is OFF by default -- CURRENT_TIER is pinned once from LOKI_SESSION_MODEL and held
- # constant every iteration; get_rarv_tier() only fires under
- # LOKI_LEGACY_TIER_SWITCHING=true (see the pin logic near run.sh:17150). And even
- # when it does fire, planning/development/fast ALL resolve to the SAME model on
- # stock config (CLAUDE_DEFAULT_*=sonnet), so the rotation changes only the
- # --effort flag, not the model. The tier -> effort map (loki_effort_for_tier,
- # autonomy/lib/claude-flags.sh):
- # - Reason phase -> planning tier (effort xhigh)
- # - Act phase -> development tier (effort high)
- # - Reflect phase -> development tier (effort high)
- # - Verify phase -> fast tier (effort medium)
- # Global tier for current iteration (set by get_rarv_tier)
- CURRENT_TIER="development"
- # Export for provider helper functions (e.g., provider_get_current_model)
- LOKI_CURRENT_TIER="$CURRENT_TIER"
- export LOKI_CURRENT_TIER
- # Get the appropriate tier based on RARV cycle step
- # Args: iteration_count (defaults to ITERATION_COUNT)
- # Returns: tier name (planning, development, fast)
- get_rarv_tier() {
- local iteration="${1:-$ITERATION_COUNT}"
- local rarv_step=$((iteration % 4))
- case $rarv_step in
- 0) # Reason phase - planning/architecture
- echo "planning"
- ;;
- 1) # Act phase - implementation
- echo "development"
- ;;
- 2) # Reflect phase - review/analysis
- echo "development"
- ;;
- 3) # Verify phase - testing/validation
- echo "fast"
- ;;
- *) # Fallback to development
- echo "development"
- ;;
- esac
- }
- # Get RARV phase name for logging
- get_rarv_phase_name() {
- local iteration="${1:-$ITERATION_COUNT}"
- local rarv_step=$((iteration % 4))
- case $rarv_step in
- 0) echo "REASON" ;;
- 1) echo "ACT" ;;
- 2) echo "REFLECT" ;;
- 3) echo "VERIFY" ;;
- *) echo "UNKNOWN" ;;
- esac
- }
- # Get provider-specific tier parameter based on current tier
- # v6.0.0: Delegates to resolve_model_for_tier() if available (dynamic resolution).
- # Falls back to static mapping for backward compatibility.
- get_provider_tier_param() {
- local tier="${1:-$CURRENT_TIER}"
- # v6.0.0: Use dynamic resolution if provider has resolve_model_for_tier
- if type resolve_model_for_tier &>/dev/null; then
- local resolved
- resolved=$(resolve_model_for_tier "$tier")
- echo "$resolved"
- return
- fi
- # Legacy fallback: static tier mapping
- case "${PROVIDER_NAME:-claude}" in
- claude)
- case "$tier" in
- planning)
- # Evidence-based routing (scoped): the official model-config
- # docs explicitly name "architecture decisions" and
- # "root-cause investigations" as where Fable 5's extra
- # investigation and self-verification pay off. So the
- # planning/architecture tier may opt in to Fable via
- # LOKI_FABLE_ARCHITECT=1. Default OFF because Fable is 2x
- # Opus per token; reserve it for the REASON/architecture
- # iterations the user explicitly wants. An explicit
- # PROVIDER_MODEL_PLANNING still wins (operator override).
- if [ -n "${PROVIDER_MODEL_PLANNING:-}" ]; then
- echo "${PROVIDER_MODEL_PLANNING}"
- elif [ "${LOKI_FABLE_ARCHITECT:-0}" = "1" ]; then
- # fable unavailable, collapse to opus. Claude Fable 5 is
- # not available at the Claude API ("use Opus 4.8"); the
- # architect opt-in now runs opus. Matches claude.sh
- # resolve_model_for_tier and the estimator/dashboard.
- echo "opus"
- else
- # v7.104.0: default planning to sonnet (was opus) to match
- # the sourced claude.sh default. This bare fallback only
- # fires when claude.sh is unsourced.
- echo "sonnet"
- fi
- ;;
- development) echo "${PROVIDER_MODEL_DEVELOPMENT:-sonnet}" ;;
- fast) echo "${PROVIDER_MODEL_FAST:-sonnet}" ;;
- # fable unavailable, collapse to opus. Without this arm an
- # unsourced-claude.sh environment (this static fallback) would
- # silently downgrade a fable-pinned tier to sonnet via the `*`
- # default. Matches resolve_model_for_tier's explicit fable) arm,
- # which now resolves to opus (Fable 5 unavailable at the API).
- fable) echo "opus" ;;
- *) echo "sonnet" ;;
- esac
- ;;
- *)
- echo "development"
- ;;
- esac
- }
- #===============================================================================
- # Provider Spawn Timeout (removed WAVE9 / provider-F2)
- #
- # A former invoke_with_timeout() helper (v6.0.0) wrapped a command in
- # `timeout <s> "$@"` with a retry loop. It was never wired to the main
- # provider invocation and is intentionally not revived, for two reasons:
- #
- # 1. No safe generous default. The main provider call is a long-running
- # autonomous coding agent. Any fixed timeout short enough to catch a
- # hang would also kill legitimate multi-minute iterations, and there is
- # no "generous enough" value that is both safe and useful by default.
- # 2. Wrong retry semantics. The helper re-ran the same command on timeout.
- # Re-running a coding agent mid-work (it may have already edited files)
- # is actively harmful, not protective.
- #
- # The main invocation is also a pipeline (`claude | tee | python3`), which a
- # positional-arg `timeout "$@"` wrapper cannot wrap at all. Interrupting a
- # hung provider is handled by the SIGINT trap (kill_provider_child) instead.
- #
- # The `loki config spawn_timeout` / `spawn_retries` knobs (autonomy/loki) and
- # the config->env mapping in this file (`'spawn_timeout':'LOKI_SPAWN_TIMEOUT'`
- # in the config loader) still export LOKI_SPAWN_TIMEOUT / LOKI_SPAWN_RETRIES,
- # but nothing consumes them now. The mapping line is intentionally left in place
- # (a config-schema test may enumerate it); the full inert-knob removal spans
- # autonomy/loki too and is a separate cross-file follow-up.
- #===============================================================================
- #===============================================================================
- # GitHub Integration Functions (v4.1.0)
- #===============================================================================
- # GitHub integration settings
- GITHUB_IMPORT=${LOKI_GITHUB_IMPORT:-false}
- GITHUB_PR=${LOKI_GITHUB_PR:-false}
- GITHUB_SYNC=${LOKI_GITHUB_SYNC:-false}
- GITHUB_REPO=${LOKI_GITHUB_REPO:-""}
- GITHUB_LABELS=${LOKI_GITHUB_LABELS:-""}
- GITHUB_MILESTONE=${LOKI_GITHUB_MILESTONE:-""}
- GITHUB_ASSIGNEE=${LOKI_GITHUB_ASSIGNEE:-""}
- GITHUB_LIMIT=${LOKI_GITHUB_LIMIT:-100}
- GITHUB_PR_LABEL=${LOKI_GITHUB_PR_LABEL:-""}
- # Check if gh CLI is available and authenticated
- check_github_cli() {
- if ! command -v gh &> /dev/null; then
- log_warn "gh CLI not found. Install with: brew install gh"
- return 1
- fi
- if ! gh auth status &> /dev/null; then
- log_warn "gh CLI not authenticated. Run: gh auth login"
- return 1
- fi
- return 0
- }
- # Get current repo from git remote or LOKI_GITHUB_REPO
- get_github_repo() {
- if [ -n "$GITHUB_REPO" ]; then
- echo "$GITHUB_REPO"
- return
- fi
- # Try to detect from git remote
- local remote_url
- remote_url=$(git remote get-url origin 2>/dev/null || echo "")
- if [ -z "$remote_url" ]; then
- return 1
- fi
- # Extract owner/repo from various URL formats
- # https://github.com/owner/repo.git
- # git@github.com:owner/repo.git
- local repo
- repo=$(echo "$remote_url" | sed -E 's/.*github.com[:/]([^/]+\/[^/]+)(\.git)?$/\1/')
- repo="${repo%.git}"
- if [ -n "$repo" ] && [[ "$repo" == *"/"* ]]; then
- echo "$repo"
- return 0
- fi
- return 1
- }
- # Import issues from GitHub as tasks
- import_github_issues() {
- if [ "$GITHUB_IMPORT" != "true" ]; then
- return 0
- fi
- if ! check_github_cli; then
- return 1
- fi
- local repo
- repo=$(get_github_repo)
- if [ -z "$repo" ]; then
- log_error "Could not determine GitHub repo. Set LOKI_GITHUB_REPO=owner/repo"
- return 1
- fi
- log_info "Importing issues from GitHub: $repo"
- # Build gh issue list command with filters
- local gh_args=("issue" "list" "--repo" "$repo" "--state" "open" "--limit" "$GITHUB_LIMIT" "--json" "number,title,body,labels,url,milestone,assignees")
- if [ -n "$GITHUB_LABELS" ]; then
- IFS=',' read -ra LABELS <<< "$GITHUB_LABELS"
- for label in "${LABELS[@]}"; do
- # Trim whitespace from label
- label=$(echo "$label" | xargs)
- gh_args+=("--label" "$label")
- done
- fi
- if [ -n "$GITHUB_MILESTONE" ]; then
- gh_args+=("--milestone" "$GITHUB_MILESTONE")
- fi
- if [ -n "$GITHUB_ASSIGNEE" ]; then
- gh_args+=("--assignee" "$GITHUB_ASSIGNEE")
- fi
- # Fetch issues with error capture
- local issues gh_error
- if ! issues=$(gh "${gh_args[@]}" 2>&1); then
- gh_error="$issues"
- if echo "$gh_error" | grep -q "rate limit"; then
- log_error "GitHub API rate limit exceeded. Wait and retry."
- else
- log_error "Failed to fetch issues: $gh_error"
- fi
- return 1
- fi
- if [ -z "$issues" ] || [ "$issues" == "[]" ]; then
- log_info "No open issues found matching filters"
- return 0
- fi
- # Convert issues to tasks
- local pending_file=".loki/queue/pending.json"
- local task_count=0
- # BUG #14 fix: Normalize to bare [] format (consistent with init_loki_dir
- # and all other queue consumers). Previously used {"tasks":[]} wrapper here
- # but bare [] everywhere else, causing format mismatch.
- if [ ! -f "$pending_file" ]; then
- echo '[]' > "$pending_file"
- elif jq -e 'type == "object"' "$pending_file" &>/dev/null; then
- # Normalize {"tasks":[...]} wrapper to bare array
- local _tmp_normalize
- _tmp_normalize=$(mktemp)
- jq 'if type == "object" then .tasks // [] else . end' "$pending_file" > "$_tmp_normalize" && mv "$_tmp_normalize" "$pending_file"
- rm -f "$_tmp_normalize"
- fi
- # Parse issues in the current shell so task_count remains accurate without
- # relying on descriptor-backed process substitution inside confinement.
- local issue_rows
- issue_rows=$(printf '%s\n' "$issues" | jq -c '.[]') || return 1
- while read -r issue; do
- [ -n "$issue" ] || continue
- local number title body full_body url labels
- number=$(echo "$issue" | jq -r '.number')
- title=$(echo "$issue" | jq -r '.title')
- full_body=$(echo "$issue" | jq -r '.body // ""')
- # Truncate body with indicator if needed
- if [ ${#full_body} -gt 500 ]; then
- body="${full_body:0:497}..."
- else
- body="$full_body"
- fi
- url=$(echo "$issue" | jq -r '.url')
- labels=$(echo "$issue" | jq -c '[.labels[].name]')
- # Check if task already exists (bare array format)
- if jq -e ".[] | select(.github_issue == $number)" "$pending_file" &>/dev/null; then
- log_info "Issue #$number already imported, skipping"
- continue
- fi
- # Determine priority from labels
- local priority="normal"
- if echo "$labels" | grep -qE '"(priority:critical|P0)"'; then
- priority="critical"
- elif echo "$labels" | grep -qE '"(priority:high|P1)"'; then
- priority="high"
- elif echo "$labels" | grep -qE '"(priority:medium|P2)"'; then
- priority="medium"
- elif echo "$labels" | grep -qE '"(priority:low|P3)"'; then
- priority="low"
- fi
- # Add task to pending queue
- local task_id="github-$number"
- local task_json
- task_json=$(jq -n \
- --arg id "$task_id" \
- --arg title "$title" \
- --arg desc "GitHub Issue #$number: $body" \
- --argjson num "$number" \
- --arg url "$url" \
- --argjson labels "$labels" \
- --arg priority "$priority" \
- --arg created "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
- '{
- id: $id,
- title: $title,
- description: $desc,
- source: "github",
- github_issue: $num,
- github_url: $url,
- labels: $labels,
- priority: $priority,
- status: "pending",
- created_at: $created
- }')
- # BUG-XC-010: Create temp file in same directory as target (avoids cross-filesystem mv).
- # v7.5.12: replace flock-only queue lock with portable mkdir-mutex via
- # safe_acquire_lock (works on macOS without util-linux flock).
- local temp_file
- temp_file=$(mktemp ".loki/queue/pending.json.tmp.XXXXXX")
- local lockfile=".loki/queue/.pending.lock"
- if type safe_acquire_lock >/dev/null 2>&1 && safe_acquire_lock "$lockfile" 5; then
- if jq ". += [$task_json]" "$pending_file" > "$temp_file" && mv "$temp_file" "$pending_file"; then
- log_info "Imported issue #$number: $title"
- task_count=$((task_count + 1))
- else
- log_warn "Failed to import issue #$number"
- fi
- safe_release_lock "$lockfile"
- else
- log_warn "Could not acquire queue lock for issue #$number, skipping"
- fi
- rm -f "$temp_file"
- done << GITHUB_ISSUES_EOF
- $issue_rows
- GITHUB_ISSUES_EOF
- log_info "Imported $task_count issues from GitHub"
- }
- # Create PR for completed feature
- create_github_pr() {
- local feature_name="$1"
- local branch_name="${2:-$(git rev-parse --abbrev-ref HEAD)}"
- if [ "$GITHUB_PR" != "true" ]; then
- return 0
- fi
- if ! check_github_cli; then
- return 1
- fi
- local repo
- repo=$(get_github_repo)
- if [ -z "$repo" ]; then
- log_error "Could not determine GitHub repo"
- return 1
- fi
- log_info "Creating PR for: $feature_name"
- # Generate PR body from completed tasks
- local pr_body=".loki/reports/pr-body.md"
- mkdir -p "$(dirname "$pr_body")"
- local version
- version=$(cat "${SCRIPT_DIR%/*}/VERSION" 2>/dev/null || echo "unknown")
- cat > "$pr_body" << EOF
- ## Summary
- Automated implementation by Loki Mode v$version ($ITERATION_COUNT iterations, provider: ${PROVIDER_NAME:-claude})
- ### Feature: $feature_name
- ### Tasks Completed
- EOF
- # Add completed tasks from ledger
- if [ -f ".loki/ledger.json" ]; then
- jq -r '.completed_tasks[]? | "- [x] \(.title // .id)"' .loki/ledger.json >> "$pr_body" 2>/dev/null || true
- fi
- cat >> "$pr_body" << EOF
- ### Quality Gates
- - Static Analysis: $([ -f ".loki/quality/static-analysis.pass" ] && echo "PASS" || echo "PENDING")
- - Unit Tests: $([ -f ".loki/quality/unit-tests.pass" ] && echo "PASS" || echo "PENDING")
- - Code Review: $([ -f ".loki/quality/code-review.pass" ] && echo "PASS" || echo "PENDING")
- ### Related Issues
- EOF
- # Find related GitHub issues
- if [ -f ".loki/ledger.json" ]; then
- jq -r '.completed_tasks[]? | select(.github_issue) | "Closes #\(.github_issue)"' .loki/ledger.json >> "$pr_body" 2>/dev/null || true
- fi
- # Proven PR (Loop 6): append the Evidence Receipt to the body file before
- # gh pr create --body-file. Default-on; LOKI_PROVEN_PR=0 -> body file bytes
- # byte-identical to before. Empty expected_head_sha by design (R-DET-1
- # run_id pointer is the anti-stale guard; the branch head is offset by the
- # session commit). Best-effort: a missing proof appends nothing extra here.
- if [ "${LOKI_PROVEN_PR:-1}" != "0" ] && declare -f render_evidence_receipt_md >/dev/null 2>&1; then
- local _bf_proof=""
- _bf_proof="$(_loki_proof_json_for_pr 2>/dev/null || true)"
- if [ -n "$_bf_proof" ]; then
- { printf '\n'; render_evidence_receipt_md "$_bf_proof" "" "" 2>/dev/null; } >> "$pr_body" 2>/dev/null || true
- fi
- fi
- # Build PR create command
- local pr_args=("pr" "create" "--repo" "$repo" "--title" "[Loki Mode] $feature_name" "--body-file" "$pr_body")
- # Add label only if specified (avoids error if label doesn't exist)
- if [ -n "$GITHUB_PR_LABEL" ]; then
- pr_args+=("--label" "$GITHUB_PR_LABEL")
- fi
- # Create PR and capture output
- local pr_url
- if ! pr_url=$(gh "${pr_args[@]}" 2>&1); then
- log_error "Failed to create PR: $pr_url"
- return 1
- fi
- log_info "PR created: $pr_url"
- }
- # Sync task status to GitHub issue
- sync_github_status() {
- local task_id="$1"
- local status="$2"
- local message="${3:-}"
- if [ "$GITHUB_SYNC" != "true" ]; then
- return 0
- fi
- if ! check_github_cli; then
- return 1
- fi
- # Extract issue number from task_id (format: github-123)
- local issue_number
- issue_number=$(echo "$task_id" | sed 's/github-//')
- if ! [[ "$issue_number" =~ ^[0-9]+$ ]]; then
- return 0 # Not a GitHub-sourced task
- fi
- local repo
- repo=$(get_github_repo)
- if [ -z "$repo" ]; then
- return 1
- fi
- # Track synced issues to avoid duplicate comments
- mkdir -p .loki/github
- local sync_log=".loki/github/synced.log"
- local sync_key="${issue_number}:${status}"
- if [ -f "$sync_log" ] && grep -qF "$sync_key" "$sync_log" 2>/dev/null; then
- return 0 # Already synced this status
- fi
- case "$status" in
- "in_progress")
- gh issue comment "$issue_number" --repo "$repo" \
- --body "**Loki Mode** -- Working on this issue (iteration $ITERATION_COUNT)" \
- 2>/dev/null || true
- ;;
- "completed")
- local branch
- branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "main")
- local commit
- commit=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
- gh issue comment "$issue_number" --repo "$repo" \
- --body "**Loki Mode** -- Implementation complete on \`$branch\` ($commit). ${message:-}" \
- 2>/dev/null || true
- ;;
- "closed")
- gh issue close "$issue_number" --repo "$repo" \
- --reason "completed" \
- --comment "**Loki Mode** -- Resolved. ${message:-}" \
- 2>/dev/null || true
- ;;
- esac
- # Record sync to avoid duplicates
- echo "$sync_key" >> "$sync_log"
- }
- # Sync all completed GitHub-sourced tasks back to their issues
- # Called after each iteration and at session end
- sync_github_completed_tasks() {
- if [ "$GITHUB_SYNC" != "true" ]; then
- return 0
- fi
- if ! check_github_cli; then
- return 0
- fi
- local completed_file=".loki/queue/completed.json"
- if [ ! -f "$completed_file" ]; then
- return 0
- fi
- # Find GitHub-sourced tasks in completed queue that haven't been synced
- python3 -c "
- import json, sys
- try:
- with open('$completed_file') as f:
- tasks = json.load(f)
- for t in tasks:
- tid = t.get('id', '')
- if tid.startswith('github-'):
- print(tid)
- except Exception:
- pass
- " 2>/dev/null | while read -r task_id; do
- sync_github_status "$task_id" "completed"
- done
- }
- # Sync GitHub-sourced tasks currently in-progress
- sync_github_in_progress_tasks() {
- if [ "$GITHUB_SYNC" != "true" ]; then
- return 0
- fi
- if ! check_github_cli; then
- return 0
- fi
- local pending_file=".loki/queue/pending.json"
- if [ ! -f "$pending_file" ]; then
- return 0
- fi
- # Find GitHub-sourced tasks in pending queue (about to be worked on)
- python3 -c "
- import json
- try:
- with open('$pending_file') as f:
- data = json.load(f)
- tasks = data.get('tasks', data) if isinstance(data, dict) else data
- for t in tasks:
- tid = t.get('id', '')
- if tid.startswith('github-'):
- print(tid)
- except Exception:
- pass
- " 2>/dev/null | while read -r task_id; do
- sync_github_status "$task_id" "in_progress"
- done
- }
- # Export tasks to GitHub issues (reverse sync)
- export_tasks_to_github() {
- if ! check_github_cli; then
- return 1
- fi
- local repo
- repo=$(get_github_repo)
- if [ -z "$repo" ]; then
- log_error "Could not determine GitHub repo"
- return 1
- fi
- local pending_file=".loki/queue/pending.json"
- if [ ! -f "$pending_file" ]; then
- log_warn "No pending tasks to export"
- return 0
- fi
- # Export non-GitHub tasks as issues (handles both bare array and wrapper formats)
- jq -c 'if type == "object" then .tasks // [] else . end | .[] | select(.source != "github")' "$pending_file" 2>/dev/null | while read -r task; do
- local title desc
- title=$(echo "$task" | jq -r '.title')
- desc=$(echo "$task" | jq -r '.description // ""')
- log_info "Creating issue: $title"
- # BUG-GH-009: Check if label exists before using --label; skip if absent
- local label_flag=""
- if gh label list --repo "$repo" 2>/dev/null | grep -q "loki-mode"; then
- label_flag="--label loki-mode"
- fi
- gh issue create --repo "$repo" \
- --title "$title" \
- --body "$desc" \
- $label_flag \
- 2>/dev/null || log_warn "Failed to create issue: $title"
- done
- }
- #===============================================================================
- # Desktop Notifications (v4.1.0)
- #===============================================================================
- # Notification settings
- NOTIFICATIONS_ENABLED=${LOKI_NOTIFICATIONS:-true}
- NOTIFICATION_SOUND=${LOKI_NOTIFICATION_SOUND:-true}
- # Send desktop notification (cross-platform)
- send_notification() {
- local title="$1"
- local message="$2"
- local urgency="${3:-normal}" # low, normal, critical
- if [ "$NOTIFICATIONS_ENABLED" != "true" ]; then
- return 0
- fi
- # Validate inputs - skip empty notifications
- if [ -z "$title" ] && [ -z "$message" ]; then
- return 0
- fi
- title="${title:-Notification}" # Default title if empty
- # macOS: use osascript
- if command -v osascript &> /dev/null; then
- # Escape backslashes first, then double quotes for AppleScript
- local escaped_title="${title//\\/\\\\}"
- escaped_title="${escaped_title//\"/\\\"}"
- local escaped_message="${message//\\/\\\\}"
- escaped_message="${escaped_message//\"/\\\"}"
- osascript -e "display notification \"$escaped_message\" with title \"Loki Mode\" subtitle \"$escaped_title\"" 2>/dev/null || true
- # Play sound if enabled (low urgency intentionally silent)
- if [ "$NOTIFICATION_SOUND" = "true" ]; then
- case "$urgency" in
- critical)
- osascript -e 'beep 3' 2>/dev/null || true
- ;;
- normal)
- osascript -e 'beep' 2>/dev/null || true
- ;;
- low)
- # Intentionally no sound for low urgency notifications
- ;;
- esac
- fi
- return 0
- fi
- # Linux: use notify-send
- if command -v notify-send &> /dev/null; then
- local notify_urgency="normal"
- case "$urgency" in
- critical) notify_urgency="critical" ;;
- low) notify_urgency="low" ;;
- *) notify_urgency="normal" ;;
- esac
- # Escape markup characters for notify-send (supports basic Pango)
- local safe_title="${title//&/&}"
- safe_title="${safe_title//</<}"
- safe_title="${safe_title//>/>}"
- local safe_message="${message//&/&}"
- safe_message="${safe_message//</<}"
- safe_message="${safe_message//>/>}"
- notify-send -u "$notify_urgency" "Loki Mode: $safe_title" "$safe_message" 2>/dev/null || true
- return 0
- fi
- # Fallback: terminal bell for critical notifications
- if [ "$urgency" = "critical" ]; then
- printf '\a' # Bell character
- fi
- return 0
- }
- notify_intervention_needed() {
- local reason="$1"
- # Delegate-then-notify: this helper ONLY fires the (gated) desktop ping. It
- # deliberately does NOT write the durable COMPLETION.txt / completion.json
- # record. Reason: notify_intervention_needed is also called from NON-terminal
- # sites (the perpetual-mode PAUSE auto-clear branch, uncertainty escalation)
- # where the run keeps going. Writing a "Needs input" durable file there would
- # falsely tell a detached user the run is done / blocked when it is not. The
- # durable intervention write now lives only at the genuinely blocking pause
- # sites (immediately before handle_pause), so the durable state matches the
- # actual run state.
- send_notification "Intervention Needed" "$reason" "critical"
- }
- notify_rate_limit() {
- local wait_time="$1"
- send_notification "Rate Limited" "Waiting ${wait_time}s before retry" "normal"
- }
- #===============================================================================
- # Delegate-then-notify: completion summary (Release 2, "delegate then notify")
- #
- # build_completion_summary <outcome> writes two durable files that survive a
- # detached (--bg) run where the terminal is gone and a bell would be useless:
- # .loki/COMPLETION.txt human plain text (no emojis, no dashes)
- # .loki/state/completion.json machine-readable record of the same facts
- # It also exports two strings for send_notification to consume:
- # _LOKI_SUMMARY_TITLE short notification subtitle
- # _LOKI_SUMMARY_BODY short notification body (outcome + branch + file count)
- #
- # All git reads are best-effort and non-fatal. The diff window is the run-start
- # SHA captured once at runner init (_LOKI_RUN_START_SHA); we REUSE it and never
- # recapture, so the reported diff matches the evidence gate's window exactly.
- #
- # This function NEVER sends a notification and NEVER gates on
- # NOTIFICATIONS_ENABLED: the files are state, not a notification, and must be
- # written even when desktop notifications are disabled. emit_completion_summary
- # below is the wrapper that writes the files AND (gated) fires the desktop ping.
- #===============================================================================
- # Read the gate escalation signal into one human line, or print nothing.
- # Shared by COMPLETION.txt's per-outcome guidance and PAUSED.md so the two never
- # disagree about why a run stopped. Best-effort: a missing or corrupt signal
- # yields an empty string, and the caller degrades to generic guidance.
- _loki_summary_gate_reason() {
- local loki_dir="${1:-${TARGET_DIR:-.}/.loki}"
- [ -s "$loki_dir/signals/GATE_ESCALATION.json" ] || return 0
- _LOKI_GR_FILE="$loki_dir/signals/GATE_ESCALATION.json" python3 -c '
- import json, os, sys
- try:
- d = json.load(open(os.environ["_LOKI_GR_FILE"]))
- except Exception:
- sys.exit(0)
- if not isinstance(d, dict):
- sys.exit(0)
- gate = str(d.get("gate", "") or "").strip()
- if not gate:
- sys.exit(0)
- count = d.get("count")
- thr = d.get("threshold")
- if isinstance(count, int) and isinstance(thr, int):
- print("%s (failed %d times, threshold %d)" % (gate, count, thr))
- else:
- print(gate)
- ' 2>/dev/null || true
- }
- # _loki_receipt_facts <loki_dir>
- #
- # Echo three tab-separated facts about THIS run's Evidence Receipt, or nothing
- # at all: "<run_id>\t<proof_dir>\t<headline>".
- #
- # WHY THIS EXISTS (#209). The receipt is generated automatically and opt-OUT
- # (LOKI_PROOF defaults to 1), but the only place its location ever reached a
- # user was the TTFV first-run block, which fires solely on a zero-config first
- # run. A user who does not already know `loki proof` exists could therefore
- # finish a successful run without ever learning a checkable receipt was written.
- #
- # HONESTY RULES, deliberately fail-silent:
- # - Resolved from the PERSISTED pointer (.loki/state/last-proof-id.txt),
- # never newest-by-mtime, so it names the run the generator actually wrote.
- # - Emits NOTHING unless the pointer, the proof dir, and proof.json all exist.
- # the summary builder (build_completion_summary) is itself invoked mid-pause
- # before the teardown receipt on the success path, where no receipt exists
- # yet -- printing a path there would be a promise, not a fact.
- # - The headline is honesty.headline, which the generator computes
- # DETERMINISTICALLY from recorded facts (VERIFIED / VERIFIED WITH GAPS /
- # NOT VERIFIED). It is never the council verdict, which the generator
- # itself labels "AI judgment, not deterministic proof". A missing or
- # unreadable headline yields an empty third field, and the callers then
- # print the path without a verdict rather than inventing one.
- # Best-effort: never fails a run.
- _loki_receipt_facts() {
- local loki_dir="${1:-}"
- [ -n "$loki_dir" ] || return 0
- local id_file="$loki_dir/state/last-proof-id.txt"
- [ -s "$id_file" ] || return 0
- local rid=""
- rid="$(cat "$id_file" 2>/dev/null || true)"
- # Confine the run id to the alphabet the generator actually mints. The
- # headline is sanitized below; the id was not, so an interior newline or a
- # tab in the pointer split the tab-separated record and handed the consumers
- # a field boundary that is not there. Reject rather than repair: a pointer
- # outside this alphabet is corrupt, and a repaired id would name a directory
- # that does not exist.
- case "$rid" in
- ''|*[!A-Za-z0-9._-]*) return 0 ;;
- esac
- [ -n "$rid" ] || return 0
- local pj="$loki_dir/proofs/$rid/proof.json"
- [ -f "$pj" ] || return 0
- local headline=""
- headline="$(python3 -E -c "import sys; sys.path[:] = [p for p in sys.path if p not in ('', '.')]
- import json, sys
- try:
- d = json.load(open(sys.argv[1]))
- h = (d.get('honesty') or {}).get('headline') or ''
- print(str(h).replace('\t', ' ').replace('\n', ' ').strip())
- except Exception:
- print('')" "$pj" 2>/dev/null || true)"
- printf '%s\t%s\t%s\n' "$rid" "$loki_dir/proofs/$rid" "$headline"
- return 0
- }
- build_completion_summary() {
- local outcome="${1:-complete}"
- local loki_dir="${TARGET_DIR:-.}/.loki"
- mkdir -p "$loki_dir/state" 2>/dev/null || true
- # Human-readable outcome label and notification title.
- local outcome_label notify_title
- case "$outcome" in
- complete) outcome_label="Completed"; notify_title="Run complete" ;;
- max_iterations) outcome_label="Max iterations"; notify_title="Run stopped (max iterations)" ;;
- stopped) outcome_label="Stopped"; notify_title="Run stopped" ;;
- failed) outcome_label="Failed"; notify_title="Run failed" ;;
- intervention) outcome_label="Needs input"; notify_title="Input needed" ;;
- # Every outcome below reached this case and fell through to the `*` arm,
- # so the user's headline label was the raw enum string --
- # "council_force_approved", "max_duration" -- at the exact moment they
- # were deciding whether to trust the build. The guidance block further
- # down already handles several of these properly; only the label was
- # missing.
- force_stopped) outcome_label="Stopped without approval"
- notify_title="Run stopped (not approved)" ;;
- budget_exceeded) outcome_label="Stopped at spend cap"
- notify_title="Run stopped (budget cap)" ;;
- max_duration) outcome_label="Time limit reached"
- notify_title="Run stopped (time limit)" ;;
- max_retries_exceeded) outcome_label="Retries exhausted"
- notify_title="Run stopped (retries exhausted)" ;;
- inconclusive_spec_contradiction) outcome_label="Spec contradiction"
- notify_title="Run stopped (spec contradiction)" ;;
- # Force-approval is NOT the same as council approval, and labelling both
- # "Completed" hid the difference on a product whose whole claim is a
- # checkable receipt. Name it.
- council_force_approved) outcome_label="Completed (force-approved)"
- notify_title="Run complete (force-approved)" ;;
- # The three gate-stuck terminals. Spelled out one literal arm each
- # rather than as a single wildcard pattern:
- # tests/test-completion-outcome-labels.sh derives outcomes from the call
- # sites and matches `^ *<outcome>\)`, so a wildcard arm renders correctly
- # at runtime while reading as "no label arm" to the guard. Literal arms
- # keep that guard strictly literal, which is what makes a NEWLY-added
- # outcome fail there instead of shipping as a raw enum. Same reason the
- # block above names each outcome.
- gate_stuck_static_analysis) outcome_label="Stopped (static analysis gate would not clear)"
- notify_title="Run stopped (gate not clearing)" ;;
- gate_stuck_mock_integrity) outcome_label="Stopped (mock integrity gate would not clear)"
- notify_title="Run stopped (gate not clearing)" ;;
- gate_stuck_mutation_integrity) outcome_label="Stopped (mutation integrity gate would not clear)"
- notify_title="Run stopped (gate not clearing)" ;;
- *) outcome_label="$outcome"; notify_title="Run finished" ;;
- esac
- # Live app URL (best-effort): if the app runner has a running app, surface
- # where the user can try it. Reads .loki/app-runner/state.json written by
- # app-runner.sh. Empty when no app is running.
- local live_app_url=""
- local _app_state_file="$loki_dir/app-runner/state.json"
- if [ -f "$_app_state_file" ]; then
- live_app_url="$(python3 -c "import json,sys
- try:
- d=json.load(open(sys.argv[1]))
- print(d.get('url','') if d.get('status')=='running' else '')
- except Exception:
- print('')" "$_app_state_file" 2>/dev/null)"
- fi
- # Time to first preview: how long until the user could SEE something running.
- #
- # WRITTEN BUT NEVER READ, until now. app-runner.sh:177 has recorded
- # seconds_to_first_preview since v8, write-once and atomically, and NOTHING
- # consumed it -- not this summary, not the receipt, not the dashboard. A
- # measurement nobody surfaces cannot influence anything, so the number that
- # research identifies as the delight peak of this whole category (a working
- # preview at roughly four minutes) was invisible to the person who waited for
- # it and to us.
- #
- # Read-only and best-effort by design: absent file means the app never
- # previewed (or the run predates the writer), and we render nothing rather
- # than a zero. A fabricated "0s to preview" would be worse than silence.
- local first_preview_s=""
- local _fp_file="$loki_dir/app-runner/first-preview.json"
- if [ -f "$_fp_file" ]; then
- first_preview_s="$(python3 -c "import json,sys
- try:
- v=json.load(open(sys.argv[1])).get('seconds_to_first_preview')
- print(int(v) if isinstance(v,(int,float)) and v>=0 else '')
- except Exception:
- print('')" "$_fp_file" 2>/dev/null)"
- fi
- # Time to first CODE CHANGE. Companion to first_preview_s for the run that
- # never previews anything, which is most of them: a scoped issue fix has no
- # app to bring up, so the preview number is empty and the user had no signal
- # at all until the run ended.
- local first_artifact_s=""
- local _fa_read="$loki_dir/state/first-artifact.json"
- if [ -f "$_fa_read" ]; then
- first_artifact_s="$(python3 -c "import json,sys
- try:
- v=json.load(open(sys.argv[1])).get('seconds_to_first_artifact')
- print(int(v) if isinstance(v,(int,float)) and v>=0 else '')
- except Exception:
- print('')" "$_fa_read" 2>/dev/null)"
- fi
- # Where the time went, per stage. Same "written but never read" story as
- # first-preview above: emit_stage_complete (run.sh:2413) has appended a
- # stage_complete record -- stage, status, duration_s, iteration -- to
- # events.jsonl since v7.91.x, and NOTHING consumed it. A user watching a
- # 322-word issue take 25+ minutes inside iteration 1 had no way to see which
- # step ate it, because the measurement existed and was never surfaced.
- #
- # Read-only aggregation over a file the run already wrote: no new subprocess
- # per stage, no new writer, one python3 pass at terminal time. We deliberately
- # render the AGENT remainder (wall clock minus summed stages) rather than
- # stages alone. The 9 emit_stage_complete sites are all post-iteration gates,
- # which sum to seconds; a table of only those would print "gates: 90s" on a
- # 25-minute run and still not answer the question. The remainder is the
- # provider/agent time, and it is usually the answer.
- #
- # Best-effort and honest about absence: no events file, no stage records, or
- # unparseable lines render NOTHING rather than a fabricated zero -- same
- # reasoning as first_preview_s. A wrong timing table is worse than silence.
- # Rework split for the user-facing summary. Reuses the SAME
- # iteration_attribution.py the prompt-side eval trend calls, so the number
- # the user reads and the number the agent steers on cannot drift apart.
- # Emits nothing unless the split is actually known.
- local _LOKI_REWORK_LINE=""
- if [ -r "${SCRIPT_DIR:-}/lib/iteration_attribution.py" ]; then
- _LOKI_REWORK_LINE="$(python3 "${SCRIPT_DIR}/lib/iteration_attribution.py" \
- --loki-dir "$loki_dir" --json 2>/dev/null | python3 -c "
- import json, sys
- try:
- d = json.load(sys.stdin)
- except Exception:
- sys.exit(0)
- prog, rew = d.get('progress', {}), d.get('rework', {})
- pc, rc = prog.get('count', 0), rew.get('count', 0)
- # Unknown-only runs render nothing: a 0/0 split is not evidence of no rework,
- # it is absence of attribution, and printing it would read as a clean bill.
- if pc + rc == 0:
- sys.exit(0)
- cost = rew.get('cost_usd', 0.0) or 0.0
- line = 'Rework: %d of %d iteration(s) redid earlier work' % (rc, pc + rc)
- if cost > 0:
- line += ' (\$%.2f)' % cost
- print(line)
- " 2>/dev/null || true)"
- fi
- local stage_timing=""
- local _ev_file="$loki_dir/events.jsonl"
- if [ -f "$_ev_file" ]; then
- stage_timing="$(LOKI_RUN_START_EPOCH="${_LOKI_RUN_START_EPOCH:-}" python3 -c "
- import json, os, sys
- tot = {}
- order = []
- first = last = None
- # events.jsonl is NEVER truncated between runs (no rm/rotate anywhere in the
- # tree), so a second 'loki start' in the same workspace would otherwise sum
- # stages from every previous run against THIS run's wall clock -- inflating
- # staged past wall and silently killing the total/remainder rows. Filter to
- # records at or after this run's start. The ISO timestamp is already on every
- # record, so this costs nothing extra.
- run_start_iso = None
- _es = (os.environ.get('LOKI_RUN_START_EPOCH') or '').strip()
- if _es:
- try:
- from datetime import datetime, timezone
- run_start_iso = datetime.fromtimestamp(float(_es), timezone.utc)
- except Exception:
- run_start_iso = None
- def _in_run(ts):
- if run_start_iso is None or not isinstance(ts, str) or not ts:
- return True # no reliable boundary: keep (old behavior)
- try:
- from datetime import datetime
- t = datetime.fromisoformat(ts.replace('Z', '+00:00'))
- return t >= run_start_iso
- except Exception:
- return True
- try:
- with open(sys.argv[1], errors='replace') as fh:
- for line in fh:
- line = line.strip()
- if not line:
- continue
- try:
- rec = json.loads(line)
- except Exception:
- continue # malformed line: skip, never abort the summary
- if not isinstance(rec, dict):
- continue
- ts = rec.get('timestamp')
- if not _in_run(ts):
- continue # record belongs to an earlier run in this workspace
- if isinstance(ts, str) and ts:
- if first is None:
- first = ts
- last = ts
- if rec.get('type') != 'stage_complete':
- continue
- d = rec.get('data') or {}
- if not isinstance(d, dict):
- continue
- name = d.get('stage')
- dur = d.get('duration_s')
- if not name or not isinstance(dur, (int, float)) or dur < 0:
- continue
- if name not in tot:
- tot[name] = 0.0
- order.append(name)
- tot[name] += float(dur)
- except Exception:
- sys.exit(0)
- if not tot:
- sys.exit(0)
- def human(s):
- s = int(round(s))
- return '%dm %02ds' % (s // 60, s % 60) if s >= 60 else '%ds' % s
- # Wall clock: prefer the run-start epoch the runner exported; else derive from
- # the first/last event timestamps. Absent both, we print stages with no total
- # rather than inventing a denominator.
- wall = None
- env_start = os.environ.get('LOKI_RUN_START_EPOCH') or ''
- try:
- if env_start.strip():
- import time
- wall = time.time() - float(env_start)
- except Exception:
- wall = None
- if wall is None and first and last:
- try:
- from datetime import datetime
- f = datetime.fromisoformat(first.replace('Z', '+00:00'))
- l = datetime.fromisoformat(last.replace('Z', '+00:00'))
- wall = (l - f).total_seconds()
- except Exception:
- wall = None
- staged = sum(tot.values())
- out = []
- for name in sorted(order, key=lambda n: -tot[n]):
- out.append(' %-22s %s' % (name.replace('_', ' '), human(tot[name])))
- if wall is not None and wall >= staged:
- rem = wall - staged
- # The bucket that answers 'where did the 25 minutes go'.
- # NOT labeled 'agent': the provider call is itself a bracketed stage above,
- # so this remainder is everything else (checklist verification, app runner,
- # playwright, council, memory). Calling it 'agent' would print two different
- # measurements under one name.
- out.append(' %-22s %s' % ('other (unaccounted)', human(rem)))
- out.append(' %-22s %s' % ('total', human(wall)))
- print('\n'.join(out))
- " "$_ev_file" 2>/dev/null)"
- fi
- # Branch + diff stats vs the run-start SHA (best-effort; non-git or empty
- # baseline yields empty values, which we render as "unknown"/"0").
- local start_sha="${_LOKI_RUN_START_SHA:-}"
- local branch="" head_sha="" diff_stat="" files_changed=0 insertions=0 deletions=0 review_cmd=""
- branch="$( (cd "${TARGET_DIR:-.}" && git rev-parse --abbrev-ref HEAD) 2>/dev/null || true )"
- [ -z "$branch" ] && branch="unknown"
- head_sha="$( (cd "${TARGET_DIR:-.}" && git rev-parse HEAD) 2>/dev/null || true )"
- # Finding #596 (HIGH): exclude .loki/ and .git/ from the summary diff/stat and
- # from the "Review the work" command we print, so the user is never told to
- # review a .loki-bloated diff and the displayed counts match the gated diff.
- local _summary_pathspec=(-- . ':(exclude).loki/' ':(exclude).git/' ':(exclude)**/.loki/**')
- if [ -n "$start_sha" ]; then
- diff_stat="$( (cd "${TARGET_DIR:-.}" && git diff --stat "${start_sha}..HEAD" "${_summary_pathspec[@]}") 2>/dev/null || true )"
- # Parse the git diff --shortstat tail for counts (locale-stable enough
- # for our display; failures leave the zeros in place).
- local shortstat
- shortstat="$( (cd "${TARGET_DIR:-.}" && git diff --shortstat "${start_sha}..HEAD" "${_summary_pathspec[@]}") 2>/dev/null || true )"
- if [ -n "$shortstat" ]; then
- files_changed="$(printf '%s\n' "$shortstat" | grep -oE '[0-9]+ file' | grep -oE '[0-9]+' | head -1)"
- insertions="$(printf '%s\n' "$shortstat" | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' | head -1)"
- deletions="$(printf '%s\n' "$shortstat" | grep -oE '[0-9]+ deletion' | grep -oE '[0-9]+' | head -1)"
- fi
- review_cmd="git diff ${start_sha}..HEAD -- . ':(exclude).loki/'"
- else
- # No baseline SHA. This is the GREENFIELD case: the run started in a repo
- # with no commits, so `git rev-parse --verify HEAD` correctly produced
- # nothing (see the start-sha capture). Everything the run built is new.
- #
- # Without this branch the counts stayed at zero and a user was told
- # "files_changed: 0" about a run that had produced real, committed,
- # working code -- observed on a PRD benchmark that built 4 files and
- # passed 28/28 of its own tests, yet reported 0. Diff against the empty
- # tree so the summary reports what was actually created.
- local _empty_tree
- _empty_tree="$( (cd "${TARGET_DIR:-.}" && git hash-object -t tree /dev/null) 2>/dev/null || true )"
- if [ -n "$_empty_tree" ] && (cd "${TARGET_DIR:-.}" && git rev-parse --verify HEAD >/dev/null 2>&1); then
- diff_stat="$( (cd "${TARGET_DIR:-.}" && git diff --stat "${_empty_tree}..HEAD" "${_summary_pathspec[@]}") 2>/dev/null || true )"
- local _shortstat_new
- _shortstat_new="$( (cd "${TARGET_DIR:-.}" && git diff --shortstat "${_empty_tree}..HEAD" "${_summary_pathspec[@]}") 2>/dev/null || true )"
- if [ -n "$_shortstat_new" ]; then
- files_changed="$(printf '%s\n' "$_shortstat_new" | grep -oE '[0-9]+ file' | grep -oE '[0-9]+' | head -1)"
- insertions="$(printf '%s\n' "$_shortstat_new" | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' | head -1)"
- deletions="$(printf '%s\n' "$_shortstat_new" | grep -oE '[0-9]+ deletion' | grep -oE '[0-9]+' | head -1)"
- fi
- review_cmd="git diff ${_empty_tree}..HEAD -- . ':(exclude).loki/'"
- else
- review_cmd="git diff HEAD -- . ':(exclude).loki/'"
- fi
- fi
- [ -z "$files_changed" ] && files_changed=0
- [ -z "$insertions" ] && insertions=0
- [ -z "$deletions" ] && deletions=0
- # Task counts: reuse the SAME queue reads as update_status_file.
- local pending=0 in_progress=0 completed=0 failed=0
- [ -f "$loki_dir/queue/pending.json" ] && pending=$(python3 -c "import json; print(len(json.load(open('$loki_dir/queue/pending.json'))))" 2>/dev/null || echo "0")
- [ -f "$loki_dir/queue/in-progress.json" ] && in_progress=$(python3 -c "import json; print(len(json.load(open('$loki_dir/queue/in-progress.json'))))" 2>/dev/null || echo "0")
- [ -f "$loki_dir/queue/completed.json" ] && completed=$(python3 -c "import json; print(len(json.load(open('$loki_dir/queue/completed.json'))))" 2>/dev/null || echo "0")
- [ -f "$loki_dir/queue/failed.json" ] && failed=$(python3 -c "import json; print(len(json.load(open('$loki_dir/queue/failed.json'))))" 2>/dev/null || echo "0")
- # Optional delegate-mode extras populated by Slice 3 (branch isolation / PR).
- local delegate_branch="${_LOKI_DELEGATE_BRANCH_NAME:-}"
- local pr_url="${_LOKI_DELEGATE_PR_URL:-}"
- local ts
- ts="$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date)"
- # v7.28.0: evidence-gate inconclusive line. When the evidence gate could not
- # establish a diff baseline (no git repo, or no run-start SHA), it records a
- # durable .loki/state/evidence-inconclusive.json instead of silently passing.
- # Surface one honest line so the user knows completion was not independently
- # verified. The record is removed by the gate on any conclusive run.
- local evidence_inconclusive_line=""
- local _inc_file="$loki_dir/state/evidence-inconclusive.json"
- if [ -f "$_inc_file" ]; then
- local _inc_reason
- _inc_reason="$(python3 -c "import json,sys
- try:
- d=json.load(open(sys.argv[1]))
- print(d.get('reason','') if d.get('inconclusive') else '')
- except Exception:
- print('')" "$_inc_file" 2>/dev/null)"
- if [ -n "$_inc_reason" ]; then
- evidence_inconclusive_line="Evidence gate: inconclusive (${_inc_reason}) - completion not independently verified"
- fi
- fi
- # P2-2: assumption-ledger summary for proof-of-done. "done" means "done, plus
- # here are the N places your spec was ambiguous and what Loki assumed."
- # spec_ledger_counts echoes "<total> <high>"; defined when spec-interrogation.sh
- # is sourced (run.sh sources it in DISCOVERY). Guarded for safety.
- local assumptions_total=0 assumptions_high=0 assumption_lines=""
- if type spec_ledger_counts &>/dev/null; then
- local _ac
- _ac="$(spec_ledger_counts 2>/dev/null || echo '0 0')"
- assumptions_total="${_ac%% *}"
- assumptions_high="${_ac##* }"
- case "$assumptions_total" in ''|*[!0-9]*) assumptions_total=0 ;; esac
- case "$assumptions_high" in ''|*[!0-9]*) assumptions_high=0 ;; esac
- if [ "$assumptions_total" != "0" ]; then
- local _ledger_md="$loki_dir/assumptions/ledger.md"
- if [ -f "$_ledger_md" ]; then
- # Pull just the "## <id> ..." headings as a terse one-per-line list.
- assumption_lines="$(grep '^## ' "$_ledger_md" 2>/dev/null | sed 's/^## / - /' || true)"
- fi
- fi
- fi
- # ---- Durable human-readable file: .loki/COMPLETION.txt --------------------
- # Presentation-only receipt: a single fixed-width label column, the live-app
- # URL elevated right under the outcome headline, and a consistent rule line.
- # Stays pure ASCII (no emoji, no dashes, no color codes) so it pastes cleanly
- # into a PR description or a chat. Same facts and values as before.
- {
- echo "Loki Mode run summary"
- echo "====================="
- echo ""
- printf '%-14s %s\n' "Outcome:" "$outcome_label"
- if [ -n "$live_app_url" ]; then
- # Compute the dashboard scheme the same way start_dashboard does
- # (url_scheme is local to that function, not visible here).
- local _dash_scheme="http"
- [ -n "${LOKI_TLS_CERT:-}" ] && [ -n "${LOKI_TLS_KEY:-}" ] && _dash_scheme="https"
- printf '%-14s %s\n' "Live app:" "$live_app_url (served locally on this machine)"
- printf '%-14s %s\n' "Dashboard:" "${_dash_scheme}://127.0.0.1:${DASHBOARD_PORT:-57374}/ (App Runner -> Live App)"
- fi
- printf '%-14s %s\n' "Branch:" "$branch"
- printf '%-14s %s\n' "Files:" "$files_changed (+$insertions / -$deletions)"
- # Only rendered when the app actually previewed. Silence beats a
- # fabricated zero for a run where nothing ever came up.
- if [ -n "$first_preview_s" ]; then
- printf '%-14s %ss\n' "First preview:" "$first_preview_s"
- fi
- # First code change, for the (far more common) run with no preview at
- # all -- a scoped issue fix produces no running app, so first_preview_s
- # is empty and the user had no signal of any kind until the end. Same
- # discipline: rendered only when actually recorded, never fabricated.
- if [ -n "$first_artifact_s" ]; then
- printf '%-14s %ss\n' "First change:" "$first_artifact_s"
- fi
- printf '%-14s %s\n' "Finished:" "$ts"
- if [ -n "$delegate_branch" ]; then
- printf '%-14s %s\n' "Delegate:" "$delegate_branch"
- fi
- if [ -n "$pr_url" ]; then
- printf '%-14s %s\n' "Pull request:" "$pr_url"
- elif [ "$outcome" = "complete" ]; then
- printf '%-14s %s\n' "Pull request:" "not opened (no GitHub remote, gh not authed, or on a default branch)"
- fi
- printf '%-14s %s\n' "Tasks:" "pending=$pending in_progress=$in_progress completed=$completed failed=$failed"
- echo ""
- if [ -n "$stage_timing" ]; then
- echo "Where the time went:"
- echo "$stage_timing"
- echo ""
- fi
- # Rework attribution. The AGENT has been steering on this since the eval
- # trend shipped, but the user never saw it -- and it is the number that
- # answers "why did this cost so much".
- #
- # iterations alone cannot distinguish real work from a gate false
- # positive that forced redos. That is not hypothetical: a measured run
- # had the agent claim done on EVERY iteration while a mock-integrity
- # false positive blocked all six, so the run looked like six iterations
- # of work and was one iteration of work plus five of harness bug.
- #
- # Renders NOTHING when the split is unknown, rather than a fabricated
- # zero -- same rule as the timing table above. A wrong attribution is
- # worse than silence because it points the user at the wrong culprit.
- if [ -n "${_LOKI_REWORK_LINE:-}" ]; then
- echo "$_LOKI_REWORK_LINE"
- echo ""
- fi
- # What to DO about it. A terminal that names a bound without naming the
- # lever leaves the user to guess which knob applies, and the wrong guess
- # is expensive: raising an iteration cap on a run that was thrashing
- # buys more thrashing at full price.
- #
- # The rework split above already distinguishes the two cases, so this
- # points at the lever that actually matches the outcome rather than
- # listing every environment variable.
- case "$outcome" in
- max_iterations)
- echo "Next: this run hit its iteration ceiling, not a verdict."
- echo " Converging (low rework above)? Raise it: LOKI_MAX_ITERATIONS=<n>"
- echo " Thrashing (high rework above)? Raising the cap buys more"
- echo " of the same. Read the last reviewer findings first:"
- echo " loki why"
- echo ""
- ;;
- budget_exceeded)
- echo "Next: the spend cap stopped this run, not a failed gate."
- echo " Raise it with LOKI_BUDGET_LIMIT=<usd>, or check the cache"
- echo " hit ratio first -- a cold cache is the usual cause of a"
- echo " surprising bill:"
- echo " loki memory economics"
- echo ""
- ;;
- force_stopped)
- echo "Next: the council stopped this run WITHOUT approving it."
- echo " The work is not verified complete. See what blocked it:"
- echo " loki why"
- echo ""
- ;;
- esac
- if [ -n "$evidence_inconclusive_line" ]; then
- echo "$evidence_inconclusive_line"
- echo ""
- fi
- if [ "$assumptions_total" != "0" ]; then
- echo "Spec assumptions recorded: $assumptions_total ($assumptions_high high-severity)"
- echo " These are places your spec was ambiguous and what Loki assumed. See .loki/assumptions/ledger.md"
- if [ -n "$assumption_lines" ]; then
- echo "$assumption_lines"
- fi
- echo ""
- fi
- echo "Review the work:"
- echo " $review_cmd"
- echo ""
- if [ -n "$diff_stat" ]; then
- echo "Diff stat:"
- echo "$diff_stat"
- else
- echo "Diff stat: (no changes detected vs run start, or git unavailable)"
- fi
- # Empty-diff remedy (mirrors emit_completion_summary's on-screen line) so
- # a --bg or dashboard user reading COMPLETION.txt gets the same honest,
- # actionable guidance as a foreground user. Never implies success.
- case "$files_changed" in
- ''|0)
- case "$outcome" in
- complete|max_iterations)
- echo ""
- echo "No file changes were produced this run. Likely the spec was too vague"
- echo "or already satisfied. Next: re-run with a more concrete spec, e.g."
- echo " loki start \"<one concrete, testable change you want>\""
- echo "or inspect what happened: loki why"
- ;;
- esac
- ;;
- esac
- # Per-outcome next step. Deliberately NOT nested inside the
- # files_changed==0 case above: a run that stops for a gate reason has
- # usually built plenty, and that is exactly the run whose user most needs
- # to be told what to do. Keying this on an empty diff would silence it
- # for the case that motivated it.
- #
- # Measured on the PRD benchmark: outcome=intervention with 9 files built
- # and 28/28 tests passing produced NO guidance at all, because the old
- # block covered only complete|max_iterations.
- case "$outcome" in
- intervention)
- echo ""
- echo "Loki stopped and needs a decision from you."
- _cs_gate="$(_loki_summary_gate_reason "$loki_dir")"
- if [ -n "$_cs_gate" ]; then
- echo " Blocked by: $_cs_gate"
- echo " The findings say exactly what to change. Fix those, then resume:"
- else
- echo " See .loki/PAUSED.md for the reason and .loki/CONTINUITY.md for context."
- echo " Resume with:"
- fi
- echo " rm .loki/PAUSE # resume this run"
- echo " touch .loki/STOP # end it instead"
- echo " Resuming without addressing the finding will stop at the same gate again."
- ;;
- failed)
- echo ""
- echo "The run failed. What went wrong:"
- echo " loki why # the classified cause, in one line"
- echo " .loki/logs/ # the full session log"
- echo "Anything already committed is listed above and is yours to keep or discard."
- ;;
- stopped|force_stopped)
- echo ""
- echo "The run was stopped before it finished."
- echo "Work committed up to that point is listed above. To continue from here:"
- echo " loki start <your spec> # a fresh run over the current tree"
- ;;
- inconclusive_spec_contradiction)
- echo ""
- echo "Loki could not reconcile parts of the spec, so it stopped rather than"
- echo "guess. The specific contradictions are in .loki/assumptions/ledger.md."
- echo "Resolve those in the spec, then re-run."
- ;;
- esac
- # Evidence Receipt (#209). The receipt is written automatically on every
- # run, success or failure, but its location only ever reached the user
- # through the zero-config first-run block. State it here, where every
- # terminal outcome passes, so a user who has never heard of `loki proof`
- # still learns a checkable receipt exists and how to re-check it.
- # Prints NOTHING when no receipt has been written yet (mid-pause, or the
- # first summary on the success path): a path that does not exist is a
- # promise, not a fact. The verdict line is the generator's deterministic
- # headline, omitted entirely when unreadable rather than invented.
- _cs_receipt="$(_loki_receipt_facts "$loki_dir" 2>/dev/null || true)"
- if [ -n "$_cs_receipt" ]; then
- _cs_rid="${_cs_receipt%% *}"
- _cs_rest="${_cs_receipt#* }"
- _cs_dir="${_cs_rest%% *}"
- _cs_headline="${_cs_rest#* }"
- echo ""
- echo "Evidence Receipt (this run):"
- if [ -n "$_cs_headline" ]; then
- echo " Verdict: $_cs_headline"
- fi
- # Gated on the PAGE, not the receipt. The helper gates on
- # proof.json, but the generator writes proof.json and THEN renders
- # index.html unwrapped (proof-generator.py), so a render failure
- # leaves the data present and the page absent. Naming a page that
- # is not there is the same class proof.ts:118 refuses ("worse than
- # an absent one") and `loki proof open` refuses with "Proof page
- # not found". Gate only this line: proof verify reads proof.json,
- # so the id and the re-check below still work without the page.
- if [ -f "$_cs_dir/index.html" ]; then
- echo " Receipt: $_cs_dir/index.html"
- fi
- echo " Re-check it yourself, do not take our word for it:"
- # Cwd-independent: the receipt is resolved from the run's target
- # dir, which need not be where the user is standing. A bare
- # `loki proof verify <id>` then fails for exactly the user we just
- # told to check our work.
- if [ "$(cd "$loki_dir/.." 2>/dev/null && pwd -P)" = "$(pwd -P)" ]; then
- echo " loki proof verify $_cs_rid"
- else
- echo " (cd $(cd "$loki_dir/.." 2>/dev/null && pwd -P) && loki proof verify $_cs_rid)"
- fi
- fi
- } > "$loki_dir/COMPLETION.txt" 2>/dev/null || true
- # ---- Durable machine-readable file: .loki/state/completion.json -----------
- _LOKI_CS_OUTCOME="$outcome" \
- _LOKI_CS_BRANCH="$branch" \
- _LOKI_CS_START_SHA="$start_sha" \
- _LOKI_CS_HEAD_SHA="$head_sha" \
- _LOKI_CS_FILES="$files_changed" \
- _LOKI_CS_INS="$insertions" \
- _LOKI_CS_DEL="$deletions" \
- _LOKI_CS_REVIEW="$review_cmd" \
- _LOKI_CS_DELEGATE_BRANCH="$delegate_branch" \
- _LOKI_CS_PR_URL="$pr_url" \
- _LOKI_CS_TS="$ts" \
- _LOKI_CS_ASSUMPTIONS_TOTAL="$assumptions_total" \
- _LOKI_CS_ASSUMPTIONS_HIGH="$assumptions_high" \
- _LOKI_CS_OUT_FILE="$loki_dir/state/completion.json" \
- _LOKI_CS_LAST_ERROR="$loki_dir/state/LAST_ERROR.json" \
- python3 -E -c "import sys; sys.path[:] = [p for p in sys.path if p not in ('', '.')]
- import json, os, tempfile
- out = os.environ['_LOKI_CS_OUT_FILE']
- def i(v):
- try: return int(v)
- except (TypeError, ValueError): return 0
- # Carry the CLASSIFIED failure reason (error_class + brief) into the machine-
- # readable completion record, from the side-record loki wrote before this summary
- # (.loki/state/LAST_ERROR.json). Without it a terminal outcome like
- # 'inconclusive_spec_contradiction' is only a label with no cause a downstream
- # (operator, next run, bench) can act on. None on a clean run.
- _err_class = None
- _err_brief = None
- try:
- with open(os.environ.get('_LOKI_CS_LAST_ERROR', '')) as _ef:
- _er = json.load(_ef)
- if isinstance(_er, dict):
- _err_class = _er.get('error_class')
- _err_brief = _er.get('brief')
- except Exception:
- pass
- rec = {
- 'outcome': os.environ.get('_LOKI_CS_OUTCOME', ''),
- 'error_class': _err_class,
- 'error_brief': _err_brief,
- 'branch': os.environ.get('_LOKI_CS_BRANCH', ''),
- 'start_sha': os.environ.get('_LOKI_CS_START_SHA', ''),
- 'head_sha': os.environ.get('_LOKI_CS_HEAD_SHA', ''),
- 'files_changed': i(os.environ.get('_LOKI_CS_FILES')),
- 'insertions': i(os.environ.get('_LOKI_CS_INS')),
- 'deletions': i(os.environ.get('_LOKI_CS_DEL')),
- 'review_cmd': os.environ.get('_LOKI_CS_REVIEW', ''),
- 'delegate_branch': os.environ.get('_LOKI_CS_DELEGATE_BRANCH', ''),
- 'pr_url': os.environ.get('_LOKI_CS_PR_URL', ''),
- 'timestamp': os.environ.get('_LOKI_CS_TS', ''),
- 'assumptions_total': i(os.environ.get('_LOKI_CS_ASSUMPTIONS_TOTAL')),
- 'assumptions_high': i(os.environ.get('_LOKI_CS_ASSUMPTIONS_HIGH')),
- }
- d = os.path.dirname(out)
- fd, tmp = tempfile.mkstemp(dir=d, suffix='.json')
- with os.fdopen(fd, 'w') as f:
- json.dump(rec, f, indent=2)
- os.replace(tmp, out)
- " 2>/dev/null || true
- # ---- P3-5: run manifest / bill-of-materials: .loki/loki-run.json ----------
- # Best-effort, auditable + reproducible record of this run. Emitted from
- # build_completion_summary because that fires on EVERY terminal path
- # (complete / max_iterations / stopped / intervention / failed), including the
- # intervention/stopped calls that bypass emit_completion_summary. The whole
- # block is wrapped so a failure can NEVER abort the run.
- {
- # Portable sha256 (macOS shasum, Linux sha256sum). Echoes empty on miss.
- _loki_sha256() {
- local _f="$1"
- [ -f "$_f" ] || { printf ''; return 0; }
- if command -v shasum >/dev/null 2>&1; then
- shasum -a 256 "$_f" 2>/dev/null | awk '{print $1}'
- elif command -v sha256sum >/dev/null 2>&1; then
- sha256sum "$_f" 2>/dev/null | awk '{print $1}'
- else
- printf ''
- fi
- }
- local _loki_ver
- _loki_ver="$(cat "$PROJECT_DIR/VERSION" 2>/dev/null || echo "unknown")"
- local _spec_path="${PRD_PATH:-}"
- [ -z "$_spec_path" ] && _spec_path="none"
- local _spec_hash=""
- [ "$_spec_path" != "none" ] && _spec_hash="$(_loki_sha256 "$_spec_path")"
- # Evidence files we reference + hash (existing artifacts, not new ones).
- local _ev_tests="$loki_dir/quality/test-results.json"
- local _ev_cov="$loki_dir/quality/coverage.json"
- local _ev_completion="$loki_dir/state/completion.json"
- _LOKI_RM_OUT="$loki_dir/loki-run.json" \
- _LOKI_RM_VERSION="$_loki_ver" \
- _LOKI_RM_SPEC_PATH="$_spec_path" \
- _LOKI_RM_SPEC_HASH="$_spec_hash" \
- _LOKI_RM_PROVIDER="${PROVIDER_NAME:-${LOKI_PROVIDER:-claude}}" \
- _LOKI_RM_TIER="${CURRENT_TIER:-unknown}" \
- _LOKI_RM_OUTCOME="$outcome" \
- _LOKI_RM_BRANCH="$branch" \
- _LOKI_RM_START_SHA="$start_sha" \
- _LOKI_RM_HEAD_SHA="$head_sha" \
- _LOKI_RM_ITERS="${ITERATION_COUNT:-0}" \
- _LOKI_RM_TS="$ts" \
- _LOKI_RM_EV_TESTS="$_ev_tests" \
- _LOKI_RM_EV_TESTS_HASH="$(_loki_sha256 "$_ev_tests")" \
- _LOKI_RM_EV_COV="$_ev_cov" \
- _LOKI_RM_EV_COV_HASH="$(_loki_sha256 "$_ev_cov")" \
- _LOKI_RM_EV_COMPLETION="$_ev_completion" \
- _LOKI_RM_NODE="$(node --version 2>/dev/null || echo '')" \
- _LOKI_RM_PYTHON="$(python3 --version 2>&1 | awk '{print $2}' || echo '')" \
- _LOKI_RM_GIT="$(git --version 2>/dev/null | awk '{print $3}' || echo '')" \
- _LOKI_RM_BUN="$(bun --version 2>/dev/null || echo '')" \
- python3 -c "
- import json, os, tempfile
- out=os.environ['_LOKI_RM_OUT']
- def s(k): return os.environ.get(k,'')
- def i(k):
- try: return int(os.environ.get(k,'0'))
- except (TypeError, ValueError): return 0
- def ev(path_k, hash_k=None):
- p=s(path_k)
- rec={'path': p, 'exists': bool(p) and os.path.isfile(p)}
- if hash_k:
- h=s(hash_k)
- if h: rec['sha256']=h
- return rec
- manifest={
- 'schema': 'loki-run-manifest/v1',
- 'loki_version': s('_LOKI_RM_VERSION'),
- 'timestamp': s('_LOKI_RM_TS'),
- 'outcome': s('_LOKI_RM_OUTCOME'),
- 'iterations': i('_LOKI_RM_ITERS'),
- 'provider': s('_LOKI_RM_PROVIDER'),
- # Tier cycles per RARV iteration (R/A/R/V); this is the LAST tier set, not
- # the only tier used. Recorded honestly as last_tier, not a single 'model'.
- 'last_tier': s('_LOKI_RM_TIER'),
- 'spec': {
- 'path': s('_LOKI_RM_SPEC_PATH'),
- 'sha256': s('_LOKI_RM_SPEC_HASH') or None,
- },
- 'git': {
- 'branch': s('_LOKI_RM_BRANCH'),
- 'start_sha': s('_LOKI_RM_START_SHA') or None,
- 'head_sha': s('_LOKI_RM_HEAD_SHA') or None,
- },
- 'tool_versions': {
- 'node': s('_LOKI_RM_NODE') or None,
- 'python': s('_LOKI_RM_PYTHON') or None,
- 'git': s('_LOKI_RM_GIT') or None,
- 'bun': s('_LOKI_RM_BUN') or None,
- },
- 'evidence': {
- 'test_results': ev('_LOKI_RM_EV_TESTS','_LOKI_RM_EV_TESTS_HASH'),
- 'coverage': ev('_LOKI_RM_EV_COV','_LOKI_RM_EV_COV_HASH'),
- 'completion': ev('_LOKI_RM_EV_COMPLETION'),
- },
- }
- d=os.path.dirname(out)
- fd, tmp=tempfile.mkstemp(dir=d, suffix='.json')
- with os.fdopen(fd,'w') as f:
- json.dump(manifest, f, indent=2)
- os.replace(tmp, out)
- " 2>/dev/null || true
- unset -f _loki_sha256 2>/dev/null || true
- } || true
- # ---- Short strings for the desktop notification --------------------------
- # Desktop body stays terse; full detail lives in COMPLETION.txt.
- _LOKI_SUMMARY_TITLE="$notify_title"
- _LOKI_SUMMARY_BODY="${outcome_label} on ${branch}: ${files_changed} files changed"
- if [ -n "$pr_url" ]; then
- _LOKI_SUMMARY_BODY="${_LOKI_SUMMARY_BODY}. PR: ${pr_url}"
- fi
- export _LOKI_SUMMARY_TITLE _LOKI_SUMMARY_BODY
- return 0
- }
- #===============================================================================
- # emit_completion_summary <outcome> [urgency]
- #
- # The single entry point every terminal state calls. It ALWAYS writes the
- # durable summary files (state, not a notification) and then fires ONE desktop
- # notification gated by the existing LOKI_NOTIFICATIONS flag (send_notification
- # already short-circuits when disabled, so the gate is implicit but explicit
- # here for clarity). Centralizing this keeps the success-only PR side effect
- # (Slice 3) in one place and prevents duplicate notifications.
- #===============================================================================
- emit_completion_summary() {
- local outcome="${1:-complete}"
- local urgency="${2:-normal}"
- build_completion_summary "$outcome"
- # Render the screenshot-worthy completion card inline on a foreground TTY run,
- # AFTER the durable files are written. The card re-reads the persisted
- # completion.json (never recomputes) so it can never diverge from the file.
- print_completion_card
- send_notification "${_LOKI_SUMMARY_TITLE:-Run finished}" "${_LOKI_SUMMARY_BODY:-}" "$urgency"
- return 0
- }
- #===============================================================================
- # print_completion_card (visible-delight completion card)
- #
- # Display-only. Renders a boxed summary card to the interactive TTY at the close
- # of a foreground run: outcome, branch, files changed (+ins/-del), the live-app
- # URL with a "try it" line when an app is running, the copy-pasteable review
- # command, and the recorded-assumptions count. This is the single most
- # screenshot-worthy moment of a run, which was previously only written to a file.
- #
- # It reads ONLY from the already-persisted .loki/state/completion.json (written
- # by build_completion_summary just before this is called) and the app-runner
- # state, so the card and the durable file are guaranteed identical. Nothing is
- # computed or written here.
- #
- # Gate (same shape as the HUD at the colors block): interactive stdout, not
- # --bg, and not opted out via LOKI_COMPLETION_CARD=0. Off-TTY / --bg / --json
- # paths emit nothing, so machine output stays byte-identical. Wrapped so any
- # internal failure still returns 0 and can never abort the completion path.
- #===============================================================================
- print_completion_card() {
- # Gate first: emit nothing unless interactive TTY, not background, not opted out.
- if ! { [ -t 1 ] && [ "${BACKGROUND_MODE:-false}" != "true" ] && [ "${LOKI_COMPLETION_CARD:-1}" != "0" ]; }; then
- return 0
- fi
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local _cj="$loki_dir/state/completion.json"
- [ -f "$_cj" ] || return 0
- # Pull the fields we render straight from the persisted record. The python3
- # call emits one field per line in a fixed order; we read them line by line
- # so empty fields (e.g. no pr_url) keep their positions (a single delimiter
- # split would collapse adjacent empties). Any failure leaves the card
- # unrendered. Trailing-newline guard: NUL-free, fields are single-line.
- local _fields
- _fields="$(python3 -E -c "import sys; sys.path[:] = [p for p in sys.path if p not in ('', '.')]
- import json,sys
- try:
- d=json.load(open(sys.argv[1]))
- except Exception:
- sys.exit(0)
- def g(k):
- v=d.get(k,'')
- return '' if v is None else str(v).replace('\n',' ')
- for k in ['outcome','branch','files_changed','insertions','deletions',
- 'review_cmd','pr_url','assumptions_total','assumptions_high']:
- print(g(k))
- " "$_cj" 2>/dev/null)" || return 0
- [ -z "$_fields" ] && return 0
- local _outcome _branch _files _ins _del _review _pr _atotal _ahigh
- {
- IFS= read -r _outcome
- IFS= read -r _branch
- IFS= read -r _files
- IFS= read -r _ins
- IFS= read -r _del
- IFS= read -r _review
- IFS= read -r _pr
- IFS= read -r _atotal
- IFS= read -r _ahigh
- } <<EOF
- $_fields
- EOF
- # Human outcome label (mirror build_completion_summary's mapping).
- local _label
- case "$_outcome" in
- complete) _label="Completed" ;;
- max_iterations) _label="Max iterations" ;;
- stopped) _label="Stopped" ;;
- force_stopped) _label="Stopped (not verified-complete)" ;;
- failed) _label="Failed" ;;
- intervention) _label="Needs input" ;;
- # Mirror of build_completion_summary's gate-stuck arms. Both must move
- # together or the card and COMPLETION.txt disagree on the same run.
- # Literal, not a glob, for the guard reason recorded there.
- gate_stuck_static_analysis) _label="Stopped (static analysis gate would not clear)" ;;
- gate_stuck_mock_integrity) _label="Stopped (mock integrity gate would not clear)" ;;
- gate_stuck_mutation_integrity) _label="Stopped (mutation integrity gate would not clear)" ;;
- *) _label="$_outcome" ;;
- esac
- # Live app URL (best-effort), same read as build_completion_summary.
- local _url="" _app_state="$loki_dir/app-runner/state.json"
- if [ -f "$_app_state" ]; then
- _url="$(python3 -c "import json,sys
- try:
- d=json.load(open(sys.argv[1]))
- print(d.get('url','') if d.get('status')=='running' else '')
- except Exception:
- print('')" "$_app_state" 2>/dev/null)" || _url=""
- fi
- # Box width matches log_header (66 inner columns). Render lines that fit;
- # this is decoration, so over-wide content is simply not boxed-truncated
- # (the durable file carries the full text).
- echo ""
- echo -e "${GREEN}+================================================================+${NC}"
- echo -e "${GREEN}|${NC} ${BOLD}Loki Mode: ${_label}${NC}"
- echo -e "${GREEN}|${NC}"
- if [ -n "$_url" ]; then
- echo -e "${GREEN}|${NC} ${BOLD}${CYAN}Your app is live at ${_url}${NC} ${DIM}- open it to try it${NC}"
- echo -e "${GREEN}|${NC}"
- fi
- echo -e "${GREEN}|${NC} Branch: ${BOLD}${_branch}${NC}"
- echo -e "${GREEN}|${NC} Files: ${BOLD}${_files}${NC} changed ${GREEN}+${_ins}${NC} / ${RED}-${_del}${NC}"
- # Empty-diff guard (honest, not fake-green): a run that ended without
- # producing ANY file change is the classic confusing new-user outcome
- # ("it said done, but nothing happened"). Never imply success on a 0-file
- # run -- surface it plainly with one actionable next step. Only for the
- # non-failure outcomes (a "failed"/"stopped" box already explains itself).
- case "$_files" in
- ''|0)
- case "$_outcome" in
- complete|max_iterations)
- echo -e "${GREEN}|${NC}"
- echo -e "${GREEN}|${NC} ${YELLOW}No file changes were produced this run.${NC} Likely the spec was too vague"
- echo -e "${GREEN}|${NC} or already satisfied. Next: re-run with a more concrete spec, e.g."
- echo -e "${GREEN}|${NC} ${BOLD}loki start \"<one concrete, testable change you want>\"${NC}"
- echo -e "${GREEN}|${NC} or inspect what happened: ${BOLD}loki why${NC}"
- ;;
- esac
- ;;
- esac
- case "$_atotal" in ''|0) : ;; *)
- echo -e "${GREEN}|${NC} Spec assumptions recorded: ${BOLD}${_atotal}${NC} (${_ahigh} high) ${DIM}see .loki/assumptions/ledger.md${NC}"
- ;;
- esac
- if [ -n "$_pr" ]; then
- echo -e "${GREEN}|${NC} Pull request: ${_pr}"
- fi
- echo -e "${GREEN}|${NC}"
- echo -e "${GREEN}|${NC} ${DIM}Review the work:${NC}"
- echo -e "${GREEN}|${NC} ${_review}"
- # NO Evidence Receipt line here, deliberately. The card renders from inside
- # emit_completion_summary, which is reached only from within run_autonomous
- # -- thousands of lines BEFORE the teardown that generates this run's
- # receipt. Announcing here would therefore be silent on a normal success
- # run (no proof exists yet), and on a SECOND run in the same directory it
- # would have been actively wrong before #211: .loki/state/last-proof-id.txt
- # was never cleared at run start, so the card would read the PREVIOUS run's
- # pointer and print that receipt, with that run's verdict, as though it
- # described this run. The pointer is now cleared during run init (search
- # "Same reasoning for the proof pointer"), which closes the cross-run leak,
- # but this site stays silent anyway: it renders thousands of lines before
- # THIS run's receipt exists, so it would print nothing on a normal success
- # run. The announcement lives at the teardown instead, after the final
- # generate_proof_of_run, where the pointer is guaranteed current.
- echo -e "${GREEN}+================================================================+${NC}"
- echo ""
- return 0
- }
- #===============================================================================
- # Rule of Two (moat P9): Loki never passes the GitHub token into the agent's environment via the
- # IMPLICIT resolution paths this module withholds (env vars, gh config store,
- # git credential.helper, SSH agent/ssh command). Not an absolute claim -- see
- # the disclosed residuals at the end of this header (an explicit named-account
- # keyring read, a direct ssh/hosts.yml/keychain read outside git); the actual
- # security boundary is a CI job holding no write token and no SSH agent while
- # the agent runs.
- #
- # The run reads untrusted issue text and spawns agents that act on it, so a
- # token in the exported environment is one prompt injection away from a push.
- # main() un-exports every GitHub token it inherited (the shell keeps the value).
- # Every provider, reviewer, helper and test the agent writes then starts without
- # it. Loki's own trusted GitHub calls get it back for the one command:
- # - direct `gh ...` calls, through the gh() wrapper defined below;
- # - _loki_with_github_tokens <cmd...>, for the post-session push/PR paths.
- # LOKI_ALLOW_AGENT_GITHUB_TOKEN=1 (exact value) restores the old inheritance
- # and prints one stderr warning that the agent holds the token.
- #
- # BACKLOG 149 (round 2, REJECT rework): the previous fix scoped GH_CONFIG_DIR
- # to a fresh empty directory but a reviewer confirmed, live, that this does not
- # close the vulnerability. Two bypasses, both reproduced empirically against
- # real gh 2.92 + a real macOS Keychain-backed `gh auth login` on 2026-09-27:
- #
- # 1. `GH_CONFIG_DIR=<empty dir> gh auth token` (real $HOME, no env token)
- # still exits 0: gh's resolution is config-dir file, THEN OS keyring
- # fallback (Keychain on macOS, libsecret on Linux). An empty config dir
- # does not stop the keyring lookup.
- # 2. git's own `credential.helper` (`gh auth setup-git` wires
- # `credential.https://github.com.helper = !gh auth git-credential`, and a
- # plain `osxkeychain`/`libsecret`/etc. helper may ALSO be configured,
- # unscoped, ahead of or behind it) is invoked directly by git on any
- # `git push`/`git credential fill` over HTTPS. This path is git-invoked,
- # not GH_CONFIG_DIR-mediated at all -- scoping GH_CONFIG_DIR does nothing
- # to it, and it resolves through the same keyring fallback OR a
- # completely separate OS-keychain entry that git's own helper (not gh's)
- # maintains independently.
- #
- # Fix, verified against `gh help environment` and empirically against both
- # bypasses (see the S-18 rework session transcript for the repro/counter-repro
- # pairs; no real credential value was ever printed during that verification):
- #
- # (a) GH_TOKEN/GITHUB_TOKEN/GH_ENTERPRISE_TOKEN/GITHUB_ENTERPRISE_TOKEN are
- # no longer merely unset -- each is set to a fresh per-process garbage
- # value (SENTINEL below). `gh help environment` documents the env token
- # as taking "precedence over previously stored credentials", and this
- # was confirmed live: with the sentinel present, `gh auth token` AND
- # `gh auth git-credential get` both echo back the garbage value, never
- # falling through to the keyring. `gh auth git-credential get` is a pure
- # read/print in this path -- it does not write to GH_CONFIG_DIR or touch
- # the keychain, so planting a sentinel has no destructive side effect;
- # a `git push` that ends up using it simply gets a 401 from GitHub, the
- # same as any other wrong password.
- # (b) The sentinel alone does not close a plain `osxkeychain` (or similar)
- # helper that is configured UNSCOPED (`credential.helper = osxkeychain`,
- # no gh involved at all) and holds its own independently-cached
- # credential -- confirmed present as a THIRD, gh-independent store on
- # the verification machine (git's own keychain entry, distinct from
- # gh's). GH_TOKEN only feeds gh's own resolution; it does not touch
- # git's credential-helper chain. So the session additionally gets
- # `credential.helper=` (empty string) injected via
- # GIT_CONFIG_COUNT/GIT_CONFIG_KEY_n/GIT_CONFIG_VALUE_n, appended after
- # any GIT_CONFIG_COUNT the operator already has set. Per gitcredentials(7):
- # "If credential.helper is configured to the empty string, this resets
- # the helper list to empty" -- this is why the reset must be UNSCOPED
- # (plain `credential.helper`, not `credential.<url>.helper`, since a
- # URL-scoped reset does not necessarily out-order every already
- # configured helper) and appended LAST (env-var config values override
- # every config FILE, per `git help config` on GIT_CONFIG_COUNT, and later
- # GIT_CONFIG_KEY_n entries are read after earlier ones). Confirmed live:
- # with this override present, `git credential fill` against github.com
- # fails closed ("could not read Username") instead of returning a
- # password line, with no fallback to any other helper. This closes the
- # git-invoked path directly, independent of GH_CONFIG_DIR or any
- # GH_TOKEN-family value.
- #
- # GH_CONFIG_DIR is still scoped to a fresh empty directory (unconditionally,
- # same as before) -- it remains correct for the plaintext-hosts.yml-read case
- # and for any gh subcommand that does not consult GH_TOKEN. $HOME itself stays
- # untouched, so Claude's own OAuth (F49) is unaffected -- nothing here reads
- # or writes $HOME/.claude or touches any Anthropic credential path.
- #
- # BACKLOG 149 (round 3, REJECT rework): round 2's fix is HTTPS-only. A
- # round-3 reviewer confirmed, live, that none of the above touches
- # SSH_AUTH_SOCK or GIT_SSH_COMMAND -- the provider session's real, untouched
- # environment always has a reachable SSH agent (by this design's own stated
- # intent of keeping the environment live for Claude's own OAuth), so
- # `git remote set-url origin git@github.com:<owner>/<repo>.git && git push`
- # authenticates exactly as an unrestricted session. Reproduced with a
- # synthetic-only ssh-agent and a throwaway key (never a real credential):
- # `ssh-add -l` against the inherited SSH_AUTH_SOCK lists the identity, and any
- # ssh-transport git operation would authenticate with it.
- #
- # Fix: SSH_AUTH_SOCK is unset (not sentineled) for the withheld session.
- # Unlike the GH_TOKEN-family vars, a present-but-wrong SSH_AUTH_SOCK does NOT
- # stop ssh from falling back: with no usable agent, ssh(1) still tries the
- # default key files (~/.ssh/id_*) and any `IdentityAgent` socket named in
- # ~/.ssh/config (and on macOS a launchd agent socket is rediscoverable). A
- # sentinel therefore buys nothing over unset, so unset is the simpler choice
- # for THIS var. Those fallbacks are why GIT_SSH_COMMAND=false (below) is the
- # part that actually closes git's ssh transport: git never runs ssh at all.
- # An agent that explicitly removes it (`env -u GIT_SSH_COMMAND git push
- # git@...`) gets ssh back with those fallbacks -- same class as `env -u
- # GH_TOKEN`, disclosed in the residual list at the end of this header. That
- # does not contradict round 2's reasoning for sentineling
- # the token vars, it applies that same reasoning to a var with different
- # fallback semantics. GIT_SSH_COMMAND is set to `false` (a POSIX builtin that
- # ignores all arguments and always exits 1): git invokes it in place of ssh
- # for any ssh-transport fetch/push, and git's own docs (`git help git-config`)
- # confirm the env var "is overridden when the environment variable is set" --
- # i.e. GIT_SSH_COMMAND (env) unconditionally outranks core.sshCommand
- # (config), so setting only the env var closes both an operator-configured
- # core.sshCommand AND an agent-configured one, with no separate
- # GIT_CONFIG_COUNT-based core.sshCommand reset needed. This means even an
- # explicit `ssh -i <path>` invocation made THROUGH git (git fetch/push/clone
- # over an ssh:// or git@ URL) fails; it does NOT stop an agent from invoking
- # the real `/usr/bin/ssh` binary directly outside of git, which is a
- # disclosed residual (see below), the same class as the already-disclosed
- # `gh auth token -u <username>` and on-disk hosts.yml/keychain reads.
- #
- # Loki's own trusted calls (gh() below, _loki_with_github_tokens) restore the
- # REAL env values (or absence) for the 4 token vars, the real GH_CONFIG_DIR
- # (or absence), the real GIT_CONFIG_COUNT/KEY/VALUE state (or absence), and
- # the real SSH_AUTH_SOCK/GIT_SSH_COMMAND state (or absence) around the one
- # trusted command, then re-apply the withhold afterward -- this repo's own
- # trusted git operations (create_session_pr, on_run_complete's push,
- # proof-check.sh) push to an https or ssh GitHub origin, so an SSH-origin
- # user's Loki-initiated push must still work; the SSH sentinel needs the same
- # re-grant path as the token vars, not a simpler one.
- #
- # Restoring credentials is only safe if the credentialed process does not
- # also load code the agent controls (round 5). Until round 5 it did: the
- # trusted push was a `git push` inside the agent's working tree, so the
- # agent's repo config (.git/hooks, core.hooksPath, core.sshCommand, a
- # repo-local credential.helper, core.fsmonitor, url.*insteadOf, include/
- # includeIf) ran holding every restored credential. Now the push goes through
- # _loki_trusted_push (fresh Loki-owned repo, origin URL read as data and
- # accepted only as a github.com https/ssh URL) and every trusted gh call runs
- # from / with GH_REPO (_loki_run_neutral), so no credentialed git or gh process
- # loads the agent's repo config. What the trusted step still trusts:
- # - the operator's GLOBAL and SYSTEM git config (they carry the real
- # credential helper). An agent that writes ~/.gitconfig, /etc/gitconfig or
- # an included file there gets the same code execution back; this is the
- # same-user filesystem class as editing ~/.ssh or a shell rc file.
- # - the branch contents, which are the agent's work by design (Loki pushes
- # them for review; nothing in them executes during the push).
- # - the origin OWNER/REPO the agent's config names: the agent can point the
- # push at another github.com repository. GitHub still enforces the
- # operator token's permissions there.
- # A non-GitHub origin is no longer pushed at all (LOKI_AUTO_PR=1 used to push
- # to any origin).
- #
- # This is hygiene against a naive injection, not an isolation boundary: code
- # running as the same user can still read the parent's environment block
- # (/proc/<pid>/environ on Linux, sudo on a hosted runner), the hosts.yml file
- # directly off disk (F49 keeps $HOME live), or invoke `git -c
- # credential.helper=...` / `security find-generic-password` /
- # `env -u GH_TOKEN -u GITHUB_TOKEN gh auth token` / `gh auth token -u
- # <username>` / `env -u GIT_SSH_COMMAND git push git@...` (ssh then falls back
- # to ~/.ssh/id_* default keys and any ~/.ssh/config IdentityAgent) / the real
- # `ssh` binary directly with an explicit `-i` key path or a re-discovered agent
- # socket (e.g. `launchctl getenv SSH_AUTH_SOCK` on macOS) / git's `ext::`
- # transport (`git -c protocol.ext.allow=always` with an `ext::<command>` URL,
- # which runs an arbitrary command as the transport) / GIT_ASKPASS (VS Code's
- # integrated terminal sets it to a helper that can answer git's credential
- # prompt over IPC; it is left untouched here) explicitly to route around
- # this. The boundary is a CI job that holds
- # no write token and no SSH agent while the agent runs (see
- # docs/examples/loki-issue-to-pr.yml.example).
- #===============================================================================
- _LOKI_WITHHELD_TOKENS=""
- _LOKI_GIT_CONFIG_COUNT_UNSUPPORTED=""
- _LOKI_GH_TOKEN_REAL_HAD=""
- _LOKI_GH_TOKEN_REAL_VAL=""
- _LOKI_GITHUB_TOKEN_REAL_HAD=""
- _LOKI_GITHUB_TOKEN_REAL_VAL=""
- _LOKI_GH_ENT_TOKEN_REAL_HAD=""
- _LOKI_GH_ENT_TOKEN_REAL_VAL=""
- _LOKI_GITHUB_ENT_TOKEN_REAL_HAD=""
- _LOKI_GITHUB_ENT_TOKEN_REAL_VAL=""
- _LOKI_GH_CONFIG_SCOPED=""
- _LOKI_GH_CONFIG_DIR_HAD=""
- _LOKI_GH_CONFIG_DIR_OLD=""
- _LOKI_GIT_CRED_SCOPED=""
- _LOKI_GIT_CONFIG_COUNT_OLD=""
- _LOKI_GIT_CONFIG_COUNT_HAD=""
- _LOKI_GIT_CRED_INDEX=""
- _LOKI_GH_SENTINEL=""
- _LOKI_SSH_AUTH_SOCK_HAD=""
- _LOKI_SSH_AUTH_SOCK_OLD=""
- _LOKI_GIT_SSH_COMMAND_HAD=""
- _LOKI_GIT_SSH_COMMAND_OLD=""
- _LOKI_SSH_WITHHELD=""
- _LOKI_GH_INHERITED=""
- # Restore the real value (or absence) of one of the 4 token vars, using the
- # per-var _HAD/_VAL pair captured by _loki_gh_capture. Bash has no portable
- # array-of-names-by-reference here, so this is spelled out per var rather than
- # looped over a dynamic name. Also used by the --bg relaunch path to hand the
- # REAL values to the relaunched child (see call site below).
- _loki_restore_one_token() {
- local _var="$1" _had="$2" _val="$3"
- if [ -n "$_had" ]; then
- export "$_var=$_val"
- else
- unset "$_var"
- fi
- }
- # Capture the REAL pre-withhold state exactly once per process: the 4 token
- # vars, GH_CONFIG_DIR, and GIT_CONFIG_COUNT. Idempotent via _LOKI_WITHHELD_TOKENS
- # as the guard, so a later _loki_gh_apply (called after a trusted command
- # restores and re-applies) never re-captures its own sentinel/scoped state as
- # if it were the original.
- _loki_gh_capture() {
- [ -z "$_LOKI_WITHHELD_TOKENS" ] || return 0
- local _v
- # GIT_CONFIG_COUNT/GIT_CONFIG_KEY_n/GIT_CONFIG_VALUE_n were introduced in
- # git 2.31.0 (2021-03-15); an older git silently ignores them, leaving the
- # credential.helper reset a no-op with no error -- the exact
- # fail-open-with-no-signal this fix must not have. Detected once per
- # process (not on every apply) from `git --version`; a `git version
- # X.Y.Z` line below 2.31 sets the flag so _loki_withhold_github_tokens can
- # warn once. Ubuntu 24.04 (2.43) and Debian bookworm (2.39), the two base
- # images this project ships (Dockerfile, Dockerfile.sandbox), are both
- # well above this floor; the warning matters for an operator's own older
- # git (e.g. RHEL 8 ships 2.18).
- if command -v git >/dev/null 2>&1; then
- local _gv _gv_major _gv_minor
- _gv="$(git --version 2>/dev/null)"
- case "$_gv" in
- git\ version\ [0-9]*.[0-9]*.*)
- _gv="${_gv#git version }"
- _gv_major="${_gv%%.*}"
- _gv="${_gv#*.}"
- _gv_minor="${_gv%%.*}"
- case "$_gv_major" in *[!0-9]*) _gv_major="" ;; esac
- case "$_gv_minor" in *[!0-9]*) _gv_minor="" ;; esac
- if [ -n "$_gv_major" ] && [ -n "$_gv_minor" ]; then
- if [ "$_gv_major" -lt 2 ] || { [ "$_gv_major" -eq 2 ] && [ "$_gv_minor" -lt 31 ]; }; then
- _LOKI_GIT_CONFIG_COUNT_UNSUPPORTED=1
- fi
- fi
- ;;
- esac
- fi
- # Cross-runtime/nested-`loki start` guard: a bash child spawned by an
- # ALREADY-withheld process (Bun's delegateToBash forwarding
- # {...process.env}, or an agent literally invoking `loki start` inside its
- # own withheld session) inherits the PARENT's sentinel/scoped state as
- # ordinary env vars. Without this guard this process would capture that
- # inherited state as if it were the operator's real original and restore
- # to it for its own trusted calls -- i.e. Loki's own push/PR would use the
- # inherited garbage sentinel instead of a real credential. Detect the
- # sentinel shape (see _LOKI_GH_SENTINEL below) and treat it, and the
- # scoped GH_CONFIG_DIR/credential.helper reset that came with it, as
- # ABSENT rather than as the real original.
- local _inherited_sentinel=""
- case "${GH_TOKEN:-}" in ghp_LOKIWITHHELDsentinel*INVALID) _inherited_sentinel=1 ;; esac
- # Nested-run rule (BACKLOG 149 round 4): an inherited sentinel means this
- # run was started from INSIDE an already-withheld session -- an agent
- # running `loki start` in its own session, or a build started from the
- # dashboard that start_dashboard spawned after the withhold. Such a run has
- # no real credentials to restore on ANY channel, and reopening any of them
- # for its "trusted" push would hand an agent-chosen repo/branch the
- # operator's credential helper or default SSH keys. So it fails CLOSED on
- # every channel: _loki_gh_restore becomes a no-op and Loki's own push/PR
- # from a nested run runs with the withheld state (and fails). Before this
- # rule the channels disagreed (HTTPS helper chain and default SSH keys
- # reopened, agent socket closed).
- [ -z "$_inherited_sentinel" ] || _LOKI_GH_INHERITED=1
- if [ -n "$_inherited_sentinel" ]; then
- : # GH_TOKEN/GITHUB_TOKEN/GH_ENTERPRISE_TOKEN/GITHUB_ENTERPRISE_TOKEN
- # all get the SAME sentinel per _loki_gh_apply, so one check covers
- # all 4 -- _LOKI_*_REAL_HAD stays unset for each, meaning "restore to
- # absent", not "restore to this garbage value". The same inherited
- # process also had SSH_AUTH_SOCK unset and GIT_SSH_COMMAND=false by
- # this same apply, so _LOKI_SSH_AUTH_SOCK_HAD/_LOKI_GIT_SSH_COMMAND_HAD
- # stay unset too (restore to absent, not to `false`) -- see the SSH
- # capture below, gated on this same flag.
- else
- if [ -n "${GH_TOKEN+x}" ]; then _LOKI_GH_TOKEN_REAL_HAD=1; _LOKI_GH_TOKEN_REAL_VAL="$GH_TOKEN"; fi
- if [ -n "${GITHUB_TOKEN+x}" ]; then _LOKI_GITHUB_TOKEN_REAL_HAD=1; _LOKI_GITHUB_TOKEN_REAL_VAL="$GITHUB_TOKEN"; fi
- if [ -n "${GH_ENTERPRISE_TOKEN+x}" ]; then _LOKI_GH_ENT_TOKEN_REAL_HAD=1; _LOKI_GH_ENT_TOKEN_REAL_VAL="$GH_ENTERPRISE_TOKEN"; fi
- if [ -n "${GITHUB_ENTERPRISE_TOKEN+x}" ]; then _LOKI_GITHUB_ENT_TOKEN_REAL_HAD=1; _LOKI_GITHUB_ENT_TOKEN_REAL_VAL="$GITHUB_ENTERPRISE_TOKEN"; fi
- fi
- # SSH sentinel (BACKLOG 149 round 3): capture the real pre-withhold state
- # of SSH_AUTH_SOCK and GIT_SSH_COMMAND, same inherited-sentinel guard as
- # the token vars above (GIT_SSH_COMMAND=false is this fix's own withheld
- # value, never an operator's real config -- an operator who genuinely
- # wants `false` as their ssh command is not a real scenario worth
- # preserving across the guard).
- if [ -z "$_inherited_sentinel" ]; then
- if [ -n "${SSH_AUTH_SOCK+x}" ]; then _LOKI_SSH_AUTH_SOCK_HAD=1; _LOKI_SSH_AUTH_SOCK_OLD="$SSH_AUTH_SOCK"; fi
- if [ -n "${GIT_SSH_COMMAND+x}" ] && [ "$GIT_SSH_COMMAND" != "false" ]; then
- _LOKI_GIT_SSH_COMMAND_HAD=1
- _LOKI_GIT_SSH_COMMAND_OLD="$GIT_SSH_COMMAND"
- fi
- fi
- for _v in GH_TOKEN GITHUB_TOKEN GH_ENTERPRISE_TOKEN GITHUB_ENTERPRISE_TOKEN; do
- _LOKI_WITHHELD_TOKENS="${_LOKI_WITHHELD_TOKENS:+$_LOKI_WITHHELD_TOKENS }$_v"
- done
- # An inherited scoped GH_CONFIG_DIR (bash's loki-gh-config.* or the Bun
- # route's loki-gh-config-*) is likewise not the operator's real original.
- # TMPDIR is normalized first: on macOS it ends in "/", so bash's own
- # mktemp path has "//" while Node's tmpdir() (Bun route) strips the slash.
- # Without accepting both, Bun's scoped dir was mistaken for the operator's.
- local _tmp_root="${TMPDIR:-/tmp}"
- _tmp_root="${_tmp_root%/}"
- case "${GH_CONFIG_DIR:-}" in
- "$_tmp_root"/loki-gh-config.* | "$_tmp_root"//loki-gh-config.* \
- | "$_tmp_root"/loki-gh-config-* | "$_tmp_root"//loki-gh-config-*) ;;
- *)
- if [ -n "${GH_CONFIG_DIR+x}" ]; then
- _LOKI_GH_CONFIG_DIR_HAD=1
- _LOKI_GH_CONFIG_DIR_OLD="$GH_CONFIG_DIR"
- fi
- ;;
- esac
- # An inherited credential.helper reset: if the sentinel was inherited AND
- # the LAST GIT_CONFIG_KEY/VALUE pair is exactly this fix's reset
- # (credential.helper=""), treat the count as if that pair were never
- # added -- i.e. capture the state BENEATH it as the real original, so
- # restore correctly removes only the inherited pair rather than an
- # operator-configured one at the same index.
- if [ -n "${GIT_CONFIG_COUNT:-}" ] && [ "${GIT_CONFIG_COUNT}" -eq "${GIT_CONFIG_COUNT}" ] 2>/dev/null; then
- local _n=$((GIT_CONFIG_COUNT - 1)) _last_key_var _last_val_var
- _last_key_var="GIT_CONFIG_KEY_${_n}"
- _last_val_var="GIT_CONFIG_VALUE_${_n}"
- if [ -n "$_inherited_sentinel" ] && [ "$_n" -ge 0 ] \
- && [ "${!_last_key_var:-}" = "credential.helper" ] && [ -z "${!_last_val_var:-}" ]; then
- if [ "$_n" -gt 0 ]; then
- _LOKI_GIT_CONFIG_COUNT_HAD=1
- _LOKI_GIT_CONFIG_COUNT_OLD="$_n"
- fi
- _LOKI_GIT_CRED_INDEX="$_n"
- else
- _LOKI_GIT_CONFIG_COUNT_HAD=1
- _LOKI_GIT_CONFIG_COUNT_OLD="$GIT_CONFIG_COUNT"
- _LOKI_GIT_CRED_INDEX="$GIT_CONFIG_COUNT"
- fi
- else
- _LOKI_GIT_CRED_INDEX=0
- fi
- # Fixed for the process lifetime once captured, so restore-then-reapply
- # (the trusted-call cycle) always re-exports the SAME sentinel rather than
- # minting a new one that a caller mid-command could not have seen.
- # Alphanumeric-only after the ghp_ prefix (no underscores): matches the
- # real gh token shape and the existing token-redaction regex in
- # autonomy/lib/proof_redact.py (gh[pousr]_[A-Za-z0-9]{20,}), so if this
- # sentinel ever leaked into a proof/receipt artifact it would still be
- # caught by the existing redaction filter rather than passing it by shape.
- _LOKI_GH_SENTINEL="ghp_LOKIWITHHELDsentinel$$${RANDOM}${RANDOM}INVALID"
- }
- # Unconditionally (re-)export the withheld/scoped state. Never captures --
- # _loki_gh_capture must run first. Silent: callers that want the one-time
- # operator-facing log line print it themselves.
- _loki_gh_apply() {
- export GH_TOKEN="$_LOKI_GH_SENTINEL" GITHUB_TOKEN="$_LOKI_GH_SENTINEL" \
- GH_ENTERPRISE_TOKEN="$_LOKI_GH_SENTINEL" GITHUB_ENTERPRISE_TOKEN="$_LOKI_GH_SENTINEL"
- if [ -z "$_LOKI_GH_CONFIG_SCOPED" ]; then
- local _empty_cfg
- _empty_cfg="$(mktemp -d "${TMPDIR:-/tmp}/loki-gh-config.XXXXXX" 2>/dev/null)" || _empty_cfg=""
- [ -n "$_empty_cfg" ] && _LOKI_GH_CONFIG_SCOPED="$_empty_cfg"
- fi
- [ -n "$_LOKI_GH_CONFIG_SCOPED" ] && export GH_CONFIG_DIR="$_LOKI_GH_CONFIG_SCOPED"
- export "GIT_CONFIG_KEY_${_LOKI_GIT_CRED_INDEX}=credential.helper"
- export "GIT_CONFIG_VALUE_${_LOKI_GIT_CRED_INDEX}="
- export GIT_CONFIG_COUNT=$((_LOKI_GIT_CRED_INDEX + 1))
- _LOKI_GIT_CRED_SCOPED=1
- # SSH sentinel (BACKLOG 149 round 3): no reachable SSH agent, and any
- # ssh-transport git operation fails closed via GIT_SSH_COMMAND=false
- # (env var, so it overrides any core.sshCommand -- see the header comment
- # above _LOKI_WITHHELD_TOKENS for the git-docs citation).
- unset SSH_AUTH_SOCK
- export GIT_SSH_COMMAND=false
- _LOKI_SSH_WITHHELD=1
- if command -v gh >/dev/null 2>&1; then
- # Repo read as data BEFORE the credentials come back; gh then runs
- # from / (round 5: gh's own git calls must not load the agent's repo
- # config while holding the real credentials).
- gh() {
- local _gh_repo
- _gh_repo="$(_loki_trusted_repo)"
- _loki_with_github_tokens _loki_run_neutral "$_gh_repo" command gh "$@"
- }
- fi
- }
- # Put back the REAL pre-withhold state around one trusted command. A no-op in
- # a run that inherited the sentinel: there is no real state to put back, and
- # the nested-run rule in _loki_gh_capture is to fail closed on every channel.
- _loki_gh_restore() {
- [ -z "$_LOKI_GH_INHERITED" ] || return 0
- _loki_restore_one_token GH_TOKEN "$_LOKI_GH_TOKEN_REAL_HAD" "$_LOKI_GH_TOKEN_REAL_VAL"
- _loki_restore_one_token GITHUB_TOKEN "$_LOKI_GITHUB_TOKEN_REAL_HAD" "$_LOKI_GITHUB_TOKEN_REAL_VAL"
- _loki_restore_one_token GH_ENTERPRISE_TOKEN "$_LOKI_GH_ENT_TOKEN_REAL_HAD" "$_LOKI_GH_ENT_TOKEN_REAL_VAL"
- _loki_restore_one_token GITHUB_ENTERPRISE_TOKEN "$_LOKI_GITHUB_ENT_TOKEN_REAL_HAD" "$_LOKI_GITHUB_ENT_TOKEN_REAL_VAL"
- if [ -n "$_LOKI_GH_CONFIG_DIR_HAD" ]; then
- export GH_CONFIG_DIR="$_LOKI_GH_CONFIG_DIR_OLD"
- else
- unset GH_CONFIG_DIR
- fi
- if [ -n "$_LOKI_GIT_CONFIG_COUNT_HAD" ]; then
- export GIT_CONFIG_COUNT="$_LOKI_GIT_CONFIG_COUNT_OLD"
- else
- unset GIT_CONFIG_COUNT
- fi
- unset "GIT_CONFIG_KEY_${_LOKI_GIT_CRED_INDEX}" "GIT_CONFIG_VALUE_${_LOKI_GIT_CRED_INDEX}"
- # SSH sentinel restore: give the trusted command back the real SSH agent
- # and ssh command, so Loki's own push/PR still works over an SSH origin.
- _loki_restore_one_token SSH_AUTH_SOCK "$_LOKI_SSH_AUTH_SOCK_HAD" "$_LOKI_SSH_AUTH_SOCK_OLD"
- _loki_restore_one_token GIT_SSH_COMMAND "$_LOKI_GIT_SSH_COMMAND_HAD" "$_LOKI_GIT_SSH_COMMAND_OLD"
- }
- _LOKI_GH_TRUSTED_DEPTH=0
- # Reentrancy: a trusted function (create_session_pr) calls `gh` directly one
- # or more times; since the gh() wrapper (_loki_gh_apply) IS
- # _loki_with_github_tokens, each of those calls re-enters this function while
- # the OUTER call is still in progress. Without depth tracking, the FIRST
- # nested `gh` call's own apply-on-exit would re-scope the environment before
- # the outer caller's second `gh` call (or any bare `git` call) runs, breaking
- # it silently -- caught in this session before landing: create_session_pr
- # calls `gh pr list` then `gh pr create`, and the nested apply between them
- # would have handed the second call sentineled/scoped state instead of real
- # credentials. Only the OUTERMOST call restores on entry and re-applies on
- # exit; a nested call finds the depth already > 0 and is a pure passthrough
- # (real credentials are already in place from the outer restore).
- _loki_with_github_tokens() {
- local _rc=0 _outermost=0
- if [ "$_LOKI_GH_TRUSTED_DEPTH" -eq 0 ]; then
- _outermost=1
- if [ -n "$_LOKI_WITHHELD_TOKENS" ]; then
- _loki_gh_restore
- fi
- fi
- _LOKI_GH_TRUSTED_DEPTH=$((_LOKI_GH_TRUSTED_DEPTH + 1))
- "$@" || _rc=$?
- _LOKI_GH_TRUSTED_DEPTH=$((_LOKI_GH_TRUSTED_DEPTH - 1))
- if [ "$_outermost" -eq 1 ] && [ -n "$_LOKI_WITHHELD_TOKENS" ]; then
- _loki_gh_apply
- fi
- return "$_rc"
- }
- #-------------------------------------------------------------------------------
- # BACKLOG 149 round 5: a credentialed git or gh process never loads the agent's
- # repo config.
- #
- # _loki_with_github_tokens hands the real credentials back for one command. If
- # that command is `git push` run inside the agent's working tree, git loads the
- # agent-writable repo config first: .git/hooks/pre-push, core.hooksPath,
- # core.sshCommand, a repo-local credential.helper, core.fsmonitor,
- # url.insteadOf/pushInsteadOf, include/includeIf. Two reviewers reproduced a
- # planted pre-push hook reading the real GH_TOKEN and SSH_AUTH_SOCK, and
- # `git credential fill` returning the operator's password. `-c
- # core.hooksPath=/dev/null` alone does not stop core.sshCommand, and a list of
- # per-key overrides would have to stay complete forever.
- #
- # So the credentialed step never runs with that config loaded:
- # - push: read remote.origin.url as DATA (no credentials), accept only a
- # github.com https/ssh URL, fetch the branch into a fresh Loki-owned repo
- # (the fetch runs upload-pack in the agent repo WITHOUT credentials), then
- # push from that repo with the credentials restored.
- # - gh: run from / with GH_REPO set to the validated origin, so gh's own git
- # calls find no repository. An explicit --repo/-R still overrides GH_REPO.
- # The operator's global/system git config still loads (it holds the real
- # credential helper); an agent that can write ~/.gitconfig is a same-user
- # filesystem residual, disclosed in the header above.
- #-------------------------------------------------------------------------------
- # Echo OWNER/REPO for a github.com origin URL, else fail. Accepted forms:
- # https://github.com/O/R[.git][/], git@github.com:O/R[.git][/],
- # ssh://git@github.com/O/R[.git][/], host matched case-insensitively. Anything
- # else (other hosts, SSH host aliases such as github.com-work, embedded
- # credentials, extra path segments, odd characters) is rejected;
- # _loki_origin_refusal says why.
- _loki_github_repo_from_url() {
- local _u="${1:-}" _lc _pre _p
- _lc="$(printf '%s' "$_u" | tr '[:upper:]' '[:lower:]')"
- case "$_lc" in
- https://github.com/*) _pre="https://github.com/" ;;
- git@github.com:*) _pre="git@github.com:" ;;
- ssh://git@github.com/*) _pre="ssh://git@github.com/" ;;
- *) return 1 ;;
- esac
- _p="${_u:${#_pre}}"
- _p="${_p%/}"
- _p="${_p%.git}"
- case "$_p" in
- */*/* | /* | */ | */.* | .* | *..* | *[!A-Za-z0-9._/-]*) return 1 ;;
- ?*/?*) printf '%s\n' "$_p" ;;
- *) return 1 ;;
- esac
- }
- # One line saying why an origin is refused. Never echoes the URL itself (it
- # may carry embedded credentials).
- _loki_origin_refusal() {
- local _lc
- _lc="$(printf '%s' "${1:-}" | tr '[:upper:]' '[:lower:]')"
- case "$_lc" in
- '') echo "this repository has no origin remote" ;;
- *@github.com[!:/]*|*://github.com[!/]*|*@github.com[!:/]*:*)
- echo "origin uses an SSH host alias or look-alike of github.com (for example github.com-work from ~/.ssh/config); Loki pushes only to a literal github.com origin, so the push never depends on agent-editable ssh config" ;;
- *github.com*)
- echo "origin names github.com but is not a plain https or ssh OWNER/REPO URL (embedded credentials, extra path segments or unusual characters are refused)" ;;
- *)
- echo "origin is not on github.com; Loki currently pushes only to github.com (GitHub Enterprise and GitLab are planned)" ;;
- esac
- }
- # Origin pin (round 6, reviewer F1): the agent could otherwise repoint origin
- # at another github.com repository and have Loki push its branch there with
- # the operator's credentials. Loki never changes origin itself, so the value
- # recorded before the first iteration (possibly "no origin") is the only one
- # a trusted push accepts. Recorded once per process.
- _LOKI_ORIGIN_PINNED=""
- _LOKI_PINNED_ORIGIN=""
- _loki_pin_origin() {
- [ -z "$_LOKI_ORIGIN_PINNED" ] || return 0
- _LOKI_PINNED_ORIGIN="$(git -C "${TARGET_DIR:-.}" config --get remote.origin.url 2>/dev/null || true)"
- _LOKI_ORIGIN_PINNED=1
- }
- # OWNER/REPO of TARGET_DIR's origin, read as data, or nothing.
- _loki_trusted_repo() {
- _loki_github_repo_from_url "$(git -C "${TARGET_DIR:-.}" config --get remote.origin.url 2>/dev/null)" 2>/dev/null || true
- }
- # _loki_run_neutral <owner/repo|""> <cmd...>: run cmd from / so no repository
- # (and so no repo config) is discovered, with GH_REPO naming the repo for gh.
- # A subshell, so functions stay callable; callers pass `command gh` to reach
- # the binary.
- _loki_run_neutral() {
- local _repo="$1"
- shift
- (
- cd / || exit 1
- unset GIT_DIR GIT_WORK_TREE
- if [ -n "$_repo" ] && [ -z "${GH_REPO:-}" ]; then
- export GH_REPO="$_repo"
- fi
- "$@"
- )
- }
- # _loki_trusted_push <runner> <repo-dir> <branch>: push <branch> of the agent
- # repo to its github.com origin without loading the agent repo's config in
- # the credentialed process. <runner> is the credential re-grant wrapper
- # (_loki_with_github_tokens, or on_run_complete's timed _loki_net). The agent
- # repo's branch gets no upstream set (it did with `push -u`); nothing reads it.
- # Optional <dest> OWNER/REPO (engine10-push.sh fork mode): the pinned origin is
- # still validated, then the push goes to https://github.com/<dest>.git, a fork
- # the push child resolved from gh and checked, never a value from the tree.
- _loki_trusted_push() {
- local _runner="$1" _dir="$2" _branch="$3" _dest="${4:-}" _url _tmp _rc=1
- _dir="$(cd "$_dir" 2>/dev/null && pwd -P)" || return 1
- git check-ref-format --branch "$_branch" >/dev/null 2>&1 || return 1
- _url="$(git -C "$_dir" config --get remote.origin.url 2>/dev/null)" || _url=""
- if ! _loki_github_repo_from_url "$_url" >/dev/null; then
- log_warn "Not pushing branch '$_branch': $(_loki_origin_refusal "$_url")."
- return 2
- fi
- if [ -n "$_LOKI_ORIGIN_PINNED" ] && [ "$_url" != "$_LOKI_PINNED_ORIGIN" ]; then
- log_warn "Not pushing branch '$_branch': origin changed during the run (it was $(_loki_github_repo_from_url "$_LOKI_PINNED_ORIGIN" 2>/dev/null || echo "not a GitHub repository") when the run started). Loki does not push to a destination the agent could have chosen."
- return 2
- fi
- # never push a default branch, for every caller (on_run_complete,
- # create_session_pr): the branch name can come from agent-writable state.
- # The default branch is resolved for the origin just validated (the pinned
- # one when pinned), gh run from / with an explicit OWNER/REPO. An
- # unresolvable default branch refuses: fail closed.
- local _repo _def=""
- case "$_branch" in
- main|master|HEAD)
- log_warn "Not pushing branch '$_branch': Loki never pushes directly to a default branch."
- return 2 ;;
- esac
- _repo="$(_loki_github_repo_from_url "$_url")"
- local _origin_url="$_url"
- if [ -n "$_dest" ]; then
- if ! _loki_github_repo_from_url "https://github.com/${_dest}.git" >/dev/null; then
- log_warn "Not pushing branch '$_branch': fork destination '$_dest' is not a plain OWNER/REPO."
- return 2
- fi
- _repo="$_dest" _url="https://github.com/${_dest}.git"
- fi
- _def="$(_loki_run_neutral "$_repo" "$_runner" gh repo view "$_repo" --json defaultBranchRef --jq '.defaultBranchRef.name' 2>/dev/null)" || _def=""
- if [ -z "$_def" ]; then
- log_warn "Not pushing branch '$_branch': could not resolve the default branch of $_repo, and Loki never risks a direct push to it."
- return 2
- fi
- if [ "$_branch" = "$_def" ]; then
- log_warn "Not pushing branch '$_branch': it is the default branch of $_repo, and Loki never pushes directly to the default branch."
- return 2
- fi
- _tmp="$(mktemp -d "${TMPDIR:-/tmp}/loki-push.XXXXXX")" || return 1
- # FC-TRUSTED-PUSH-PARTIAL-CLONE: a partial agent clone (extensions.partialclone,
- # or a remote.<n>.promisor true/yes/on/1 on newer git; both read as data)
- # holds objects it never downloaded, and upload-pack does not lazy-fetch
- # them, so the branch fetch below dies. Seed the temp repo with the
- # validated origin (_origin_url, captured before a fork _dest overwrites
- # _url) as a promisor remote and fetch its refs first. The agent pack is
- # thin (deltas against blobs the temp repo only knows by id), so the client
- # lazy-fetches those delta bases from loki-base during the agent fetch: that
- # fetch therefore runs under "$_runner" too. The credentials stay on the
- # client side only: the agent-dir upload-pack, the one process that reads
- # agent config, is started through `env -i` with no credentials, no helper
- # config, no HOME and no lazy fetch. GIT_TERMINAL_PROMPT=0 so a refused
- # token never waits on a username. Non-partial repos skip all of this.
- local _partial="" _up=""
- if [ -n "$(git -C "$_dir" config --get extensions.partialclone 2>/dev/null)" ] \
- || git -C "$_dir" config --get-regexp '^remote\..*\.promisor$' 2>/dev/null | grep -Eqi ' (true|yes|on|1)$'; then
- _partial=1
- _up="/usr/bin/env -i PATH=/usr/bin:/bin GIT_NO_LAZY_FETCH=1 $(printf '%q' "$(command -v git)") upload-pack"
- fi
- # --update-shallow: a shallow agent repo's history ends in shallow roots,
- # which a plain fetch rejects while still exiting 0. The rev-parse makes a
- # silently empty fetch fail here instead of at the push.
- if git init -q --template= "$_tmp" >/dev/null 2>&1 \
- && { [ -z "$_partial" ] || {
- git -C "$_tmp" config core.repositoryformatversion 1 \
- && git -C "$_tmp" config extensions.partialclone loki-base \
- && git -C "$_tmp" config remote.loki-base.url "$_origin_url" \
- && git -C "$_tmp" config remote.loki-base.promisor true \
- && git -C "$_tmp" config remote.loki-base.partialclonefilter blob:none \
- && GIT_TERMINAL_PROMPT=0 "$_runner" git -C "$_tmp" fetch -q --no-tags --filter=blob:none loki-base "+refs/heads/*:refs/remotes/loki-base/*" >/dev/null 2>&1; }; } \
- && { if [ -n "$_partial" ]; then
- GIT_TERMINAL_PROMPT=0 "$_runner" git -C "$_tmp" fetch -q --no-tags --update-shallow --upload-pack "$_up" "$_dir" "+refs/heads/$_branch:refs/heads/$_branch" >/dev/null 2>&1
- else
- git -C "$_tmp" fetch -q --no-tags --update-shallow "$_dir" "+refs/heads/$_branch:refs/heads/$_branch" >/dev/null 2>&1
- fi; } \
- && git -C "$_tmp" rev-parse -q --verify "refs/heads/$_branch" >/dev/null 2>&1; then
- _rc=0
- # `-C "$_tmp"`: git starts repository discovery in the fresh repo, so
- # the agent repo is never found, whatever the caller's cwd.
- "$_runner" git -C "$_tmp" push -q "$_url" "refs/heads/$_branch:refs/heads/$_branch" || _rc=$?
- fi
- rm -rf "$_tmp"
- return "$_rc"
- }
- _loki_withhold_github_tokens() {
- local _v _held=""
- # Before any provider runs, so the pin is the operator's origin, not one
- # an agent set. Also under the opt-out: it guards the push destination.
- _loki_pin_origin
- # Operator opt-out (exact value 1): keep the earlier behavior, where the
- # agent inherits the token and the real gh config. That is a Rule of Two
- # exposure, so say so. No GH_CONFIG_DIR/credential.helper scoping under
- # the opt-out either -- the point of this knob is the old, fully-inherited
- # behavior.
- if [ "${LOKI_ALLOW_AGENT_GITHUB_TOKEN:-}" = "1" ]; then
- for _v in GH_TOKEN GITHUB_TOKEN GH_ENTERPRISE_TOKEN GITHUB_ENTERPRISE_TOKEN; do
- [ -n "${!_v:-}" ] && _held="${_held:+$_held }$_v"
- done
- [ -z "$_held" ] || printf '%s\n' "WARNING: LOKI_ALLOW_AGENT_GITHUB_TOKEN=1: the agent session holds the GitHub token ($_held); an injected prompt can push with it (Rule of Two exposure)." >&2
- return 0
- fi
- local _first_time=""
- [ -n "$_LOKI_WITHHELD_TOKENS" ] || _first_time=1
- # gh documents (`gh help environment`) that an env token "takes precedence
- # over previously stored credentials", so a garbage-but-present value
- # makes gh's own resolution (both `gh auth token` and `gh auth
- # git-credential`, the latter reachable directly by git's
- # credential.helper) return the garbage value instead of falling through
- # to a keyring/hosts.yml lookup. GH_CONFIG_DIR stays scoped to a fresh
- # empty dir (the plaintext-hosts.yml-read case). credential.helper is
- # additionally reset to empty (gitcredentials(7)): this is git-invoked,
- # not gh-mediated, so the sentinel does not reach a plain
- # osxkeychain/libsecret helper on its own.
- _loki_gh_capture
- _loki_gh_apply
- if [ -n "$_first_time" ]; then
- if [ -n "$_LOKI_GH_INHERITED" ]; then
- printf '%s\n' "WARNING: this run was started from inside an already-withheld Loki session (inherited Rule of Two sentinel). It has no real GitHub or SSH credentials to restore, so its own push and PR steps will fail closed. Start builds that need to push from a normal shell." >&2
- else
- log_info "Withheld from agent sessions (Rule of Two): $_LOKI_WITHHELD_TOKENS (sentineled, not merely unset), gh config store, git credential.helper, SSH agent (SSH_AUTH_SOCK unset, GIT_SSH_COMMAND=false). Loki's own push and PR steps still use the real credentials."
- fi
- if [ -n "$_LOKI_GIT_CONFIG_COUNT_UNSUPPORTED" ]; then
- printf '%s\n' "WARNING: git < 2.31 detected -- GIT_CONFIG_COUNT/GIT_CONFIG_KEY/GIT_CONFIG_VALUE (used to reset credential.helper) are silently ignored on this git version. The git-invoked credential-helper bypass (BACKLOG 149 round 2) is NOT closed on this host; upgrade git to 2.31+ to close it. The GH_TOKEN-family sentinel, GH_CONFIG_DIR scoping, and the SSH_AUTH_SOCK/GIT_SSH_COMMAND withhold above are unaffected and still apply (GIT_SSH_COMMAND is a plain env var, not a GIT_CONFIG_* mechanism, so it needs no version floor)." >&2
- fi
- fi
- }
- #===============================================================================
- # on_run_complete (Slice 3: opt-in local git output on success)
- #
- # Called from every SUCCESS exit BEFORE emit_completion_summary so the PR url it
- # discovers is folded into the summary. Default behavior is a no-op: it only
- # acts when LOKI_DELEGATE_PR=1.
- #
- # LOKI_DELEGATE_PR=1 opens a LOCAL pull request from the user's machine, only if:
- # - this is a GitHub repo (gh + a github.com remote), AND
- # - `gh auth status` succeeds, AND
- # - the current branch is not main/master (never PR a default branch to itself)
- # It mirrors the proven pattern at autonomy/loki:5524-5527: push the branch,
- # then `gh pr create --head <branch>`. NO auto-merge. Every call is best-effort
- # (`|| true`); failures never block completion. This is a single sanctioned
- # local network call, never CI.
- #
- # Reconciliation with the existing GITHUB_PR path (run.sh create_github_pr,
- # invoked after run_autonomous returns when LOKI_GITHUB_PR=true): if GITHUB_PR
- # is already true we DEFER to that path and do nothing here, so a user who set
- # both knobs never gets a double PR.
- #===============================================================================
- # Write the PR url where an OUT-OF-PROCESS caller can read it.
- #
- # _LOKI_DELEGATE_PR_URL is exported, which reaches children but NOT a sibling
- # step. A GitHub composite action runs `loki start` in one step and reports the
- # result in the next, so an exported variable is invisible to it and the action
- # would have to re-derive the url and could get it wrong. Persisting it means a
- # caller reports what ACTUALLY happened.
- #
- # Best-effort by construction: a failure here must never affect a run whose PR
- # was already opened successfully.
- _loki_persist_pr_url() {
- local _u="${1:-}"
- [ -n "$_u" ] || return 0
- mkdir -p ".loki/state" 2>/dev/null || return 0
- printf '%s\n' "$_u" > ".loki/state/pr-url.txt" 2>/dev/null || true
- return 0
- }
- on_run_complete() {
- # DEFAULT ON as of v9.43.0.
- #
- # This shipped OFF, and the product literally printed "Pull request: not
- # opened (set LOKI_DELEGATE_PR=1 to open one)" -- it knew what the user
- # wanted and asked them to go read documentation instead of doing it. For
- # the core use case ("I give it a GitHub issue and it resolves"), the PR IS
- # the deliverable, so shipping it off meant shipping the product off.
- #
- # Safe to default on because every guard below was already built and is
- # unchanged: requires a GitHub repo AND `gh auth status` AND a non-default
- # branch; it opens a PR and NEVER merges; every call is best-effort so a
- # failure cannot block completion. Opt out with LOKI_DELEGATE_PR=0.
- if [ "${LOKI_DELEGATE_PR:-1}" != "1" ]; then
- return 0
- fi
- # Defer to the existing dedicated PR path to avoid a double PR.
- if [ "${GITHUB_PR:-false}" = "true" ]; then
- return 0
- fi
- # Network-call timeout guard: a stalled network / auth prompt would
- # otherwise hang the completion path indefinitely in --bg. Run each network
- # call through `timeout 30` when available; fall back to the bare call if
- # timeout is not installed (a local wrapper keeps this set -u safe on bash
- # 3.2, where an empty array expansion would error). Keeps every existing
- # `|| true` non-fatal behavior.
- # The push and gh calls below are the trusted post-session step, so they
- # get the withheld GitHub token back (a plain `timeout 30 gh` would exec
- # the binary and bypass the gh() wrapper).
- # Round 5: every call runs from / (_loki_run_neutral) with GH_REPO read as
- # data first, so neither gh nor git loads the agent's repo config while
- # holding the credentials; the push goes through _loki_trusted_push.
- _loki_net() {
- local _net_repo
- _net_repo="$(_loki_trusted_repo)"
- if command -v timeout >/dev/null 2>&1; then
- _loki_with_github_tokens _loki_run_neutral "$_net_repo" timeout 30 "$@"
- else
- _loki_with_github_tokens _loki_run_neutral "$_net_repo" "$@"
- fi
- }
- # Require gh + auth.
- if ! command -v gh >/dev/null 2>&1; then
- return 0
- fi
- if ! (cd "${TARGET_DIR:-.}" && _loki_net gh auth status) >/dev/null 2>&1; then
- return 0
- fi
- # Require a GitHub remote (skip silently on non-GitHub repos).
- local remote_url
- remote_url="$( (cd "${TARGET_DIR:-.}" && git config --get remote.origin.url) 2>/dev/null || true )"
- case "$remote_url" in
- *github.com*) : ;;
- *) return 0 ;;
- esac
- # Resolve current branch; never PR a default branch to itself.
- local branch
- branch="$( (cd "${TARGET_DIR:-.}" && git rev-parse --abbrev-ref HEAD) 2>/dev/null || true )"
- case "$branch" in
- ""|main|master|HEAD) return 0 ;;
- esac
- # the repository's ACTUAL default branch (develop, trunk, ...) is
- # refused inside _loki_trusted_push, for every caller.
- log_info "LOKI_DELEGATE_PR=1: opening a local pull request for branch '$branch'..."
- # Push, then create. Non-interactive (no tty in --bg). Best-effort, each
- # network call bounded by the timeout guard above.
- # A refused or failed push is reported (the refusal names its reason on
- # stderr) and ends the delegate step: there is nothing to open a PR from.
- local _push_rc=0
- _loki_trusted_push _loki_net "${TARGET_DIR:-.}" "$branch" >/dev/null || _push_rc=$?
- if [ "$_push_rc" -ne 0 ]; then
- [ "$_push_rc" -eq 2 ] || log_warn "LOKI_DELEGATE_PR=1: pushing branch '$branch' failed; not opening a pull request."
- return 0
- fi
- # Explicit --repo on every gh call (not only GH_REPO): the repo the push
- # just validated, so no gh subcommand can fall back to cwd detection.
- local _pr_repo
- _pr_repo="$(_loki_trusted_repo)"
- local pr_title
- pr_title="Loki Mode: ${branch}"
- local pr_url=""
- # ENT-4 (idempotent PR): reuse an existing OPEN PR for this head instead of
- # attempting a second create on a platform retry / resume.
- local existing_pr
- existing_pr="$( (cd "${TARGET_DIR:-.}" && _loki_net gh pr list --repo "$_pr_repo" --head "$branch" --state open --json url --jq '.[0].url') 2>/dev/null || true )"
- if [ -n "$existing_pr" ]; then
- _LOKI_DELEGATE_PR_URL="$existing_pr"
- export _LOKI_DELEGATE_PR_URL
- _loki_persist_pr_url "$existing_pr"
- log_info "LOKI_DELEGATE_PR=1: PR already exists for branch '$branch': $existing_pr (skipping create)."
- return 0
- fi
- # Proven PR (Loop 6): append the Evidence Receipt to the inline body.
- # Default-on; LOKI_PROVEN_PR=0 -> body byte-identical to before. This path
- # runs gh from a `cd "${TARGET_DIR:-.}"` subshell, so resolve the proof
- # relative to TARGET_DIR (not the bare-relative helper). Empty
- # expected_head_sha by design (R-DET-1 run_id pointer is the anti-stale guard).
- local _del_body="Opened by Loki Mode (delegate mode). Review locally before merge."
- if [ "${LOKI_PROVEN_PR:-1}" != "0" ] && declare -f render_evidence_receipt_md >/dev/null 2>&1; then
- local _del_loki="${TARGET_DIR:-.}/.loki"
- local _del_idfile="$_del_loki/state/last-proof-id.txt"
- if [ -s "$_del_idfile" ]; then
- local _del_rid=""
- _del_rid="$(cat "$_del_idfile" 2>/dev/null || true)"
- if [ -n "$_del_rid" ] && [ -f "$_del_loki/proofs/$_del_rid/proof.json" ]; then
- local _del_receipt=""
- _del_receipt="$(render_evidence_receipt_md "$_del_loki/proofs/$_del_rid/proof.json" "" "" 2>/dev/null || true)"
- if [ -n "$_del_receipt" ]; then
- _del_body="${_del_body}
- ${_del_receipt}"
- fi
- fi
- fi
- fi
- pr_url="$( (cd "${TARGET_DIR:-.}" && _loki_net gh pr create --repo "$_pr_repo" --title "$pr_title" --body "$_del_body" --head "$branch") 2>/dev/null || true )"
- if [ -n "$pr_url" ]; then
- # Export so build_completion_summary folds the url into the summary.
- _LOKI_DELEGATE_PR_URL="$pr_url"
- export _LOKI_DELEGATE_PR_URL
- _loki_persist_pr_url "$pr_url"
- log_info "Pull request opened: $pr_url"
- else
- log_warn "LOKI_DELEGATE_PR=1: gh pr create did not return a URL (a PR may already exist for this branch)."
- fi
- return 0
- }
- #===============================================================================
- # Parallel Workflow Functions (Git Worktrees)
- #===============================================================================
- # Production bridge for the Bun execution-manifest intelligence. The feature is
- # explicitly opt-in; with LOKI_EXEC_MANIFEST unset this performs no I/O and the
- # legacy parallel workflow remains byte-for-byte on its old path.
- _loki_exec_manifest() {
- [ "${LOKI_EXEC_MANIFEST:-0}" = "1" ] || return 0
- bun --no-env-file --config=/dev/null "${SCRIPT_DIR}/../loki-ts/dist/loki.js" internal exec-manifest "$@"
- }
- init_exec_manifest() {
- [ "${LOKI_EXEC_MANIFEST:-0}" = "1" ] || return 0
- local base_sha plan_file
- # create_worktree's built-in streams branch from main (with HEAD only as its
- # final fallback), so the manifest must pin that exact production base.
- base_sha=$(git -C "$TARGET_DIR" rev-parse main 2>/dev/null) || \
- base_sha=$(git -C "$TARGET_DIR" rev-parse HEAD 2>/dev/null) || return 1
- mkdir -p "${TARGET_DIR}/.loki"
- plan_file=$(mktemp "${TARGET_DIR}/.loki/.exec-manifest-plan.XXXXXX") || return 1
- LOKI_PLAN_FILE="$plan_file" LOKI_BASE_SHA="$base_sha" \
- LOKI_PARALLEL_TESTING_VALUE="$PARALLEL_TESTING" \
- LOKI_PARALLEL_DOCS_VALUE="$PARALLEL_DOCS" \
- LOKI_PARALLEL_BLOG_VALUE="$PARALLEL_BLOG" python3 <<'PY'
- import json, os
- streams = []
- if os.environ["LOKI_PARALLEL_TESTING_VALUE"] == "true":
- streams.append({"name": "testing", "paths": ["tests", "loki-ts/tests"],
- "acceptance": "test stream exits successfully"})
- if os.environ["LOKI_PARALLEL_DOCS_VALUE"] == "true":
- streams.append({"name": "docs", "paths": ["docs", "README.md", "CHANGELOG.md"],
- "acceptance": "documentation stream exits successfully"})
- if os.environ["LOKI_PARALLEL_BLOG_VALUE"] == "true":
- streams.append({"name": "blog", "paths": ["blog"],
- "acceptance": "blog stream exits successfully"})
- with open(os.environ["LOKI_PLAN_FILE"], "w") as f:
- json.dump({"baseSha": os.environ["LOKI_BASE_SHA"],
- "integrationOwner": "parallel-orchestrator",
- "streams": streams,
- "env": {"LOKI_EXEC_MANIFEST": "1"}}, f)
- PY
- _loki_exec_manifest plan "$plan_file" "${TARGET_DIR}/.loki" >/dev/null
- local rc=$?
- rm -f "$plan_file"
- return "$rc"
- }
- validate_exec_manifest_result() {
- local stream_name="$1" branch="$2"
- [ "${LOKI_EXEC_MANIFEST:-0}" = "1" ] || return 0
- local base_sha result_file
- base_sha="${WORKTREE_BASE_SHAS[$stream_name]:-}"
- [ -n "$base_sha" ] || base_sha=$(git -C "$TARGET_DIR" merge-base "$branch" HEAD 2>/dev/null) || return 1
- result_file=$(mktemp "${TARGET_DIR}/.loki/.exec-manifest-result.XXXXXX") || return 1
- LOKI_RESULT_FILE="$result_file" LOKI_RESULT_STREAM="$stream_name" \
- LOKI_RESULT_BASE="$base_sha" LOKI_RESULT_BRANCH="$branch" \
- LOKI_RESULT_REPO="$TARGET_DIR" python3 -E - <<'PY'
- import sys; sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- import json, os, subprocess
- paths = subprocess.check_output(
- ["git", "-C", os.environ["LOKI_RESULT_REPO"], "diff", "--name-only",
- f'{os.environ["LOKI_RESULT_BASE"]}..{os.environ["LOKI_RESULT_BRANCH"]}'],
- text=True).splitlines()
- with open(os.environ["LOKI_RESULT_FILE"], "w") as f:
- json.dump({"name": os.environ["LOKI_RESULT_STREAM"],
- "baseSha": os.environ["LOKI_RESULT_BASE"],
- "changedPaths": paths, "acceptanceMet": True}, f)
- PY
- local outcome rc=0
- outcome=$(_loki_exec_manifest validate "$result_file" "${TARGET_DIR}/.loki" 2>&1) || rc=$?
- rm -f "$result_file"
- if [ "$rc" -ne 0 ]; then
- log_error "Execution manifest rejected $stream_name: $outcome"
- return 1
- fi
- log_info "Execution manifest accepted: $stream_name"
- return 0
- }
- # Check if parallel mode is supported (bash 4+ required for associative arrays)
- check_parallel_support() {
- if [ "$BASH_VERSION_MAJOR" -lt 4 ] 2>/dev/null; then
- log_error "Parallel mode requires bash 4.0+ (current: $BASH_VERSION)"
- log_error "Parallel mode uses associative arrays which require bash 4+"
- log_error ""
- log_error "How to upgrade:"
- log_error " macOS: brew install bash && sudo chsh -s /opt/homebrew/bin/bash"
- log_error " Ubuntu: sudo apt install bash"
- log_error " WSL: Usually has bash 4+ by default"
- log_error ""
- log_error "Or run without --parallel flag for sequential mode (works with bash 3.2+)"
- return 1
- fi
- return 0
- }
- # npm install cache keyed on sha256(package-lock.json + package.json),
- # stored under the project's .loki/cache/install/. Hit: copy cached node_modules.
- # Miss or unusable entry: real install, then publish the entry atomically
- # (temp dir + rename) so concurrent worktrees never see a partial entry.
- # ponytail: npm only; pip installs into the active interpreter, so there is no
- # per-worktree venv to cache. No lockfile means no cache (always install).
- _loki_npm_install_cached() {
- local cache_root="$1" wt="$2" key entry tmp aside old
- if [ ! -f "$wt/package-lock.json" ]; then
- (cd "$wt" && npm install --silent 2>/dev/null) || true
- return 0
- fi
- # node version + platform in the key: native addons are ABI specific
- key="$({ cat "$wt/package-lock.json" "$wt/package.json" 2>/dev/null; node -v 2>/dev/null; uname -sm; } | { shasum -a 256 2>/dev/null || sha256sum; } | cut -d' ' -f1)"
- if [ -n "$key" ]; then
- entry="$cache_root/$key"
- if [ -f "$entry/.complete" ] && [ -d "$entry/node_modules" ]; then
- rm -rf "$wt/node_modules"
- if cp -R "$entry/node_modules" "$wt/node_modules" 2>/dev/null; then
- return 0
- fi
- rm -rf "$wt/node_modules"
- fi
- # Evict only an unusable entry (never a complete one another creator
- # just published), renamed aside first so the removal is not racy.
- if [ -d "$entry" ] && { [ ! -f "$entry/.complete" ] || [ ! -d "$entry/node_modules" ]; }; then
- aside="$cache_root/.evict.$key.$$.$RANDOM"
- mv "$entry" "$aside" 2>/dev/null && rm -rf "$aside" 2>/dev/null
- fi
- fi
- (cd "$wt" && npm install --silent 2>/dev/null) || true
- if [ -n "$key" ] && [ -d "$wt/node_modules" ]; then
- mkdir -p "$cache_root" 2>/dev/null || return 0
- tmp="$(mktemp -d "$cache_root/.tmp.XXXXXX" 2>/dev/null)" || return 0
- if cp -R "$wt/node_modules" "$tmp/node_modules" 2>/dev/null && : >"$tmp/.complete"; then
- # mv onto an existing directory would nest tmp inside it, so
- # publish only when absent, then clean any nesting from a lost race.
- if [ -e "$entry" ]; then
- rm -rf "$tmp"
- else
- mv "$tmp" "$entry" 2>/dev/null
- rm -rf "${entry:?}/${tmp##*/}" "$tmp" 2>/dev/null
- fi
- # ponytail: keep the 3 newest entries by mtime; no LRU touch on hit
- for old in $(ls -t "$cache_root" 2>/dev/null | tail -n +4); do
- aside="$cache_root/.tmp.evict.$old.$$"
- mv "$cache_root/$old" "$aside" 2>/dev/null && rm -rf "$aside"
- done
- else
- rm -rf "$tmp"
- fi
- fi
- return 0
- }
- # copy .loki into a worktree without the install cache (the cache is
- # often hundreds of MB; copy-then-delete would cost more than it saves).
- # Children are copied one by one, so no tar/rsync flavour issues (bash 3.2 safe).
- _loki_copy_state_no_install_cache() {
- local src="$1" dst="$2" e c
- mkdir -p "$dst" || return 1
- for e in "$src"/* "$src"/.[!.]*; do
- [ -e "$e" ] || [ -L "$e" ] || continue
- if [ "${e##*/}" = "cache" ] && [ -d "$e" ] && [ ! -L "$e" ]; then
- mkdir -p "$dst/cache"
- for c in "$e"/* "$e"/.[!.]*; do
- [ -e "$c" ] || [ -L "$c" ] || continue
- [ "${c##*/}" = "install" ] && continue
- cp -R "$c" "$dst/cache/" 2>/dev/null || true
- done
- else
- cp -R "$e" "$dst/" 2>/dev/null || true
- fi
- done
- return 0
- }
- # Create a worktree for a specific stream
- create_worktree() {
- local stream_name="$1"
- local branch_name="${2:-}"
- local project_name=$(basename "$TARGET_DIR")
- local worktree_path="${TARGET_DIR}/../${project_name}-${stream_name}"
- if [ -d "$worktree_path" ]; then
- log_info "Worktree already exists: $stream_name"
- WORKTREE_PATHS[$stream_name]="$worktree_path"
- if [ "${LOKI_EXEC_MANIFEST:-0}" = "1" ]; then
- WORKTREE_BASE_SHAS[$stream_name]="$(python3 -c "import json; print(json.load(open('${TARGET_DIR}/.loki/manifest/exec-manifest.json'))['base_sha'])" 2>/dev/null)"
- fi
- return 0
- fi
- log_step "Creating worktree: $stream_name"
- local wt_exit=1
- if [ -n "$branch_name" ]; then
- # Create new branch
- git -C "$TARGET_DIR" worktree add "$worktree_path" -b "$branch_name" 2>/dev/null && wt_exit=0 || \
- { git -C "$TARGET_DIR" worktree add "$worktree_path" "$branch_name" 2>/dev/null && wt_exit=0; }
- else
- # BUG-PAR-001: Testing/docs worktrees use -b parallel-<stream> main (not bare main checkout)
- # This avoids "already checked out" errors and keeps each worktree on its own branch
- git -C "$TARGET_DIR" worktree add "$worktree_path" -b "parallel-${stream_name}" main 2>/dev/null && wt_exit=0 || \
- { git -C "$TARGET_DIR" worktree add "$worktree_path" "parallel-${stream_name}" 2>/dev/null && wt_exit=0; } || \
- { git -C "$TARGET_DIR" worktree add "$worktree_path" HEAD 2>/dev/null && wt_exit=0; }
- fi
- if [ $wt_exit -eq 0 ]; then
- WORKTREE_PATHS[$stream_name]="$worktree_path"
- WORKTREE_BASE_SHAS[$stream_name]="$(git -C "$worktree_path" rev-parse HEAD 2>/dev/null)"
- # Copy .loki state to worktree
- if [ -d "$TARGET_DIR/.loki" ]; then
- # never copy the install cache into the worktree
- _loki_copy_state_no_install_cache "$TARGET_DIR/.loki" "$worktree_path/.loki"
- fi
- # Initialize environment (detect and run appropriate install)
- (
- cd "$worktree_path" || exit 1
- if [ -f "package.json" ]; then
- _loki_npm_install_cached "${TARGET_DIR}/.loki/cache/install" "$worktree_path"
- elif [ -f "requirements.txt" ]; then
- pip install -r requirements.txt -q 2>/dev/null || true
- elif [ -f "Cargo.toml" ]; then
- cargo build --quiet 2>/dev/null || true
- fi
- ) &
- # Capture install PID for cleanup on exit
- WORKTREE_INSTALL_PIDS+=($!)
- register_pid "$!" "worktree-install" "stream=$stream_name"
- log_info "Created worktree: $worktree_path"
- return 0
- else
- log_error "Failed to create worktree: $stream_name"
- # BUG-PU-001: Clean up partial worktree on creation failure
- if [ -d "$worktree_path" ]; then
- git -C "$TARGET_DIR" worktree remove "$worktree_path" --force 2>/dev/null || \
- rm -rf "$worktree_path" 2>/dev/null || true
- fi
- # Clean up any orphaned branch created during the attempt
- if [ -n "$branch_name" ]; then
- git -C "$TARGET_DIR" branch -D "$branch_name" 2>/dev/null || true
- else
- git -C "$TARGET_DIR" branch -D "parallel-${stream_name}" 2>/dev/null || true
- fi
- return 1
- fi
- }
- # Remove a worktree
- remove_worktree() {
- local stream_name="$1"
- local worktree_path="${WORKTREE_PATHS[$stream_name]:-}"
- if [ -z "$worktree_path" ] || [ ! -d "$worktree_path" ]; then
- return 0
- fi
- log_step "Removing worktree: $stream_name"
- # Kill any running Claude session
- local pid="${WORKTREE_PIDS[$stream_name]:-}"
- if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
- kill "$pid" 2>/dev/null || true
- wait "$pid" 2>/dev/null || true
- fi
- # Remove worktree (with safety check for rm -rf)
- git -C "$TARGET_DIR" worktree remove "$worktree_path" --force 2>/dev/null || {
- # BUG-PAR-005: Safety check uses dirname with trailing / to prevent prefix-match false positives
- # e.g. TARGET_DIR=/foo/bar must not match /foo/bar-other
- local parent_dir
- parent_dir="$(dirname "$TARGET_DIR")/"
- if [[ -n "$worktree_path" && "$worktree_path" != "/" && "$worktree_path" == "${parent_dir}"* ]]; then
- rm -rf "$worktree_path" 2>/dev/null
- else
- log_warn "Skipping unsafe rm -rf for path: $worktree_path"
- fi
- }
- unset "WORKTREE_PATHS[$stream_name]"
- unset "WORKTREE_PIDS[$stream_name]"
- unset "WORKTREE_BASE_SHAS[$stream_name]"
- log_info "Removed worktree: $stream_name"
- }
- # Compute the effective parallel-session cap for the current scheduling pass.
- # Default-off contract: when LOKI_DYNAMIC_CONCURRENCY is not "1" this echoes
- # exactly MAX_PARALLEL_SESSIONS with zero file reads and zero subprocesses, so
- # the spawn decision is byte-identical to the pre-feature behavior.
- # When enabled, it starts from the configured ceiling and scales DOWN based on
- # .loki/state/resources.json. All reads are best-effort: a missing, empty, or
- # unparseable file (or non-numeric values) leaves the cap at the ceiling. The
- # result is always clamped to the range [1, ceiling] and never exceeds it.
- effective_session_cap() {
- # Fast default-off path: identical to today, no I/O, no subprocesses.
- if [ "${DYNAMIC_CONCURRENCY:-0}" != "1" ]; then
- echo "$MAX_PARALLEL_SESSIONS"
- return 0
- fi
- # Ceiling is the upper bound when dynamic scaling is on.
- local ceiling="${MAX_PARALLEL_SESSIONS_CEILING:-$MAX_PARALLEL_SESSIONS}"
- # Guard against a non-numeric or sub-1 ceiling override.
- case "$ceiling" in
- ''|*[!0-9]*) ceiling="$MAX_PARALLEL_SESSIONS" ;;
- esac
- [ "$ceiling" -lt 1 ] 2>/dev/null && ceiling=1
- local cap="$ceiling"
- local resources_file=".loki/state/resources.json"
- # No resource data -> best-effort, leave at ceiling.
- if [ ! -f "$resources_file" ]; then
- echo "$cap"
- return 0
- fi
- # Read usage and status best-effort. Defaults keep the cap at the ceiling
- # if the file is empty, malformed, or missing keys.
- local cpu_usage mem_usage status
- cpu_usage=$(python3 -c "import json; print(json.load(open('$resources_file')).get('cpu', {}).get('usage_percent', 0))" 2>/dev/null || echo "0")
- mem_usage=$(python3 -c "import json; print(json.load(open('$resources_file')).get('memory', {}).get('usage_percent', 0))" 2>/dev/null || echo "0")
- status=$(python3 -c "import json; print(json.load(open('$resources_file')).get('overall_status', 'ok'))" 2>/dev/null || echo "ok")
- # usage_percent can be a float (e.g. 85.3). Reduce to an integer part for
- # comparison and fall back to 0 if anything is non-numeric.
- cpu_usage="${cpu_usage%%.*}"
- mem_usage="${mem_usage%%.*}"
- case "$cpu_usage" in ''|*[!0-9]*) cpu_usage=0 ;; esac
- case "$mem_usage" in ''|*[!0-9]*) mem_usage=0 ;; esac
- local crit="${CONCURRENCY_CRITICAL_THRESHOLD:-95}"
- local cpu_thr="${CONCURRENCY_CPU_THRESHOLD:-85}"
- local mem_thr="${CONCURRENCY_MEM_THRESHOLD:-85}"
- if [ "$cpu_usage" -ge "$crit" ] || [ "$mem_usage" -ge "$crit" ]; then
- # Critical pressure: drop to a single session.
- cap=1
- elif [ "$cpu_usage" -ge "$cpu_thr" ] || [ "$mem_usage" -ge "$mem_thr" ] || [ "$status" != "ok" ]; then
- # Elevated pressure or a non-ok overall status: halve (integer floor).
- cap=$(( ceiling / 2 ))
- fi
- # Clamp to [1, ceiling]. Never runaway, never zero.
- [ "$cap" -lt 1 ] && cap=1
- [ "$cap" -gt "$ceiling" ] && cap="$ceiling"
- echo "$cap"
- return 0
- }
- # Auto-flag parity for parallel worktree Claude sessions (wave-5 fix).
- # The main RARV loop applies adaptive cost/resilience flags to its claude
- # invocation (run.sh:16007-16030 effort/max-budget/fallback) plus an MCP
- # bundle, but the parallel worktree spawn shipped a bare invocation that missed
- # them entirely. This populates the global _LOKI_WT_AUTO_FLAGS array with ONLY
- # the flags that are safe on the plain (non stream-json) worktree invocation:
- # --effort adaptive reasoning depth (dev tier)
- # --max-budget-usd per-call hard backstop
- # --fallback-model resilience to model overload/unavailability
- # --mcp-config (+ --strict-mcp-config) the same MCP bundle the Bun
- # route emits; a SUPERSET of the bash main loop, which does
- # NOT emit --mcp-config (see ledger run.sh:15990-15996).
- # Worktree dev streams benefit from the bundled MCP servers.
- # We deliberately do NOT replicate the stream-json-coupled flags
- # (--include-hook-events, --include-partial-messages) nor --output-format /
- # --session-id: this invocation logs free-form text, not parsed stream-json, so
- # those would be invalid or unwanted here. Each flag is gated on CLI support +
- # its opt-out env var, matching the main loop. Extracted from spawn_worktree_session
- # so it is unit-testable (tests/test-worktree-auto-flags.sh).
- _loki_build_worktree_claude_flags() {
- _LOKI_WT_AUTO_FLAGS=()
- # Non-claude providers never receive these flags.
- if [ "${PROVIDER_NAME:-claude}" != "claude" ]; then
- return 0
- fi
- # Dev-tier model param drives the fallback derivation (worktree streams are
- # development work). Resolve via the provider helper when available.
- local _loki_wt_primary=""
- if type provider_get_tier_param >/dev/null 2>&1; then
- _loki_wt_primary="$(provider_get_tier_param development 2>/dev/null || true)"
- fi
- if [ "${LOKI_AUTO_EFFORT:-on}" != "off" ] \
- && type loki_effort_for_tier >/dev/null 2>&1 \
- && type loki_claude_flag_supported >/dev/null 2>&1 \
- && loki_claude_flag_supported "--effort"; then
- local _loki_wt_effort
- _loki_wt_effort="$(loki_effort_for_tier development "${DETECTED_COMPLEXITY:-${LOKI_COMPLEXITY:-standard}}")"
- [ -n "$_loki_wt_effort" ] && _LOKI_WT_AUTO_FLAGS+=("--effort" "$_loki_wt_effort")
- fi
- if [ "${LOKI_AUTO_BUDGET:-on}" != "off" ] \
- && type loki_remaining_budget >/dev/null 2>&1 \
- && type loki_claude_flag_supported >/dev/null 2>&1 \
- && loki_claude_flag_supported "--max-budget-usd"; then
- local _loki_wt_rem
- _loki_wt_rem="$(loki_remaining_budget)"
- [ -n "$_loki_wt_rem" ] && _LOKI_WT_AUTO_FLAGS+=("--max-budget-usd" "$_loki_wt_rem")
- fi
- if [ "${LOKI_AUTO_FALLBACK:-on}" != "off" ] \
- && [ -n "$_loki_wt_primary" ] \
- && type loki_fallback_for_primary >/dev/null 2>&1 \
- && type loki_claude_flag_supported >/dev/null 2>&1 \
- && loki_claude_flag_supported "--fallback-model"; then
- local _loki_wt_fb
- _loki_wt_fb="$(loki_fallback_for_primary "$_loki_wt_primary")"
- [ -n "$_loki_wt_fb" ] && _LOKI_WT_AUTO_FLAGS+=("--fallback-model" "$_loki_wt_fb")
- fi
- if type loki_mcp_config_argv >/dev/null 2>&1 \
- && type loki_claude_flag_supported >/dev/null 2>&1 \
- && loki_claude_flag_supported "--mcp-config"; then
- local _loki_wt_mcp
- if _loki_wt_mcp="$(loki_mcp_config_argv)" && [ -n "$_loki_wt_mcp" ]; then
- _LOKI_WT_AUTO_FLAGS+=("--mcp-config")
- local _loki_wt_mcp_path
- for _loki_wt_mcp_path in $_loki_wt_mcp; do
- _LOKI_WT_AUTO_FLAGS+=("$_loki_wt_mcp_path")
- done
- # --strict-mcp-config only alongside a real bundle, never bare.
- if [ "${LOKI_STRICT_MCP:-1}" != "0" ] \
- && loki_claude_flag_supported "--strict-mcp-config"; then
- _LOKI_WT_AUTO_FLAGS+=("--strict-mcp-config")
- fi
- fi
- fi
- return 0
- }
- # Spawn a Claude session in a worktree
- spawn_worktree_session() {
- local stream_name="$1"
- local task_prompt="$2"
- local worktree_path="${WORKTREE_PATHS[$stream_name]:-}"
- if [ -z "$worktree_path" ] || [ ! -d "$worktree_path" ]; then
- log_error "Worktree not found: $stream_name"
- return 1
- fi
- # Check if session limit reached
- local active_count=0
- for pid in "${WORKTREE_PIDS[@]}"; do
- if kill -0 "$pid" 2>/dev/null; then
- ((active_count++))
- fi
- done
- local session_cap
- session_cap=$(effective_session_cap)
- if [ "$active_count" -ge "$session_cap" ]; then
- # BUG-PAR-014: Max-sessions rejection queues spawn for retry
- log_warn "Max parallel sessions reached ($session_cap). Queuing $stream_name for retry."
- mkdir -p "${TARGET_DIR:-.}/.loki/signals"
- echo "{\"stream\":\"$stream_name\",\"task\":\"$(echo "$task_prompt" | head -c 200)\",\"timestamp\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}" \
- > "${TARGET_DIR:-.}/.loki/signals/SPAWN_QUEUED_${stream_name}"
- return 1
- fi
- local log_file="$worktree_path/.loki/logs/session-${stream_name}.log"
- mkdir -p "$(dirname "$log_file")"
- # Check provider parallel support
- if [ "${PROVIDER_HAS_PARALLEL:-false}" != "true" ]; then
- log_warn "Provider ${PROVIDER_NAME:-unknown} does not support parallel sessions"
- log_warn "Running sequentially instead (degraded mode)"
- return 1
- fi
- log_step "Spawning ${PROVIDER_DISPLAY_NAME:-Claude} session: $stream_name"
- # Build the worktree auto-flag set (effort/max-budget/fallback/mcp-config)
- # BEFORE the ( subshell so it is inherited (a `bash -c` would not). Populates
- # the global _LOKI_WT_AUTO_FLAGS array. See _loki_build_worktree_claude_flags.
- _loki_build_worktree_claude_flags
- (
- cd "$worktree_path" || exit 1
- _wt_exit=0
- # Provider-specific invocation for parallel sessions
- case "${PROVIDER_NAME:-claude}" in
- claude)
- # caveman ACTIVATION (free-form): a parallel worktree dev stream
- # is free-form generation; its output goes to a log, not a parsed
- # sentinel. Activate compression at the configured level when
- # warranted (empty -> bare invocation, byte-identical to before).
- # Type-guarded; inlined per-invocation (not exported).
- local _loki_wt_cm=""
- if type loki_caveman_activate_env >/dev/null 2>&1; then
- _loki_wt_cm="$(loki_caveman_activate_env)"
- fi
- # Expand the auto-flag array with the bash-3.2 empty-array guard
- # (${arr[@]+...}) so a bare "${arr[@]}" under set -u does not
- # abort with "unbound variable" when no flags were collected.
- _LOKI_PERM_ARGV=(--dangerously-skip-permissions)
- ! type _loki_claude_permission_argv >/dev/null 2>&1 || _loki_claude_permission_argv
- if [ -n "$_loki_wt_cm" ]; then
- SHELL="$(_loki_claude_shell_value)" CAVEMAN_DEFAULT_MODE="$_loki_wt_cm" claude "${_LOKI_PERM_ARGV[@]}" \
- "${_LOKI_WT_AUTO_FLAGS[@]+"${_LOKI_WT_AUTO_FLAGS[@]}"}" \
- -p "Loki Mode: $task_prompt. Read .loki/CONTINUITY.md for context." \
- >> "$log_file" 2>&1 || _wt_exit=$?
- else
- SHELL="$(_loki_claude_shell_value)" claude "${_LOKI_PERM_ARGV[@]}" \
- "${_LOKI_WT_AUTO_FLAGS[@]+"${_LOKI_WT_AUTO_FLAGS[@]}"}" \
- -p "Loki Mode: $task_prompt. Read .loki/CONTINUITY.md for context." \
- >> "$log_file" 2>&1 || _wt_exit=$?
- fi
- ;;
- *)
- log_error "Unknown provider: ${PROVIDER_NAME}"
- _wt_exit=1
- ;;
- esac
- # Completion signaling (v6.7.0)
- if [ $_wt_exit -eq 0 ]; then
- # BUG-PAR-006: git add excludes .env, *.key, *.pem, credentials*
- git -C "$worktree_path" add -A \
- ':!.env' ':!*.key' ':!*.pem' ':!credentials*' 2>/dev/null
- git -C "$worktree_path" commit -m "feat($stream_name): worktree work complete" 2>/dev/null || true
- # BUG-PAR-008: Signal files written atomically (temp + mv)
- mkdir -p "${TARGET_DIR:-.}/.loki/signals"
- local _sig_tmp
- _sig_tmp=$(mktemp "${TARGET_DIR:-.}/.loki/signals/.tmp.XXXXXX") || true
- cat > "$_sig_tmp" <<EOSIG
- {"stream":"$stream_name","branch":"$(git -C "$worktree_path" branch --show-current 2>/dev/null)","worktree":"$worktree_path","timestamp":"$(date -u +%Y-%m-%dT%H:%M:%SZ)","exit_code":$_wt_exit}
- EOSIG
- mv "$_sig_tmp" "${TARGET_DIR:-.}/.loki/signals/MERGE_REQUESTED_${stream_name}" 2>/dev/null || \
- cp "$_sig_tmp" "${TARGET_DIR:-.}/.loki/signals/MERGE_REQUESTED_${stream_name}" 2>/dev/null
- rm -f "$_sig_tmp" 2>/dev/null
- echo "WORKTREE_COMPLETE: $stream_name" >> "$log_file"
- else
- # BUG-PAR-008: Signal files written atomically (temp + mv)
- mkdir -p "${TARGET_DIR:-.}/.loki/signals"
- local _fail_tmp
- _fail_tmp=$(mktemp "${TARGET_DIR:-.}/.loki/signals/.tmp.XXXXXX") || true
- echo "{\"stream\":\"$stream_name\",\"status\":\"failed\",\"exit_code\":$_wt_exit,\"timestamp\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}" \
- > "$_fail_tmp"
- mv "$_fail_tmp" "${TARGET_DIR:-.}/.loki/signals/WORKTREE_FAILED_${stream_name}" 2>/dev/null || \
- cp "$_fail_tmp" "${TARGET_DIR:-.}/.loki/signals/WORKTREE_FAILED_${stream_name}" 2>/dev/null
- rm -f "$_fail_tmp" 2>/dev/null
- fi
- ) &
- local pid=$!
- WORKTREE_PIDS[$stream_name]=$pid
- register_pid "$pid" "worktree-session" "stream=$stream_name"
- log_info "Session spawned: $stream_name (PID: $pid)"
- return 0
- }
- # Merge a completed worktree back to main branch (v6.7.0)
- # Usage: merge_worktree <stream_name>
- merge_worktree() {
- local stream_name="$1"
- local signal_file="${TARGET_DIR:-.}/.loki/signals/MERGE_REQUESTED_${stream_name}"
- if [ ! -f "$signal_file" ]; then
- log_error "No merge signal found for: $stream_name"
- return 1
- fi
- # BUG-PAR-013: Signal file parsing falls back to jq when python3 unavailable
- local branch worktree_path
- branch=$(python3 -c "import json; print(json.load(open('$signal_file'))['branch'])" 2>/dev/null) || \
- branch=$(jq -r '.branch' "$signal_file" 2>/dev/null) || true
- worktree_path=$(python3 -c "import json; print(json.load(open('$signal_file'))['worktree'])" 2>/dev/null) || \
- worktree_path=$(jq -r '.worktree' "$signal_file" 2>/dev/null) || true
- if [ -z "$branch" ]; then
- log_error "Could not determine branch for: $stream_name"
- return 1
- fi
- log_step "Merging worktree: $stream_name (branch: $branch)"
- # Validate the actual branch result at the integration seam, immediately
- # before any checkout or merge can mutate the integration tree.
- if ! validate_exec_manifest_result "$stream_name" "$branch"; then
- return 1
- fi
- # BUG-PAR-009: Verify git checkout main before merge
- local current_branch
- current_branch=$(git -C "${TARGET_DIR:-.}" branch --show-current 2>/dev/null)
- if [ "$current_branch" != "main" ]; then
- log_info "Switching to main before merge (was on: $current_branch)"
- if ! git -C "${TARGET_DIR:-.}" checkout main 2>/dev/null; then
- log_error "Failed to checkout main for merge: $stream_name"
- return 1
- fi
- fi
- if git -C "${TARGET_DIR:-.}" merge --no-ff "$branch" -m "merge($stream_name): auto-merge from parallel worktree" 2>&1; then
- log_info "Merge successful: $stream_name"
- # Clean up signal and worktree
- rm -f "$signal_file"
- if [ -n "$worktree_path" ] && [ -d "$worktree_path" ]; then
- git -C "${TARGET_DIR:-.}" worktree remove "$worktree_path" --force 2>/dev/null || true
- git -C "${TARGET_DIR:-.}" branch -d "$branch" 2>/dev/null || true
- fi
- return 0
- else
- log_error "Merge conflict for $stream_name - manual resolution needed"
- git -C "${TARGET_DIR:-.}" merge --abort 2>/dev/null || true
- return 1
- fi
- }
- # Check and process all pending merge signals (v6.7.0)
- process_pending_merges() {
- local signals_dir="${TARGET_DIR:-.}/.loki/signals"
- local merged=0
- local failed=0
- for signal_file in "$signals_dir"/MERGE_REQUESTED_*; do
- case "$signal_file" in *.ack) continue ;; esac
- [ -f "$signal_file" ] || continue
- local stream_name
- stream_name=$(basename "$signal_file" | sed 's/MERGE_REQUESTED_//')
- if merge_worktree "$stream_name"; then
- ((merged++))
- else
- ((failed++))
- fi
- done
- if [ $merged -gt 0 ] || [ $failed -gt 0 ]; then
- log_info "Merge results: $merged successful, $failed failed"
- fi
- }
- # List all active worktrees
- list_worktrees() {
- log_header "Active Worktrees"
- git -C "$TARGET_DIR" worktree list 2>/dev/null
- echo ""
- log_info "Tracked sessions:"
- for stream in "${!WORKTREE_PIDS[@]}"; do
- local pid="${WORKTREE_PIDS[$stream]}"
- local status="stopped"
- if kill -0 "$pid" 2>/dev/null; then
- status="running"
- fi
- echo " [$stream] PID: $pid - $status"
- done
- }
- # Check for completed features ready to merge
- check_merge_queue() {
- local signals_dir="$TARGET_DIR/.loki/signals"
- if [ ! -d "$signals_dir" ]; then
- return 0
- fi
- for signal in "$signals_dir"/MERGE_REQUESTED_*; do
- # Skip our own acknowledgement markers (*.ack) so they are not treated
- # as signals themselves.
- case "$signal" in *.ack) continue ;; esac
- if [ -f "$signal" ]; then
- local feature=$(basename "$signal" | sed 's/MERGE_REQUESTED_//')
- if [ "$AUTO_MERGE" = "true" ]; then
- # Auto-merge consumes the signal (merge_feature rm's it), so the
- # log line fires exactly once per completed feature.
- log_info "Merge requested: $feature"
- merge_feature "$feature"
- else
- # Auto-merge is OFF (e.g. `--pr` without `--ship`): the PR is left
- # for a human to merge, so the signal is INTENTIONALLY not consumed.
- # Log ONCE (guarded by a sibling .ack marker) instead of re-logging
- # the same "Merge requested" line every orchestrator pass -- that
- # re-log was a 90-minute "looks stuck" spam while real work was
- # already done. The signal itself is preserved untouched.
- if [ ! -f "${signal}.ack" ]; then
- log_info "Merge requested: $feature (PR ready; auto-merge off -- merge the PR, or use --ship to auto-merge)"
- : > "${signal}.ack" 2>/dev/null || true
- fi
- fi
- fi
- done
- }
- # AI-powered conflict resolution (inspired by Auto-Claude)
- resolve_conflicts_with_ai() {
- local feature="$1"
- local conflict_files=$(git diff --name-only --diff-filter=U 2>/dev/null)
- if [ -z "$conflict_files" ]; then
- return 0
- fi
- log_step "AI-powered conflict resolution for: $feature"
- for file in $conflict_files; do
- log_info "Resolving conflicts in: $file"
- # Get conflict markers
- local conflict_content=$(cat "$file")
- # Use AI to resolve conflict (provider-aware)
- local resolution=""
- local conflict_prompt="You are resolving a git merge conflict. The file below contains conflict markers.
- Your task is to merge both changes intelligently, preserving functionality from both sides.
- FILE: $file
- CONTENT:
- $conflict_content
- Output ONLY the resolved file content with no conflict markers. No explanations."
- case "${PROVIDER_NAME:-claude}" in
- claude)
- # EMBED 2 (v7.33.0): --bare on this cheap NON-MAIN subcall.
- # Reasoning: $conflict_prompt is fully self-contained -- it
- # carries the complete instruction set AND the entire conflicted
- # file content inline, and the agent's output is captured to a
- # variable (the shell, not the agent, writes the resolved file).
- # It needs no hooks, LSP, CLAUDE.md auto-discovery, or MCP, so
- # --bare is safe and cheaper. Gated + opt-out LOKI_BARE_SUBCALLS=0.
- _LOKI_PERM_ARGV=(--dangerously-skip-permissions)
- ! type _loki_claude_permission_argv >/dev/null 2>&1 || _loki_claude_permission_argv
- local _cr_argv=("${_LOKI_PERM_ARGV[@]}")
- if type loki_subcall_bare_enabled >/dev/null 2>&1 && loki_subcall_bare_enabled; then
- _cr_argv+=("--bare")
- fi
- # caveman HARD-SUPPRESS (parsed output): the output is captured as
- # the EXACT resolved file content (the shell writes it verbatim).
- # Compressing prose into the merged source would corrupt the file,
- # so disable caveman unconditionally here. No-op when absent.
- resolution=$(SHELL="$(_loki_claude_shell_value)" CAVEMAN_DEFAULT_MODE=off claude "${_cr_argv[@]}" -p "$conflict_prompt" --output-format text < /dev/null 2>/dev/null)
- ;;
- *)
- log_error "Unknown provider: ${PROVIDER_NAME}"
- return 1
- ;;
- esac
- if [ -n "$resolution" ]; then
- echo "$resolution" > "$file"
- git add "$file"
- log_info "Resolved: $file"
- else
- log_error "AI resolution failed for: $file"
- return 1
- fi
- done
- return 0
- }
- # Merge a completed feature branch (with AI conflict resolution)
- # BUG-PAR-011: Not in a subshell -- uses git -C instead of cd
- # BUG-PAR-003: Strips feature- prefix to avoid feature/feature-auth double-prefix
- merge_feature() {
- local feature="$1"
- # BUG-PAR-003: Strip feature- prefix if present to avoid double-prefix (feature/feature-auth)
- local clean_feature="${feature#feature-}"
- local branch="feature/$clean_feature"
- # The session signal is authoritative for worktree branches (for example
- # parallel-testing). The historical feature/<name> convention remains the
- # fallback for legacy feature streams.
- local _mf_signal="$TARGET_DIR/.loki/signals/MERGE_REQUESTED_$feature"
- if [ -f "$_mf_signal" ]; then
- local _mf_branch=""
- _mf_branch=$(LOKI_SIGNAL_FILE="$_mf_signal" python3 -c \
- "import json,os; print(json.load(open(os.environ['LOKI_SIGNAL_FILE'])).get('branch',''))" 2>/dev/null || true)
- [ -n "$_mf_branch" ] && branch="$_mf_branch"
- fi
- log_step "Merging feature: $clean_feature"
- # This is the autonomous orchestrator's real integration seam. Reject a
- # stale, out-of-scope, unknown, or unsuccessful stream before checkout/merge.
- if ! validate_exec_manifest_result "$feature" "$branch"; then
- return 1
- fi
- # BUG-PAR-011: Ensure we're on main using git -C (no subshell)
- git -C "$TARGET_DIR" checkout main 2>/dev/null
- # Attempt merge with no-ff for clear history
- if git -C "$TARGET_DIR" merge "$branch" --no-ff -m "feat: Merge $clean_feature" 2>/dev/null; then
- log_info "Merged cleanly: $clean_feature"
- else
- # Merge has conflicts - try AI resolution
- log_warn "Merge conflicts detected - attempting AI resolution"
- if resolve_conflicts_with_ai "$clean_feature"; then
- # AI resolved conflicts, commit the merge
- git -C "$TARGET_DIR" commit -m "feat: Merge $clean_feature (AI-resolved conflicts)"
- audit_agent_action "git_commit" "Committed changes" "merge=$clean_feature,resolution=ai"
- log_info "Merged with AI conflict resolution: $clean_feature"
- else
- # AI resolution failed, abort merge
- log_error "AI conflict resolution failed: $clean_feature"
- git -C "$TARGET_DIR" merge --abort 2>/dev/null || true
- return 1
- fi
- fi
- # Remove signal (and its log-once .ack marker, if any)
- rm -f "$TARGET_DIR/.loki/signals/MERGE_REQUESTED_$feature" \
- "$TARGET_DIR/.loki/signals/MERGE_REQUESTED_$feature.ack"
- # Remove worktree
- remove_worktree "feature-$clean_feature"
- # Delete branch
- git -C "$TARGET_DIR" branch -d "$branch" 2>/dev/null || true
- # DOCS_NEEDED signal: triggers the parallel docs worktree to run `loki docs update`
- # and regenerate documentation for recently changed files.
- mkdir -p "$TARGET_DIR/.loki/signals"
- touch "$TARGET_DIR/.loki/signals/DOCS_NEEDED"
- }
- # Initialize parallel workflow streams
- init_parallel_streams() {
- # Check bash version
- if ! check_parallel_support; then
- return 1
- fi
- log_header "Initializing Parallel Workflows"
- if ! init_exec_manifest; then
- log_error "Failed to initialize execution manifest"
- return 1
- fi
- local active_streams=0
- # Create testing worktree (always tracks main)
- if [ "$PARALLEL_TESTING" = "true" ]; then
- create_worktree "testing"
- ((active_streams++))
- fi
- # Create documentation worktree
- if [ "$PARALLEL_DOCS" = "true" ]; then
- create_worktree "docs"
- ((active_streams++))
- fi
- # Create blog worktree if enabled
- if [ "$PARALLEL_BLOG" = "true" ]; then
- create_worktree "blog"
- ((active_streams++))
- fi
- log_info "Initialized $active_streams parallel streams"
- list_worktrees
- }
- # Spawn feature worktree from task
- # Cleanup all worktrees on exit
- cleanup_parallel_streams() {
- log_header "Cleaning Up Parallel Streams"
- # Kill background install processes
- for pid in "${WORKTREE_INSTALL_PIDS[@]}"; do
- if kill -0 "$pid" 2>/dev/null; then
- kill "$pid" 2>/dev/null || true
- fi
- unregister_pid "$pid"
- done
- WORKTREE_INSTALL_PIDS=()
- # Kill all sessions
- for stream in "${!WORKTREE_PIDS[@]}"; do
- local pid="${WORKTREE_PIDS[$stream]}"
- if kill -0 "$pid" 2>/dev/null; then
- log_step "Stopping session: $stream"
- kill "$pid" 2>/dev/null || true
- fi
- unregister_pid "$pid"
- done
- # Wait for all to finish
- wait 2>/dev/null || true
- # Optionally remove worktrees (keep by default for inspection)
- # Uncomment to auto-cleanup:
- # for stream in "${!WORKTREE_PATHS[@]}"; do
- # remove_worktree "$stream"
- # done
- log_info "Parallel streams stopped"
- }
- # Orchestrator loop for parallel mode
- run_parallel_orchestrator() {
- log_header "Parallel Orchestrator Started"
- # Initialize streams - exit if bash version is too old
- if ! init_parallel_streams; then
- log_error "Failed to initialize parallel streams"
- log_error "Falling back to sequential mode"
- PARALLEL_MODE=false
- return 1
- fi
- # Spawn testing session
- if [ "$PARALLEL_TESTING" = "true" ] && [ -n "${WORKTREE_PATHS[testing]:-}" ]; then
- spawn_worktree_session "testing" "Run all tests continuously. Watch for changes. Report failures to .loki/state/test-results.json"
- fi
- # Spawn docs session
- if [ "$PARALLEL_DOCS" = "true" ] && [ -n "${WORKTREE_PATHS[docs]:-}" ]; then
- spawn_worktree_session "docs" "Documentation maintenance stream. Steps: 1) Run 'loki docs generate' if .loki/docs/ does not exist. 2) Watch for .loki/signals/DOCS_NEEDED file. When found, run 'loki docs update' and remove the signal file. 3) After each doc update, run 'loki docs check' and report coverage. 4) Focus on documenting new files, changed APIs, and architectural decisions."
- fi
- # Main orchestrator loop
- local running=true
- # BUG-PAR-004: Orchestrator trap handles SIGTERM properly (cleanup + restore global trap + exit)
- trap 'running=false; cleanup_parallel_streams; trap cleanup INT TERM; exit 0' TERM
- trap 'running=false; cleanup_parallel_streams' INT
- while $running; do
- # Check for merge requests
- check_merge_queue
- # BUG-PAR-014: Retry queued spawns when sessions free up
- local active_count=0
- for _qpid in "${WORKTREE_PIDS[@]}"; do
- if kill -0 "$_qpid" 2>/dev/null; then
- ((active_count++))
- fi
- done
- local _session_cap
- _session_cap=$(effective_session_cap)
- if [ "$active_count" -lt "$_session_cap" ]; then
- for queued_signal in "${TARGET_DIR:-.}"/.loki/signals/SPAWN_QUEUED_*; do
- [ -f "$queued_signal" ] || continue
- local queued_stream
- queued_stream=$(basename "$queued_signal" | sed 's/SPAWN_QUEUED_//')
- local queued_task=""
- queued_task=$(python3 -c "import json; print(json.load(open('$queued_signal'))['task'])" 2>/dev/null) || \
- queued_task=$(jq -r '.task' "$queued_signal" 2>/dev/null) || true
- if [ -n "$queued_task" ] && [ -n "${WORKTREE_PATHS[$queued_stream]:-}" ]; then
- rm -f "$queued_signal"
- spawn_worktree_session "$queued_stream" "$queued_task" && \
- log_info "Retried queued spawn: $queued_stream"
- fi
- done
- fi
- # Check session health
- for stream in "${!WORKTREE_PIDS[@]}"; do
- local pid="${WORKTREE_PIDS[$stream]}"
- if ! kill -0 "$pid" 2>/dev/null; then
- log_warn "Session ended: $stream"
- unset "WORKTREE_PIDS[$stream]"
- fi
- done
- # Update orchestrator state
- local state_file="$TARGET_DIR/.loki/state/parallel-streams.json"
- mkdir -p "$(dirname "$state_file")"
- # BUG-PAR-007: Empty worktree map produces valid JSON
- local worktree_json=""
- if [ ${#WORKTREE_PATHS[@]} -gt 0 ]; then
- worktree_json=$(for stream in "${!WORKTREE_PATHS[@]}"; do
- local path="${WORKTREE_PATHS[$stream]}"
- local pid="null"
- if [ -n "${WORKTREE_PIDS[$stream]+x}" ]; then
- pid="${WORKTREE_PIDS[$stream]}"
- fi
- local status="stopped"
- if [ "$pid" != "null" ] && kill -0 "$pid" 2>/dev/null; then
- status="running"
- fi
- echo " \"$stream\": {\"path\": \"$path\", \"pid\": $pid, \"status\": \"$status\"},"
- done | sed '$ s/,$//')
- fi
- cat > "$state_file" << EOF
- {
- "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
- "worktrees": {
- ${worktree_json}
- },
- "active_sessions": ${#WORKTREE_PIDS[@]},
- "max_sessions": $MAX_PARALLEL_SESSIONS
- }
- EOF
- sleep 30
- done
- }
- #===============================================================================
- # Prerequisites Check
- #===============================================================================
- check_prerequisites() {
- log_header "Checking Prerequisites"
- local missing=()
- # Check Provider CLI (uses PROVIDER_CLI from loaded provider config)
- local cli_name="${PROVIDER_CLI:-claude}"
- local display_name="${PROVIDER_DISPLAY_NAME:-Claude Code}"
- log_step "Checking $display_name CLI..."
- if command -v "$cli_name" &> /dev/null; then
- local version=$("$cli_name" --version 2>/dev/null | head -1 || echo "unknown")
- log_info "$display_name CLI: $version"
- else
- missing+=("$cli_name")
- log_error "$display_name CLI not found"
- case "$cli_name" in
- claude)
- log_info "Install: https://claude.ai/code or npm install -g @anthropic-ai/claude-code"
- ;;
- opencode)
- log_info "Install: npm install -g opencode-ai"
- ;;
- *)
- log_info "Install the $cli_name CLI for your provider"
- ;;
- esac
- fi
- # Check Python 3
- log_step "Checking Python 3..."
- if command -v python3 &> /dev/null; then
- local py_version=$(python3 --version 2>&1)
- log_info "Python: $py_version"
- else
- missing+=("python3")
- log_error "Python 3 not found"
- fi
- # Check Git
- log_step "Checking Git..."
- if command -v git &> /dev/null; then
- local git_version=$(git --version)
- log_info "Git: $git_version"
- else
- missing+=("git")
- log_error "Git not found"
- fi
- # Check Node.js (optional but recommended)
- log_step "Checking Node.js (optional)..."
- if command -v node &> /dev/null; then
- local node_version=""
- if node_version=$(node --version 2>/dev/null); then
- _LOKI_NODE_PROBE_STATUS="available"
- _LOKI_NODE_PROBE_VERSION="$node_version"
- log_info "Node.js: $node_version"
- else
- _LOKI_NODE_PROBE_STATUS="probe_failed"
- log_warn "Node.js executable found, but its version probe failed"
- fi
- else
- _LOKI_NODE_PROBE_STATUS="missing"
- log_warn "Node.js not found (optional, needed for some builds)"
- fi
- # Check npm (optional)
- if command -v npm &> /dev/null; then
- local npm_version=""
- if npm_version=$(npm --version 2>/dev/null); then
- _LOKI_NPM_PROBE_STATUS="available"
- _LOKI_NPM_PROBE_VERSION="$npm_version"
- log_info "npm: $npm_version"
- else
- _LOKI_NPM_PROBE_STATUS="probe_failed"
- log_warn "npm executable found, but its version probe failed"
- fi
- else
- _LOKI_NPM_PROBE_STATUS="missing"
- fi
- # Check curl (for web fetches)
- log_step "Checking curl..."
- if command -v curl &> /dev/null; then
- log_info "curl: available"
- else
- missing+=("curl")
- log_error "curl not found"
- fi
- # Check jq (optional but helpful)
- log_step "Checking jq (optional)..."
- if command -v jq &> /dev/null; then
- log_info "jq: available"
- else
- log_warn "jq not found (optional, for JSON parsing)"
- fi
- # Summary
- echo ""
- if [ ${#missing[@]} -gt 0 ]; then
- log_error "Missing required tools: ${missing[*]}"
- log_info "Please install the missing tools and try again."
- return 1
- else
- log_info "All required prerequisites are installed!"
- return 0
- fi
- }
- #===============================================================================
- # Skill Installation Check
- #===============================================================================
- check_skill_installed() {
- log_header "Checking Loki Mode Skill"
- # Build skill locations array dynamically based on provider
- local skill_locations=()
- # Add provider-specific skill directory if set (e.g., ~/.claude/skills for Claude)
- if [ -n "${PROVIDER_SKILL_DIR:-}" ]; then
- skill_locations+=("${PROVIDER_SKILL_DIR}/loki-mode/SKILL.md")
- fi
- # Add local project skill locations
- skill_locations+=(
- ".claude/skills/loki-mode/SKILL.md"
- "$PROJECT_DIR/SKILL.md"
- )
- for loc in "${skill_locations[@]}"; do
- if [ -f "$loc" ]; then
- log_info "Skill found: $loc"
- return 0
- fi
- done
- # For providers without a native skill directory, this is expected
- if [ -z "${PROVIDER_SKILL_DIR:-}" ]; then
- log_info "Provider ${PROVIDER_NAME:-unknown} has no native skill directory"
- log_info "Skill will be passed via prompt injection"
- else
- log_warn "Loki Mode skill not found in standard locations"
- fi
- log_info "The skill will be used from: $PROJECT_DIR/SKILL.md"
- if [ -f "$PROJECT_DIR/SKILL.md" ]; then
- log_info "Using skill from project directory"
- return 0
- else
- log_error "SKILL.md not found!"
- return 1
- fi
- }
- #===============================================================================
- # Initialize Loki Directory
- #===============================================================================
- init_loki_dir() {
- log_header "Initializing Loki Mode Directory"
- # Clean up stale control files ONLY if no other session is running.
- # Deleting these while another session is active would destroy its signals.
- #
- # v7.5.12: PID-liveness probe replaces flock-based "is the lock held?"
- # check. The mkdir-mutex used by safe_acquire_lock is not introspectable
- # the same way (no FD to non-blocking-poll), but the PID file is the
- # source of truth for liveness anyway -- a stale lockdir without a
- # live owner means the session is gone, so cleanup is safe.
- #
- # Per-session locking (v6.4.0): When LOKI_SESSION_ID is set, only clean up
- # that session's files. Global control files (PAUSE/STOP) are only cleaned
- # when NO sessions are active.
- local lock_file can_cleanup=false
- if [ -n "${LOKI_SESSION_ID:-}" ]; then
- # Per-session: PID-liveness probe
- lock_file=".loki/sessions/${LOKI_SESSION_ID}/session.lock"
- local session_pid_file=".loki/sessions/${LOKI_SESSION_ID}/loki.pid"
- local existing_pid=""
- if [ -f "$session_pid_file" ]; then
- existing_pid=$(cat "$session_pid_file" 2>/dev/null)
- fi
- if [ -z "$existing_pid" ] || ! kill -0 "$existing_pid" 2>/dev/null; then
- can_cleanup=true
- fi
- if [ "$can_cleanup" = "true" ]; then
- rm -f "$session_pid_file" 2>/dev/null
- rm -f "$lock_file" 2>/dev/null
- rm -rf "${lock_file}.lockdir" 2>/dev/null
- fi
- else
- # Global: PID-liveness probe
- lock_file=".loki/session.lock"
- local existing_pid=""
- if [ -f ".loki/loki.pid" ]; then
- existing_pid=$(cat ".loki/loki.pid" 2>/dev/null)
- fi
- if [ -z "$existing_pid" ] || ! kill -0 "$existing_pid" 2>/dev/null; then
- can_cleanup=true
- fi
- if [ "$can_cleanup" = "true" ]; then
- # v7.4.16: extended stale-signal cleanup. Pre-v7.4.16 only
- # PAUSE / STOP / HUMAN_INPUT.md were cleaned -- but
- # PAUSE_AT_CHECKPOINT, PAUSED.md, and COMPLETED were added
- # to the signal-file family later without updating this
- # cleanup. A stale PAUSE_AT_CHECKPOINT from a prior session
- # (created by Ctrl+C in checkpoint mode) caused fresh
- # `loki start` to pause immediately when PRD-driven mode
- # auto-switched to checkpoint. User-reported regression.
- rm -f .loki/PAUSE .loki/STOP .loki/HUMAN_INPUT.md 2>/dev/null
- rm -f .loki/PAUSE_AT_CHECKPOINT .loki/PAUSED.md .loki/COMPLETED 2>/dev/null
- rm -f .loki/loki.pid 2>/dev/null
- rm -f .loki/session.lock 2>/dev/null
- rm -rf .loki/session.lock.lockdir 2>/dev/null
- fi
- fi
- # crash.sh's disclosure sentinel can have left a regular FILE at .loki/config
- # (LOKI_DIR is the project .loki); mkdir -p of the config DIRECTORY then fails
- # with "File exists". Fold the file into the directory (back-compat sentinel only),
- # but ONLY when it holds nothing except that sentinel, and NEVER for ~/.loki (a run
- # started from $HOME): there config is the user's real settings FILE (telemetry opt-out).
- if [ -f .loki/config ] && [ ! -L .loki/config ] \
- && [ "$(cd .loki 2>/dev/null && pwd -P)" != "$(cd "${HOME:-/nonexistent}/.loki" 2>/dev/null && pwd -P)" ] \
- && [ -z "$(grep -v -e '^DISCLOSURE_SHOWN=true$' -e '^[[:space:]]*$' .loki/config 2>/dev/null)" ]; then
- mv .loki/config .loki/config.disclosure.tmp 2>/dev/null \
- && mkdir -p .loki/config 2>/dev/null \
- && mv .loki/config.disclosure.tmp .loki/config/disclosure 2>/dev/null
- fi
- # Retry once: a concurrent creator of the same dir can make the first mkdir -p
- # fail with "File exists" (seen on CI, leaked to the quiet quick terminal).
- mkdir -p .loki/{state,queue,messages,logs,config,prompts,artifacts,scripts} 2>/dev/null \
- || mkdir -p .loki/{state,queue,messages,logs,config,prompts,artifacts,scripts}
- mkdir -p .loki/queue
- mkdir -p .loki/state/checkpoints
- mkdir -p .loki/artifacts/{releases,reports,backups}
- mkdir -p .loki/memory/{ledgers,handoffs,learnings,episodic,semantic,skills}
- # metrics/rewards was created and emptied every run and NOTHING ever wrote
- # or read it: the only reference in the whole codebase was the rm -f below.
- # A directory that exists only to be deleted is not a feature, and
- # documenting it as one told the agent something false.
- mkdir -p .loki/metrics/efficiency
- # Clear stale metrics from previous sessions so loki metrics shows current run data (#75)
- rm -f .loki/metrics/efficiency/iteration-*.json 2>/dev/null || true
- # Same reasoning for the proof pointer (#211). .loki/state/last-proof-id.txt
- # is written by generate_proof_of_run and was never cleared, so a run that
- # died before generating a proof left the PREVIOUS run's id behind. Every
- # reader then resolved a receipt describing different work, and
- # the summary builder (build_completion_summary) would print it under the
- # literal heading "Evidence
- # Receipt (this run):". A receipt naming the wrong run is worse than no
- # receipt: absence reads as "no data", a stale one reads as evidence.
- rm -f .loki/state/last-proof-id.txt 2>/dev/null || true
- mkdir -p .loki/rules
- mkdir -p .loki/signals
- # BUG-XC-008: Initialize queue files only if missing or invalid JSON
- for queue in pending in-progress completed failed dead-letter; do
- local qfile=".loki/queue/${queue}.json"
- if [ ! -f "$qfile" ] || ! python3 -c "import json; json.load(open('$qfile'))" 2>/dev/null; then
- echo "[]" > "$qfile"
- fi
- done
- # Initialize orchestrator state if it doesn't exist
- if [ ! -f ".loki/state/orchestrator.json" ]; then
- cat > ".loki/state/orchestrator.json" << EOF
- {
- "version": "$(cat "$PROJECT_DIR/VERSION" 2>/dev/null || echo "2.2.0")",
- "currentPhase": "BOOTSTRAP",
- "startedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
- "agents": {},
- "metrics": {
- "tasksCompleted": 0,
- "tasksFailed": 0,
- "retries": 0
- }
- }
- EOF
- fi
- # Write pricing.json with provider-specific model rates
- _write_pricing_json
- # Write budget.json if a budget limit is configured
- if [ -n "$BUDGET_LIMIT" ]; then
- # Validate budget limit is numeric before writing JSON
- if ! echo "$BUDGET_LIMIT" | grep -qE '^[0-9]+(\.[0-9]+)?$'; then
- log_warn "Invalid BUDGET_LIMIT '$BUDGET_LIMIT', ignoring (no cap set)"
- BUDGET_LIMIT=""
- fi
- # Mirror the CLI guard: a non-positive cap (0/0.00) would make
- # check_budget_limit pause before any work runs. Treat it as "no cap"
- # rather than a silent pre-work pause.
- if [ -n "$BUDGET_LIMIT" ] && ! awk -v b="$BUDGET_LIMIT" 'BEGIN{exit !(b+0 > 0)}'; then
- log_warn "BUDGET_LIMIT '$BUDGET_LIMIT' is not greater than 0, ignoring (no cap set)"
- BUDGET_LIMIT=""
- fi
- fi
- if [ -n "$BUDGET_LIMIT" ]; then
- cat > ".loki/metrics/budget.json" << BUDGET_EOF
- {
- "limit": $BUDGET_LIMIT,
- "budget_limit": $BUDGET_LIMIT,
- "budget_used": 0,
- "created_at": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- }
- BUDGET_EOF
- log_info "Budget limit set: \$$BUDGET_LIMIT (source: ${BUDGET_SOURCE:-default})"
- fi
- # v7.0.0 Phase 2: remote->local hydrate. Runs ONCE at session boot (not
- # per iteration) to pull semantic patterns + procedural skills from the
- # managed store into .loki/memory/semantic/patterns.json and
- # .loki/memory/skills/*.json. Gated on parent + MEMORY + HYDRATE; all
- # three must be "true". 10s hard timeout so a slow remote never blocks
- # startup. Idempotent (sentinel: .loki/managed/hydrate.lock).
- if [ "$LOKI_MANAGED_AGENTS" = "true" ] \
- && [ "$LOKI_MANAGED_MEMORY" = "true" ] \
- && [ "$LOKI_MANAGED_MEMORY_HYDRATE" = "true" ]; then
- local _hydrate_target="${TARGET_DIR:-$(pwd)}"
- local _hydrate_out
- _hydrate_out=$(
- cd "$PROJECT_DIR" 2>/dev/null && \
- LOKI_TARGET_DIR="$_hydrate_target" \
- timeout 10 python3 -m memory.managed_memory.retrieve --hydrate 2>/dev/null || true
- )
- if [ -n "$_hydrate_out" ]; then
- log_info "Managed hydrate: $_hydrate_out"
- else
- LOKI_TARGET_DIR="$_hydrate_target" \
- python3 -c "from memory.managed_memory.events import emit_managed_event; emit_managed_event('managed_memory_hydrate_timeout', {'phase': 'init'})" 2>/dev/null || true
- fi
- fi
- log_info "Loki directory initialized: .loki/"
- }
- # Write .loki/pricing.json. The model rows derive from loki-ts/data/model-pricing.json
- # (single source of truth, also read by budget.ts); nothing is hand-kept here.
- # Readers: dashboard/server.py (_load_pricing_from_file, /api/pricing).
- # Fail-safe: an unreadable source (or no python3) writes an empty models map, which the dashboard
- # treats as "use my static defaults".
- _write_pricing_json() {
- local provider="${LOKI_PROVIDER:-claude}"
- local updated
- updated=$(date -u +%Y-%m-%d)
- PRICING_SRC="${SCRIPT_DIR}/../loki-ts/data/model-pricing.json" \
- PRICING_PROVIDER="$provider" PRICING_UPDATED="$updated" \
- python3 - > ".loki/pricing.json" << 'PRICING_PY' || printf '{"provider": "%s", "models": {}}\n' "$provider" > ".loki/pricing.json"
- import json, os, sys
- src = os.environ["PRICING_SRC"]
- models = {}
- try:
- with open(src, encoding="utf-8") as fh:
- rows = json.load(fh)["pricing"]
- for key, row in rows.items():
- if not isinstance(row, dict) or "input" not in row or "output" not in row:
- continue
- owner = row.get("provider") or ("opencode" if key.startswith("gpt-") else "claude")
- out = {"input": row["input"], "output": row["output"]}
- if owner == "claude":
- for field in ("cache_read", "cache_write", "cache_write_1h"):
- if field in row:
- out[field] = row[field]
- tier = row.get("over_100k")
- if isinstance(tier, dict) and "input" in tier and "output" in tier:
- out["over_100k"] = {"input": tier["input"], "output": tier["output"]}
- out["label"] = row.get("label", key)
- out["provider"] = owner
- models[key] = out
- except (OSError, ValueError, KeyError, AttributeError) as exc:
- sys.stderr.write("loki: warning: cannot derive .loki/pricing.json models from %s: %s\n" % (src, exc))
- models = {}
- print(json.dumps({
- "provider": os.environ["PRICING_PROVIDER"],
- "updated": os.environ["PRICING_UPDATED"],
- "source": "static",
- "models": models,
- }, indent=2))
- PRICING_PY
- log_info "Pricing data written: .loki/pricing.json (provider: ${provider})"
- }
- #===============================================================================
- # Copy Skill Files to Project Directory
- #===============================================================================
- copy_skill_files() {
- # Copy skill files from the CLI package to the project's .loki/ directory.
- # This makes the CLI self-contained - no need to install Claude Code skill separately.
- # All providers (Claude, opencode) use the same .loki/skills/ location.
- local skills_src="$PROJECT_DIR/skills"
- local skills_dst=".loki/skills"
- if [ ! -d "$skills_src" ]; then
- log_warn "Skills directory not found at $skills_src"
- return 1
- fi
- # Create destination and copy skill files
- mkdir -p "$skills_dst"
- # Copy all skill markdown files
- local copied=0
- for skill_file in "$skills_src"/*.md; do
- if [ -f "$skill_file" ]; then
- cp "$skill_file" "$skills_dst/"
- ((copied++))
- fi
- done
- # Also copy SKILL.md to .loki/ and rewrite paths for workspace access
- if [ -f "$PROJECT_DIR/SKILL.md" ]; then
- # Rewrite skill paths from skills/ to .loki/skills/
- # ONE regex, not a hardcoded filename list. The old form named 8 skills
- # explicitly plus a `Read skills/` catchall, so any NEW skill silently
- # kept an unrewritten path, and three bare paths in SKILL.md survived
- # because they say "See", not "Read". The negated class avoids rewriting
- # an already-correct `.loki/skills/`.
- # Sentinel-protect, rewrite, restore. This is PORTABLE and idempotent.
- #
- # The old form named 8 skills explicitly plus a `Read skills/` catchall,
- # so any NEW skill silently kept an unrewritten path, and bare paths
- # introduced by "See skills/..." survived because they are not "Read".
- # A `sed -E 's|(^|[^.])skills/|...'` one-liner looks tidier but BSD sed
- # rejects it ("RE error: parentheses not balanced"), which would fail
- # SILENTLY on macOS and leave every path unrewritten -- verified by
- # running it. Protecting the already-correct prefixes first is what lets
- # the bare rewrite be unconditional.
- sed -e 's|\.loki/skills/|@@LOKI_S@@|g' \
- -e 's|\.loki/references/|@@LOKI_R@@|g' \
- -e 's|skills/|.loki/skills/|g' \
- -e 's|references/|.loki/references/|g' \
- -e 's|@@LOKI_S@@|.loki/skills/|g' \
- -e 's|@@LOKI_R@@|.loki/references/|g' \
- "$PROJECT_DIR/SKILL.md" > ".loki/SKILL.md"
- fi
- # ALSO copy references/. The copied skills cite references/*.md 21 times
- # across 8 files, every one of which existed in the repo and was NEVER
- # copied, so the agent followed 21 dead paths and silently lost the guidance
- # this function believes it shipped. Copying skills without their references
- # is shipping half a manual.
- local refs_src="$PROJECT_DIR/references"
- local refs_dst=".loki/references"
- local refs_copied=0
- if [ -d "$refs_src" ]; then
- mkdir -p "$refs_dst"
- for ref_file in "$refs_src"/*.md; do
- if [ -f "$ref_file" ]; then
- cp "$ref_file" "$refs_dst/" && refs_copied=$((refs_copied + 1))
- fi
- done
- fi
- log_info "Copied $copied skill files to .loki/skills/ and $refs_copied references to .loki/references/"
- }
- #===============================================================================
- # Task Status Monitor
- #===============================================================================
- update_status_file() {
- # Create a human-readable status file
- local status_file=".loki/STATUS.txt"
- # Get current phase
- local current_phase="UNKNOWN"
- if [ -f ".loki/state/orchestrator.json" ]; then
- current_phase=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('currentPhase', 'UNKNOWN'))" 2>/dev/null || echo "UNKNOWN")
- fi
- # Count tasks in each queue
- local pending=0 in_progress=0 completed=0 failed=0
- [ -f ".loki/queue/pending.json" ] && pending=$(python3 -c "import json; print(len(json.load(open('.loki/queue/pending.json'))))" 2>/dev/null || echo "0")
- [ -f ".loki/queue/in-progress.json" ] && in_progress=$(python3 -c "import json; print(len(json.load(open('.loki/queue/in-progress.json'))))" 2>/dev/null || echo "0")
- [ -f ".loki/queue/completed.json" ] && completed=$(python3 -c "import json; print(len(json.load(open('.loki/queue/completed.json'))))" 2>/dev/null || echo "0")
- [ -f ".loki/queue/failed.json" ] && failed=$(python3 -c "import json; print(len(json.load(open('.loki/queue/failed.json'))))" 2>/dev/null || echo "0")
- cat > "$status_file" << EOF
- ╔════════════════════════════════════════════════════════════════╗
- ║ LOKI MODE STATUS ║
- ╚════════════════════════════════════════════════════════════════╝
- Updated: $(date)
- Phase: $current_phase
- Tasks:
- ├─ Pending: $pending
- ├─ In Progress: $in_progress
- ├─ Completed: $completed
- └─ Failed: $failed
- Monitor: watch -n 2 cat .loki/STATUS.txt
- EOF
- }
- #===============================================================================
- # Phase Management (Dashboard Integration)
- #===============================================================================
- # Track last known phase to detect changes
- LAST_KNOWN_PHASE=""
- #===============================================================================
- # Dashboard State Writer (Real-time sync with web dashboard)
- #===============================================================================
- write_dashboard_state() {
- # Write comprehensive dashboard state to JSON for web dashboard consumption
- local output_file=".loki/dashboard-state.json"
- # Get current phase and version
- local current_phase="BOOTSTRAP"
- local version="unknown"
- local started_at=""
- local tasks_completed=0
- local tasks_failed=0
- if [ -f ".loki/state/orchestrator.json" ]; then
- _dv=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('currentPhase', 'BOOTSTRAP'))" 2>/dev/null) && current_phase="$_dv"
- _dv=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('version', 'unknown'))" 2>/dev/null) && version="$_dv"
- _dv=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('startedAt', ''))" 2>/dev/null) && started_at="$_dv"
- _dv=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('metrics', {}).get('tasksCompleted', 0))" 2>/dev/null) && tasks_completed="$_dv"
- _dv=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('metrics', {}).get('tasksFailed', 0))" 2>/dev/null) && tasks_failed="$_dv"
- fi
- # Emit phase change event if phase has changed (checked in background monitor loop)
- if [ -n "$LAST_KNOWN_PHASE" ] && [ "$current_phase" != "$LAST_KNOWN_PHASE" ]; then
- emit_event_json "phase_change" \
- "from=$LAST_KNOWN_PHASE" \
- "to=$current_phase" \
- "iteration=$ITERATION_COUNT"
- fi
- LAST_KNOWN_PHASE="$current_phase"
- # Get task counts from queues
- local pending_tasks="[]"
- local in_progress_tasks="[]"
- local completed_tasks="[]"
- local failed_tasks="[]"
- local review_tasks="[]"
- # Read queue files, normalizing {"tasks":[...]} format to plain array
- [ -f ".loki/queue/pending.json" ] && pending_tasks=$(jq 'if type == "object" then .tasks // [] else . end' ".loki/queue/pending.json" 2>/dev/null || echo "[]")
- [ -f ".loki/queue/in-progress.json" ] && in_progress_tasks=$(jq 'if type == "object" then .tasks // [] else . end' ".loki/queue/in-progress.json" 2>/dev/null || echo "[]")
- [ -f ".loki/queue/completed.json" ] && completed_tasks=$(jq 'if type == "object" then .tasks // [] else . end' ".loki/queue/completed.json" 2>/dev/null || echo "[]")
- [ -f ".loki/queue/failed.json" ] && failed_tasks=$(jq 'if type == "object" then .tasks // [] else . end' ".loki/queue/failed.json" 2>/dev/null || echo "[]")
- [ -f ".loki/queue/review.json" ] && review_tasks=$(jq 'if type == "object" then .tasks // [] else . end' ".loki/queue/review.json" 2>/dev/null || echo "[]")
- # Get agents state
- local agents="[]"
- [ -f ".loki/state/agents.json" ] && agents=$(cat ".loki/state/agents.json" 2>/dev/null || echo "[]")
- # Get resources state
- local cpu_usage=0
- local mem_usage=0
- local resource_status="ok"
- if [ -f ".loki/state/resources.json" ]; then
- cpu_usage=$(python3 -c "import json; print(json.load(open('.loki/state/resources.json')).get('cpu', {}).get('usage_percent', 0))" 2>/dev/null || echo "0")
- mem_usage=$(python3 -c "import json; print(json.load(open('.loki/state/resources.json')).get('memory', {}).get('usage_percent', 0))" 2>/dev/null || echo "0")
- resource_status=$(python3 -c "import json; print(json.load(open('.loki/state/resources.json')).get('overall_status', 'ok'))" 2>/dev/null || echo "ok")
- fi
- # Check human intervention signals
- local mode="autonomous"
- if [ -f ".loki/PAUSE" ]; then
- mode="paused"
- elif [ -f ".loki/STOP" ]; then
- mode="stopped"
- fi
- # Get complexity tier
- local complexity="${DETECTED_COMPLEXITY:-standard}"
- # Get RARV cycle step from actual phase tracking (falls back to iteration-based)
- local rarv_step=${RARV_CURRENT_STEP:-$((ITERATION_COUNT % 4))}
- local rarv_stages='["reason", "act", "reflect", "verify"]'
- # Get memory system stats (if available)
- local episodic_count=0
- local semantic_count=0
- local procedural_count=0
- [ -d ".loki/memory/episodic" ] && episodic_count=$(find ".loki/memory/episodic" -type f -name "*.json" 2>/dev/null | wc -l | tr -d ' ')
- [ -d ".loki/memory/semantic" ] && semantic_count=$(find ".loki/memory/semantic" -type f -name "*.json" 2>/dev/null | wc -l | tr -d ' ')
- [ -d ".loki/memory/skills" ] && procedural_count=$(find ".loki/memory/skills" -type f -name "*.json" 2>/dev/null | wc -l | tr -d ' ')
- # Get quality gates status (if available)
- local quality_gates='null'
- if [ -f ".loki/state/quality-gates.json" ]; then
- quality_gates=$(cat ".loki/state/quality-gates.json" 2>/dev/null || echo 'null')
- fi
- # Get Completion Council state (v5.25.0)
- local council_state='{"enabled":false}'
- if [ -f ".loki/council/state.json" ]; then
- council_state=$(cat ".loki/council/state.json" 2>/dev/null || echo '{"enabled":false}')
- fi
- # PRD Checklist summary (v5.44.0)
- local checklist_summary='null'
- if [ -f ".loki/checklist/verification-results.json" ]; then
- checklist_summary=$(cat ".loki/checklist/verification-results.json" 2>/dev/null || echo "null")
- fi
- # App Runner state (v5.45.0)
- local app_runner_state='{"status":"not_initialized"}'
- if [ -f ".loki/app-runner/state.json" ]; then
- app_runner_state=$(cat ".loki/app-runner/state.json" 2>/dev/null || echo '{"status":"error"}')
- fi
- # Playwright verification results (v5.46.0)
- local playwright_results='null'
- if [ -f ".loki/verification/playwright-results.json" ]; then
- playwright_results=$(cat ".loki/verification/playwright-results.json" 2>/dev/null || echo "null")
- fi
- # Get budget status (if configured)
- local budget_json="null"
- if [ -f ".loki/metrics/budget.json" ]; then
- budget_json=$(cat ".loki/metrics/budget.json" 2>/dev/null || echo "null")
- fi
- # Get context window tracking state (v5.40.0)
- local context_state="null"
- if [ -f ".loki/context/tracking.json" ]; then
- context_state=$(cat ".loki/context/tracking.json" 2>/dev/null || echo "null")
- fi
- # Get notification summary (v5.40.0)
- local notification_summary='{"total":0,"unacknowledged":0,"critical":0,"warning":0,"info":0}'
- if [ -f ".loki/notifications/active.json" ]; then
- notification_summary=$(python3 -c "
- import json,sys
- try:
- data=json.load(open('.loki/notifications/active.json'))
- print(json.dumps(data.get('summary',{'total':0,'unacknowledged':0})))
- except: print('{\"total\":0,\"unacknowledged\":0}')
- " 2>/dev/null || echo '{"total":0,"unacknowledged":0}')
- fi
- # Write comprehensive JSON state (atomic via temp file + mv).
- # v7.7.5 fix: previously used `${output_file}.tmp` (no PID suffix). When two
- # background processes both called write_dashboard_state concurrently, they
- # raced on the same .tmp filename -- one would clobber the other's content,
- # the loser's `mv` would fail with "No such file or directory" because the
- # winner already moved the shared .tmp away. This flooded the agent output
- # with `mv: rename .loki/dashboard-state.json.tmp ...` errors and made
- # Loki sessions appear broken. Now each process gets a unique tmp suffix.
- local project_name=$(basename "$(pwd)")
- local project_path=$(pwd)
- local _tmp_state="${output_file}.tmp.$$.$RANDOM"
- # BUG #49 fix: Escape project path/name for JSON to handle special chars
- # (spaces, quotes, backslashes in directory names)
- local project_name_escaped
- local project_path_escaped
- project_name_escaped=$(printf '%s' "$project_name" | sed 's/\\/\\\\/g; s/"/\\"/g')
- project_path_escaped=$(printf '%s' "$project_path" | sed 's/\\/\\\\/g; s/"/\\"/g')
- cat > "$_tmp_state" << EOF
- {
- "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
- "version": "$version",
- "project": {
- "name": "$project_name_escaped",
- "path": "$project_path_escaped"
- },
- "mode": "$mode",
- "provider": "${PROVIDER_NAME:-claude}",
- "phase": "$current_phase",
- "complexity": "$complexity",
- "iteration": $ITERATION_COUNT,
- "startedAt": "$started_at",
- "rarv": {
- "currentStep": $rarv_step,
- "stages": $rarv_stages
- },
- "tasks": {
- "pending": $pending_tasks,
- "inProgress": $in_progress_tasks,
- "review": $review_tasks,
- "completed": $completed_tasks,
- "failed": $failed_tasks
- },
- "agents": $agents,
- "metrics": {
- "tasksCompleted": $tasks_completed,
- "tasksFailed": $tasks_failed,
- "cpuUsage": $cpu_usage,
- "memoryUsage": $mem_usage,
- "resourceStatus": "$resource_status"
- },
- "memory": {
- "episodic": $episodic_count,
- "semantic": $semantic_count,
- "procedural": $procedural_count
- },
- "qualityGates": $quality_gates,
- "council": $council_state,
- "checklist": $checklist_summary,
- "appRunner": $app_runner_state,
- "playwright": $playwright_results,
- "budget": $budget_json,
- "context": $context_state,
- "tokens": $(python3 -c "
- import json
- try:
- t = json.load(open('.loki/context/tracking.json'))
- totals = t.get('totals', {})
- print(json.dumps({'input': totals.get('total_input', 0), 'output': totals.get('total_output', 0), 'cost_usd': totals.get('total_cost_usd', 0)}))
- except: print('null')
- " 2>/dev/null || echo "null"),
- "notifications": $notification_summary
- }
- EOF
- # v7.7.5 fix: silence mv stderr so any residual race (different process
- # already swapped a newer file in) doesn't flood the agent output. The
- # PID-suffixed tmp eliminates the race; the 2>/dev/null is belt-and-
- # suspenders. `|| rm -f "$_tmp_state" 2>/dev/null` cleans up the tmp
- # on the rare failure rather than leaking it.
- mv "$_tmp_state" "$output_file" 2>/dev/null || rm -f "$_tmp_state" 2>/dev/null
- }
- #===============================================================================
- # Context Window Tracking (v5.40.0)
- #===============================================================================
- # Track context window usage (provider-agnostic)
- track_context_usage() {
- local iteration="$1"
- mkdir -p .loki/context
- local provider_arg="${LOKI_PROVIDER:-claude}"
- local window_arg="${LOKI_CONTEXT_WINDOW_SIZE:-0}"
- python3 "${SCRIPT_DIR}/context-tracker.py" \
- --iteration "$iteration" \
- --loki-dir ".loki" \
- --provider "$provider_arg" \
- --window-size "$window_arg" 2>/dev/null || true
- }
- # Check notification triggers against current state
- check_notification_triggers() {
- local iteration="$1"
- mkdir -p .loki/notifications
- python3 "${SCRIPT_DIR}/notification-checker.py" \
- --iteration "$iteration" \
- --loki-dir ".loki" 2>/dev/null || true
- }
- #===============================================================================
- # Task Queue Auto-Tracking (for degraded mode providers)
- #===============================================================================
- # Derive a plain-language, honest summary of the spec this run is building from,
- # for the dashboard iteration card (Task G). Mirrors the "Building:" start
- # headline (run.sh:17061) and proof-generator's source resolution so the card,
- # the banner, and the proof artifact all describe the same real intent.
- #
- # Echoes two tab-separated fields: <human spec label>\t<spec kind>, where kind is
- # one of: brief | prd | codebase-analysis. The label is plain English with no
- # RARV jargon. Best-effort and never fails (used only to enrich a card).
- _loki_iteration_spec_summary() {
- local prd="${1:-}"
- local label="" kind=""
- # 1. A recorded one-line brief (`loki start "<brief>"`) is the user's own
- # words; show them verbatim, truncated to stay on one tidy line.
- if [ -f ".loki/state/brief.txt" ]; then
- local _brief
- _brief="$(tr '\n' ' ' < .loki/state/brief.txt 2>/dev/null | sed 's/ */ /g; s/^ //; s/ $//')"
- if [ -n "$_brief" ]; then
- if [ "${#_brief}" -gt 80 ]; then
- _brief="${_brief:0:77}..."
- fi
- printf '%s\t%s' "$_brief" "brief"
- return 0
- fi
- fi
- # 2. A generated PRD means there was no user spec; we are reverse-engineering
- # one from the code. Say exactly that rather than naming the internal file.
- case "$prd" in
- ""|*.loki/generated-prd.md|*.loki/generated-prd.json)
- label="the codebase (no spec provided)"
- kind="codebase-analysis"
- ;;
- *)
- # 3. A real user PRD / spec file: name it by basename, no path noise.
- label="$(basename "$prd" 2>/dev/null || printf '%s' "$prd")"
- kind="prd"
- ;;
- esac
- printf '%s\t%s' "$label" "$kind"
- }
- # Track iteration start - create task in in-progress queue
- track_iteration_start() {
- local iteration="$1"
- local prd="${2:-}"
- local task_id="iteration-$iteration"
- mkdir -p .loki/queue
- # Record iteration start time for efficiency tracking (SYN-018)
- record_iteration_start
- # Emit iteration start event for dashboard
- emit_event_json "iteration_start" \
- "iteration=$iteration" \
- "provider=${PROVIDER_NAME:-claude}" \
- "prd=${prd:-Codebase Analysis}"
- # Also emit to pending dir for OTEL bridge
- if [ -n "${LOKI_OTEL_ENDPOINT:-}" ]; then
- emit_event_pending "iteration_start" \
- "iteration=$iteration" \
- "provider=${PROVIDER_NAME:-claude}"
- fi
- # Read next pending task for context (enrich iteration with PRD task details)
- local next_task_context=""
- if [[ -f ".loki/queue/pending.json" ]]; then
- next_task_context=$(python3 -c "
- import json
- try:
- with open('.loki/queue/pending.json') as f:
- tasks = json.load(f)
- if isinstance(tasks, dict):
- tasks = tasks.get('tasks', [])
- pending = [t for t in tasks if isinstance(t, dict) and t.get('status','pending') == 'pending']
- if pending:
- t = pending[0]
- print(json.dumps({
- 'current_task': t.get('title',''),
- 'description': t.get('description',''),
- 'acceptance_criteria': t.get('acceptance_criteria', []),
- 'user_story': t.get('user_story', ''),
- 'source': t.get('source', ''),
- 'project': t.get('project', '')
- }))
- except: pass
- " 2>/dev/null || true)
- fi
- # Create task entry (escape PRD path for safe JSON embedding)
- local prd_escaped
- prd_escaped=$(printf '%s' "${prd:-Codebase Analysis}" | sed 's/\\/\\\\/g; s/"/\\"/g; s/\t/\\t/g')
- # Task G: derive a plain-language, honest summary of what this run is building
- # from, so the card title/description describe the real work instead of the
- # generic "Iteration N" / "RARV iteration N" placeholder. Best-effort.
- local _spec_label="" _spec_kind=""
- local _spec_summary
- _spec_summary=$(_loki_iteration_spec_summary "$prd")
- _spec_label="${_spec_summary%%$'\t'*}"
- _spec_kind="${_spec_summary##*$'\t'}"
- # Escape for safe embedding in the python -c literals below.
- local _spec_label_esc _spec_kind_esc
- _spec_label_esc=$(printf '%s' "$_spec_label" | sed 's/\\/\\\\/g; s/"/\\"/g; s/\t/\\t/g')
- _spec_kind_esc=$(printf '%s' "$_spec_kind" | sed 's/\\/\\\\/g; s/"/\\"/g; s/\t/\\t/g')
- # Build enriched task JSON with pending task context.
- # Must initialize to empty: this script runs under `set -u` (line 152),
- # so `local task_json` without a value leaves it unset. When the pending
- # queue is empty, the enrichment `if` is skipped and the `-z` check below
- # would fire on an unset variable and kill the run.
- local task_json=""
- if [[ -n "${next_task_context:-}" ]]; then
- task_json=$(python3 -c "
- import json, sys
- ctx = json.loads('''$next_task_context''')
- # Task G: the card must describe the REAL work in plain language, not the
- # generic 'Iteration N' / 'RARV iteration N' placeholder. Prefer the pending
- # PRD task's own title/description/criteria. When those are absent, fall back to
- # a plain-language summary derived from the spec this run is building from, and
- # OMIT acceptance criteria rather than show RARV phase-name boilerplate (only
- # state what is really known).
- spec_label = '${_spec_label_esc}'
- spec_kind = '${_spec_kind_esc}'
- if spec_kind == 'codebase-analysis':
- fallback_title = 'Analyzing the codebase and generating a spec'
- fallback_desc = 'Reading the existing code to reverse-engineer a spec, then building against it.'
- elif spec_kind == 'brief':
- fallback_title = 'Building: ' + spec_label
- fallback_desc = 'Building from your brief: ' + spec_label
- else:
- fallback_title = 'Building from ' + spec_label
- fallback_desc = 'Implementing the spec in ' + spec_label + ' and verifying it.'
- task = {
- 'id': 'iteration-$iteration',
- 'type': 'iteration',
- 'title': ctx.get('current_task') or fallback_title,
- 'description': ctx.get('description') or fallback_desc,
- 'status': 'in_progress',
- 'priority': 'medium',
- 'startedAt': '$(date -u +%Y-%m-%dT%H:%M:%SZ)',
- 'provider': '${PROVIDER_NAME:-claude}',
- 'acceptance_criteria': ctx.get('acceptance_criteria') or [],
- 'notes': [],
- 'logs': [{
- 'timestamp': '$(date -u +%Y-%m-%dT%H:%M:%SZ)',
- 'iteration': $iteration,
- 'level': 'info',
- 'phase': 'BOOTSTRAP',
- 'message': 'Iteration $iteration started'
- }]
- }
- if ctx.get('user_story'):
- task['user_story'] = ctx['user_story']
- if ctx.get('source'):
- task['source'] = ctx['source']
- if ctx.get('project'):
- task['project'] = ctx['project']
- print(json.dumps(task, indent=2))
- " 2>/dev/null) || task_json=""
- fi
- # Fallback when there is no pending PRD task (codebase-analysis run or empty
- # queue). Task G: this was the worst placeholder card -- 'Iteration N' /
- # 'RARV iteration N' with RARV-phase-name acceptance criteria, which tells a
- # watching user nothing about the real work. Build an honest, plain-language
- # card from the spec summary instead, and omit acceptance criteria (we have no
- # real per-iteration criteria here -- do not invent RARV jargon).
- if [[ -z "${task_json:-}" ]]; then
- local _start_ts="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- task_json=$(python3 -c "
- import json
- spec_label = '${_spec_label_esc}'
- spec_kind = '${_spec_kind_esc}'
- if spec_kind == 'codebase-analysis':
- title = 'Analyzing the codebase and generating a spec'
- desc = 'Reading the existing code to reverse-engineer a spec, then building against it.'
- elif spec_kind == 'brief':
- title = 'Building: ' + spec_label
- desc = 'Building from your brief: ' + spec_label
- else:
- title = 'Building from ' + spec_label
- desc = 'Implementing the spec in ' + spec_label + ' and verifying it.'
- task = {
- 'id': '$task_id',
- 'type': 'iteration',
- 'title': title,
- 'description': desc,
- 'status': 'in_progress',
- 'priority': 'medium',
- 'startedAt': '$_start_ts',
- 'provider': '${PROVIDER_NAME:-claude}',
- 'acceptance_criteria': [],
- 'notes': [],
- 'logs': [{
- 'timestamp': '$_start_ts',
- 'iteration': $iteration,
- 'level': 'info',
- 'phase': 'BOOTSTRAP',
- 'message': 'Iteration $iteration started'
- }]
- }
- print(json.dumps(task, indent=2))
- " 2>/dev/null) || task_json=""
- fi
- # Last-resort safety net if python is unavailable. Still avoids the generic
- # 'RARV iteration' wording and omits boilerplate acceptance criteria.
- if [[ -z "${task_json:-}" ]]; then
- local _start_ts2="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- local _safe_title
- case "$_spec_kind" in
- codebase-analysis) _safe_title="Analyzing the codebase and generating a spec" ;;
- brief) _safe_title="Building from your brief" ;;
- *) _safe_title="Building from the spec" ;;
- esac
- task_json=$(cat <<EOF
- {
- "id": "$task_id",
- "type": "iteration",
- "title": "$_safe_title",
- "description": "$_safe_title (iteration $iteration).",
- "status": "in_progress",
- "priority": "medium",
- "startedAt": "$_start_ts2",
- "provider": "${PROVIDER_NAME:-claude}",
- "acceptance_criteria": [],
- "notes": [],
- "logs": [
- {
- "timestamp": "$_start_ts2",
- "iteration": $iteration,
- "level": "info",
- "phase": "BOOTSTRAP",
- "message": "Iteration $iteration started"
- }
- ]
- }
- EOF
- )
- fi
- # Add to in-progress queue
- # BUG-XC-003: atomic queue modification.
- # v7.5.12: portable mkdir-mutex via safe_acquire_lock (no flock needed).
- # v7.5.12 Dev11 (R1 HIGH): gate the read-modify-write on acquire SUCCESS.
- # The prior `safe_acquire_lock ... || true` then unconditional
- # `safe_release_lock` mutated state on timeout AND released the OTHER
- # holder's lock -- a mutex correctness violation. Mirror the working
- # pattern at line 1845 (acquire-success guarded RMW + release inside).
- local in_progress_file=".loki/queue/in-progress.json"
- local lockfile=".loki/queue/.in-progress.lock"
- if type safe_acquire_lock >/dev/null 2>&1 && safe_acquire_lock "$lockfile" 5; then
- if [ -f "$in_progress_file" ]; then
- local existing=$(cat "$in_progress_file")
- if [ "$existing" = "[]" ] || [ -z "$existing" ]; then
- echo "[$task_json]" > "$in_progress_file"
- else
- # Append to existing array
- echo "$existing" | python3 -c "
- import sys, json
- data = json.load(sys.stdin)
- data.append($task_json)
- print(json.dumps(data, indent=2))
- " > "$in_progress_file" 2>/dev/null || echo "[$task_json]" > "$in_progress_file"
- fi
- else
- echo "[$task_json]" > "$in_progress_file"
- fi
- safe_release_lock "$lockfile"
- else
- log_warn "could not acquire in-progress lock; skipping update"
- fi
- # BUG-ST-014: Atomic current-task.json update via temp file + mv
- local ct_tmp=".loki/queue/current-task.json.tmp.$$"
- echo "$task_json" > "$ct_tmp"
- mv -f "$ct_tmp" .loki/queue/current-task.json
- }
- # Track iteration completion - move task to completed queue
- # v7.8.1: staleness-aware generated-PRD reuse helpers.
- # Hash stdin with whatever digest tool is available (mirrors the existing
- # stat -f%z || stat -c%s dual-probe portability pattern). Echoes a short hash.
- _loki_hash_stdin() {
- if command -v shasum >/dev/null 2>&1; then
- shasum -a 256 | cut -c1-16
- elif command -v sha256sum >/dev/null 2>&1; then
- sha256sum | cut -c1-16
- else
- cksum | tr -d ' ' | cut -c1-16
- fi
- }
- # Compute a cheap, clone-stable signature of the codebase so we can tell whether
- # it changed since the generated PRD was last written. Git repos: HEAD sha +
- # dirty flag (.loki/.git churn filtered out). Non-git: a hash of sorted
- # path+size pairs PLUS file content (v7.32.3, #569: path+size alone was blind to
- # a same-size content edit, so a stale PRD could be silently reused with a
- # false "codebase unchanged" disclosure). Content hashing is clone-stable
- # (mtime is not, which is why mtime was never used). Three content tiers:
- # 1. full content hash ("files:") when the tree is both at-or-under
- # LOKI_PRD_SIG_CONTENT_BUDGET bytes (default 50MB) AND at-or-under
- # LOKI_PRD_SIG_CONTENT_MAXFILES files (default 20000). Detects any edit.
- # 2. sampled content hash ("files-sampled:", #171) when the tree exceeds
- # either bound: hashes the head + tail (first 4096 + last 4096 bytes) of
- # every file. Catches same-size edits at the start or end of a file
- # without a full read of a huge tree. Residual honest gap: a same-size
- # edit in the MIDDLE of a large file (>8192 bytes) that touches neither
- # 4KB window is still invisible. Far narrower than the old size-blind
- # fallback, which missed ALL same-size edits.
- # 3. (historical) "files-shallow:" was the old content-blind fallback. It is
- # no longer emitted, but is still ACCEPTED when read from a stored pre-#171
- # signature so the first post-upgrade run reuses instead of falsely
- # claiming "codebase changed" (one-run format-transition, see consumer).
- # Echoes the signature.
- compute_codebase_signature() {
- local dir="${1:-.}"
- ( cd "$dir" 2>/dev/null || exit 0
- if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
- # Content-identity signature (gitc:): the hash is over the WORKING-TREE
- # CONTENT of every tracked + untracked-not-ignored file, independent of
- # the commit boundary. Whether a file is committed or sitting dirty in
- # the worktree yields the SAME value. This is what makes reuse robust to
- # Loki's OWN session commit (commit_session_changes, default-on as of
- # v7.73.0): a rerun whose only "change" is that the prior run committed
- # work it had already analyzed still classifies as reuse, not a spurious
- # "codebase changed -> update". A genuine source edit (committed OR
- # uncommitted) changes a blob hash and is still detected, and a new
- # untracked file is detected too. .loki/ is excluded (runtime state).
- # Paths are enumerated NUL-safe, .loki dropped, sorted, then hashed in
- # ONE batched `git hash-object --stdin-paths` pass (order-preserving),
- # so cost is one git process regardless of file count.
- local gitc gitc_paths gitc_deleted gitc_paths_file gitc_deleted_file
- # Tracked files removed from the worktree (but not staged): they have no
- # content to hash and would make the batched hash-object abort mid-list,
- # truncating the output and misaligning the path<->hash pairing. Drop
- # them; their removal from the list is itself the detected change.
- gitc_deleted=$(git ls-files --deleted -z 2>/dev/null | tr '\0' '\n')
- # Exclude .loki/ AND Loki's own session-end artifacts (HANDOFF.md,
- # USAGE.md) written+committed to the repo ROOT by the completion path
- # (LOKI_HANDOFF / commit_session_changes, both default-on). Those are
- # runtime OUTPUT of the session, not user codebase; counting them would
- # make the next PRD-reuse check see a spurious "codebase changed" and
- # flip reuse->update (the signature is persisted per-iteration BEFORE
- # the after-loop HANDOFF write, so it can never account for them).
- gitc_paths=$( { git ls-files -z 2>/dev/null; git ls-files --others --exclude-standard -z 2>/dev/null; } \
- | tr '\0' '\n' | grep -vE '(^|/)\.loki(/|$)|^HANDOFF\.md$|^USAGE\.md$' | LC_ALL=C sort -u )
- mkdir -p .loki/tmp 2>/dev/null || { echo "gitc:"; exit 0; }
- gitc_paths_file=$(mktemp ".loki/tmp/signature-paths.XXXXXX") || { echo "gitc:"; exit 0; }
- gitc_deleted_file=$(mktemp ".loki/tmp/signature-deleted.XXXXXX") || {
- rm -f "$gitc_paths_file"
- echo "gitc:"
- exit 0
- }
- printf '%s\n' "$gitc_paths" > "$gitc_paths_file"
- printf '%s\n' "$gitc_deleted" > "$gitc_deleted_file"
- if [ -n "$gitc_deleted" ]; then
- gitc_paths=$(grep -vxF -f "$gitc_deleted_file" "$gitc_paths_file" || true)
- printf '%s\n' "$gitc_paths" > "$gitc_paths_file"
- fi
- if [ -z "$gitc_paths" ]; then
- # No tracked or untracked content (empty/fresh repo): a stable
- # constant so two empty-tree runs still compare equal (reuse).
- gitc=$(printf '' | _loki_hash_stdin)
- else
- gitc=$(printf '%s\n' "$gitc_paths" | git hash-object --stdin-paths 2>/dev/null \
- | paste -d'\t' - "$gitc_paths_file" \
- | LC_ALL=C sort | _loki_hash_stdin)
- fi
- rm -f "$gitc_paths_file" "$gitc_deleted_file"
- echo "gitc:${gitc}"
- else
- local listing count total_sz budget maxfiles
- listing=$(find . \
- -type d \( -name .loki -o -name .git -o -name node_modules -o -name dist \
- -o -name build -o -name .next -o -name target -o -name vendor \
- -o -name __pycache__ -o -name .venv -o -name venv \) -prune -o \
- -type f -print 2>/dev/null \
- | grep -vE '^\./(HANDOFF|USAGE)\.md$' \
- | while IFS= read -r f; do
- local sz
- sz=$(stat -f%z "$f" 2>/dev/null || stat -c%s "$f" 2>/dev/null || echo 0)
- printf '%s\t%s\n' "$f" "$sz"
- done | LC_ALL=C sort)
- # grep -c prints 0 itself on no match (exit 1); '|| true' avoids the
- # old '|| echo 0' double-zero that embedded a newline on empty trees
- count=$(printf '%s\n' "$listing" | grep -c . || true)
- total_sz=$(printf '%s\n' "$listing" | awk -F'\t' '{s+=$2} END {printf "%d", s}')
- budget="${LOKI_PRD_SIG_CONTENT_BUDGET:-52428800}"
- maxfiles="${LOKI_PRD_SIG_CONTENT_MAXFILES:-20000}"
- if [ "${total_sz:-0}" -le "$budget" ] 2>/dev/null \
- && [ "${count:-0}" -le "$maxfiles" ] 2>/dev/null; then
- # Tier 1 -- full content pass: stream all file contents through one
- # hash in the same sorted order as the listing. Detects any edit,
- # including same-size ones. xargs -0 batches the reads into a
- # handful of cat invocations, so cost scales with BYTES (which the
- # budget above bounds), not file count: a fork-per-file loop here
- # measured ~38s of added startup on a 30k-small-file tree. Renames
- # and content swaps are still caught by the listing hash below.
- local content_hash
- content_hash=$(printf '%s\n' "$listing" | cut -f1 | tr '\n' '\0' \
- | xargs -0 cat 2>/dev/null | _loki_hash_stdin)
- echo "files:$(printf '%s' "$listing" | _loki_hash_stdin):${count}:${content_hash}"
- else
- # Tier 2 -- sampled content pass (#171): the tree is over the byte
- # budget OR over the file-count cap, so a full read would be slow.
- # Hash the head + tail (first 4096 + last 4096 bytes) of every file
- # instead. This catches same-size edits at the start or end of a
- # file (which the old size-blind "files-shallow:" missed entirely),
- # at a fixed <=8KB-per-file cost. -n 64 batches files per sh fork
- # to avoid a fork-per-file storm on large trees. Same sorted order
- # as the listing keeps the hash deterministic. Residual honest gap:
- # a same-size edit in the middle of a >8KB file is still invisible.
- local sample_hash
- sample_hash=$(printf '%s\n' "$listing" | cut -f1 | tr '\n' '\0' \
- | xargs -0 -n 64 sh -c 'for f in "$@"; do head -c 4096 -- "$f" 2>/dev/null; tail -c 4096 -- "$f" 2>/dev/null; done' _ 2>/dev/null \
- | _loki_hash_stdin)
- echo "files-sampled:$(printf '%s' "$listing" | _loki_hash_stdin):${count}:${sample_hash}"
- fi
- fi
- )
- }
- # Recompute the PRE-content-hash git-mode signature ("git:<HEAD>:<dirty>") for a
- # one-time format transition: a signature recorded by an older Loki (HEAD +
- # porcelain) must still be comparable on the first run after the upgrade to the
- # new content-identity "gitc:" format, or decide would falsely flip to "update".
- # Echoes the legacy-format value, or "" when not inside a git work tree.
- _loki_compute_legacy_git_signature() {
- local dir="${1:-.}"
- ( cd "$dir" 2>/dev/null || exit 0
- git rev-parse --is-inside-work-tree >/dev/null 2>&1 || exit 0
- local head dirty porcelain
- head=$(git rev-parse HEAD 2>/dev/null || echo "nohead")
- porcelain=$(git status --porcelain 2>/dev/null | grep -vE '(^...?\.loki/|/\.loki/| \.loki/|\.git/)' || true)
- if [ -z "$porcelain" ]; then
- dirty="clean"
- else
- dirty=$(printf '%s' "$porcelain" | _loki_hash_stdin)
- fi
- echo "git:${head}:${dirty}"
- )
- }
- # Content hash of the generated PRD file itself (NOT the codebase). Used to
- # detect that a user hand-edited the generated PRD: when the file no longer
- # matches the prd_sha Loki recorded after it last wrote the file, the PRD is
- # user-owned and must be used as-is, never silently overwritten. Echoes "" when
- # no generated PRD file is present.
- _loki_prd_file_hash() {
- local loki_dir="${1:-.}/.loki"
- local f=""
- if [ -f "$loki_dir/generated-prd.md" ]; then
- f="$loki_dir/generated-prd.md"
- elif [ -f "$loki_dir/generated-prd.json" ]; then
- f="$loki_dir/generated-prd.json"
- fi
- [ -n "$f" ] || { echo ""; return 0; }
- _loki_hash_stdin < "$f"
- }
- # Decide what to do with a previously generated PRD on a no-PRD run.
- # Echoes one of: reuse | update | generate | user_owned. Never fails the run.
- # Precedence: force-regen > user_owned (hand-edited) > reuse/update > generate.
- # - LOKI_PRD_REGEN=1 (or --regen-prd/--fresh-prd, which set it) -> generate.
- # - no generated PRD present -> generate (first run).
- # - generated PRD present but its content hash differs from the recorded
- # prd_sha -> user_owned (the user hand-edited it; use as-is, do not rewrite).
- # - generated PRD present, no recorded signature -> update (have a PRD but no
- # provenance: reconcile incrementally rather than trust-blindly or discard).
- # - signature matches current codebase -> reuse (unchanged).
- # - signature differs -> update (codebase changed; update incrementally).
- decide_generated_prd_action() {
- local loki_dir="${TARGET_DIR:-.}/.loki"
- if [ "${LOKI_PRD_REGEN:-}" = "1" ]; then
- echo "generate"; return 0
- fi
- if [ ! -f "$loki_dir/generated-prd.md" ] && [ ! -f "$loki_dir/generated-prd.json" ]; then
- echo "generate"; return 0
- fi
- local sig_file="$loki_dir/state/prd-signature.json"
- if [ ! -f "$sig_file" ]; then
- echo "update"; return 0
- fi
- # source:"user" short-circuit (LOCK 2): an explicit user-provided PRD was
- # persisted into the canonical slot. Always use it as-is, never enter the
- # signature-diff update path -- even if the file hash drifted (hand-edit) or
- # the codebase changed since. --fresh-prd/LOKI_PRD_REGEN (checked above)
- # still wins and forces a regenerate. A missing/empty source falls through
- # to the generated-PRD logic below (defensive: correctness never depends on
- # a backfilled field).
- local prd_source
- prd_source=$(LOKI_SIG_FILE="$sig_file" python3 -c "
- import json, os
- try:
- print(json.load(open(os.environ['LOKI_SIG_FILE'])).get('source',''))
- except Exception:
- print('')
- " 2>/dev/null)
- if [ "$prd_source" = "user" ]; then
- echo "user_owned"; return 0
- fi
- local stored stored_prd_sha current cur_prd_sha
- stored=$(LOKI_SIG_FILE="$sig_file" python3 -c "
- import json, os
- try:
- print(json.load(open(os.environ['LOKI_SIG_FILE'])).get('signature',''))
- except Exception:
- print('')
- " 2>/dev/null)
- stored_prd_sha=$(LOKI_SIG_FILE="$sig_file" python3 -c "
- import json, os
- try:
- print(json.load(open(os.environ['LOKI_SIG_FILE'])).get('prd_sha',''))
- except Exception:
- print('')
- " 2>/dev/null)
- [ -z "$stored" ] && { echo "update"; return 0; }
- # Hand-edit detection (precedence above reuse/update): if we recorded a
- # prd_sha and the file no longer matches it, the user edited it themselves.
- # Treat as user-owned: use as-is, never regenerate over their changes.
- if [ -n "$stored_prd_sha" ]; then
- cur_prd_sha=$(_loki_prd_file_hash "${TARGET_DIR:-.}")
- if [ -n "$cur_prd_sha" ] && [ "$cur_prd_sha" != "$stored_prd_sha" ]; then
- echo "user_owned"; return 0
- fi
- fi
- current=$(compute_codebase_signature "${TARGET_DIR:-.}")
- if [ "$stored" = "$current" ]; then
- echo "reuse"
- else
- # v7.32.3 format transition (#569): a stored pre-content-hash signature
- # ("files:<listing>:<count>", 3 fields) compared against the new 4-field
- # format would falsely claim "codebase changed" on the first post-upgrade
- # run. When the new signature extends the stored one (same listing
- # fields), the tree is unchanged at the old format's trust level: reuse,
- # honestly. The next persist upgrades the stored format. A same-size edit
- # made BEFORE the upgrade stays invisible for this one run, exactly as it
- # was on the old version (no regression, no false disclosure).
- case "$stored" in
- files:*:*)
- # Require the legitimate old 3-field format (files:HASH:COUNT,
- # exactly 2 colons), not a truncated/corrupted 2-field value
- # (council hardening: corruption must fall to update, as before).
- if [ "$(printf '%s' "$stored" | tr -dc ':' | wc -c | tr -d ' ')" = "2" ] \
- && [ "${current#"${stored}":}" != "$current" ]; then
- echo "reuse"; return 0
- fi
- ;;
- files-shallow:*:*)
- # #171 format transition: a stored pre-#171 size-blind signature
- # ("files-shallow:<listing>:<count>", 3 fields) compared against
- # the new sampled signature ("files-sampled:<listing>:<count>:
- # <samplehash>") would falsely claim "codebase changed" on the
- # first post-upgrade run. When the listing-hash and count match
- # (i.e. the stored value with its prefix swapped to files-sampled:
- # is a prefix of the current sampled signature), the tree is
- # unchanged at the OLD format's trust level (paths+sizes): reuse,
- # honestly. The next persist upgrades the stored format to the
- # sampled tier. A same-size edit made BEFORE the upgrade stays
- # invisible for this one run, exactly as on the old version (no
- # regression, no false disclosure).
- if [ "$(printf '%s' "$stored" | tr -dc ':' | wc -c | tr -d ' ')" = "2" ]; then
- local stored_sampled="files-sampled:${stored#files-shallow:}"
- case "$current" in
- files-sampled:*)
- if [ "${current#"${stored_sampled}":}" != "$current" ]; then
- echo "reuse"; return 0
- fi
- ;;
- esac
- fi
- ;;
- git:*)
- # git-mode format transition: a stored pre-content-hash signature
- # ("git:<HEAD>:<dirty>") cannot be compared directly against the
- # new content-identity "gitc:" format, so the first run after the
- # upgrade would falsely claim "codebase changed". Recompute the
- # OLD-format signature and compare against the stored value: if it
- # matches, the tree is unchanged at the old format's trust level
- # (HEAD + dirty porcelain) -> reuse, honestly. The next persist
- # upgrades the stored format to "gitc:". Only honor this when the
- # current signature is the new git format (a real format change),
- # never when both are old git: (that path already matched above).
- case "$current" in
- gitc:*)
- local legacy
- legacy=$(_loki_compute_legacy_git_signature "${TARGET_DIR:-.}")
- if [ -n "$legacy" ] && [ "$legacy" = "$stored" ]; then
- echo "reuse"; return 0
- fi
- ;;
- esac
- ;;
- esac
- echo "update"
- fi
- }
- # Persist the current codebase signature after a clean no-PRD iteration that has
- # a generated PRD, so the next run can decide reuse vs update. Best-effort; never
- # fails the run. Only records on exit_code==0 (do not bless a broken iteration).
- persist_prd_signature_if_present() {
- local exit_code="${1:-0}"
- [ "$exit_code" = "0" ] || return 0
- # Hand-edited (user-owned) PRD: do NOT rewrite the signature. Re-hashing the
- # user's edited file would re-baseline its content as the new prd_sha, so the
- # next run would fall through to plain reuse with the wrong (non-user-owned)
- # disclosure. Preserve the prior Loki-authored prd_sha/generated_at so every
- # subsequent run keeps detecting user_owned until --fresh-prd forces a regen.
- [ "${GENERATED_PRD_ACTION:-}" = "user_owned" ] && return 0
- # only for no-PRD runs whose generated PRD exists
- case "${prd_path:-}" in
- ""|*.loki/generated-prd.md|*.loki/generated-prd.json) ;;
- *) return 0 ;;
- esac
- local loki_dir="${TARGET_DIR:-.}/.loki"
- [ -f "$loki_dir/generated-prd.md" ] || [ -f "$loki_dir/generated-prd.json" ] || return 0
- local sig
- sig=$(compute_codebase_signature "${TARGET_DIR:-.}")
- [ -n "$sig" ] || return 0
- mkdir -p "$loki_dir/state" 2>/dev/null || return 0
- local mode="files"; case "$sig" in git:*|gitc:*) mode="git" ;; esac
- # Record the content hash of the PRD file Loki just wrote so a later
- # hand-edit by the user is detectable (decide_generated_prd_action). This
- # runs AFTER the agent's own PRD writes, so Loki's updates are not mistaken
- # for user edits.
- local prd_sha; prd_sha=$(_loki_prd_file_hash "${TARGET_DIR:-.}")
- local tmp="$loki_dir/state/.prd-signature.json.tmp.$$"
- # git-mode format upgrade (old "git:<HEAD>:<dirty>" -> new content-identity
- # "gitc:..."): when this run is a reuse honored via the decide transition
- # (the recomputed legacy signature still matches the stored one), the PRD
- # content did not change, so the generated_at date must be preserved across
- # the upgrade, exactly like the files: -> files-sampled: upgrade clauses.
- # Recompute the legacy value once and pass a match flag to the persist below.
- local git_upgrade_match=""
- case "$sig" in
- gitc:*)
- local _stored_sig
- _stored_sig=$(LOKI_SIG_FILE="$loki_dir/state/prd-signature.json" python3 -c "
- import json, os
- try:
- print(json.load(open(os.environ['LOKI_SIG_FILE'])).get('signature',''))
- except Exception:
- print('')
- " 2>/dev/null)
- case "$_stored_sig" in
- git:*)
- local _legacy
- _legacy=$(_loki_compute_legacy_git_signature "${TARGET_DIR:-.}")
- [ -n "$_legacy" ] && [ "$_legacy" = "$_stored_sig" ] && git_upgrade_match=1
- ;;
- esac
- ;;
- esac
- # Preserve generated_at when the codebase signature is unchanged so the
- # reuse disclosure ("generated on <date>") stays honest across reuse runs;
- # only stamp a new date when the PRD content actually changed (sig differs).
- LOKI_SIG="$sig" LOKI_SIG_MODE="$mode" LOKI_SIG_VER="$(get_version 2>/dev/null || echo unknown)" \
- LOKI_PRD_SHA="$prd_sha" LOKI_SIG_FILE="$loki_dir/state/prd-signature.json" \
- LOKI_GIT_UPGRADE_MATCH="$git_upgrade_match" \
- python3 -c "
- import json, os, datetime
- sig = os.environ['LOKI_SIG']
- prev = {}
- try:
- prev = json.load(open(os.environ['LOKI_SIG_FILE']))
- except Exception:
- prev = {}
- prev_at = prev.get('generated_at') if isinstance(prev, dict) else None
- prev_sig = prev.get('signature') if isinstance(prev, dict) else None
- # Unchanged, OR the v7.32.3 files-signature format upgrade (#569): the new
- # 4-field signature extends an old 3-field one whose listing fields match.
- # Preserve the date in both cases; the PRD content did not change.
- _legacy_upgrade = (
- isinstance(prev_sig, str) and prev_sig.startswith('files:')
- and prev_sig.count(':') == 2
- and sig.startswith(prev_sig + ':')
- )
- # #171 format upgrade: a stored pre-#171 size-blind 'files-shallow:<listing>:
- # <count>' (3 fields) reused into the new sampled 'files-sampled:<listing>:
- # <count>:<samplehash>' whose listing fields match. Same trust level (the
- # decide returned reuse), so the PRD content did not change: preserve the date.
- _sampled_upgrade = (
- isinstance(prev_sig, str) and prev_sig.startswith('files-shallow:')
- and prev_sig.count(':') == 2
- and sig.startswith('files-sampled:' + prev_sig[len('files-shallow:'):] + ':')
- )
- # git-mode format upgrade (old 'git:<HEAD>:<dirty>' -> new content-identity
- # 'gitc:...'): the caller recomputed the legacy signature and confirmed it still
- # matches the stored one (decide returned reuse), so the PRD content did not
- # change: preserve the date across the one-time upgrade.
- _git_upgrade = (
- bool(os.environ.get('LOKI_GIT_UPGRADE_MATCH'))
- and isinstance(prev_sig, str) and prev_sig.startswith('git:')
- and sig.startswith('gitc:')
- )
- if prev_at and (prev_sig == sig or _legacy_upgrade or _sampled_upgrade or _git_upgrade):
- generated_at = prev_at
- else:
- generated_at = datetime.datetime.now(datetime.timezone.utc).isoformat().replace('+00:00','Z')
- rec = {
- 'signature': sig,
- 'generated_at': generated_at,
- 'prd_path': '.loki/generated-prd.md',
- 'prd_sha': os.environ.get('LOKI_PRD_SHA',''),
- 'mode': os.environ['LOKI_SIG_MODE'],
- 'loki_version': os.environ['LOKI_SIG_VER'],
- 'source': 'generated',
- }
- print(json.dumps(rec))
- " > "$tmp" 2>/dev/null && mv -f "$tmp" "$loki_dir/state/prd-signature.json" 2>/dev/null || rm -f "$tmp" 2>/dev/null
- }
- # Persist an explicit user-provided PRD into the canonical generated-PRD slot so
- # later no-file runs continue from it (brownfield reuse), and stamp source:"user"
- # so it is always treated as user-owned (reuse/use-as-is), never incrementally
- # updated. Echoes the canonical relative path (".loki/generated-prd.md") on a
- # successful persist so the caller can repoint prd_path; echoes nothing (empty)
- # and changes no state on any failure (the caller then keeps the original path).
- #
- # $1 = the original user PRD file path (the arg passed to run_autonomous).
- # Reads/writes under "${TARGET_DIR:-.}/.loki" to stay aligned with
- # decide_generated_prd_action and _loki_prd_file_hash (which anchor there too).
- persist_user_prd() {
- local src="$1"
- [ -n "$src" ] || { echo ""; return 0; }
- [ -f "$src" ] || { echo ""; return 0; }
- # Skip when the source already IS the canonical generated PRD (a no-op copy,
- # and the no-file reuse path owns that case). Mirrors the persist guard.
- case "$src" in
- *.loki/generated-prd.md|*.loki/generated-prd.json) echo ""; return 0 ;;
- esac
- local loki_dir="${TARGET_DIR:-.}/.loki"
- mkdir -p "$loki_dir" "$loki_dir/state" 2>/dev/null || { echo ""; return 0; }
- # Atomic copy: write to a temp file in the destination dir, then mv into
- # place so a concurrent reader never sees a half-written PRD.
- local dest="$loki_dir/generated-prd.md"
- local tmp_prd="$loki_dir/.generated-prd.md.tmp.$$"
- cp -f "$src" "$tmp_prd" 2>/dev/null || { rm -f "$tmp_prd" 2>/dev/null; echo ""; return 0; }
- mv -f "$tmp_prd" "$dest" 2>/dev/null || { rm -f "$tmp_prd" 2>/dev/null; echo ""; return 0; }
- # Content hash of the PRD we just wrote (over the copied file) + the current
- # codebase signature, recorded directly (NOT via persist_prd_signature_if_present,
- # whose guards skip non-canonical/user paths and whose user_owned early-return
- # would skip it anyway). source:"user" makes decide_generated_prd_action short
- # circuit to user_owned on every later no-file run (LOCK 2).
- local prd_sha sig mode
- prd_sha=$(_loki_prd_file_hash "${TARGET_DIR:-.}")
- sig=$(compute_codebase_signature "${TARGET_DIR:-.}")
- mode="files"; case "$sig" in git:*|gitc:*) mode="git" ;; esac
- local sig_tmp="$loki_dir/state/.prd-signature.json.tmp.$$"
- LOKI_SIG="$sig" LOKI_SIG_MODE="$mode" \
- LOKI_SIG_VER="$(get_version 2>/dev/null || echo unknown)" \
- LOKI_PRD_SHA="$prd_sha" LOKI_ORIGIN_PATH="$src" \
- python3 -c "
- import json, os, datetime
- rec = {
- 'signature': os.environ.get('LOKI_SIG',''),
- 'generated_at': datetime.datetime.now(datetime.timezone.utc).isoformat().replace('+00:00','Z'),
- 'prd_path': '.loki/generated-prd.md',
- 'prd_sha': os.environ.get('LOKI_PRD_SHA',''),
- 'mode': os.environ.get('LOKI_SIG_MODE','files'),
- 'loki_version': os.environ.get('LOKI_SIG_VER','unknown'),
- 'source': 'user',
- 'origin_path': os.environ.get('LOKI_ORIGIN_PATH',''),
- }
- print(json.dumps(rec))
- " > "$sig_tmp" 2>/dev/null \
- && mv -f "$sig_tmp" "$loki_dir/state/prd-signature.json" 2>/dev/null \
- || rm -f "$sig_tmp" 2>/dev/null
- echo ".loki/generated-prd.md"
- }
- # generate_proof_of_run: thin fire-and-forget wrapper around the standalone
- # proof-of-run generator (autonomy/lib/proof-generator.py). Runs on both
- # success and failure session ends. The generator owns the schema, redaction
- # chokepoint, and HTML rendering; this wrapper only resolves the path and
- # invokes python3. Never fails the session (|| true at the call site).
- # NOTE: no inline python here on purpose -- keep this wrapper apostrophe-free
- # to avoid the bash single-quote trap.
- generate_proof_of_run() {
- local _result="${1:-0}"
- local gen="$SCRIPT_DIR/lib/proof-generator.py"
- [ -f "$gen" ] || return 0
- local loki_dir="${TARGET_DIR:-.}/.loki"
- [ -d "$loki_dir" ] || return 0
- local ver provider
- ver="$(get_version 2>/dev/null || echo unknown)"
- provider="${PROVIDER_NAME:-claude}"
- # Proven PR (Loop 6 / Slice A2): resolve a deterministic run_id so the
- # proof + the PR Evidence Receipt agree, and persist a stable pointer the PR
- # sites read (never newest-by-mtime). The generator otherwise mints a fresh
- # _gen_run_id() when LOKI_SESSION_ID is unset (the `loki start ./prd.md`
- # case), which the later PR step could not know. Gated on the SAME flag as the
- # receipt so LOKI_PROVEN_PR=0 is a byte-identical no-op (no --run-id passed,
- # no pointer written): the pointer is only ever READ by the renderer, which
- # is itself off under that flag.
- if [ "${LOKI_PROVEN_PR:-1}" != "0" ]; then
- local _rid=""
- if declare -f _loki_trust_run_id >/dev/null 2>&1; then
- _rid="$(_loki_trust_run_id 2>/dev/null || true)"
- fi
- # Fall back to a locally minted id when no persisted per-run id exists.
- # Do NOT call _loki_trust_run_id --new here: that would clobber the
- # trust-events run-id file; this is a read-or-mint-local resolution.
- if [ -z "$_rid" ]; then
- _rid="proof-$(date -u +%Y%m%d%H%M%S 2>/dev/null || echo 0)-$$-${RANDOM:-0}"
- fi
- ITERATION_COUNT="${ITERATION_COUNT:-0}" \
- PROVIDER_NAME="$provider" \
- PRD_PATH="${prd_path:-}" \
- python3 -E "$gen" \
- --loki-dir "$loki_dir" \
- --loki-version "$ver" \
- --provider "$provider" \
- --session-exit-code "$_result" \
- --run-id "$_rid" \
- --quiet >/dev/null 2>&1 || true
- # Persist the resolved run_id atomically (.tmp + mv) so the PR sites read
- # the exact run the generator wrote, never an mtime guess.
- local _id_dir="$loki_dir/state"
- local _id_file="$_id_dir/last-proof-id.txt"
- mkdir -p "$_id_dir" 2>/dev/null || true
- if printf '%s' "$_rid" > "${_id_file}.tmp" 2>/dev/null; then
- mv -f "${_id_file}.tmp" "$_id_file" 2>/dev/null || rm -f "${_id_file}.tmp" 2>/dev/null || true
- fi
- # Opt-in build-outcome analytics (default OFF; strict second-layer gate in
- # telemetry.sh). Reads the just-written receipt's allowlisted scalars only;
- # trust-core untouched. Fire-and-forget, never fails the session.
- if declare -f loki_emit_build_verified >/dev/null 2>&1; then
- loki_emit_build_verified "$loki_dir/proofs/$_rid/proof.json" 2>/dev/null || true
- fi
- return 0
- fi
- ITERATION_COUNT="${ITERATION_COUNT:-0}" \
- PROVIDER_NAME="$provider" \
- PRD_PATH="${prd_path:-}" \
- python3 -E "$gen" \
- --loki-dir "$loki_dir" \
- --loki-version "$ver" \
- --provider "$provider" \
- --session-exit-code "$_result" \
- --quiet >/dev/null 2>&1 || true
- return 0
- }
- # capture_preedit_snapshot: freeze the agent's raw diff BEFORE anything else
- # touches the tree, so quality numbers measure the agent and not the
- # agent-plus-whoever-fixed-it (autonomy/lib/preedit_snapshot.py owns the schema
- # and the write-once rule; this is only the call site).
- #
- # WHY THIS IS NOT INSIDE generate_proof_of_run, even though the receipt is the
- # obvious neighbour. Two reasons, both measured in this file:
- #
- # 1. TOO LATE AT THE LATE PROOF SITES. commit_session_changes commits the
- # session's work, and the module's default baseline is `git diff HEAD`.
- # After that commit `git diff HEAD` is EMPTY, so a capture at the teardown
- # proof site would freeze an empty diff -- and because the snapshot is
- # write-once by design, that empty capture would be permanent and
- # unrecoverable. run.sh already documents this mutation window itself: the
- # comment above the final generate_proof_of_run call says "HANDOFF.md and
- # commit_session_changes can change the worktree after the earlier receipt".
- # The receipt can be regenerated against a later tree; the snapshot cannot.
- # 2. WRONG GATE. Every generate_proof_of_run call site is gated on
- # LOKI_PROOF!=0, and the run_id resolution only exists inside its
- # LOKI_PROVEN_PR!=0 branch. Authorship evidence and shareable proofs are
- # different concerns, so a user who turns off proofs must not silently lose
- # the ability to tell agent output from human edits.
- #
- # So the capture happens EARLIER, immediately after run_autonomous returns,
- # before any post-processing step can modify the diff.
- #
- # Baseline: _LOKI_RUN_START_SHA (exported at runner init, persisted to
- # .loki/state/start-sha) is passed when available, so the snapshot is anchored
- # to the run's own starting commit rather than to a moving HEAD. That makes the
- # capture correct even if a later caller fires after a commit. Falls back to the
- # module's `git diff HEAD` default when no baseline resolved (greenfield repos
- # with no commits write an empty file there by design).
- #
- # run_id: read-path _loki_trust_run_id ONLY, never --new (minting here would
- # clobber the trust-events id file). If it resolves empty we SKIP: a snapshot
- # filed under an id nothing else references is worse than no snapshot, because
- # verdict.py would count it as authorship evidence that no receipt can join to.
- #
- # Guarded and non-fatal throughout: a diagnostic must never break the run it is
- # diagnosing. Write-once makes repeat calls free (later ones return "exists"),
- # so the earliest caller wins and extra call sites cost nothing.
- capture_preedit_snapshot() {
- local snap="$SCRIPT_DIR/lib/preedit_snapshot.py"
- [ -f "$snap" ] || return 0
- command -v python3 >/dev/null 2>&1 || return 0
- # Match _loki_trust_run_id's dir expression, not generate_proof_of_run's:
- # with LOKI_DIR set, ${TARGET_DIR:-.}/.loki would write the snapshot beside
- # a run-id file that lives somewhere else.
- local loki_dir="${LOKI_DIR:-${TARGET_DIR:-.}/.loki}"
- [ -d "$loki_dir" ] || return 0
- local _rid=""
- if declare -f _loki_trust_run_id >/dev/null 2>&1; then
- _rid="$(_loki_trust_run_id 2>/dev/null || true)"
- fi
- [ -n "$_rid" ] || return 0
- # Resolve the baseline from the PERSISTED file when the exported variable is
- # not visible. _LOKI_RUN_START_SHA is exported inside run_autonomous, and in
- # PARALLEL_MODE run_autonomous runs in a subshell -- an export from a
- # subshell never reaches the parent, so at that call site the variable is
- # empty and only the file survives. Same read the pause path already does.
- # Without this the parallel branch would silently fall back to `git diff
- # HEAD` instead of the anchored baseline this function documents.
- local _base="${_LOKI_RUN_START_SHA:-}"
- [ -n "$_base" ] || _base="$(cat "$loki_dir/state/start-sha" 2>/dev/null || true)"
- # LOKI_PREEDIT_CWD is load-bearing: the module defaults cwd to os.getcwd(),
- # and if that is not the target repo capture returns not_a_git_repo and
- # writes nothing SILENTLY -- a call site that looks wired but never fires.
- LOKI_DIR="$loki_dir" \
- LOKI_PREEDIT_CWD="${TARGET_DIR:-.}" \
- LOKI_RUN_START_SHA="$_base" \
- python3 "$snap" capture "$_rid" >/dev/null 2>&1 || true
- return 0
- }
- # print_ttfv_next_steps: R7 zero-config first-run "what next / go deeper"
- # message. The wording MUST match what actually ran, so it branches on the mode:
- # - brief: a one-line brief ran on the lightweight profile (council off,
- # simple tier, capped iterations). Proof contains diffs, cost, time
- # (council verdicts are absent because the council was disabled).
- # - repo: a no-arg in-repo run analyzed the codebase and ran at full depth
- # (council on). Proof contains diffs, cost, time, and council
- # verdicts.
- # This function only prints; the caller owns the TTY gate. Never fails the run.
- # Usage: print_ttfv_next_steps <mode> <result>
- print_ttfv_next_steps() {
- local mode="${1:-}"
- local result="${2:-0}"
- local loki_dir="${TARGET_DIR:-.}/.loki"
- local proofs_dir="$loki_dir/proofs"
- echo ""
- echo "============================================================"
- if [ "$result" = "0" ]; then
- echo " First pass complete. Here is what you have:"
- else
- echo " First pass ended early. Here is what was produced:"
- fi
- echo "============================================================"
- echo ""
- echo " What I did:"
- if [ "$mode" = "brief" ] || [ "$mode" = "repo-fast" ]; then
- if [ "$mode" = "repo-fast" ]; then
- echo " - Inspected your repository on a fast, lightweight first"
- echo " pass (council off, simple tier, capped iterations)."
- else
- echo " - Worked from your one-line brief on a fast, lightweight first"
- echo " pass (council off, simple tier, capped iterations)."
- fi
- echo " - Generated a proof-of-run (diffs, cost, time)."
- else
- echo " - Analyzed your codebase and generated a PRD, then ran a full"
- echo " first pass (council on, full RARV-C depth)."
- echo " - Generated a proof-of-run (diffs, cost, time, council verdicts)."
- fi
- echo ""
- echo " See the visible artifact (proof-of-run):"
- if [ -d "$proofs_dir" ]; then
- local latest
- latest=$(ls -1t "$proofs_dir" 2>/dev/null | head -1)
- if [ -n "$latest" ]; then
- echo " loki proof open $latest"
- echo " (or open $proofs_dir/$latest/index.html)"
- else
- echo " loki proof list"
- fi
- else
- echo " loki proof list"
- fi
- echo ""
- if [ "$mode" = "brief" ] || [ "$mode" = "repo-fast" ]; then
- echo " Go deeper (full RARV-C depth, council-gated):"
- echo " loki start # continue / harden this project"
- echo " loki start ./prd.md # build from a full PRD"
- else
- echo " Next steps:"
- echo " loki start ./prd.md # build from a full PRD"
- echo " loki start \"<one line>\" # fast first pass from a brief"
- fi
- # Frictionless discovery: surface 'loki why' so users learn the
- # what-happened/what-to-do diagnosis without having to know it exists.
- echo " loki why # explain this outcome + what to do next"
- echo ""
- return 0
- }
- # _read_iteration_cost <iteration>
- # Emit "input output cost cache_read cache_creation" for the given iteration,
- # preferring the authoritative result-cost file written by the embedded stream
- # parser (Claude'\''s own total_cost_usd + usage, slug/symlink-independent) over
- # the context-tracker-derived estimate in tracking.json. Falls back to
- # tracking.json when no result-cost file exists, and to all zeros otherwise.
- # Best-effort: any parse failure yields "0 0 0 0 0" and never aborts.
- _read_iteration_cost() {
- local iteration="$1"
- local result_cost_file=".loki/metrics/result-cost-${iteration}.json"
- if [ -f "$result_cost_file" ]; then
- python3 -c "
- import json
- def _c(v):
- return v if isinstance(v, (int, float)) and not isinstance(v, bool) else 'unmeasured'
- try:
- d = json.load(open('$result_cost_file'))
- print(
- d.get('input_tokens', 0) or 0,
- d.get('output_tokens', 0) or 0,
- _c(d.get('total_cost_usd')),
- d.get('cache_read_tokens', 0) or 0,
- d.get('cache_creation_tokens', 0) or 0,
- )
- except Exception:
- print(0, 0, 'unmeasured', 0, 0)
- " 2>/dev/null || echo "0 0 unmeasured 0 0"
- elif [ -f ".loki/context/tracking.json" ]; then
- python3 -c "
- import json
- def _c(v):
- return v if isinstance(v, (int, float)) and not isinstance(v, bool) else 'unmeasured'
- try:
- t = json.load(open('.loki/context/tracking.json'))
- iters = t.get('per_iteration', [])
- match = [i for i in iters if i.get('iteration') == $iteration]
- if match:
- m = match[-1]
- print(
- m.get('input_tokens', 0),
- m.get('output_tokens', 0),
- _c(m.get('cost_usd')),
- m.get('cache_read_tokens', 0),
- m.get('cache_creation_tokens', 0),
- )
- else:
- print(0, 0, 'unmeasured', 0, 0)
- except Exception:
- print(0, 0, 'unmeasured', 0, 0)
- " 2>/dev/null || echo "0 0 unmeasured 0 0"
- else
- echo "0 0 unmeasured 0 0"
- fi
- }
- track_iteration_complete() {
- local iteration="$1"
- local exit_code="${2:-0}"
- local task_id="iteration-$iteration"
- mkdir -p .loki/queue
- # Calculate iteration duration (SYN-018)
- local duration_ms
- duration_ms=$(get_iteration_duration_ms)
- # Emit iteration complete event for dashboard
- local status_str
- [ "$exit_code" = "0" ] && status_str="completed" || status_str="failed"
- emit_event_json "iteration_complete" \
- "iteration=$iteration" \
- "status=$status_str" \
- "exitCode=$exit_code" \
- "provider=${PROVIDER_NAME:-claude}"
- # Also emit to pending dir for OTEL bridge
- if [ -n "${LOKI_OTEL_ENDPOINT:-}" ]; then
- emit_event_pending "iteration_complete" \
- "iteration=$iteration" \
- "status=$status_str" \
- "exit_code=$exit_code"
- fi
- # Emit learning signals based on outcome (SYN-018)
- if [ "$exit_code" = "0" ]; then
- # Success pattern for completed iteration
- emit_learning_signal success_pattern \
- --source cli \
- --action "iteration_complete" \
- --pattern-name "rarv_iteration" \
- --action-sequence '["reason", "act", "reflect", "verify"]' \
- --duration "$((duration_ms / 1000))" \
- --outcome success \
- --context "{\"iteration\":$iteration,\"provider\":\"${PROVIDER_NAME:-claude}\"}"
- # Tool efficiency signal
- emit_learning_signal tool_efficiency \
- --source cli \
- --action "iteration_complete" \
- --tool-name "${PROVIDER_NAME:-claude}" \
- --execution-time-ms "$duration_ms" \
- --outcome success \
- --context "{\"iteration\":$iteration}"
- else
- # Error pattern for failed iteration
- emit_learning_signal error_pattern \
- --source cli \
- --action "iteration_complete" \
- --error-type "IterationFailure" \
- --error-message "Iteration $iteration failed with exit code $exit_code" \
- --recovery-steps '["Check logs", "Review error output", "Retry iteration"]' \
- --context "{\"iteration\":$iteration,\"provider\":\"${PROVIDER_NAME:-claude}\",\"exit_code\":$exit_code}"
- # Tool efficiency signal with failure
- emit_learning_signal tool_efficiency \
- --source cli \
- --action "iteration_failed" \
- --tool-name "${PROVIDER_NAME:-claude}" \
- --execution-time-ms "$duration_ms" \
- --outcome failure \
- --context "{\"iteration\":$iteration,\"exit_code\":$exit_code}"
- fi
- # Track context window usage FIRST to get token data (v5.42.0)
- track_context_usage "$iteration"
- # Write efficiency tracking file for /api/cost endpoint.
- # The recorded model MUST be the one that actually ran this iteration. The
- # dispatch exports LOKI_CURRENT_MODEL="$tier_param" (the exact --model value)
- # right before the provider call, AFTER every mutation (opus-pin force,
- # LOKI_MAX_TIER clamp, mid-flight override). That is the single source of
- # truth -- reading it avoids re-deriving the model here and drifting from
- # dispatch. Hardcoding PROVIDER_MODEL_DEVELOPMENT (the old code) mislabeled
- # every non-development iteration: a haiku/fast pin was recorded "sonnet" and
- # an opus pin (tier_param forced to opus) was recorded "sonnet" too, making
- # the model-equivalence bench unfalsifiable. Resolver is the fallback for the
- # policy-blocked path where no dispatch ran (LOKI_CURRENT_MODEL unset).
- mkdir -p .loki/metrics/efficiency
- local model_tier="${LOKI_CURRENT_MODEL:-}"
- if [ -z "$model_tier" ]; then
- model_tier="$(get_provider_tier_param "${CURRENT_TIER:-development}" 2>/dev/null)"
- fi
- # Fallback to the old per-provider default only if both the dispatched model
- # and the resolver are unavailable (e.g. provider config not sourced) so we
- # never write an empty model.
- if [ -z "$model_tier" ]; then
- model_tier="${PROVIDER_MODEL_DEVELOPMENT:-sonnet}"
- fi
- local phase="${LAST_KNOWN_PHASE:-}"
- [ -z "$phase" ] && phase=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('currentPhase', 'unknown'))" 2>/dev/null || echo "unknown")
- # Read token data, preferring Claude'\''s authoritative result-cost file over
- # the context-tracker estimate (v7.28.0 cost-capture fix). See
- # _read_iteration_cost for precedence rationale.
- # v6.82.0: also capture cache_read_tokens / cache_creation_tokens for
- # prompt-cache hit-rate analysis (S1.1 prompt restructure).
- local iter_input=0 iter_output=0 iter_cost=0 iter_cost_line=""
- local iter_cache_read=0 iter_cache_creation=0
- iter_cost_line="$(_read_iteration_cost "$iteration")"
- read -r iter_input iter_output iter_cost iter_cache_read iter_cache_creation << ITER_COST_EOF
- $iter_cost_line
- ITER_COST_EOF
- # Unknown cost is omitted with a reason, never written as 0 (readers treat absence as unmeasured).
- local iter_cost_json
- case "${iter_cost:-unmeasured}" in
- unmeasured) iter_cost_json=' "cost_unmeasured_reason": "no dollar figure recorded for this iteration",
- ' ;;
- *) iter_cost_json=" \"cost_usd\": ${iter_cost},
- " ;;
- esac
- cat > ".loki/metrics/efficiency/iteration-${iteration}.json" << EFF_EOF
- {
- "iteration": $iteration,
- "model": "$model_tier",
- "phase": "$phase",
- "duration_ms": $duration_ms,
- "provider": "${PROVIDER_NAME:-claude}",
- "status": "$status_str",
- "input_tokens": ${iter_input:-0},
- "output_tokens": ${iter_output:-0},
- "cache_read_tokens": ${iter_cache_read:-0},
- "cache_creation_tokens": ${iter_cache_creation:-0},
- ${iter_cost_json} "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- }
- EFF_EOF
- # Check notification triggers (v5.40.0)
- check_notification_triggers "$iteration"
- # Sync completed GitHub tasks back to issues (v5.41.0)
- sync_github_completed_tasks
- # Get task from in-progress
- local in_progress_file=".loki/queue/in-progress.json"
- local completed_file=".loki/queue/completed.json"
- local failed_file=".loki/queue/failed.json"
- # Initialize files if needed
- [ ! -f "$completed_file" ] && echo "[]" > "$completed_file"
- [ ! -f "$failed_file" ] && echo "[]" > "$failed_file"
- # Build the completed iteration record + move it out of in-progress, ATOMICALLY.
- # v7.104.3 task-list accuracy fix: the old writer emitted a THIN body
- # {id,type,title:"Iteration N",status,exitCode,provider} with no description
- # and no logs, and then DELETED the rich in-progress record -- so the done
- # column showed empty cards. We now LIFT the honest per-iteration parts (logs,
- # startedAt) from the in-progress record BEFORE removing it, and give the card
- # an HONEST iteration-scoped title/description derived from values in scope
- # ($phase/$exit_code/$duration_ms). We deliberately do NOT reuse the borrowed
- # PRD-story title: because pending stories never leave the queue, the
- # in-progress title is always pending[0] ("server.js..."), so carrying it onto
- # N done cards would falsely imply that story was built N times (fake-green).
- # We also upsert-by-id (no cross-sub-run "iteration-1 x5" accumulation) and
- # keep completed/failed mutually exclusive per id.
- local target_file="$completed_file"
- local other_file="$failed_file"
- [ "$exit_code" != "0" ] && { target_file="$failed_file"; other_file="$completed_file"; }
- local _completed_ts; _completed_ts="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- _LOKI_TASK_ID="$task_id" \
- _LOKI_ITER="$iteration" \
- _LOKI_PHASE="$phase" \
- _LOKI_EXIT="$exit_code" \
- _LOKI_DUR="$duration_ms" \
- _LOKI_PROVIDER="${PROVIDER_NAME:-claude}" \
- _LOKI_COMPLETED_TS="$_completed_ts" \
- _LOKI_TARGET="$target_file" \
- _LOKI_OTHER="$other_file" \
- _LOKI_INPROG="$in_progress_file" \
- python3 -c "
- import json, os
- tid = os.environ['_LOKI_TASK_ID']
- it = os.environ['_LOKI_ITER']
- phase = os.environ.get('_LOKI_PHASE') or 'unknown'
- exit_code = os.environ.get('_LOKI_EXIT', '1')
- dur = os.environ.get('_LOKI_DUR', '')
- provider = os.environ.get('_LOKI_PROVIDER', 'claude')
- completed_ts = os.environ['_LOKI_COMPLETED_TS']
- target = os.environ['_LOKI_TARGET']
- other = os.environ['_LOKI_OTHER']
- inprog = os.environ['_LOKI_INPROG']
- ok = (exit_code == '0')
- def load(p):
- try:
- with open(p) as f:
- d = json.load(f)
- return d if isinstance(d, list) else (d.get('tasks', []) if isinstance(d, dict) else [])
- except Exception:
- return []
- # Lift the honest body from the in-progress record before removing it.
- prior = next((t for t in load(inprog) if isinstance(t, dict) and t.get('id') == tid), {})
- logs = prior.get('logs') if isinstance(prior.get('logs'), list) else []
- started = prior.get('startedAt')
- dur_s = ''
- try:
- if dur not in ('', None):
- _ms = int(float(dur))
- dur_s = f\", {_ms}ms\" if _ms < 1000 else f\", {round(_ms/1000)}s\"
- except Exception:
- dur_s = ''
- if ok:
- title = f'Iteration {it} complete - {phase}'
- desc = f'Iteration {it} finished cleanly in the {phase} phase (exit 0{dur_s}).'
- else:
- title = f'Iteration {it} failed (exit {exit_code})'
- desc = f'Iteration {it} ended in the {phase} phase with exit code {exit_code}{dur_s}.'
- entry = {
- 'id': tid,
- 'type': 'iteration',
- 'title': title,
- 'description': desc,
- 'status': 'completed' if ok else 'failed',
- 'phase': phase,
- 'completedAt': completed_ts,
- 'exitCode': int(exit_code) if str(exit_code).lstrip('-').isdigit() else exit_code,
- 'provider': provider,
- 'logs': logs,
- }
- if started:
- entry['startedAt'] = started
- # Upsert-by-id into target (drop any stale same-id), cap at 50.
- data = [t for t in load(target) if not (isinstance(t, dict) and t.get('id') == tid)]
- data.append(entry)
- data = data[-50:]
- with open(target, 'w') as f:
- json.dump(data, f, indent=2)
- # Mutual exclusion: remove this id from the OTHER terminal file.
- odata = [t for t in load(other) if not (isinstance(t, dict) and t.get('id') == tid)]
- with open(other, 'w') as f:
- json.dump(odata, f, indent=2)
- # Remove from in-progress.
- idata = [t for t in load(inprog) if not (isinstance(t, dict) and t.get('id') == tid)]
- with open(inprog, 'w') as f:
- json.dump(idata, f, indent=2)
- " 2>/dev/null || echo "[{\"id\":\"$task_id\",\"type\":\"iteration\",\"title\":\"Iteration $iteration\",\"status\":\"$([ "$exit_code" = "0" ] && echo completed || echo failed)\",\"completedAt\":\"$_completed_ts\",\"exitCode\":$exit_code,\"provider\":\"${PROVIDER_NAME:-claude}\"}]" > "$target_file"
- # BUG-ST-014: Atomic current-task.json clear via temp file + mv
- local ct_tmp=".loki/queue/current-task.json.tmp.$$"
- echo "{}" > "$ct_tmp"
- mv -f "$ct_tmp" .loki/queue/current-task.json
- # Write-back completed BMAD stories to source artifacts (v6.29.0)
- if [ "$exit_code" = "0" ]; then
- bmad_write_back
- fi
- }
- start_status_monitor() {
- loki_background_services_enabled || return 0
- log_step "Starting status monitor..."
- # Initial update
- update_status_file
- update_agents_state
- write_dashboard_state
- # Background update loop (2-second interval for realtime dashboard)
- (
- while true; do
- update_status_file
- update_agents_state
- write_dashboard_state
- sleep 2
- done
- ) &
- STATUS_MONITOR_PID=$!
- register_pid "$STATUS_MONITOR_PID" "status-monitor"
- log_info "Status monitor started"
- log_info "Monitor progress: ${CYAN}watch -n 2 cat .loki/STATUS.txt${NC}"
- }
- stop_status_monitor() {
- if [ -n "$STATUS_MONITOR_PID" ]; then
- kill "$STATUS_MONITOR_PID" 2>/dev/null || true
- wait "$STATUS_MONITOR_PID" 2>/dev/null || true
- unregister_pid "$STATUS_MONITOR_PID"
- fi
- stop_resource_monitor
- }
- #===============================================================================
- # Web Dashboard
- #===============================================================================
- update_agents_state() {
- # Aggregate agent information from .agent/sub-agents/*.json into .loki/state/agents.json
- local agents_dir=".agent/sub-agents"
- local output_file=".loki/state/agents.json"
- # Initialize empty array if no agents directory
- if [ ! -d "$agents_dir" ]; then
- echo "[]" > "$output_file"
- return
- fi
- # Find all agent JSON files and aggregate them
- local agents_json="["
- local first=true
- for agent_file in "$agents_dir"/*.json; do
- # Skip if no JSON files exist
- [ -e "$agent_file" ] || continue
- # Read agent JSON
- local agent_data=$(cat "$agent_file" 2>/dev/null)
- if [ -n "$agent_data" ]; then
- # Add comma separator for all but first entry
- if [ "$first" = true ]; then
- first=false
- else
- agents_json="${agents_json},"
- fi
- agents_json="${agents_json}${agent_data}"
- fi
- done
- agents_json="${agents_json}]"
- # Write aggregated data (atomic via temp file + mv)
- local tmp_file="${output_file}.tmp.$$"
- echo "$agents_json" > "$tmp_file"
- mv -f "$tmp_file" "$output_file" 2>/dev/null || rm -f "$tmp_file"
- }
- #===============================================================================
- # Resource Monitoring
- #===============================================================================
- check_system_resources() {
- # Check CPU and memory usage and write status to .loki/state/resources.json
- local output_file=".loki/state/resources.json"
- # Get CPU usage (average across all cores)
- local cpu_usage=0
- if [[ "$OSTYPE" == "darwin"* ]]; then
- # macOS: get CPU idle from top header, calculate usage = 100 - idle
- local idle=$(top -l 2 -n 0 | grep "CPU usage" | tail -1 | awk -F'[:,]' '{for(i=1;i<=NF;i++) if($i ~ /idle/) print $(i)}' | awk '{print int($1)}')
- cpu_usage=$((100 - ${idle:-0}))
- elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
- # Linux: use top or mpstat
- cpu_usage=$(top -bn2 | grep "Cpu(s)" | tail -1 | sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | awk '{print int(100 - $1)}')
- else
- cpu_usage=0
- fi
- # Get memory usage
- local mem_usage=0
- if [[ "$OSTYPE" == "darwin"* ]]; then
- # macOS: use vm_stat
- local page_size=$(pagesize)
- local vm_stat=$(vm_stat)
- local pages_free=$(echo "$vm_stat" | awk '/Pages free/ {print $3}' | tr -d '.')
- local pages_active=$(echo "$vm_stat" | awk '/Pages active/ {print $3}' | tr -d '.')
- local pages_inactive=$(echo "$vm_stat" | awk '/Pages inactive/ {print $3}' | tr -d '.')
- local pages_speculative=$(echo "$vm_stat" | awk '/Pages speculative/ {print $3}' | tr -d '.')
- local pages_wired=$(echo "$vm_stat" | awk '/Pages wired down/ {print $4}' | tr -d '.')
- local total_pages=$((pages_free + pages_active + pages_inactive + pages_speculative + pages_wired))
- local used_pages=$((pages_active + pages_wired))
- mem_usage=$((used_pages * 100 / total_pages))
- elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
- # Linux: use free
- mem_usage=$(free | grep Mem | awk '{print int($3/$2 * 100)}')
- else
- mem_usage=0
- fi
- # Determine status
- local cpu_status="ok"
- local mem_status="ok"
- local overall_status="ok"
- local warning_message=""
- if [ "$cpu_usage" -ge "$RESOURCE_CPU_THRESHOLD" ]; then
- cpu_status="high"
- overall_status="warning"
- warning_message="CPU usage is ${cpu_usage}% (threshold: ${RESOURCE_CPU_THRESHOLD}%). Consider reducing parallel agent count or pausing non-critical tasks."
- fi
- if [ "$mem_usage" -ge "$RESOURCE_MEM_THRESHOLD" ]; then
- mem_status="high"
- overall_status="warning"
- if [ -n "$warning_message" ]; then
- warning_message="${warning_message} Memory usage is ${mem_usage}% (threshold: ${RESOURCE_MEM_THRESHOLD}%)."
- else
- warning_message="Memory usage is ${mem_usage}% (threshold: ${RESOURCE_MEM_THRESHOLD}%). Consider reducing parallel agent count or cleaning up resources."
- fi
- fi
- # Write JSON status
- cat > "$output_file" << EOF
- {
- "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
- "cpu": {
- "usage_percent": $cpu_usage,
- "threshold_percent": $RESOURCE_CPU_THRESHOLD,
- "status": "$cpu_status"
- },
- "memory": {
- "usage_percent": $mem_usage,
- "threshold_percent": $RESOURCE_MEM_THRESHOLD,
- "status": "$mem_status"
- },
- "overall_status": "$overall_status",
- "warning_message": "$warning_message"
- }
- EOF
- # Log warning if resources are high
- if [ "$overall_status" = "warning" ]; then
- log_warn "RESOURCE WARNING: $warning_message"
- fi
- }
- start_resource_monitor() {
- loki_background_services_enabled || return 0
- log_step "Starting resource monitor (checks every ${RESOURCE_CHECK_INTERVAL}s)..."
- # Initial check
- check_system_resources
- # Background monitoring loop. BACKLOG 22: a bare `sleep N` inside this
- # subshell survives a TERM to the subshell -- caught traps do not carry
- # into a foreground child, so the sleep reparents to init and outlives the
- # run for up to RESOURCE_CHECK_INTERVAL seconds. Backgrounding the sleep
- # and recording its PID lets a TERM trap kill it directly; `wait` returns
- # immediately once the trap fires, so shutdown is not delayed either.
- (
- _loki_rm_sleep_pid=""
- trap '[ -n "$_loki_rm_sleep_pid" ] && kill "$_loki_rm_sleep_pid" 2>/dev/null; exit 0' TERM
- while true; do
- sleep "$RESOURCE_CHECK_INTERVAL" &
- _loki_rm_sleep_pid=$!
- wait "$_loki_rm_sleep_pid"
- _loki_rm_sleep_pid=""
- check_system_resources
- done
- ) &
- RESOURCE_MONITOR_PID=$!
- register_pid "$RESOURCE_MONITOR_PID" "resource-monitor"
- log_info "Resource monitor started (CPU threshold: ${RESOURCE_CPU_THRESHOLD}%, Memory threshold: ${RESOURCE_MEM_THRESHOLD}%)"
- log_info "Check status: ${CYAN}cat .loki/state/resources.json${NC}"
- }
- stop_resource_monitor() {
- if [ -n "$RESOURCE_MONITOR_PID" ]; then
- kill "$RESOURCE_MONITOR_PID" 2>/dev/null || true
- wait "$RESOURCE_MONITOR_PID" 2>/dev/null || true
- unregister_pid "$RESOURCE_MONITOR_PID"
- fi
- }
- #===============================================================================
- # Audit Logging (Enterprise Security)
- #===============================================================================
- audit_log() {
- # Log security-relevant events for enterprise compliance
- local event_type="$1"
- local event_data="$2"
- local audit_file=".loki/logs/audit-$(date +%Y%m%d).jsonl"
- if [ "$AUDIT_LOG_ENABLED" != "true" ]; then
- return
- fi
- mkdir -p .loki/logs
- local log_entry
- if command -v jq >/dev/null 2>&1; then
- log_entry=$(jq -n --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg evt "$event_type" --arg data "$event_data" --arg user "$(whoami)" --argjson pid "$$" '{timestamp:$ts,event:$evt,data:$data,user:$user,pid:$pid}')
- else
- local safe_type safe_data
- safe_type=$(printf '%s' "$event_type" | sed 's/["\\]/\\&/g; s/\n/\\n/g')
- safe_data=$(printf '%s' "$event_data" | sed 's/["\\]/\\&/g; s/\n/\\n/g')
- log_entry="{\"timestamp\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"event\":\"$safe_type\",\"data\":\"$safe_data\",\"user\":\"$(whoami)\",\"pid\":$$}"
- fi
- echo "$log_entry" >> "$audit_file"
- }
- #===============================================================================
- # Engine-owned workspace git-init (Plan #16, Option A-1)
- #===============================================================================
- # Resolve a path to its physical absolute form (symlinks + .. collapsed) using
- # whatever is available; falls back to the input unchanged. Portable across the
- # BSD (macOS) and GNU userlands the engine runs on.
- _loki_resolve_path() {
- local p="${1:-}"
- [ -n "$p" ] || { printf '%s' ""; return 0; }
- if command -v realpath >/dev/null 2>&1; then
- realpath "$p" 2>/dev/null && return 0
- fi
- # python3 is a hard engine dependency; use it as the portable fallback.
- python3 - "$p" <<'PYRESOLVE' 2>/dev/null && return 0
- import os, sys
- print(os.path.realpath(sys.argv[1]))
- PYRESOLVE
- printf '%s' "$p"
- }
- # True (0) when TARGET_DIR is an engine-owned, freshly-minted build workspace
- # that Loki may auto-git-init without surprising a user. Two honest signals:
- # 1. LOKI_AUTO_GIT_INIT=1 -- explicit opt-in (non-SaaS automation).
- # 2. LOKI_TARGET_DIR is set AND realpath-contained under one of the
- # colon-separated LOKI_WORKSPACE_ROOTS dirs (the v7.91.0 SaaS route: the
- # BFF mints <root>/<buildId> and the server pins LOKI_TARGET_DIR to it).
- # A user's own folder (`loki start ./prd.md`, no workspace, roots unset) is
- # NEVER engine-owned -- it must not get a silent .git. Realpath containment (not
- # a prefix string match) so /root/build-other does not match /root/build.
- _loki_workspace_is_engine_owned() {
- [ "${LOKI_AUTO_GIT_INIT:-0}" = "1" ] && return 0
- local roots_raw="${LOKI_WORKSPACE_ROOTS:-}"
- [ -n "${LOKI_TARGET_DIR:-}" ] || return 1
- [ -n "$roots_raw" ] || return 1
- local ws_real root_real
- ws_real="$(_loki_resolve_path "${TARGET_DIR:-.}")"
- [ -n "$ws_real" ] || return 1
- local IFS=':'
- local root
- for root in $roots_raw; do
- [ -n "$root" ] || continue
- root_real="$(_loki_resolve_path "$root")"
- [ -n "$root_real" ] || continue
- # Exact match or contained: ws == root, or ws starts with root + "/".
- if [ "$ws_real" = "$root_real" ] || case "$ws_real" in "$root_real"/*) true ;; *) false ;; esac; then
- return 0
- fi
- done
- return 1
- }
- # Engine-owned workspaces share their host with the Autonomi control plane.
- # Claude Bash commands in those workspaces must load Loki's trusted PreToolUse
- # guard explicitly, rather than relying on mutable project settings.
- _loki_host_guard_required() {
- case "${LOKI_HOST_GUARD:-0}" in
- 1|true|yes|on) return 0 ;;
- esac
- [ -n "${LOKI_TARGET_DIR:-}" ] || return 1
- [ -n "${LOKI_WORKSPACE_ROOTS:-}" ] || return 1
- LOKI_AUTO_GIT_INIT=0 _loki_workspace_is_engine_owned
- }
- # Host-guard hook timeout, seconds. Mirrors hookTimeoutSeconds in
- # loki-ts/src/runner/providers.ts (same env var, default 30, positive int <= 99999).
- _loki_hook_timeout() {
- case "${LOKI_HOOK_TIMEOUT:-}" in
- [1-9] | [1-9][0-9] | [1-9][0-9][0-9] | [1-9][0-9][0-9][0-9] | [1-9][0-9][0-9][0-9][0-9]) printf '%s\n' "$LOKI_HOOK_TIMEOUT" ;;
- *) printf '%s\n' 30 ;;
- esac
- }
- _loki_prepare_host_guard() {
- _loki_host_guard_required || return 0
- if [ "${PROVIDER_NAME:-claude}" != "claude" ]; then
- log_error "Host command guard is required, but provider '${PROVIDER_NAME:-unknown}' has no enforced PreToolUse hook. Refusing to start."
- return 1
- fi
- if ! command -v python3 >/dev/null 2>&1; then
- log_error "Host command guard requires python3 for deterministic command inspection. Refusing to start."
- return 1
- fi
- local hook_path="${PROJECT_DIR}/autonomy/hooks/validate-bash.sh"
- if [ ! -r "$hook_path" ]; then
- log_error "Host command guard hook is missing or unreadable: $hook_path"
- return 1
- fi
- if ! type loki_claude_flag_supported >/dev/null 2>&1 \
- || ! loki_claude_flag_supported "--settings"; then
- log_error "Host command guard requires a Claude CLI with --settings support. Refusing to start."
- return 1
- fi
- local settings_json
- settings_json=$(_LOKI_HOST_GUARD_HOOK="$hook_path" _LOKI_HOOK_TIMEOUT="$(_loki_hook_timeout)" python3 -c '
- import json, os, shlex
- command = "bash " + shlex.quote(os.environ["_LOKI_HOST_GUARD_HOOK"])
- print(json.dumps({"hooks": {"PreToolUse": [{
- "matcher": "Bash",
- "hooks": [{"type": "command", "command": command, "timeout": int(os.environ["_LOKI_HOOK_TIMEOUT"])}],
- }]}}))
- ' 2>/dev/null) || settings_json=""
- if [ -z "$settings_json" ]; then
- log_error "Host command guard settings could not be generated. Refusing to start."
- return 1
- fi
- export LOKI_HOST_GUARD=1
- export LOKI_HOST_GUARD_SETTINGS_JSON="$settings_json"
- return 0
- }
- # Plan #16 Option A-1: establish git in an engine-owned build workspace so the
- # review/verify gate (run_code_review) and branch-isolation chain can actually
- # run. Without git history, run_code_review's diff resolves empty and the gate
- # SKIPS (silently reporting PASS) -- so a build that was never reviewed could
- # earn a VERIFIED receipt. This makes the gate RUN; it does NOT force a green
- # (a build whose review/tests fail still gets an honest verdict).
- #
- # Constraints honored:
- # - Engine-owned workspaces ONLY (see _loki_workspace_is_engine_owned). A
- # user's own folder is never silently git-init'd.
- # - Already a git repo (user's repo OR the engine source tree) -> NO-OP. Only
- # a non-git workspace is initialized.
- # - One INITIAL commit (not a bare init): a zero-commit unborn HEAD breaks the
- # start-SHA capture (git rev-parse HEAD) and re-trips the HEAD~1 skip. The
- # commit uses --allow-empty because a greenfield workspace at build start
- # holds only .loki/ (the spec lands in .loki/specs/), which .loki/.gitignore
- # excludes -> nothing to stage -> a bare commit would fail and leave an
- # unborn HEAD. --allow-empty guarantees a real HEAD either way.
- # - Neutral repo-local identity (loki-build) so the initial commit AND the
- # later commit_session_changes commit never inherit a developer's global git
- # identity. Repo-local sticks on a freshly-init'd workspace (no revert hook).
- # - Secrets never committed: brownfield files are staged through the same
- # secret-scan guard (_commit_path_looks_secret / _commit_scan_secret_file)
- # used by commit_session_changes; any offender unstages the whole set and
- # the initial commit falls back to --allow-empty (HEAD still established).
- maybe_git_init_engine_workspace() {
- local ws="${TARGET_DIR:-.}"
- if ! command -v git >/dev/null 2>&1; then
- loki_is_supervised_simple_web && log_error "Git is required for supervised review baselines"
- loki_is_supervised_simple_web && return 1
- return 0
- fi
- if [ ! -d "$ws" ]; then
- loki_is_supervised_simple_web && return 1
- return 0
- fi
- # Already a git repo (user repo or engine source tree): do nothing.
- if git -C "$ws" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
- if loki_is_supervised_simple_web \
- && ! git -C "$ws" rev-parse --verify HEAD >/dev/null 2>&1; then
- log_error "Supervised workspace has no Git baseline commit"
- return 1
- fi
- return 0
- fi
- if ! _loki_workspace_is_engine_owned; then
- loki_is_supervised_simple_web && log_error "Supervised workspace is not eligible for Git baseline initialization"
- loki_is_supervised_simple_web && return 1
- return 0
- fi
- log_info "Engine-owned workspace is not a git repo; initializing for review/verify gates"
- if ! git -C "$ws" init -q >/dev/null 2>&1; then
- if loki_is_supervised_simple_web; then
- log_error "git init failed in supervised engine workspace"
- return 1
- fi
- log_warn "git init failed in engine workspace; review gate will skip (non-fatal)"
- return 0
- fi
- # Neutral repo-local identity for the initial AND session-end commits.
- git -C "$ws" config user.name "loki-build" >/dev/null 2>&1 || true
- git -C "$ws" config user.email "loki-build@autonomi.dev" >/dev/null 2>&1 || true
- # Self-ignore .loki/ runtime state so no commit ever stages it (mirrors
- # setup_agent_branch). Idempotent.
- mkdir -p "$ws/.loki" 2>/dev/null || true
- [ -f "$ws/.loki/.gitignore" ] || printf '*\n' > "$ws/.loki/.gitignore" 2>/dev/null || true
- # Stage everything except .loki/ and an obvious-secret-path denylist (same
- # first-cut excludes commit_session_changes uses). The scan loop below is the
- # real guarantee for nested/weak secrets.
- git -C "$ws" add -A \
- ':!.loki' ':!.loki/' \
- ':!.env' ':!.env.*' ':!*.env' \
- ':!*.key' ':!*.pem' ':!*.p12' ':!*.keystore' \
- ':!id_rsa*' ':!*.token' ':!credentials*' >/dev/null 2>&1 || true
- # Secret-scan staged files. ANY offender -> unstage all (safe default: never
- # commit a possible secret). The --allow-empty commit below still runs so a
- # real HEAD is established regardless.
- local _offenders=""
- local _f _staged_paths
- _staged_paths=$(mktemp "$ws/.loki/.baseline-staged.XXXXXX") || {
- log_error "Initial Git baseline could not allocate a build-local scan file"
- return 1
- }
- if ! git -C "$ws" diff --cached --name-only -z > "$_staged_paths" 2>/dev/null; then
- rm -f "$_staged_paths"
- log_error "Initial Git baseline could not enumerate staged files"
- return 1
- fi
- while IFS= read -r -d '' _f; do
- [ -f "$ws/$_f" ] || continue
- if _commit_path_looks_secret "$ws/$_f" || _commit_scan_secret_file "$ws/$_f"; then
- _offenders="${_offenders}${_offenders:+, }${_f}"
- fi
- done < "$_staged_paths"
- rm -f "$_staged_paths"
- if [ -n "$_offenders" ]; then
- git -C "$ws" reset >/dev/null 2>&1 || true
- log_warn "Initial commit: possible secret in ${_offenders}; left unstaged (baseline commit will be empty)"
- fi
- # ONE initial commit. --allow-empty: a greenfield workspace stages nothing
- # (only .loki/, which is ignored), so a bare commit would fail and leave an
- # unborn HEAD -- the exact failure mode this whole block exists to avoid.
- if git -C "$ws" commit --allow-empty -q -m "loki: initial build workspace baseline" >/dev/null 2>&1; then
- log_info "Initialized git in engine workspace (initial baseline commit created)"
- audit_log "WORKSPACE_GIT_INIT" "workspace=$ws"
- else
- if loki_is_supervised_simple_web; then
- log_error "Initial supervised Git baseline commit failed"
- return 1
- fi
- log_warn "Initial baseline commit failed; review gate may skip (non-fatal)"
- fi
- if loki_is_supervised_simple_web \
- && ! git -C "$ws" rev-parse --verify HEAD >/dev/null 2>&1; then
- log_error "Supervised Git baseline could not be verified"
- return 1
- fi
- return 0
- }
- #===============================================================================
- # Branch Protection for Agent Changes
- #===============================================================================
- setup_agent_branch() {
- # Create an isolated feature branch for agent changes off the branch Loki
- # was run from. This keeps the user's working branch clean and leaves work
- # on a feature branch ready to PR.
- # Controlled by LOKI_BRANCH_PROTECTION env var (default: true). Set it to
- # "false" to opt out fully and work on the current branch (back-compat).
- local branch_protection="${LOKI_BRANCH_PROTECTION:-true}"
- # Set to 1 only once THIS run has a current snapshot (mint or resume
- # union); _loki_record_session_created records nothing otherwise, since a
- # leftover or stale snapshot would make it claim a user's file.
- _LOKI_SNAPSHOT_THIS_RUN=0
- if [ "$branch_protection" != "true" ]; then
- log_info "Branch protection disabled (LOKI_BRANCH_PROTECTION=${branch_protection})"
- return 0
- fi
- # Need git to do anything here.
- command -v git >/dev/null 2>&1 || { log_warn "git not available - skipping branch protection"; return 0; }
- # Ensure we are inside a git repository
- if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
- log_warn "Not a git repository - skipping branch protection"
- return 0
- fi
- # Self-ignore .loki/ so NO git add (ours or the user's own `git add -A`)
- # can ever stage runtime state (checkpoints, semantic memory, etc.). This is
- # robust regardless of the repo's own .gitignore and applies brownfield and
- # greenfield. Idempotent: write only when missing. Never fatal.
- mkdir -p .loki 2>/dev/null || true
- [ -f .loki/.gitignore ] || printf '*\n' > .loki/.gitignore 2>/dev/null || true
- # Capture the ref Loki was run from. Detached HEAD yields the literal "HEAD".
- # symbolic-ref also names an unborn branch (fresh `git init`), where
- # rev-parse printed "HEAD" and failed, recording "HEAD" twice as the base.
- local cur=""
- cur="$(git symbolic-ref --short -q HEAD 2>/dev/null || echo HEAD)"
- # Detached HEAD: do NOT branch, do NOT fabricate a base (LOCK A2/A6).
- if [ "$cur" = "HEAD" ]; then
- log_info "Detached HEAD; staying on current commit, no feature branch created"
- return 0
- fi
- # Already on a loki branch (session-* or delegate-*): idempotent reuse, do
- # not nest a branch off a loki branch (LOCK A5/A7).
- # Both reuse paths add what is untracked now to the pre-existing list (a
- # file the user made between sessions is theirs), except the files an
- # unfinished previous session created (_loki_resume_snapshot).
- case "$cur" in
- loki/*)
- log_info "Already on loki branch ${cur}"
- mkdir -p .loki/state 2>/dev/null || true
- printf '%s\n' "$cur" > .loki/state/agent-branch.txt 2>/dev/null || true
- _loki_resume_snapshot
- return 0
- ;;
- esac
- # Resume reuse: if a prior session recorded a checkout-able branch, reuse it
- # instead of minting a new one (LOCK A5).
- local recorded=""
- if [ -s .loki/state/agent-branch.txt ]; then
- recorded="$(cat .loki/state/agent-branch.txt 2>/dev/null || true)"
- if [ -n "$recorded" ] && git rev-parse --verify "$recorded" >/dev/null 2>&1; then
- # --no-overwrite-ignore: git's default checkout silently replaces a
- # gitignored user file at a path the session branch tracks, and the
- # file is gone when the user switches back. On conflict the resume
- # is refused and a fresh session branch (with a fresh snapshot) is
- # minted below.
- local resume_err="" resume_files=""
- if resume_err="$(git checkout --no-overwrite-ignore "$recorded" 2>&1 >/dev/null)"; then
- log_info "Resuming on recorded agent branch: ${recorded}"
- _loki_resume_snapshot
- return 0
- fi
- # git lists the blocking paths as tab-indented lines.
- resume_files="$(printf '%s\n' "$resume_err" | sed -n 's/^[[:space:]][[:space:]]*//p' | head -5 | paste -sd, - | sed 's/,/, /g')"
- if [ -n "$resume_files" ]; then
- log_warn "Recorded agent branch ${recorded} could not be checked out because it would overwrite your ignored file(s): ${resume_files} - creating a new one; the earlier commits stay on ${recorded}"
- else
- log_warn "Recorded agent branch ${recorded} could not be checked out - creating a new one; the earlier commits stay on ${recorded}"
- fi
- fi
- fi
- # Fresh run: mint and check out the feature branch, then persist its base
- # (fresh-run-only, LOCK A2).
- local timestamp
- timestamp=$(date +%s)
- local branch_name="loki/session-${timestamp}-$$"
- mkdir -p .loki/state 2>/dev/null || true
- log_info "Branch protection enabled - creating agent branch: $branch_name (base: $cur)"
- # Record the user's own untracked and gitignored files so
- # commit_session_changes and the receipt leave them alone.
- local snap_ok=1 leftover="" gittool=""
- if ! _loki_snapshot_or_fail_closed; then
- snap_ok=0
- rm -f .loki/state/preexisting-untracked.sha.z 2>/dev/null
- # BACKLOG 88: the receipt (workspace_diff._preexisting_untracked) reads
- # preexisting-untracked.z on its own, independent of
- # _LOKI_SNAPSHOT_THIS_RUN below. Leaving it absent after a failed
- # snapshot (old code: `rm -f` here too) makes the receipt treat EVERY
- # currently-untracked file as this run's own work, even though nothing
- # is actually committed -- a pre-existing file the user never touched
- # gets named as the run's. `ls-files --others --exclude-standard` needs
- # no --no-renames or --ignored=matching, so it still works on the git
- # that just failed the richer `status` call above, giving the receipt a
- # names-only fallback list. Ignored files are not covered by this
- # fallback (old-git territory), so it is partial, not a full snapshot;
- # the commit path is untouched (snap_ok stays 0, _LOKI_SNAPSHOT_THIS_RUN
- # is never set below), so this still fails closed.
- gittool="$(_loki_snapshot_git_tool)" && \
- "$gittool" ls-files --others --exclude-standard -z > .loki/state/preexisting-untracked.z.tmp 2>/dev/null && \
- mv .loki/state/preexisting-untracked.z.tmp .loki/state/preexisting-untracked.z \
- || rm -f .loki/state/preexisting-untracked.z .loki/state/preexisting-untracked.z.tmp 2>/dev/null
- log_warn "Could not record pre-existing untracked files; this session will commit nothing (review and commit manually)"
- fi
- # Create and checkout the feature branch
- if ! git checkout -b "$branch_name" 2>/dev/null; then
- log_error "Failed to create agent branch: $branch_name"
- return 1
- fi
- # A new session branch starts with an empty session-created record. Files
- # an unfinished earlier session left (reached here after a refused resume)
- # were just snapshotted as pre-existing: kept on disk, never committed.
- leftover="$(_loki_nul_names .loki/state/session-created.z)"
- if [ -n "$leftover" ]; then
- log_warn "Left uncommitted from an earlier unfinished session, now treated as your files: $leftover"
- fi
- rm -f .loki/state/session-created.z 2>/dev/null
- # BACKLOG 130(c): seal the now-empty session-created.z record so this
- # fresh mint's OWN in-process memory of "absent" is what every later
- # record call and commit_session_changes verify against for the rest of
- # this run -- not the file's on-disk content, which an agent process could
- # otherwise repopulate with a planted path before this run ever trusts it.
- _loki_session_created_seal # BACKLOG-130C-MINT-SEAL
- # BACKLOG 68 rework: a fresh mint starts with no tracked-since anchor and no
- # in-flight marker (never seed the anchor from session-start-sha -- an
- # anchor==start would let a later resume's "anchor..HEAD" diff include this
- # session's OWN commits, pruning them from the next session's union as if
- # the user had tracked them between sessions).
- rm -f .loki/state/tracked-since.sha .loki/state/turn-in-flight 2>/dev/null
- if [ "$snap_ok" = 1 ]; then _LOKI_SNAPSHOT_THIS_RUN=1; fi
- # Every mint (a refused resume included) records its own base and the
- # commit it starts at (the empty tree when unborn); the resume paths above
- # keep both. A base left by an earlier session can name a branch deleted
- # since, and _loki_untrack_agent_committed_user_files then could not find
- # the user files the agent commits.
- printf '%s\n' "$cur" > .loki/state/base-branch.txt 2>/dev/null
- { git rev-parse --verify -q HEAD || git hash-object -t tree /dev/null; } \
- > .loki/state/session-start-sha 2>/dev/null
- # Store the branch name for later use (PR creation, cleanup)
- printf '%s\n' "$branch_name" > .loki/state/agent-branch.txt 2>/dev/null
- log_info "Agent branch created: $branch_name"
- audit_log "BRANCH_PROTECTION" "branch=$branch_name"
- echo "$branch_name"
- }
- # Snapshot, or fail closed. Any snapshot failure (for example git older than
- # 2.18, whose status lacks --no-renames / --ignored=matching) leaves a marker
- # that makes commit_session_changes commit nothing: a missing snapshot must
- # only ever mean an older session, never "sweep everything".
- _loki_snapshot_or_fail_closed() {
- if _loki_snapshot_preexisting "$@"; then
- rm -f .loki/state/preexisting-untracked.failed 2>/dev/null
- return 0
- fi
- mkdir -p .loki/state 2>/dev/null
- : > .loki/state/preexisting-untracked.failed 2>/dev/null
- return 1
- }
- # BACKLOG 70: preexisting-untracked.z and its .sha.z sibling live in
- # .loki/state, which the agent process (and anything it spawns) can write. A
- # co-located adversarial agent can delete .sha.z to blind
- # workspace_diff._preexisting_untracked's mismatch check (silently suppressing
- # every preexisting_modified disclosure), or append/edit .z entries to hide its
- # own new files from the receipt or falsely disown them. Full prevention against
- # a same-UID process is not achievable (it can read and rewrite anything this
- # process can); the realistic bar, and the one BACKLOG 70 asks for, is
- # DISCLOSURE: this process is the only one that can remember what it itself
- # just wrote, so it seals a sha256 of both files in its own memory right after
- # writing them, and re-derives + compares that digest at every later read site.
- # A mismatch (or the sealed file now missing) is reported as tampering and
- # treated as a failed snapshot (fail closed: no session commit), never patched
- # over silently. Cleared whenever the files are deliberately removed (line
- # ~9165) so that path is never reported as tampering.
- _LOKI_SNAPSHOT_SEAL=""
- # BACKLOG 130(c): .loki/state/session-created.z is NOT one of the two files
- # _LOKI_SNAPSHOT_SEAL above covers. It records the paths THIS session created
- # (unioned after every provider turn by _loki_record_session_created), and
- # _loki_snapshot_preexisting's resume union (line ~9683) trusts its content
- # directly: any path it lists is EXCLUDED from the new preexisting-untracked.z
- # snapshot, i.e. treated as "the session's own, never the user's". An agent
- # that plants a user's file path into session-created.z therefore gets that
- # exact path permanently exempted from pre-existing-file protection on the
- # next resume -- the file is swept into `git add -A` and the session commit as
- # if the session had created it. A separate seal, sibling to
- # _LOKI_SNAPSHOT_SEAL, closes this WITHIN one process's lifetime: sealed right
- # after every legitimate write this process makes (a successful record, the
- # fresh-mint delete, and this process's first look at the file on resume),
- # verified before any LATER point in the SAME process trusts the content,
- # fail-closed (delete the file, clear the seal) on a mismatch. Kept SEPARATE
- # from _LOKI_SNAPSHOT_SEAL rather than folded into one combined digest: this
- # file legitimately changes every provider turn, so a shared digest would
- # falsely flag tampering on every normal turn, and the fresh-mint `rm` (line
- # ~9241) runs after preexisting-untracked.z is sealed, which would falsely
- # flag every fresh mint too.
- # ACCEPTED GAP (disclosure, not prevention, same bar as BACKLOG 70's own): a
- # seal lives only in this process's memory, so it cannot verify content a
- # PRIOR process wrote before this one starts (setup_agent_branch's resume path
- # seals its own first read of the file rather than verifying it, since no
- # prior seal survives to check against) -- tampering already on disk before
- # this process begins is trusted at that one read, same as
- # _LOKI_SNAPSHOT_SEAL's own resume-time trust of preexisting-untracked.z's
- # prior content. What this DOES close: an agent process co-located with THIS
- # run can no longer plant a path into session-created.z between this run's own
- # writes and its own later reads (the resume union's exclude use, the next
- # record's merge, or the commit-time gate) without it being caught before that
- # content is trusted.
- _LOKI_SESSION_CREATED_SEAL=""
- # BACKLOG 129 (follow-up to BACKLOG 70/BACKLOG 70-awk): the DIGEST computation
- # above is hardened (fixed-root-path-first, no awk), but until this function
- # was added the snapshot WRITER (_loki_untracked_merge) and READER
- # (_loki_covered_paths), plus the hash-snapshot helper that produces .sha.z,
- # still ran bare `python3 -E`. -E only blocks PYTHON* env vars; it does NOT
- # stop user (or Homebrew) site-packages from loading, and Python execs every
- # line starting "import " in every .pth file found there on every startup
- # (confirmed empirically: a .pth planted in user site-packages fired and
- # silently rewrote the writer's declared output file after the legitimate
- # write completed). A same-UID agent that plants such a .pth can forge the
- # CONTENT of the snapshot BEFORE the (correctly hardened) seal ever computes a
- # digest over it -- the seal then faithfully approves the forgery. Resolved
- # the same way as _loki_snapshot_hash_tool: fixed root-owned candidate paths
- # first (a materially higher bar than a user-writable site-packages
- # directory), each PROBED with a real run (not just `-x`, which a Command Line
- # Tools-less Mac's /usr/bin shims would still pass while failing to execute
- # and possibly popping an install dialog), then a PATH walk restricted to
- # absolute entries as a last resort. -I -S (isolated mode, and -S also skips
- # `import site` entirely) is used at every call site below: -I alone still
- # loads site-packages (verified: `python3 -I` still lists Homebrew's
- # site-packages on this host), so -S is required to close the .pth vector,
- # not just narrow it. Accepted gap, same shape as _loki_snapshot_hash_tool's:
- # replacing the interpreter binary AT ONE OF THE FIXED PATHS this resolves to
- # first requires root; a host with none of the fixed candidates falls through
- # to the PATH-walk, which still trusts whatever sits first on an absolute PATH
- # entry under that name. A further, host-specific gap: on macOS these fixed
- # paths are typically Command Line Tools / Xcode shims that dispatch into
- # /Applications/Xcode.app -- root-owned on a normal install, but the fixed
- # path is only as trustworthy as that dispatch target on any host where it is
- # not.
- _loki_snapshot_py_tool() {
- local c
- for c in /usr/bin/python3 /bin/python3; do
- [ -x "$c" ] && [ ! -d "$c" ] && "$c" -I -S -c '' >/dev/null 2>&1 && { printf '%s\n' "$c"; return 0; }
- done
- local dir
- local IFS=:
- for dir in $PATH; do
- case "$dir" in
- /*) ;;
- *) continue ;;
- esac
- if [ -x "$dir/python3" ] && [ ! -d "$dir/python3" ] \
- && "$dir/python3" -I -S -c '' >/dev/null 2>&1; then
- printf '%s\n' "$dir/python3"
- return 0
- fi
- done
- return 1
- }
- # BACKLOG 129: same rationale and pattern as _loki_snapshot_py_tool above, for
- # `git`. Every call in this snapshot writer/reader chain
- # (_loki_untracked_status, _loki_tracked_by_user_since_anchor,
- # _loki_snapshot_preexisting's own `git rev-parse --show-toplevel`) used a bare
- # `git` -- an unscoped PATH lookup a same-UID attacker can shadow with a fake
- # binary placed in a user-writable directory ahead of the real one, exactly
- # the class of bypass _loki_snapshot_hash_tool's own comment already documents
- # for sha256sum/shasum. Reproduced empirically: a fake `git` ahead on PATH
- # that filters one path out of `git status`'s output caused that path to be
- # completely absent from the sealed snapshot. `_loki_untrack_agent_committed_
- # user_files`'s git calls (feeding the READER path, _loki_covered_paths) and
- # the commit-time `git reset`/`git commit`/`git update-index` calls in
- # commit_session_changes are DELIBERATELY left unresolved here (out of scope
- # for this fix): a fake git on PATH already defeats the commit gate directly
- # at commit time regardless of what the snapshot says, so resolving only the
- # reader's git feed would not change that attacker's outcome. Tracked as a
- # follow-up (process-wide resolved git) rather than folded into this change.
- _loki_snapshot_git_tool() {
- local c
- for c in /usr/bin/git /bin/git; do
- [ -x "$c" ] && [ ! -d "$c" ] && "$c" --version >/dev/null 2>&1 && { printf '%s\n' "$c"; return 0; }
- done
- local dir
- local IFS=:
- for dir in $PATH; do
- case "$dir" in
- /*) ;;
- *) continue ;;
- esac
- if [ -x "$dir/git" ] && [ ! -d "$dir/git" ] && "$dir/git" --version >/dev/null 2>&1; then
- printf '%s\n' "$dir/git"
- return 0
- fi
- done
- return 1
- }
- # _loki_snapshot_hash_tool: resolve an absolute path to sha256sum or shasum.
- # Fixed, root-owned candidate paths are tried FIRST, before any PATH walk: a
- # same-UID agent that can write to a user-writable directory ahead of the real
- # tool on PATH (Homebrew's prefix, e.g. /opt/homebrew/bin, is user-writable by
- # default on macOS and typically precedes /usr/bin) could otherwise drop its
- # own sha256sum/shasum there that reads the two snapshot files and echoes back
- # whatever hash it wants -- the same class of bypass as the .pth hijack this
- # whole function exists to close, just with a fake binary as the carrier
- # instead of a fake module. Trying the standard system locations first closes
- # that: they require root to write to, a materially higher bar than a
- # user-writable site-packages directory or a user-writable PATH entry.
- # A PATH walk (D7: never trust a bare command name -- restricted to absolute
- # entries, matching autonomy/loki's loki_verify_tool_path) is kept only as a
- # last resort for a host that keeps sha256sum/shasum somewhere nonstandard;
- # document there that this fallback still trusts whatever is first on an
- # absolute PATH entry, which a user-writable directory ahead of the system
- # dirs can still shadow. Prints nothing (caller then falls back) when no tool
- # is found anywhere.
- _loki_snapshot_hash_tool() {
- local c
- for c in /usr/bin/sha256sum /bin/sha256sum /sbin/sha256sum \
- /usr/bin/shasum /bin/shasum; do
- [ -x "$c" ] && [ ! -d "$c" ] && { printf '%s\n' "$c"; return 0; }
- done
- # Last resort: PATH walk, absolute entries only. Still trusts whatever
- # sits first on PATH under that name; see comment above.
- local name dir
- local IFS=:
- for name in sha256sum shasum; do
- for dir in $PATH; do
- case "$dir" in
- /*) ;;
- *) continue ;;
- esac
- if [ -x "$dir/$name" ] && [ ! -d "$dir/$name" ]; then
- printf '%s\n' "$dir/$name"
- return 0
- fi
- done
- done
- return 1
- }
- # _loki_snapshot_digest: sha256 of preexisting-untracked.z + .sha.z content
- # (each independently, "-" standing in for "file absent" so a delete is part
- # of the digest, not indistinguishable from an empty file; "?" standing in for
- # "file present but the hash tool failed to produce a real digest" -- a
- # nonzero exit, empty output, or output that fails validation below -- so a
- # transient read error never seals or compares as an empty string).
- #
- # HIGH-severity reviewer finding on the original python3 -E version: -E only
- # ignores PYTHON* env vars, it does NOT stop Python from loading user (or, on
- # macOS with Homebrew, system) site-packages, and Python executes every .pth
- # file found there on every startup. A same-UID agent process can plant a .pth
- # file there that hijacks this script's own print() to emit a cached, stale
- # (pre-tamper) digest -- fully defeating the seal while `git`/file state show
- # real tampering happened. Fixed by computing the digest with sha256sum/shasum
- # instead: neither has a .pth-style module-loading path for that hijack to
- # attach to. shasum is a Perl script on macOS (not a compiled binary), but its
- # @INC search path is root-owned system Perl library directories, not a user
- # site-packages directory, so the .pth carrier does not transfer to it.
- #
- # SECOND reviewer finding, on the shasum/sha256sum version that first closed
- # the .pth hole: the digest was piped through a bare `awk '{print $1}')`
- # resolved via an unscoped PATH lookup (`awk '{...}'` runs whatever `awk`
- # comes first on PATH). A fake `awk` placed in a user-writable directory ahead
- # of the real one (the same Homebrew-prefix-precedes-/usr/bin shape as the
- # hash-tool bypass above) can intercept the pipe and print a cached, stale
- # digest, defeating the seal exactly as before -- the fix reintroduced the
- # same bypass class it was written to close, just moved one command over.
- # Fixed by dropping awk entirely: the first whitespace-delimited field is
- # taken with the shell builtin `${h%% *}` (parameter expansion, no external
- # process, nothing on PATH to hijack), and the result is validated as exactly
- # 64 lowercase hex characters before being trusted as a real sha256 digest --
- # anything else (empty, wrong length, non-hex, a hand-rolled fake tool that
- # does not bother to format like sha256sum/shasum) becomes "?". The hash
- # tool's own exit status is also checked explicitly now: a piped
- # `cmd | awk ...` hid a nonzero exit from the hash tool behind awk's exit
- # status; capturing to a plain variable surfaces it.
- #
- # Resolved via _loki_snapshot_hash_tool above (fixed root-owned paths first,
- # PATH walk restricted to absolute entries only as a last resort), not a bare
- # `command -v`, so this also closes the sibling D7-pattern gap: an unscoped
- # lookup could otherwise resolve to a same-named binary planted in a
- # user-writable directory ahead of the real tool on PATH.
- #
- # Accepted gaps (disclosure, not prevention, per BACKLOG 70's own bar, same as
- # documented at _LOKI_SNAPSHOT_SEAL above): a same-UID ptrace attack on this
- # process or on the sha256sum/shasum child process itself; replacing the
- # sha256sum/shasum binary AT ONE OF THE FIXED SYSTEM PATHS this resolves to
- # first (requires root, a materially higher bar than a user-writable
- # site-packages .pth or a user-writable PATH entry); on a host with none of
- # the fixed candidates, the PATH-walk fallback inside _loki_snapshot_hash_tool
- # still trusts whatever sits first on an absolute PATH entry under that name,
- # which a user-writable directory ahead of the system dirs (e.g. Homebrew's
- # prefix on macOS) can shadow -- not fully closed, only narrowed to hosts
- # lacking the standard tool paths; the python3 fallback below, kept only for
- # the extremely rare host with neither tool anywhere, is resolved via
- # _loki_snapshot_py_tool and run with -I -S (BACKLOG 131(b) -- see the comment
- # at that fallback for why -S, not just -I, is required to close the .pth
- # class), so it carries the same fixed-root-path-first resolution and
- # site-packages exclusion as the other hardened call sites in this file. It
- # still inherits _loki_snapshot_py_tool's own accepted gaps (documented at
- # that function): replacing the interpreter AT ONE OF ITS FIXED PATHS requires
- # root, and its own PATH-walk fallback still trusts whatever sits first on an
- # absolute PATH entry on a host lacking those fixed paths.
- _loki_snapshot_digest() {
- local tool
- tool="$(_loki_snapshot_hash_tool)" || tool=""
- if [ -n "$tool" ]; then
- local out="" path h
- for path in ".loki/state/preexisting-untracked.z" ".loki/state/preexisting-untracked.sha.z"; do
- if [ -f "$path" ]; then
- h=""
- case "${tool##*/}" in
- shasum) h="$("$tool" -a 256 -- "$path" 2>/dev/null)" || h="" ;;
- *) h="$("$tool" -- "$path" 2>/dev/null)" || h="" ;;
- esac
- # No awk: take the first whitespace-delimited field with a
- # shell builtin (nothing on PATH to hijack), then require
- # exactly 64 lowercase hex characters. Anything else -- empty
- # output, a nonzero exit already caught above, an unexpected
- # format, a hand-rolled fake tool -- becomes "?" rather than
- # being trusted as a real digest. "?" is distinct from "-"
- # (file absent) and from any real hex digest, so a later
- # successful read of the same file mismatches it and is still
- # disclosed as tampering.
- h="${h%% *}"
- if [ "${#h}" -eq 64 ]; then
- case "$h" in
- *[!0123456789abcdef]*) h="?" ;;
- esac
- else
- h="?"
- fi
- out="$out $h"
- else
- out="$out -"
- fi
- done
- printf '%s\n' "${out# }"
- return 0
- fi
- # Fallback: neither sha256sum nor shasum found anywhere checked above
- # (extremely rare on macOS/Linux). Resolved via _loki_snapshot_py_tool
- # (fixed root-owned path first) and run with -I -S, the same BACKLOG 129
- # pattern used at _loki_untracked_merge / _loki_covered_paths: -I alone is
- # NOT enough -- a reviewer reproduced, on a sibling function in this same
- # tamper-detection system, that a .pth file planted in Homebrew python3's
- # own (user-writable) site-packages directory still fires under `python3
- # -I` alone, since -I implies -s (skip user site-packages) but does not
- # skip the resolved interpreter's OWN site-packages. Only -S (skip ALL
- # site-packages, including the interpreter's own) closes that. A resolve
- # failure degrades this fallback to printing nothing, same as any other
- # failure path here (caller treats empty as "could not seal"). Additional
- # accepted gap specific to this fallback (not present on the tool-based
- # path above, which distinguishes "-" from "?"): OSError covers BOTH
- # "file does not exist" and "file exists but is unreadable" here, so an
- # unreadable file on this path is indistinguishable from an absent one --
- # both print "-". The tool-based path above tells them apart ("-" for
- # absent, "?" for present-but-unreadable, caught by the seal-time
- # fail-closed check). Narrowing this further (os.path.lexists to emit "?"
- # for present-but-unreadable) is straightforward but left undone here: this
- # fallback only runs on a host with neither sha256sum nor shasum anywhere,
- # already the rare case this whole function treats as a residual gap.
- local pytool=""
- pytool="$(_loki_snapshot_py_tool)" || return 1
- "$pytool" -I -S -c 'import sys, hashlib
- sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- out = []
- for path in sys.argv[1:]:
- try:
- with open(path, "rb") as fh:
- out.append(hashlib.sha256(fh.read()).hexdigest())
- except OSError:
- out.append("-")
- print(" ".join(out))' \
- ".loki/state/preexisting-untracked.z" ".loki/state/preexisting-untracked.sha.z" 2>/dev/null
- }
- # _loki_session_created_digest: sha256 of session-created.z alone ("-" for
- # absent, "?" for present-but-unreadable/unhashable, same sentinel contract as
- # _loki_snapshot_digest above). Deliberately a thin sibling, not a parameterized
- # reuse of _loki_snapshot_digest: that function's exact text (the `for path in
- # ...` line and the python fallback's trailing argv line) is matched by
- # tests/test-branch-lifecycle.sh's RED mutation harness, so leaving it
- # untouched keeps that harness's anchors valid. Resolved via the SAME hardened
- # tool resolvers (_loki_snapshot_hash_tool, _loki_snapshot_py_tool) as the
- # sibling above -- no fresh/unhardened python3 or shasum call site.
- _loki_session_created_digest() {
- local tool
- tool="$(_loki_snapshot_hash_tool)" || tool=""
- local path=".loki/state/session-created.z"
- # A FIFO or ANY symlink at $path (dangling or not -- [ -L ] rejects both,
- # deliberately broader than just the dangling case: the only way this
- # file is ever legitimately written is _loki_untracked_merge's own
- # write-to-.tmp-then-`mv -f`, which always leaves a plain regular file at
- # $path, never a symlink, so a symlink here is never this session's own
- # legitimate output) is neither absent ([ -f ] is false for both) nor a
- # hashable regular file -- falling through to the absent-file branch below
- # would print the same "-" sentinel every fresh mint already seals as its
- # normal starting state, silently trusting an agent-planted FIFO (whose
- # blocked writer feeds content into the merge's later open() call) or a
- # dangling symlink (whose target read fails and is treated as "-" too).
- # Caught here, before either digest branch, with a sentinel ("?") that can
- # never equal "-", a real hex digest, or empty.
- if [ -L "$path" ] || { [ -e "$path" ] && [ ! -f "$path" ]; }; then printf '%s\n' "?"; return 0; fi # BACKLOG-130C-NONREG-CHECK
- if [ -n "$tool" ]; then
- local h=""
- if [ -f "$path" ]; then
- case "${tool##*/}" in
- shasum) h="$("$tool" -a 256 -- "$path" 2>/dev/null)" || h="" ;;
- *) h="$("$tool" -- "$path" 2>/dev/null)" || h="" ;;
- esac
- h="${h%% *}"
- if [ "${#h}" -eq 64 ]; then
- case "$h" in
- *[!0123456789abcdef]*) h="?" ;;
- esac
- else
- h="?"
- fi
- printf '%s\n' "$h"
- else
- printf '%s\n' "-"
- fi
- return 0
- fi
- # Fallback: neither sha256sum nor shasum found anywhere (same rare-host
- # case _loki_snapshot_digest's own fallback documents). Resolved via
- # _loki_snapshot_py_tool, run with -I -S (see that function's comment for
- # why -S, not just -I, is required to close the .pth vector).
- local pytool=""
- pytool="$(_loki_snapshot_py_tool)" || return 1
- "$pytool" -I -S -c 'import sys, hashlib
- sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- try:
- with open(sys.argv[1], "rb") as fh:
- print(hashlib.sha256(fh.read()).hexdigest())
- except OSError:
- print("-")' \
- "$path" 2>/dev/null
- }
- # _loki_session_created_seal: record this process's own memory of the current
- # session-created.z content, right after every point that legitimately changes
- # it (a successful union in _loki_record_session_created, the fresh-mint
- # delete, a resume, and the post-commit/no-op delete in
- # commit_session_changes). Same fail-closed shape as _loki_snapshot_seal: an
- # empty or "?" digest at seal time is itself suspicious (a legitimate write or
- # delete should never immediately be unreadable), so it is rejected rather
- # than sealed, which would let seal and a later verify silently agree on the
- # sentinel and disarm the guard for the rest of the session.
- _loki_session_created_seal() {
- local d
- d="$(_loki_session_created_digest)"
- case "$d" in
- ""|*'?'*)
- # Fail closed, not just "untrusted": an empty seal makes
- # _loki_session_created_verify's `[ -n "$_LOKI_SESSION_CREATED_SEAL" ]
- # || return 0` skip every later check for the rest of this
- # process's life. Leaving the file on disk here would fail OPEN --
- # unreadable-but-present content would then be silently trusted by
- # every later read site with no seal ever able to catch it. Delete
- # it instead, matching _loki_session_created_verify's own
- # fail-closed action on a real mismatch -- and, like that function's
- # own re-seal after its delete, set the seal to the digest's real
- # "absent" sentinel ("-"), not empty: an EMPTY seal here re-arms
- # nothing (the guard above short-circuits as "never sealed" on
- # every later check for the rest of this process), so an agent that
- # recreates the file with a planted path in this window would get
- # it silently merged in as trusted on the next successful record.
- # "-" is a real, non-empty seal value that the next verify's live
- # digest of a freshly-recreated file cannot match, keeping the
- # guard armed against exactly that recreation.
- rm -f .loki/state/session-created.z 2>/dev/null
- _LOKI_SESSION_CREATED_SEAL="-"
- log_warn "Could not seal the session-created record (hash tool failed or produced no readable digest); discarding it (its files are treated as yours, left uncommitted)"
- audit_log "SESSION_CREATED_SEAL_FAILED" "_loki_session_created_digest returned empty or an unreadable-file sentinel at seal time"
- ;;
- *)
- _LOKI_SESSION_CREATED_SEAL="$d"
- ;;
- esac
- }
- # _loki_session_created_verify: compare the live session-created.z digest
- # against the sealed one. No seal recorded (never sealed this process, or
- # deliberately cleared) is not tampering: returns success, does nothing --
- # EVERY caller that trusts session-created.z's content must therefore have
- # sealed it first (mint, resume, and after each successful record all do). A
- # seal that no longer matches IS tampering: logs and discloses it (same
- # DISCLOSURE bar as BACKLOG 70, not prevention against a same-UID process), and
- # fails closed by deleting the untrusted file outright -- its entries then
- # become "the user's, not recorded as this session's", the safe direction,
- # since a false claim of user ownership only leaves a file uncommitted on disk,
- # never deletes it, while a false claim of session ownership can get a user's
- # file swept into a commit and removed from disk on a later checkout of the
- # base. Called BEFORE any later point in the SAME process treats the file's
- # content as trustworthy (verify-before-trust, not verify-after-use):
- # _loki_record_session_created verifies at its very top, before its own
- # self-referential merge (base="$out"), and unconditionally -- not gated on
- # _LOKI_SNAPSHOT_THIS_RUN, since an interrupt/cleanup call can still fire after
- # that flag was cleared mid-session; commit_session_changes verifies at its
- # very top, before every one of its several early returns (the untrack
- # failure, the git 2.25 unstage failure, a secret-scan abort, a failed `git
- # commit`) that would otherwise skip its own rm -f cleanup and leave a
- # tampered file on disk for the next resume. _loki_resume_snapshot does NOT
- # call this function: it is this process's FIRST read of a file the PREVIOUS
- # (dead) process wrote, so there is no seal from that process to verify
- # against here; it instead calls _loki_session_created_seal directly to
- # establish THIS process's own baseline before anything (including
- # _loki_snapshot_preexisting's resume-union exclude use, right after) reads
- # the file. See the ACCEPTED GAP note at _LOKI_SESSION_CREATED_SEAL above.
- _loki_session_created_verify() {
- local live="" path=".loki/state/session-created.z"
- [ -n "$_LOKI_SESSION_CREATED_SEAL" ] || return 0
- live="$(_loki_session_created_digest)"
- if [ "$live" = "$_LOKI_SESSION_CREATED_SEAL" ]; then
- return 0
- fi
- log_warn "session-created.z changed after this session recorded it (possible tampering by an agent process); discarding the record (its files are treated as yours, left uncommitted)"
- audit_log "SESSION_CREATED_TAMPERED" "session-created.z changed after being sealed: live=$live,sealed=$_LOKI_SESSION_CREATED_SEAL"
- rm -f "$path" 2>/dev/null
- _LOKI_SESSION_CREATED_SEAL=""
- _loki_session_created_seal
- return 1
- }
- # _loki_snapshot_seal: record this process's own memory of what it just wrote.
- # Called only right after a successful _loki_snapshot_preexisting. Minor
- # reviewer note: a legitimate snapshot write should never hit an unreadable-
- # file condition immediately after writing it, so a "?" in the digest AT SEAL
- # TIME (as opposed to at a later verify) is itself suspicious -- both seal and
- # a later verify could then read "?", compare equal, and silently disarm the
- # guard for the rest of the session (the same failure mode
- # _loki_snapshot_verify's mismatch path exists to catch, just never triggered
- # because both sides agree on the sentinel). Closed cheaply: reject an empty
- # or "?"-containing digest at seal time the same way _loki_snapshot_verify
- # rejects a mismatch -- log and disclose it, write the .failed marker, AND
- # clear _LOKI_SNAPSHOT_THIS_RUN directly here (belt-and-braces: the actual
- # gate that stops a commit). The call site right after `_loki_snapshot_seal`
- # in _loki_snapshot_preexisting also checks `_LOKI_SNAPSHOT_SEAL` and returns
- # non-zero on failure, which is what actually propagates to
- # _loki_snapshot_or_fail_closed / _loki_resume_snapshot and keeps their
- # existing callers from ever setting _LOKI_SNAPSHOT_THIS_RUN=1 in the first
- # place (see the comment there) -- so the in-function clear here is redundant
- # with that, not the sole mechanism, kept only as defense in depth in case a
- # future caller invokes this function directly. The call site above is
- # intentionally left as a bare `_loki_snapshot_seal` (no `|| return`):
- # tests/test-branch-lifecycle.sh's RED harness matches that exact line to
- # strip the guard; the propagation check right after it carries its own
- # trailing-comment marker the RED harness also strips (see the comment at
- # that check, in _loki_snapshot_preexisting, for the exact text -- not
- # repeated here so a grep for it does not also match this sentence), so
- # removing either half of the guard is enough to reproduce the pre-fix
- # silent-degrade behavior in that test.
- _loki_snapshot_seal() {
- local d
- d="$(_loki_snapshot_digest)"
- case "$d" in
- ""|*'?'*)
- _LOKI_SNAPSHOT_SEAL=""
- _LOKI_SNAPSHOT_THIS_RUN=0
- log_warn "Could not seal the pre-existing-file snapshot (hash tool failed or produced no readable digest); treating this session as unsnapshotted (nothing will be committed)"
- audit_log "SNAPSHOT_SEAL_FAILED" "_loki_snapshot_digest returned empty or an unreadable-file sentinel at seal time"
- mkdir -p .loki/state 2>/dev/null
- : > .loki/state/preexisting-untracked.failed 2>/dev/null
- ;;
- *)
- _LOKI_SNAPSHOT_SEAL="$d"
- ;;
- esac
- }
- # _loki_snapshot_verify: compare the live files against the sealed digest.
- # No seal recorded yet (never snapshotted this process, or deliberately
- # cleared) is not tampering: returns success and does nothing. A seal that no
- # longer matches IS tampering: logs it (disclosure is the bar BACKLOG 70 asks
- # for; a same-UID process can rewrite anything else this one could check), and
- # forces every fail-closed path a missing/failed snapshot already takes --
- # clears the seal so a caller cannot re-check a stale value, clears
- # _LOKI_SNAPSHOT_THIS_RUN so commit_session_changes and
- # _loki_record_session_created both treat this as an unsnapshotted session, and
- # (re)writes preexisting-untracked.failed so BACKLOG 90's existing message
- # fires. Every consumer of the snapshot/hash files calls this first and treats
- # non-zero as "no snapshot".
- _loki_snapshot_verify() {
- local live=""
- [ -n "$_LOKI_SNAPSHOT_SEAL" ] || return 0
- live="$(_loki_snapshot_digest)"
- if [ "$live" = "$_LOKI_SNAPSHOT_SEAL" ]; then
- return 0
- fi
- log_warn "Pre-existing-file snapshot or its hash file changed after this session recorded it (possible tampering by an agent process); treating this session as unsnapshotted (nothing will be committed)"
- # Record both digests (before clearing the seal below) so an operator
- # reading the audit log can tell a genuine content tamper apart from a
- # transient hash-tool failure that just produced a different digest.
- audit_log "SNAPSHOT_TAMPERED" "preexisting-untracked.z or .sha.z changed after being sealed: live=$live,sealed=$_LOKI_SNAPSHOT_SEAL"
- _LOKI_SNAPSHOT_SEAL=""
- _LOKI_SNAPSHOT_THIS_RUN=0
- mkdir -p .loki/state 2>/dev/null
- : > .loki/state/preexisting-untracked.failed 2>/dev/null
- return 1
- }
- # _loki_snapshot_preexisting [union]
- # Write .loki/state/preexisting-untracked.z: every path git does not track,
- # untracked and gitignored, repo-wide, NUL-delimited, relative to the repo top.
- # commit_session_changes unstages these and the receipt
- # (autonomy/lib/workspace_diff.py) does not claim them. --ignored=matching lists
- # a directory only when an ignore pattern matches it, so node_modules/ is one
- # "dir/" entry covering its subtree, while logs/ holding only *.log files is
- # listed file by file (a new logs/app.json is still the agent's). "union" keeps
- # the paths already recorded, minus any that became tracked because the USER
- # committed them between sessions (BACKLOG 68 rework). Distinguishing that from
- # "the AGENT itself committed or staged this pre-existing file during THIS
- # session, then got killed" is the entire point of the anchor below: only a
- # path git ADDED strictly after the last point this run itself could reach is
- # ever pruned. A path the agent tracked before that point is left IN the union
- # so it still flows into _loki_untrack_agent_committed_user_files's disclosure
- # (warn, remove from the branch tip, keep on disk) -- never silently swept in.
- #
- # The anchor is .loki/state/tracked-since.sha, a commit SHA written by
- # _loki_record_session_created (after each provider turn) and by
- # commit_session_changes (at session end): the newest point this run itself
- # recorded. .loki/state/turn-in-flight marks a provider turn as currently
- # running (set right before invocation, cleared by the same record call that
- # advances the anchor); its presence means a turn may have committed something
- # since the last anchor write, so pruning is skipped entirely rather than risk
- # treating an agent's mid-turn commit as the user's. Any anchor-resolution
- # failure (missing anchor, anchor not an ancestor of HEAD, merge-base failure)
- # also skips pruning: the safe direction is the old behavior (false blame, file
- # kept on disk), never silent adoption.
- #
- # The sibling .sha.z (content hash per file entry) lets the receipt list a
- # pre-existing file the run changed; it is removed first, so a failed hash step
- # means no detection, never stale hashes. Returns non-zero, leaving any earlier
- # list in place, when git cannot list. Seals a digest of both files on success
- # (BACKLOG 70) so later reads can detect tampering.
- _loki_snapshot_preexisting() {
- local snap=".loki/state/preexisting-untracked.z" top="" base="" exclude="" tracked="" gittool="" pytool=""
- rm -f "${snap%.z}.sha.z" 2>/dev/null
- gittool="$(_loki_snapshot_git_tool)" || return 1
- top="$("$gittool" rev-parse --show-toplevel 2>/dev/null)" || return 1
- mkdir -p .loki/state 2>/dev/null || return 1
- # "union" never adds a path listed in session-created.z: those files were
- # made by a session that did not finish (interrupt, pod loss), so the
- # session that finishes the work commits them. If the user edits one of
- # them between sessions it is still the session's file and is committed
- # with the edit. Recorded entries only block additions; a path this run can
- # prove the USER tracked between sessions is the one case they DO remove
- # (see _loki_tracked_by_user_since_anchor below).
- if [ "${1:-}" = union ]; then
- base="$snap"
- exclude=".loki/state/session-created.z"
- tracked="$snap.tracked"
- if ! _loki_tracked_by_user_since_anchor "$top" "$tracked"; then
- rm -f "$tracked"
- tracked=""
- fi
- fi
- if ! _loki_untracked_status "$snap.status" \
- || ! _loki_untracked_merge "$snap.status" "$base" "$exclude" exact "$snap" "$tracked"; then
- rm -f "$tracked" "$snap.status" "$snap.tmp"
- return 1
- fi
- rm -f "$tracked" "$snap.status"
- # BACKLOG 129: write_snapshot_hashes (the only code path "hash-snapshot"
- # reaches) does pure file I/O (open/os.path/hashlib), no sibling import
- # from autonomy/lib and no subprocess/git call, so -I -S is safe here: it
- # never needs the script's own directory on sys.path or anything from
- # site-packages. This writes .sha.z, one of the two files the seal above
- # digests, so it is in scope of the same forgery class as the merge/covered
- # sites below. A failed resolve here degrades to the existing log_warn path
- # (receipt loses preexisting_modified detection) rather than blocking the
- # snapshot -- matching this call's pre-existing non-fatal-on-failure shape.
- pytool="$(_loki_snapshot_py_tool)" && "$pytool" -I -S "$SCRIPT_DIR/lib/workspace_diff.py" hash-snapshot "$top" "$snap" >/dev/null 2>&1 \
- || log_warn "Could not hash your pre-existing untracked files; the receipt cannot list the ones this run changes"
- _loki_snapshot_seal
- # A seal failure (empty digest or an unreadable-file "?", see the comment
- # at _loki_snapshot_seal) leaves _LOKI_SNAPSHOT_SEAL empty; propagate that
- # as a failure of this function too, so both callers'
- # (_loki_snapshot_or_fail_closed and _loki_resume_snapshot) existing
- # fail-closed handling applies, instead of silently reporting success on a
- # session this process could not actually seal. This is the line that
- # actually makes a seal failure fail closed (_loki_snapshot_seal's own
- # in-function clear is defense in depth, not the mechanism -- see the
- # comment there). This line is deliberately also stripped by
- # tests/test-branch-lifecycle.sh's BACKLOG-70 RED mutation, by its trailing
- # comment marker (kept off this sentence on purpose, so a grep for the
- # marker text matches only the one real code line below, not this prose),
- # same as the guard calls above it: without it, on a mutated copy that
- # also strips `_loki_snapshot_seal`, _LOKI_SNAPSHOT_SEAL stays at its
- # global initial value ("") and this check alone would still return 1 and
- # fail closed, masking the RED reproduction the test relies on to prove
- # the mutation of the ACTUAL guard calls is non-vacuous.
- [ -n "$_LOKI_SNAPSHOT_SEAL" ] || return 1 # BACKLOG-70-SEAL-CHECK
- return 0
- }
- # _loki_untracked_status <out>: the one enumeration behind both the snapshot
- # and the session-created record, so the two always agree: raw
- # `git status --porcelain -z` records for the whole repo, .loki excluded.
- # Non-zero, writing nothing, when git cannot list.
- # BACKLOG 129: resolved via _loki_snapshot_git_tool (fixed root-owned path
- # first, PATH walk restricted to absolute entries as a last resort), not a
- # bare `git`, which a same-UID attacker can shadow with a fake binary on a
- # user-writable PATH entry ahead of the real one -- reproduced empirically: a
- # fake git filtering one path out of its `status` output made that path
- # entirely absent from the sealed snapshot this function ultimately feeds.
- _loki_untracked_status() {
- local top="" prefix="" gittool=""
- local genv=() fk="" fn="" n=0
- gittool="$(_loki_snapshot_git_tool)" || return 1
- # Version floor: GIT_CONFIG_COUNT needs git >= 2.31 and GIT_NO_LAZY_FETCH
- # (no hook-running lazy fetch) needs git >= 2.44. Unparseable also refuses.
- local gver="" gmaj="" gmin=""
- gver="$(LC_ALL=C "$gittool" --version 2>/dev/null)" || gver=""
- gver="${gver#git version }"
- gmaj="${gver%%.*}"
- gmin="${gver#*.}"
- gmin="${gmin%%.*}"
- case "${gmaj}${gmin}" in
- '' | *[!0-9]*) gmaj="" ;;
- esac
- if [ -z "$gmaj" ] || [ -z "$gmin" ] || [ "$gmaj" -lt 2 ] || { [ "$gmaj" -eq 2 ] && [ "$gmin" -lt 44 ]; }; then
- echo "snapshot: git >= 2.44 required (got '${gver:-unknown}'); refusing to run status unprotected" >&2
- return 1
- fi
- # Inherited GIT_DIR / GIT_WORK_TREE would repoint every call below.
- top="$(env -u GIT_DIR -u GIT_WORK_TREE "$gittool" rev-parse --show-toplevel 2>/dev/null)" || return 1
- prefix="$(env -u GIT_DIR -u GIT_WORK_TREE "$gittool" rev-parse --show-prefix 2>/dev/null)" || return 1
- # BACKLOG 131c: a repo-local core.fsmonitor is a command git runs on
- # status; the agent can write .git/config, so disable it (and the
- # untracked cache it could have poisoned) for this one call.
- # A filter driver named by attributes (clean/smudge/process) is also a
- # command git runs when status compares a stat-dirty tracked file, so every
- # configured driver is blanked: clean/smudge/process become empty (never
- # `cat`, which resolves from PATH) and required is false.
- # The overrides travel as GIT_CONFIG_KEY_n / GIT_CONFIG_VALUE_n, never as
- # `-c key=value`: git splits -c at the first "=", so a driver named "x=y"
- # would be missed. GIT_NO_LAZY_FETCH stops a partial-clone status from
- # fetching a missing blob over an agent-written core.sshCommand. Inherited
- # GIT_DIR, GIT_WORK_TREE, GIT_CONFIG_PARAMETERS and GIT_CONFIG_GLOBAL are
- # dropped and GIT_CONFIG_COUNT is overwritten, never appended to. LC_ALL=C
- # keeps the name lookup byte-exact for non-UTF-8 driver names.
- genv=(env -u GIT_DIR -u GIT_WORK_TREE -u GIT_CONFIG_PARAMETERS -u GIT_CONFIG_GLOBAL
- LC_ALL=C GIT_NO_LAZY_FETCH=1)
- # The NUL-delimited listing goes to a file beside the output path, read by
- # a loop in this shell (it appends to genv and n); removed right after the
- # loop, before any later return path.
- : > "${1}.filters" || return 1
- "${genv[@]}" "$gittool" -C "$top" config -z --name-only --get-regexp \
- '^filter\..*\.(clean|smudge|process)$' > "${1}.filters" 2>/dev/null || true
- while IFS= read -r -d '' fk; do
- fn="${fk%.*}"
- genv+=("GIT_CONFIG_KEY_${n}=${fk}" "GIT_CONFIG_VALUE_${n}=")
- n=$((n + 1))
- genv+=("GIT_CONFIG_KEY_${n}=${fn}.required" "GIT_CONFIG_VALUE_${n}=false")
- n=$((n + 1))
- done < "${1}.filters"
- rm -f "${1}.filters"
- # Prove the overrides are visible with a per-call random nonce (a constant
- # could be planted in .git/config by the agent), or refuse to run.
- local nonce=""
- nonce="$(od -An -tx8 -N8 /dev/urandom 2>/dev/null | tr -d ' \n')"
- [ -n "$nonce" ] || return 1
- genv+=("GIT_CONFIG_KEY_${n}=loki.snapshot.sentinel" "GIT_CONFIG_VALUE_${n}=${nonce}")
- n=$((n + 1))
- genv+=("GIT_CONFIG_COUNT=${n}")
- if [ "$("${genv[@]}" "$gittool" -C "$top" config --get loki.snapshot.sentinel 2>/dev/null)" != "$nonce" ]; then
- echo "snapshot: git does not honor GIT_CONFIG_COUNT; refusing to run status unprotected" >&2
- return 1
- fi
- "${genv[@]}" "$gittool" -C "$top" -c core.fsmonitor=false -c core.untrackedCache=false \
- --no-optional-locks status --porcelain -z --no-renames -uall \
- --ignored=matching --ignore-submodules=all -- ":(exclude,literal)${prefix}.loki" \
- > "$1" 2>/dev/null && return 0
- rm -f "$1"
- return 1
- }
- # _loki_untracked_merge <status> <base> <exclude> exact|cover <out> [tracked]
- # Atomically write <out>: the NUL-delimited paths of <base> minus any path
- # <tracked> lists (BACKLOG 68: entries the USER tracked between sessions, as
- # resolved by _loki_tracked_by_user_since_anchor -- never the whole index),
- # plus every untracked ("??") or ignored ("!!") path in <status> that <exclude>
- # does not hold. With "cover", an <exclude> entry ending in "/" also holds
- # every path below it (workspace_diff._covered). An empty or missing <base> or
- # <exclude> is an empty list; any other read or write failure is non-zero and
- # leaves <out> as it was. Sorted bytewise, like `LC_ALL=C sort -z -u`. One
- # python process: bash 3.2 has no associative arrays, and a bash loop cost
- # 300ms per 6,000 entries on /bin/bash. -I -S and no cwd on sys.path: the
- # cwd is the agent's repo.
- # BACKLOG 129: this is the snapshot WRITER -- its <out> becomes
- # preexisting-untracked.z, one of the two files the seal above digests. A bare
- # `python3 -E` here still loads user site-packages (-E only blocks PYTHON* env
- # vars), so a same-UID .pth planted there can run arbitrary code during this
- # call and rewrite <out> after the legitimate write, forging exactly the bytes
- # the seal is about to trust -- reproduced empirically (a .pth that stripped
- # an entry from the just-written output). Resolved via _loki_snapshot_py_tool
- # (fixed root-owned path first) and run with -I -S: -I alone is not enough
- # (verified: `python3 -I` alone still lists Homebrew's user-writable
- # site-packages on this host), -S additionally skips `import site` so no .pth
- # anywhere is ever processed. A resolve failure fails this function closed
- # (return 1, same as any other read/write failure here) rather than silently
- # falling back to a bare, unresolved `python3`.
- _loki_untracked_merge() {
- local pytool=""
- pytool="$(_loki_snapshot_py_tool)" || return 1
- "$pytool" -I -S -c 'import sys
- sys.path[:] = [p for p in sys.path if p not in ("", ".")]
- import os
- status, base, exclude, mode, out = sys.argv[1:6]
- tracked = sys.argv[6] if len(sys.argv) > 6 else ""
- def entries(path, missing_ok=True):
- try:
- with open(path, "rb") as fh:
- return [p for p in fh.read().split(b"\0") if p]
- except FileNotFoundError:
- if missing_ok:
- return []
- raise
- held = set(entries(exclude)) if exclude else set()
- def covered(path):
- if path in held:
- return True
- cut = path.find(b"/") if mode == "cover" else -1
- while cut != -1:
- if path[:cut + 1] in held:
- return True
- cut = path.find(b"/", cut + 1)
- return False
- paths = set(entries(base)) if base else set()
- if tracked:
- # Exact match only: a "dir/" entry is never dropped even if a file under it
- # is now tracked, so an ignored directory (node_modules/) never re-exposes
- # the rest of its subtree (BACKLOG 66/78 territory).
- paths -= set(entries(tracked))
- for rec in entries(status, missing_ok=False):
- # The session-created record ("cover" mode) never holds a directory entry:
- # a whole-directory entry (logs/, out/) would stop the resume union from
- # adopting a file the user later creates inside it, and a .gitignore
- # rewrite would then sweep that file. Adopting such a directory as
- # pre-existing fails safe (the agent files inside stay uncommitted).
- if mode == "cover" and rec.endswith(b"/"):
- continue
- if rec[:3] in (b"?? ", b"!! ") and not covered(rec[3:]):
- paths.add(rec[3:])
- with open(out + ".tmp", "wb") as fh:
- fh.write(b"".join(p + b"\0" for p in sorted(paths)))
- os.replace(out + ".tmp", out)' "$@" 2>/dev/null
- }
- # _loki_tracked_by_user_since_anchor <top> <out> (BACKLOG 68 rework)
- # Write <out>: the NUL-delimited paths git added to the index between the
- # recorded "tracked-since" anchor and HEAD -- i.e. what a resume union may
- # safely treat as "the USER tracked this between sessions" and prune. Returns
- # non-zero (writing nothing) whenever that inference is not safe, which leaves
- # the caller pruning nothing (the old, safe-by-default behavior):
- # - no anchor recorded yet (an older session, or nothing survived a turn)
- # - a provider turn is marked in-flight (.loki/state/turn-in-flight): a turn
- # can commit or stage a file and then be SIGKILLed before the next record
- # call advances the anchor, so anything since the LAST anchor is unproven
- # - the anchor does not resolve, or is not an ancestor of HEAD (branch
- # switched, history rewritten, anchor stale)
- # --diff-filter=A, --no-renames: only additions, matching
- # _loki_untrack_agent_committed_user_files's own diff so the two never
- # disagree about which paths are "added since X".
- # BACKLOG 129: <out> here is subtracted from the snapshot base inside
- # _loki_untracked_merge (the "tracked" argument) before that WRITER seals its
- # result -- a fake git that lists an extra path here can prune a real
- # untracked path (e.g. a secret) out of the sealed snapshot, the same attack
- # as a forged `git status`, just one function over. Resolved via
- # _loki_snapshot_git_tool like the other git calls in this writer chain.
- _loki_tracked_by_user_since_anchor() {
- local top="$1" out="$2" anchor_file=".loki/state/tracked-since.sha" anchor="" gittool=""
- [ -f .loki/state/turn-in-flight ] && return 1
- [ -s "$anchor_file" ] || return 1
- anchor="$(cat "$anchor_file" 2>/dev/null)"
- [ -n "$anchor" ] || return 1
- gittool="$(_loki_snapshot_git_tool)" || return 1
- "$gittool" rev-parse --verify -q "$anchor" >/dev/null 2>&1 || return 1
- "$gittool" merge-base --is-ancestor "$anchor" HEAD 2>/dev/null || return 1
- "$gittool" -C "$top" diff --name-only -z --no-renames --diff-filter=A "$anchor" HEAD > "$out.tmp" 2>/dev/null \
- || { rm -f "$out.tmp"; return 1; }
- mv -f "$out.tmp" "$out"
- return 0
- }
- # _loki_advance_tracked_since_anchor
- # Move .loki/state/tracked-since.sha to the current HEAD and clear
- # .loki/state/turn-in-flight, but ONLY once the anchor write is confirmed on
- # disk. Called after every provider turn (_loki_record_session_created) and at
- # normal session end (commit_session_changes), i.e. at every point this run
- # can prove no provider turn is currently running -- PROVIDED the anchor write
- # itself lands, since _loki_tracked_by_user_since_anchor trusts both signals
- # together (anchor position + in-flight marker) to prove a commit happened
- # strictly after the anchor with no turn running. Clearing the marker on a
- # failed write would decouple them: a stale anchor stays on disk while the
- # marker falsely reports "no turn was in flight", so a later resume could
- # prove something it can't actually prove and prune an agent's own commit with
- # no disclosure (this is BACKLOG 68's failure mode, reintroduced). A HEAD that
- # does not resolve (unborn branch) is not a write failure -- there is nothing
- # to anchor yet -- so the marker still clears; the anchor stays "not set" as
- # before. Always returns 0 (both call sites are bare, unchecked calls): the
- # in-flight marker, not the return code, is the safety signal on a failed
- # write.
- _loki_advance_tracked_since_anchor() {
- local sha=""
- sha="$(git rev-parse --verify -q HEAD 2>/dev/null)" || true
- if [ -n "$sha" ]; then
- if ! { printf '%s\n' "$sha" > .loki/state/tracked-since.sha.tmp 2>/dev/null \
- && mv -f .loki/state/tracked-since.sha.tmp .loki/state/tracked-since.sha 2>/dev/null; }; then
- # Anchor write failed (disk full, a stray .tmp collision, a
- # concurrent writer). Deliberately leave turn-in-flight SET and
- # return here, before the unconditional clear below: the safe
- # direction is to keep the old fallback-to-no-pruning path active
- # for one more check, never to clear a marker the anchor write did
- # not earn.
- rm -f .loki/state/tracked-since.sha.tmp 2>/dev/null
- return 0
- fi
- fi
- rm -f .loki/state/turn-in-flight 2>/dev/null
- return 0
- }
- # _loki_record_session_created
- # Union into .loki/state/session-created.z (NUL-delimited, repo-top-relative,
- # atomic) the paths git does not track now that the pre-existing snapshot does
- # not cover: the files this session created. The resume union never adds these
- # to the snapshot, so a session that is interrupted (cleanup) or killed (pod
- # loss: no trap runs, hence the call after every provider turn) still has its
- # own files committed by the session that finishes. commit_session_changes
- # clears the record after a normal commit; a new session branch starts empty.
- # Records nothing unless this run took a current snapshot (a leftover or stale
- # one would make it claim a user's file). On a failure the record is left as it
- # was, never partial, and preexisting-untracked.failed is never touched; the
- # next successful call recomputes everything, so only a failure on the final
- # call before a kill loses paths (those are then treated as the user's: kept on
- # disk, not committed).
- _loki_record_session_created() {
- local snap=".loki/state/preexisting-untracked.z" out=".loki/state/session-created.z"
- # BACKLOG 130(c): verify session-created.z BEFORE this run's THIS_RUN gate
- # or its own merge trust the file -- verify-before-trust, not verify-after-
- # use. Runs unconditionally (not gated on THIS_RUN=1): an interrupt/cleanup
- # call can still fire after THIS_RUN was cleared mid-session (e.g. by
- # _loki_snapshot_verify below on an earlier turn), and that call must not
- # skip this check just because the gate below is about to return early. On
- # a mismatch the file is already deleted by the time this returns (fail
- # closed inside _loki_session_created_verify itself, not via this return
- # value alone -- every call site of this function ignores its return with
- # `|| true`, same as _loki_snapshot_verify above); the merge below then
- # recomputes cleanly from a clean slate (base="$out" is now absent), so a
- # legitimate record right after a caught tamper still records this turn's
- # real files instead of hard-failing the whole call.
- # ACCEPTED GAP (verify-to-use window, disclosed here since ordering is the
- # whole point of this fix): this verify call and the merge's own read of
- # "$out" as `base` (inside _loki_untracked_merge, below) are not atomic --
- # _loki_advance_tracked_since_anchor, a git rev-parse, and a whole-repo
- # `git status` all run in between. A same-UID write landing in that window
- # is merged, then sealed as trusted by the seal call after a successful
- # merge. This is the SAME shape of gap _LOKI_SNAPSHOT_SEAL's own
- # verify/merge pair already accepts for preexisting-untracked.z (see
- # _loki_untracked_merge's own BACKLOG 129 comment). Closing it would mean
- # re-verifying again immediately before _loki_untracked_merge's read, inside
- # that function -- deliberately not done here: _loki_untracked_merge's exact
- # text is what tests/test-branch-lifecycle.sh's RED mutation harness matches
- # by name-anchor, and threading a second, narrower verify through its
- # shared codepath (also called with mode=exact for the OTHER snapshot) risks
- # that anchor for a gap this narrow. Disclosure, not full prevention, is the
- # same bar BACKLOG 70 itself accepts.
- _loki_session_created_verify || true
- [ "${_LOKI_SNAPSHOT_THIS_RUN:-0}" = 1 ] || return 0
- # BACKLOG 70: detect tampering with the sealed snapshot as early as
- # possible (this runs after every provider turn, not just at commit time).
- _loki_snapshot_verify || return 1
- # BACKLOG 68 rework: this call is the point right after a provider turn
- # ends, so it also advances the tracked-since anchor and clears the
- # in-flight marker -- whatever the agent committed or staged during the
- # turn that just finished is now provably NOT mid-turn, so a later resume's
- # anchor..HEAD diff may safely be checked against it.
- _loki_advance_tracked_since_anchor
- if [ -f "$snap" ] && _loki_untracked_status "$out.status" \
- && _loki_untracked_merge "$out.status" "$out" "$snap" cover "$out"; then
- rm -f "$out.status"
- # BACKLOG 130(c): seal this process's own memory of the record it just
- # wrote, so the NEXT call to this function (or commit_session_changes)
- # verifies against what THIS process actually wrote, not whatever an
- # agent process may write to the file in between.
- _loki_session_created_seal
- return 0
- fi
- rm -f "$out.status" "$out.tmp" 2>/dev/null
- log_warn "Could not record the files this session created; if the run stops before the next record, they are treated as yours and left uncommitted"
- return 1
- }
- # _loki_resume_snapshot: the resume paths' union, naming the files carried over
- # from an unfinished previous session (this session's commit includes them).
- _loki_resume_snapshot() {
- local carried=""
- # BACKLOG 130(c): seal THIS process's own first look at session-created.z
- # before anything below reads or trusts it (verify-before-trust). This is
- # a resuming process picking up a file the PREVIOUS (now-dead) process
- # wrote; there is no seal from that process to verify against here (a
- # seal never survives past the process that set it) -- sealing now is what
- # lets THIS process detect an agent tampering with the file for the rest
- # of ITS OWN lifetime (every subsequent record call, and the commit-time
- # check), the same disclosure bar BACKLOG 70 already accepts for the
- # sibling snapshot: tampering already on disk before this process starts
- # is an accepted gap, never silently trusted for
autonomy/sandbox.sh · Bundled script · 2,265 lines · 2 findings marked
- #!/usr/bin/env bash
- #===============================================================================
- # Loki Mode - Docker Sandbox Manager
- # Provides isolated container execution for enhanced security
- #
- # Usage:
- # ./autonomy/sandbox.sh start [OPTIONS] [PRD_PATH]
- # ./autonomy/sandbox.sh stop
- # ./autonomy/sandbox.sh status
- # ./autonomy/sandbox.sh shell
- #
- # Environment Variables:
- # LOKI_SANDBOX_IMAGE - Docker image to use (default: loki-mode:sandbox)
- # LOKI_SANDBOX_NETWORK - Network mode: bridge, none, host (default: bridge)
- # LOKI_SANDBOX_ALLOW_HOSTS - Opt-in egress allowlist (comma-separated hostnames);
- # blocks link-local and private ranges (default: unset)
- # LOKI_SANDBOX_CPUS - CPU limit (default: 2)
- # LOKI_SANDBOX_MEMORY - Memory limit (default: 4g)
- # LOKI_SANDBOX_READONLY - Mount project as read-only (default: false)
- #
- # Security Features:
- # - Seccomp profile restricts dangerous syscalls
- # - No new privileges flag prevents privilege escalation
- # - Dropped capabilities reduce attack surface
- # - Resource limits prevent DoS
- # - Optional read-only filesystem
- # - API keys mounted read-only
- #===============================================================================
- set -euo pipefail
- # Colors
- RED='\033[0;31m'
- GREEN='\033[0;32m'
- YELLOW='\033[1;33m'
- BLUE='\033[0;34m'
- CYAN='\033[0;36m'
- NC='\033[0m'
- BOLD='\033[1m'
- # Configuration
- SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
- SKILL_DIR="$(dirname "$SCRIPT_DIR")"
- PROJECT_DIR="${LOKI_PROJECT_DIR:-$(pwd)}"
- # Normalize PROJECT_DIR (remove trailing slash)
- PROJECT_DIR="${PROJECT_DIR%/}"
- # Container name includes path hash to avoid collisions between similarly-named projects
- # macOS uses md5 instead of md5sum
- PROJECT_HASH=$(echo "$PROJECT_DIR" | (md5sum 2>/dev/null || md5 -r 2>/dev/null || echo "$$") | cut -c1-8)
- CONTAINER_NAME="loki-sandbox-$(basename "$PROJECT_DIR" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9-]/-/g')-${PROJECT_HASH}"
- # Sandbox settings
- SANDBOX_IMAGE="${LOKI_SANDBOX_IMAGE:-loki-mode:sandbox}"
- SANDBOX_NETWORK="${LOKI_SANDBOX_NETWORK:-bridge}"
- SANDBOX_CPUS="${LOKI_SANDBOX_CPUS:-2}"
- SANDBOX_MEMORY="${LOKI_SANDBOX_MEMORY:-4g}"
- SANDBOX_READONLY="${LOKI_SANDBOX_READONLY:-false}"
- # API ports
- API_PORT="${LOKI_DASHBOARD_PORT:-57374}"
- DASHBOARD_PORT="${LOKI_DASHBOARD_PORT:-57374}"
- # Security: Prompt injection disabled by default for enterprise security
- PROMPT_INJECTION_ENABLED="${LOKI_PROMPT_INJECTION:-false}"
- #===============================================================================
- # Docker Credential Mount Presets
- #===============================================================================
- # Preset definitions: "host_path:container_path:mode"
- # Container runs as user 'loki' (UID 1000), so mount to /home/loki/
- # Uses functions instead of declare -A for bash 3.2 compatibility (macOS default)
- _get_mount_preset() {
- case "$1" in
- gh) echo "$HOME/.config/gh:/home/loki/.config/gh:ro" ;;
- git) echo "$HOME/.gitconfig:/home/loki/.gitconfig:ro" ;;
- ssh) echo "$HOME/.ssh/known_hosts:/home/loki/.ssh/known_hosts:ro" ;;
- aws) echo "$HOME/.aws:/home/loki/.aws:ro" ;;
- azure) echo "$HOME/.azure:/home/loki/.azure:ro" ;;
- kube) echo "$HOME/.kube:/home/loki/.kube:ro" ;;
- terraform) echo "$HOME/.terraform.d:/home/loki/.terraform.d:ro" ;;
- gcloud) echo "$HOME/.config/gcloud:/home/loki/.config/gcloud:ro" ;;
- npm) echo "$HOME/.npmrc:/home/loki/.npmrc:ro" ;;
- *) echo "" ;;
- esac
- }
- # Environment variables auto-passed per preset (comma-separated)
- _get_env_preset() {
- case "$1" in
- aws) echo "AWS_REGION,AWS_PROFILE,AWS_DEFAULT_REGION" ;;
- azure) echo "AZURE_SUBSCRIPTION_ID,AZURE_TENANT_ID" ;;
- gcloud) echo "GOOGLE_PROJECT,GOOGLE_REGION,GCLOUD_PROJECT" ;;
- terraform) echo "TF_VAR_*" ;;
- *) echo "" ;;
- esac
- }
- #===============================================================================
- # Utility Functions
- #===============================================================================
- log_info() {
- echo -e "${BLUE}[SANDBOX]${NC} $1"
- }
- log_success() {
- echo -e "${GREEN}[SANDBOX]${NC} $1"
- }
- log_warn() {
- echo -e "${YELLOW}[SANDBOX]${NC} $1"
- }
- log_error() {
- echo -e "${RED}[SANDBOX]${NC} $1" >&2
- }
- #===============================================================================
- # A5: ALLOWED_PATHS + BLOCKED_COMMANDS enforcement (sandbox-local)
- #
- # sandbox.sh runs as its own process (set -euo pipefail) and does NOT source
- # run.sh, so the run.sh helpers are out of scope. These are self-contained
- # mirrors that read the same env vars (LOKI_ALLOWED_PATHS / LOKI_BLOCKED_COMMANDS)
- # so the sandbox -- the real security boundary -- enforces them at the point
- # where run.sh/sandbox.sh actually controls the surface: which host paths get
- # bind-mounted, and the operator-supplied `loki sandbox run <cmd>` argv.
- #
- # HONEST SCOPE: this does NOT restrict provider-driven agent writes inside the
- # container (those are contained by cap-drop/seccomp/read-only mounts, not a
- # path allowlist). It is defense-in-depth on the surfaces run.sh owns.
- #===============================================================================
- # Return 0 if target is within LOKI_ALLOWED_PATHS (or the allowlist is unset/
- # empty -> no restriction). realpath -m collapses ../ traversal before the
- # prefix check, so allowed/../../etc cannot slip past.
- _sandbox_path_within_allowed() {
- local target="$1"
- local allow="${LOKI_ALLOWED_PATHS:-}"
- [ -z "$allow" ] && return 0
- [ -z "$target" ] && return 1
- local norm_target
- norm_target="$(realpath -m -- "$target" 2>/dev/null || true)"
- if [ -z "$norm_target" ]; then
- norm_target="$(LOKI_AP_T="$target" python3 -c 'import os; print(os.path.realpath(os.environ["LOKI_AP_T"]))' 2>/dev/null || true)"
- fi
- [ -z "$norm_target" ] && return 1
- local entry norm_entry
- IFS=',' read -ra _AP_ARRAY <<< "$allow"
- for entry in "${_AP_ARRAY[@]}"; do
- entry="${entry#"${entry%%[![:space:]]*}"}"
- entry="${entry%"${entry##*[![:space:]]}"}"
- [ -z "$entry" ] && continue
- # Tilde-expand allowlist entries so "~/proj" matches a resolved $HOME path.
- # SC2088 disabled intentionally: we are pattern-MATCHING a literal "~/" at
- # the start of a config-supplied string (no expansion wanted here), then
- # expanding it ourselves via $HOME. The directive sits in front of the whole
- # if-block (it cannot precede an elif branch -- SC1123).
- # shellcheck disable=SC2088
- if [[ "$entry" == "~/"* ]]; then
- entry="$HOME/${entry#\~/}"
- elif [[ "$entry" == "~" ]]; then
- entry="$HOME"
- fi
- norm_entry="$(realpath -m -- "$entry" 2>/dev/null || true)"
- if [ -z "$norm_entry" ]; then
- norm_entry="$(LOKI_AP_E="$entry" python3 -c 'import os; print(os.path.realpath(os.environ["LOKI_AP_E"]))' 2>/dev/null || true)"
- fi
- [ -z "$norm_entry" ] && continue
- if [ "$norm_target" = "$norm_entry" ] || [ "${norm_target#"${norm_entry}/"}" != "$norm_target" ]; then
- return 0
- fi
- done
- return 1
- }
- # Return 1 (block) if the command string contains a blocked pattern. Mirrors
- # run.sh check_command_allowed; default list matches run.sh's BLOCKED_COMMANDS.
- _sandbox_command_allowed() {
- local command="$1"
- # NOTE: the default list is assigned to a separate var first. It cannot be an
- # inline ${VAR:-default} default because the default value contains a literal
- # "}" (the fork-bomb ":(){ :|:& };:"), which would prematurely terminate the
- Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
- Line 186 on GitHub
- # ${...} parameter expansion and corrupt the list.
- local _default_blocked='rm -rf /,dd if=,mkfs,:(){ :|:& };:'
- Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
- Line 188 on GitHub
- local blocked_list="${LOKI_BLOCKED_COMMANDS:-$_default_blocked}"
- local blocked
- IFS=',' read -ra _BC_ARRAY <<< "$blocked_list"
- for blocked in "${_BC_ARRAY[@]}"; do
- [ -z "$blocked" ] && continue
- if [[ "$command" == *"$blocked"* ]]; then
- log_error "SECURITY: blocked dangerous command (matched '$blocked'): $command"
- return 1
- fi
- done
- return 0
- }
- # Check if a port is available
- check_port_available() {
- local port="$1"
- if command -v lsof &>/dev/null; then
- ! lsof -i ":$port" &>/dev/null
- elif command -v nc &>/dev/null; then
- ! nc -z localhost "$port" 2>/dev/null
- else
- # Assume available if we can't check
- return 0
- fi
- }
- # Validate project directory
- validate_project_dir() {
- if [[ ! -d "$PROJECT_DIR" ]]; then
- log_error "Project directory does not exist: $PROJECT_DIR"
- return 1
- fi
- if [[ ! -r "$PROJECT_DIR" ]]; then
- log_error "Project directory is not readable: $PROJECT_DIR"
- return 1
- fi
- if [[ "$SANDBOX_READONLY" != "true" ]] && [[ ! -w "$PROJECT_DIR" ]]; then
- log_warn "Project directory is not writable: $PROJECT_DIR"
- log_info "Consider using LOKI_SANDBOX_READONLY=true"
- fi
- return 0
- }
- # Warn about API keys based on provider
- warn_missing_api_keys() {
- local provider="$1"
- case "$provider" in
- claude)
- if [[ -z "${ANTHROPIC_API_KEY:-}" ]]; then
- log_warn "ANTHROPIC_API_KEY not set - Claude commands will fail inside container"
- fi
- ;;
- opencode)
- if [[ -z "${OPENROUTER_API_KEY:-}" ]]; then
- log_info "OPENROUTER_API_KEY not set - opencode needs a key for the provider in its model id (e.g. openrouter/<vendor>/<model>)"
- fi
- ;;
- esac
- }
- # Cleanup handler for signals
- cleanup_container() {
- log_warn "Interrupted - cleaning up container..."
- docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
- _sandbox_egress_down
- exit 130
- }
- check_docker() {
- if ! command -v docker &> /dev/null; then
- log_error "Docker not found. Install Docker to use sandbox mode."
- log_error " macOS: brew install --cask docker"
- log_error " Linux: curl -fsSL https://get.docker.com | sh"
- exit 1
- fi
- if ! docker info &> /dev/null; then
- log_error "Docker daemon not running. Start Docker Desktop or dockerd."
- exit 1
- fi
- }
- build_sandbox_image() {
- local dockerfile="$SKILL_DIR/docker/Dockerfile.sandbox"
- if [[ ! -f "$dockerfile" ]]; then
- log_error "Sandbox Dockerfile not found at $dockerfile"
- exit 1
- fi
- log_info "Building sandbox image..."
- docker build -t "$SANDBOX_IMAGE" -f "$dockerfile" "$SKILL_DIR"
- log_success "Sandbox image built: $SANDBOX_IMAGE"
- }
- ensure_image() {
- if ! docker image inspect "$SANDBOX_IMAGE" &> /dev/null; then
- log_warn "Sandbox image not found. Building..."
- build_sandbox_image
- fi
- }
- #===============================================================================
- # Git Worktree Sandbox (Fallback for non-Docker environments)
- #===============================================================================
- # Worktree sandbox settings
- WORKTREE_PREFIX="loki-sandbox"
- WORKTREE_BASE="${LOKI_WORKTREE_BASE:-${TMPDIR:-/tmp}}"
- WORKTREE_STATE_FILE="${PROJECT_DIR}/.loki/sandbox/worktree-state.json"
- # Check if Docker Desktop Sandbox is available (microVM-based isolation)
- is_docker_desktop_sandbox_available() {
- command -v docker &>/dev/null && docker sandbox version &>/dev/null 2>&1
- }
- # Check if Docker is available (non-fatal version)
- is_docker_available() {
- command -v docker &>/dev/null && docker info &>/dev/null 2>&1
- }
- # Check if git worktree is available
- is_git_available() {
- command -v git &>/dev/null && git rev-parse --git-dir &>/dev/null 2>&1
- }
- # Detect which sandbox mode to use
- # Priority: docker-desktop > docker > worktree
- detect_sandbox_mode() {
- local requested="${1:-auto}"
- case "$requested" in
- docker-desktop)
- if is_docker_desktop_sandbox_available; then
- echo "docker-desktop"
- else
- log_error "Docker Desktop Sandbox not available (requires Docker Desktop 4.58+)"
- return 1
- fi
- ;;
- docker)
- if is_docker_available; then
- echo "docker"
- else
- log_error "Docker requested but not available"
- return 1
- fi
- ;;
- worktree)
- if is_git_available; then
- echo "worktree"
- else
- log_error "Git not available for worktree sandbox"
- return 1
- fi
- ;;
- auto|*)
- if is_docker_desktop_sandbox_available; then
- echo "docker-desktop"
- elif is_docker_available; then
- echo "docker"
- elif is_git_available; then
- log_warn "Docker not available - using worktree sandbox (soft isolation)"
- echo "worktree"
- else
- log_error "Neither Docker nor Git available for sandbox mode"
- return 1
- fi
- ;;
- esac
- }
- # Create a worktree sandbox
- create_worktree_sandbox() {
- local prd_path="${1:-}"
- local provider="${LOKI_PROVIDER:-claude}"
- local timestamp
- timestamp=$(date +%Y%m%d-%H%M%S)
- local sandbox_name="${WORKTREE_PREFIX}-${timestamp}"
- local sandbox_branch="${WORKTREE_PREFIX}-${timestamp}"
- local sandbox_path="${WORKTREE_BASE}/${sandbox_name}"
- log_warn ""
- log_warn "=========================================="
- log_warn " WORKTREE SANDBOX - SOFT ISOLATION ONLY"
- log_warn "=========================================="
- log_warn ""
- log_warn "This provides workspace isolation but NOT:"
- log_warn " - Filesystem isolation (can access any file)"
- log_warn " - Network isolation (full network access)"
- log_warn " - Process isolation (no resource limits)"
- log_warn ""
- log_warn "For full isolation, install Docker."
- log_warn ""
- log_info "Creating worktree sandbox..."
- log_info " Location: $sandbox_path"
- log_info " Branch: $sandbox_branch"
- # Check for existing sandbox
- if [[ -f "$WORKTREE_STATE_FILE" ]]; then
- local existing_path
- existing_path=$(jq -r '.sandbox_path // empty' "$WORKTREE_STATE_FILE" 2>/dev/null)
- if [[ -n "$existing_path" ]] && [[ -d "$existing_path" ]]; then
- log_warn "Existing sandbox found: $existing_path"
- log_info "Use 'loki sandbox stop' to stop it first."
- return 1
- fi
- fi
- # Create branch for sandbox
- if ! git branch "$sandbox_branch" HEAD 2>/dev/null; then
- log_error "Failed to create sandbox branch. Are you in a git repository?"
- return 1
- fi
- # Create worktree
- if ! git worktree add "$sandbox_path" "$sandbox_branch" 2>/dev/null; then
- git branch -D "$sandbox_branch" 2>/dev/null
- log_error "Failed to create worktree at $sandbox_path"
- return 1
- fi
- # Set up sandbox environment
- mkdir -p "$sandbox_path/.loki/"{state,logs,signals,queue,memory}
- # Copy essential files
- for file in ".loki/CONTINUITY.md" ".loki/config.yaml" "SKILL.md" "CLAUDE.md"; do
- if [[ -f "$PROJECT_DIR/$file" ]]; then
- mkdir -p "$(dirname "$sandbox_path/$file")"
- cp "$PROJECT_DIR/$file" "$sandbox_path/$file" 2>/dev/null || true
- fi
- done
- # Create sandbox marker
- cat > "$sandbox_path/.loki/SANDBOX_MODE" << EOF
- ISOLATION_TYPE=worktree
- CREATED_AT=$(date -Iseconds)
- PARENT_DIR=$PROJECT_DIR
- EOF
- # Save state using python3 for safe JSON encoding (avoids shell injection
- # from paths containing quotes, backslashes, or other special characters)
- mkdir -p "$(dirname "$WORKTREE_STATE_FILE")"
- python3 -c "
- import json, sys, datetime
- data = {
- 'sandbox_path': sys.argv[1],
- 'sandbox_branch': sys.argv[2],
- 'created_at': datetime.datetime.now().astimezone().isoformat(),
- 'provider': sys.argv[3],
- 'prd_path': sys.argv[4],
- 'status': 'created',
- 'isolation_type': 'worktree'
- }
- with open(sys.argv[5], 'w') as f:
- json.dump(data, f, indent=4)
- f.write('\n')
- " "$sandbox_path" "$sandbox_branch" "$provider" "$prd_path" "$WORKTREE_STATE_FILE"
- log_success "Worktree sandbox created: $sandbox_path"
- return 0
- }
- # Start loki in worktree sandbox
- start_worktree_sandbox() {
- local prd_path="${1:-}"
- local provider="${LOKI_PROVIDER:-claude}"
- # Create sandbox if needed
- if ! [[ -f "$WORKTREE_STATE_FILE" ]]; then
- create_worktree_sandbox "$prd_path" || return 1
- fi
- local sandbox_path
- sandbox_path=$(jq -r '.sandbox_path' "$WORKTREE_STATE_FILE")
- if [[ ! -d "$sandbox_path" ]]; then
- log_error "Sandbox path does not exist: $sandbox_path"
- rm -f "$WORKTREE_STATE_FILE"
- return 1
- fi
- log_info "Starting Loki in worktree sandbox..."
- log_info " Path: $sandbox_path"
- log_info " Provider: $provider"
- # Build loki command
- local loki_cmd="$SKILL_DIR/autonomy/run.sh"
- if [[ -n "$prd_path" ]]; then
- if [[ -f "$sandbox_path/$prd_path" ]]; then
- loki_cmd="$loki_cmd $prd_path"
- elif [[ -f "$prd_path" ]]; then
- cp "$prd_path" "$sandbox_path/" 2>/dev/null || true
- loki_cmd="$loki_cmd $(basename "$prd_path")"
- fi
- fi
- loki_cmd="$loki_cmd --provider $provider"
- # Set environment
- export LOKI_SANDBOX_MODE=true
- export LOKI_SANDBOX_TYPE=worktree
- export LOKI_NOTIFICATIONS=false
- log_info ""
- log_info "Commands (in another terminal):"
- log_info " loki sandbox status - Check status"
- log_info " loki sandbox prompt 'msg' - Send prompt"
- log_info " loki sandbox stop - Stop sandbox"
- log_info ""
- # Run loki in sandbox directory
- cd "$sandbox_path" && $loki_cmd
- }
- # Stop worktree sandbox
- stop_worktree_sandbox() {
- if [[ ! -f "$WORKTREE_STATE_FILE" ]]; then
- log_warn "No active worktree sandbox found"
- return 0
- fi
- local sandbox_path
- sandbox_path=$(jq -r '.sandbox_path' "$WORKTREE_STATE_FILE")
- local sandbox_branch
- sandbox_branch=$(jq -r '.sandbox_branch' "$WORKTREE_STATE_FILE")
- log_info "Stopping worktree sandbox..."
- # Send stop signal
- if [[ -d "$sandbox_path" ]]; then
- touch "$sandbox_path/.loki/STOP" 2>/dev/null || true
- fi
- # Cleanup
- if [[ "${LOKI_SANDBOX_CLEANUP:-true}" == "true" ]]; then
- log_info "Cleaning up worktree..."
- if [[ -n "$sandbox_path" ]] && [[ -d "$sandbox_path" ]]; then
- git worktree remove "$sandbox_path" --force 2>/dev/null || rm -rf "$sandbox_path" 2>/dev/null
- fi
- if [[ -n "$sandbox_branch" ]]; then
- git branch -D "$sandbox_branch" 2>/dev/null || true
- fi
- git worktree prune 2>/dev/null || true
- rm -f "$WORKTREE_STATE_FILE"
- log_success "Worktree sandbox cleaned up"
- else
- log_info "Sandbox preserved at: $sandbox_path"
- log_info "Run 'loki sandbox cleanup' to remove."
- fi
- }
- # Worktree sandbox status
- worktree_sandbox_status() {
- if [[ ! -f "$WORKTREE_STATE_FILE" ]]; then
- log_info "No active worktree sandbox"
- return 0
- fi
- local sandbox_path
- sandbox_path=$(jq -r '.sandbox_path' "$WORKTREE_STATE_FILE")
- local sandbox_branch
- sandbox_branch=$(jq -r '.sandbox_branch' "$WORKTREE_STATE_FILE")
- local created_at
- created_at=$(jq -r '.created_at' "$WORKTREE_STATE_FILE")
- echo ""
- echo -e "${BOLD}Worktree Sandbox Status${NC}"
- echo "========================"
- echo ""
- echo -e " Path: ${CYAN}$sandbox_path${NC}"
- echo -e " Branch: $sandbox_branch"
- echo -e " Created: $created_at"
- if [[ -d "$sandbox_path" ]]; then
- local disk_usage
- disk_usage=$(du -sh "$sandbox_path" 2>/dev/null | cut -f1)
- echo -e " Disk: $disk_usage"
- if [[ -f "$sandbox_path/.loki/STOP" ]]; then
- echo -e " Status: ${YELLOW}Stopping${NC}"
- else
- echo -e " Status: ${GREEN}Active${NC}"
- fi
- else
- echo -e " Status: ${RED}Missing${NC}"
- fi
- echo ""
- echo -e "${YELLOW}[SOFT ISOLATION]${NC} - No filesystem/network/process isolation"
- echo ""
- }
- # Send prompt to worktree sandbox
- worktree_sandbox_prompt() {
- local prompt="$*"
- # Security check: prompt injection disabled by default
- if [[ "$PROMPT_INJECTION_ENABLED" != "true" ]]; then
- log_error "Prompt injection is disabled for security"
- log_info ""
- log_info "To enable, set LOKI_PROMPT_INJECTION=true"
- log_info " Example: LOKI_PROMPT_INJECTION=true loki sandbox prompt 'your message'"
- log_info ""
- log_warn "WARNING: Only enable in trusted environments"
- return 1
- fi
- if [[ -z "$prompt" ]]; then
- log_error "Usage: loki sandbox prompt <your message>"
- return 1
- fi
- if [[ ! -f "$WORKTREE_STATE_FILE" ]]; then
- log_error "No active worktree sandbox"
- return 1
- fi
- local sandbox_path
- sandbox_path=$(jq -r '.sandbox_path' "$WORKTREE_STATE_FILE")
- if [[ ! -d "$sandbox_path" ]]; then
- log_error "Sandbox path does not exist"
- return 1
- fi
- # Use printf to safely write prompt without interpretation
- printf '%s\n' "$prompt" > "$sandbox_path/.loki/HUMAN_INPUT.md"
- log_success "Prompt sent to worktree sandbox"
- log_info "Check $sandbox_path/.loki/logs/ for response"
- }
- # Cleanup orphaned worktrees
- cleanup_worktrees() {
- log_info "Scanning for orphaned sandbox worktrees..."
- local found=0
- while IFS= read -r line; do
- if [[ "$line" == *"$WORKTREE_PREFIX"* ]]; then
- log_info " Found: $line"
- found=$((found+1))
- fi
- done < <(git worktree list 2>/dev/null)
- if [[ $found -eq 0 ]]; then
- log_info "No sandbox worktrees found"
- return 0
- fi
- log_info "Pruning worktrees..."
- git worktree prune 2>/dev/null || true
- # Clean up orphaned branches
- local branches
- branches=$(git branch --list "${WORKTREE_PREFIX}*" 2>/dev/null)
- while IFS= read -r branch; do
- branch=$(echo "$branch" | tr -d '* ')
- if [[ -n "$branch" ]]; then
- log_info " Removing branch: $branch"
- git branch -D "$branch" 2>/dev/null || true
- fi
- done <<< "$branches"
- rm -f "$WORKTREE_STATE_FILE" 2>/dev/null
- log_success "Cleanup complete"
- }
- #===============================================================================
- # Docker Desktop Sandbox (microVM-based isolation via Docker Desktop 4.58+)
- #===============================================================================
- #
- # v8.1 DEPRECATION SPLIT (binary-hosting is legacy; isolation is NOT):
- # This path creates a microVM from Docker Desktop's "claude" sandbox TEMPLATE,
- # which bakes the `claude` CLI binary into the VM so the agent can run inside it.
- # With v8's SDK route, the judge/text sites (and, under LOKI_SDK_MODE, the whole
- # engine) run pure-HTTPS with only ANTHROPIC_API_KEY -- NO in-VM `claude` binary
- # is needed. So the *binary-hosting* purpose of this template is obsolete.
- #
- # What is NOT obsolete, and matters MORE with an in-process SDK: isolating
- # untrusted CODE EXECUTION (npm install / build / test / the generated app).
- # In-process SDK judging runs at HOST privilege, so wrapping the code-exec in a
- # container/VM is still the right posture. The forward path is the existing
- # `start_sandbox` container/worktree isolation (the automatic fallback at the
- # `docker sandbox create` failure below); a future release ships a lightweight
- # `bun + SDK` sandbox image with no bundled agent binary.
- #
- # This release: annotation + a one-shot deprecation notice on selection only. No
- # behavior change, no removal. See tests/test-sandbox-deprecation.sh.
- # Desktop sandbox name follows same pattern as container name
- DESKTOP_SANDBOX_NAME="$CONTAINER_NAME"
- # Emit a one-shot deprecation notice for the microVM binary-hosting path. Names
- # the binary-hosting only and states explicitly that ISOLATION is retained (so it
- # is never misread as "the sandbox is going away"). Best-effort + non-blocking:
- # telemetry must never abort a sandbox start. Mirrors the cli_command_deprecated
- # pattern (event type/source/action + a human log line). Opt out with
- # LOKI_SANDBOX_DEPRECATION_QUIET=1.
- _desktop_deprecation_notice() {
- [ "${LOKI_SANDBOX_DEPRECATION_QUIET:-0}" = "1" ] && return 0
- log_warn "DEPRECATED: the Docker Desktop microVM 'claude' template hosts the"
- log_warn " claude BINARY in-VM, which the v8 SDK route no longer needs."
- log_warn " Code-execution ISOLATION is RETAINED via the container/worktree"
- log_warn " sandbox (unchanged); only the binary-hosting template is legacy."
- # Best-effort telemetry (never abort the start on a telemetry miss).
- if [ -f "$SKILL_DIR/events/emit.sh" ]; then
- bash "$SKILL_DIR/events/emit.sh" sandbox cli sandbox_microvm_binary_deprecated \
- isolation_retained=true path=docker-desktop >/dev/null 2>&1 || true
- fi
- return 0
- }
- # Install provider CLI inside sandbox if not pre-installed
- _desktop_install_provider_cli() {
- local sandbox_name="$1" provider="$2"
- case "$provider" in
- opencode)
- if ! docker sandbox exec "$sandbox_name" which opencode &>/dev/null 2>&1; then
- log_info "Installing opencode in sandbox (one-time)..."
- docker sandbox exec "$sandbox_name" npm install -g opencode-ai 2>&1 | tail -1
- fi
- ;;
- # claude is pre-installed in the sandbox template
- esac
- }
- # Build environment variable args for docker sandbox exec
- # Uses printf %q to escape values and prevent shell expansion corruption
- # (API keys can contain $, !, and other special characters)
- _desktop_build_env_args() {
- DESKTOP_ENV_ARGS=()
- local _escaped
- if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then
- printf -v _escaped '%s' "$ANTHROPIC_API_KEY"
- DESKTOP_ENV_ARGS+=("-e" "ANTHROPIC_API_KEY=$_escaped")
- fi
- if [[ -n "${OPENAI_API_KEY:-}" ]]; then
- printf -v _escaped '%s' "$OPENAI_API_KEY"
- DESKTOP_ENV_ARGS+=("-e" "OPENAI_API_KEY=$_escaped")
- fi
- if [[ -n "${GOOGLE_API_KEY:-}" ]]; then
- printf -v _escaped '%s' "$GOOGLE_API_KEY"
- DESKTOP_ENV_ARGS+=("-e" "GOOGLE_API_KEY=$_escaped")
- fi
- if [[ -n "${GITHUB_TOKEN:-}" ]]; then
- printf -v _escaped '%s' "$GITHUB_TOKEN"
- DESKTOP_ENV_ARGS+=("-e" "GITHUB_TOKEN=$_escaped")
- fi
- if [[ -n "${GH_TOKEN:-}" ]]; then
- printf -v _escaped '%s' "$GH_TOKEN"
- DESKTOP_ENV_ARGS+=("-e" "GH_TOKEN=$_escaped")
- fi
- # Forward all LOKI_* env vars via --env-file to avoid shell expansion issues.
- # Security: LOKI_* values may include secrets, so the env-file is created with
- # mktemp (unpredictable name, no symlink race) and restricted to mode 600 before
- # anything is written into it. The desktop path consumes this file via a
- # foreground/blocking "docker sandbox exec" (see start_docker_desktop_sandbox),
- # so the file is no longer needed once the script exits -- an EXIT trap is the
- # correct cleanup point here. The trap value is baked in (double quotes) so the
- # local path is captured now rather than re-evaluated at exit when it is out of
- # scope. We append our removal so we do not clobber any existing INT/TERM trap.
- local _env_file
- _env_file="$(mktemp "${TMPDIR:-/tmp}/loki-sandbox-env.XXXXXX")"
- chmod 600 "$_env_file"
- local _has_loki_vars=false
- local var
- while IFS= read -r var; do
- if [[ -n "$var" ]]; then
- printf '%s=%s\n' "$var" "${!var}" >> "$_env_file"
- _has_loki_vars=true
- fi
- done < <(compgen -v LOKI_ 2>/dev/null || true)
- if [[ "$_has_loki_vars" == "true" ]] && [[ -f "$_env_file" ]]; then
- DESKTOP_ENV_ARGS+=("--env-file" "$_env_file")
- # shellcheck disable=SC2064
- trap "rm -f -- '$_env_file'" EXIT
- else
- # No LOKI_ vars were forwarded; the file is unused, remove it immediately.
- rm -f -- "$_env_file"
- fi
- }
- start_docker_desktop_sandbox() {
- local prd_path="${1:-}"
- local provider="${LOKI_PROVIDER:-claude}"
- validate_project_dir || return 1
- warn_missing_api_keys "$provider"
- # v8.1: one-shot deprecation notice for the microVM binary-hosting path
- # (isolation is retained; only the bundled-binary template is legacy).
- _desktop_deprecation_notice
- # Check if sandbox already exists
- local sandbox_exists=false
- if docker sandbox ls 2>/dev/null | grep -q "$DESKTOP_SANDBOX_NAME"; then
- sandbox_exists=true
- fi
- if [[ "$sandbox_exists" == "false" ]]; then
- log_info "Creating Docker Desktop Sandbox (microVM)..."
- log_info " Workspace: $PROJECT_DIR"
- log_info " Agent: claude (provider: $provider)"
- docker sandbox create --name "$DESKTOP_SANDBOX_NAME" claude "$PROJECT_DIR" || {
- log_error "Failed to create Docker Desktop Sandbox"
- log_info "Falling back to Docker container sandbox..."
- start_sandbox "$@"
- return $?
- }
- log_success "Sandbox created: $DESKTOP_SANDBOX_NAME"
- else
- log_info "Using existing sandbox: $DESKTOP_SANDBOX_NAME"
- fi
- # Install provider CLI if needed (opencode is not pre-installed)
- _desktop_install_provider_cli "$DESKTOP_SANDBOX_NAME" "$provider"
- # Build environment variable args
- _desktop_build_env_args
- # Build loki command - use SKILL_DIR (the loki-mode install directory) rather than
- # PROJECT_DIR, which is the user's project and may not contain autonomy/run.sh
- local loki_cmd="bash ${SKILL_DIR}/autonomy/run.sh"
- if [[ -n "$prd_path" ]]; then
- local abs_prd
- abs_prd=$(cd "$(dirname "$prd_path")" && pwd)/$(basename "$prd_path")
- loki_cmd="$loki_cmd $abs_prd"
- fi
- loki_cmd="LOKI_PROVIDER=$provider LOKI_SANDBOX_MODE=true LOKI_DASHBOARD=false $loki_cmd"
- log_info ""
- log_info "Starting Loki Mode in Docker Desktop Sandbox..."
- log_info " Isolation: microVM (hypervisor-based)"
- log_info " Provider: $provider"
- log_info " Sandbox: $DESKTOP_SANDBOX_NAME"
- log_info " Workspace: $PROJECT_DIR (synced at same path)"
- log_info ""
- log_info " Private Docker daemon available inside sandbox"
- log_info " Dashboard: run 'loki dashboard' on host (reads synced .loki/ files)"
- log_info ""
- log_info "Commands:"
- log_info " loki sandbox status - Check sandbox status"
- log_info " loki sandbox shell - Open shell in sandbox"
- log_info " loki sandbox stop - Stop sandbox"
- log_info ""
- # Execute loki inside sandbox (use -it only when stdin is a terminal)
- local tty_args="-i"
- if [ -t 0 ]; then
- tty_args="-it"
- fi
- docker sandbox exec $tty_args \
- -w "$PROJECT_DIR" \
- ${DESKTOP_ENV_ARGS[@]+"${DESKTOP_ENV_ARGS[@]}"} \
- "$DESKTOP_SANDBOX_NAME" \
- bash -c "$loki_cmd"
- }
- # shellcheck disable=SC2120
- stop_docker_desktop_sandbox() {
- local remove="${1:-false}"
- if ! docker sandbox ls 2>/dev/null | grep -q "$DESKTOP_SANDBOX_NAME"; then
- log_error "Sandbox not found: $DESKTOP_SANDBOX_NAME"
- return 1
- fi
- # Signal loki to stop gracefully
- docker sandbox exec "$DESKTOP_SANDBOX_NAME" \
- bash -c "touch ${PROJECT_DIR}/.loki/STOP 2>/dev/null" 2>/dev/null || true
- log_info "Stopping sandbox: $DESKTOP_SANDBOX_NAME"
- docker sandbox stop "$DESKTOP_SANDBOX_NAME" 2>/dev/null || true
- if [[ "$remove" == "true" ]]; then
- log_info "Removing sandbox: $DESKTOP_SANDBOX_NAME"
- docker sandbox rm "$DESKTOP_SANDBOX_NAME" 2>/dev/null || true
- fi
- log_success "Sandbox stopped"
- }
- docker_desktop_sandbox_status() {
- local found=false
- while IFS= read -r line; do
- if echo "$line" | grep -q "$DESKTOP_SANDBOX_NAME"; then
- found=true
- echo "$line"
- fi
- done < <(docker sandbox ls 2>/dev/null)
- if [[ "$found" == "false" ]]; then
- log_info "No Docker Desktop Sandbox found for this project"
- log_info "Expected name: $DESKTOP_SANDBOX_NAME"
- return 1
- fi
- # If sandbox is running, get loki status
- if docker sandbox ls 2>/dev/null | grep "$DESKTOP_SANDBOX_NAME" | grep -q "running"; then
- echo ""
- docker sandbox exec -w "$PROJECT_DIR" "$DESKTOP_SANDBOX_NAME" \
- bash -c "bash ${PROJECT_DIR}/autonomy/loki status" 2>/dev/null || true
- fi
- }
- docker_desktop_sandbox_shell() {
- if ! docker sandbox ls 2>/dev/null | grep "$DESKTOP_SANDBOX_NAME" | grep -q "running"; then
- log_error "Sandbox is not running: $DESKTOP_SANDBOX_NAME"
- log_info "Start it with: loki sandbox start"
- return 1
- fi
- log_info "Opening shell in Docker Desktop Sandbox..."
- docker sandbox exec -it -w "$PROJECT_DIR" "$DESKTOP_SANDBOX_NAME" bash
- }
- docker_desktop_sandbox_logs() {
- local lines="${1:-100}"
- if ! docker sandbox ls 2>/dev/null | grep -q "$DESKTOP_SANDBOX_NAME"; then
- log_error "Sandbox not found: $DESKTOP_SANDBOX_NAME"
- return 1
- fi
- docker sandbox exec -w "$PROJECT_DIR" "$DESKTOP_SANDBOX_NAME" \
- bash -c "cat .loki/logs/*.log 2>/dev/null | tail -${lines}" || \
- log_warn "No logs found in .loki/logs/"
- }
- docker_desktop_sandbox_prompt() {
- local message="${1:-}"
- if [[ -z "$message" ]]; then
- log_error "Usage: loki sandbox prompt <message>"
- return 1
- fi
- if ! docker sandbox ls 2>/dev/null | grep "$DESKTOP_SANDBOX_NAME" | grep -q "running"; then
- log_error "Sandbox is not running"
- return 1
- fi
- _desktop_build_env_args
- docker sandbox exec -w "$PROJECT_DIR" \
- ${DESKTOP_ENV_ARGS[@]+"${DESKTOP_ENV_ARGS[@]}"} \
- "$DESKTOP_SANDBOX_NAME" \
- bash -c "printf '%s\n' \"\$1\" > ${PROJECT_DIR}/.loki/HUMAN_INPUT.md" -- "$message"
- log_success "Prompt sent to sandbox"
- }
- docker_desktop_sandbox_run() {
- local cmd="${*}"
- if [[ -z "$cmd" ]]; then
- log_error "Usage: loki sandbox run <command>"
- return 1
- fi
- if ! docker sandbox ls 2>/dev/null | grep "$DESKTOP_SANDBOX_NAME" | grep -q "running"; then
- log_error "Sandbox is not running"
- return 1
- fi
- _desktop_build_env_args
- docker sandbox exec -w "$PROJECT_DIR" \
- ${DESKTOP_ENV_ARGS[@]+"${DESKTOP_ENV_ARGS[@]}"} \
- "$DESKTOP_SANDBOX_NAME" \
- bash -c "$cmd"
- }
- #===============================================================================
- # Docker Container Sandbox (standard Docker - fallback from Docker Desktop)
- #===============================================================================
- # Resolve Docker credential mount presets
- # Reads from: .loki/config/settings.json dockerMounts, LOKI_DOCKER_MOUNTS env var
- # Returns: string of -v and -e flags for docker run
- resolve_docker_mounts() {
- RESOLVED_MOUNTS=()
- # Read configured presets (JSON array of strings)
- local presets_json=""
- if [[ -f "${PROJECT_DIR}/.loki/config/settings.json" ]]; then
- presets_json=$(python3 -c "
- import json, sys
- try:
- with open(sys.argv[1] + '/.loki/config/settings.json') as f:
- print(json.dumps(json.load(f).get('dockerMounts', [])))
- except: print('[]')
- " "${PROJECT_DIR}" 2>/dev/null || echo "[]")
- fi
- # Override with env var if set
- if [[ -n "${LOKI_DOCKER_MOUNTS:-}" ]]; then
- presets_json="$LOKI_DOCKER_MOUNTS"
- fi
- # Parse and resolve
- if [[ "$presets_json" != "[]" ]] && [[ "$presets_json" != "" ]]; then
- local preset_names
- preset_names=$(python3 -c "
- import json, sys
- try:
- data = json.loads(sys.argv[1])
- if isinstance(data, list):
- for item in data:
- print(item)
- except: pass
- " "$presets_json" 2>/dev/null || echo "")
- local name
- while IFS= read -r name; do
- [[ -z "$name" ]] && continue
- local preset_value
- preset_value="$(_get_mount_preset "$name")"
- if [[ -z "$preset_value" ]]; then
- log_warn "Unknown Docker mount preset: $name"
- continue
- fi
- IFS=':' read -ra parts <<< "$preset_value"
- local host_path="${parts[0]}"
- local container_path="${parts[1]}"
- local mode="${parts[2]:-ro}"
- # Safe tilde expansion (no eval) - SC2088 disabled: tilde is intentionally
- # treated as a literal string here for safe expansion without eval
- # shellcheck disable=SC2088
- if [[ "$host_path" == "~/"* ]]; then
- host_path="$HOME/${host_path#\~/}"
- elif [[ "$host_path" == "~" ]]; then
- host_path="$HOME"
- fi
- # Only mount if host path exists
- if [[ -e "$host_path" ]]; then
- RESOLVED_MOUNTS+=("-v" "${host_path}:${container_path}:${mode}")
- log_info " Mount preset [$name]: $host_path -> $container_path ($mode)"
- fi
- # Add associated env vars
- local env_list
- env_list="$(_get_env_preset "$name")"
- if [[ -n "$env_list" ]]; then
- IFS=',' read -ra env_names <<< "$env_list"
- local env_name
- for env_name in "${env_names[@]}"; do
- if [[ "$env_name" == *"*" ]]; then
- # Wildcard: pass all matching env vars
- local prefix="${env_name%\*}"
- while IFS='=' read -r key val; do
- [[ "$key" == "$prefix"* ]] && [[ -n "$val" ]] && {
- RESOLVED_MOUNTS+=("-e" "$key")
- log_info " Env wildcard [$name]: passing $key"
- }
- done < <(env)
- elif [[ -n "${!env_name:-}" ]]; then
- RESOLVED_MOUNTS+=("-e" "$env_name")
- fi
- done
- fi
- done <<< "$preset_names"
- fi
- }
- #-------------------------------------------------------------------------------
- # Opt-in egress allowlist (LOKI_SANDBOX_ALLOW_HOSTS, comma-separated hostnames).
- # Unset: nothing below runs and the docker run argv is unchanged. Set: the sandbox
- # joins an --internal network (no route out) and reaches the world only through a
- # filtering proxy sidecar that permits the listed hosts and refuses any target
- # resolving to link-local, loopback or private ranges.
- #-------------------------------------------------------------------------------
- SANDBOX_ALLOW_HOSTS="${LOKI_SANDBOX_ALLOW_HOSTS:-}"
- EGRESS_PROXY_PORT=3128
- EGRESS_BLOCKED_CIDRS="169.254.0.0/16 fe80::/10 fec0::/10 100.64.0.0/10 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 127.0.0.0/8 ::1/128 fc00::/7"
- # Print the validated allowlist one host per line. Rejects IP literals and
- # anything that is not a plain hostname, so a literal metadata IP can never be
- # allowlisted. Returns 1 on an invalid entry or an empty list.
- _sandbox_egress_hosts() {
- local raw="${1:-$SANDBOX_ALLOW_HOSTS}" h count=0
- local -a parts
- IFS=',' read -ra parts <<< "$raw"
- for h in "${parts[@]}"; do
- h="${h//[[:space:]]/}"
- [[ -z "$h" ]] && continue
- if [[ ! "$h" =~ ^[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?(\.[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?)+$ ]] \
- || [[ "$h" =~ ^[0-9.]+$ ]]; then
- log_error "LOKI_SANDBOX_ALLOW_HOSTS: invalid host (hostnames only, no IPs): $h"
- return 1
- fi
- printf '%s\n' "$h" | tr '[:upper:]' '[:lower:]'
- count=$((count+1))
- done
- [[ $count -gt 0 ]]
- }
- # Filtering proxy (CONNECT and plain HTTP). Decision function is check_target so
- # tests can exercise it with an injected resolver; no network needed.
- _sandbox_egress_proxy_source() {
- cat <<'PYEOF'
- import ipaddress, os, socket, sys, threading
- ALLOW = {h.strip().lower() for h in os.environ.get("ALLOW_HOSTS", "").split(",") if h.strip()}
- BLOCKED = [ipaddress.ip_network(c) for c in os.environ.get("BLOCKED_CIDRS", "").split()]
- def ip_blocked(ip):
- a = ipaddress.ip_address(ip)
- if getattr(a, "ipv4_mapped", None):
- a = a.ipv4_mapped
- return (not a.is_global or getattr(a, "is_site_local", False) or a.is_private or a.is_link_local or a.is_loopback or a.is_unspecified
- or a.is_multicast or a.is_reserved or any(a in n for n in BLOCKED))
- def check_target(host, resolve=None, port=443):
- """Return (ok, reason). Allowed only if host is listed, port is 80 or 443 and every resolved IP is global."""
- if port not in (80, 443):
- return False, "port not allowed"
- host = host.strip().lower().rstrip(".")
- if host not in ALLOW:
- return False, "host not in allowlist"
- resolve = resolve or (lambda h: [i[4][0] for i in socket.getaddrinfo(h, None)])
- try:
- ips = resolve(host)
- except Exception:
- return False, "resolution failed"
- if not ips:
- return False, "no address"
- for ip in ips:
- if ip_blocked(ip):
- return False, "resolves to blocked range " + ip
- return True, ips[0]
- def pipe(a, b):
- try:
- while True:
- d = a.recv(65536)
- if not d:
- break
- b.sendall(d)
- except OSError:
- pass
- finally:
- for s in (a, b):
- try:
- s.shutdown(socket.SHUT_RDWR)
- except OSError:
- pass
- def handle(c):
- try:
- buf = b""
- while b"\r\n\r\n" not in buf and len(buf) < 65536:
- d = c.recv(4096)
- if not d:
- return
- buf += d
- head, _, rest = buf.partition(b"\r\n\r\n")
- line = head.split(b"\r\n")[0].decode("latin1").split()
- if len(line) < 3:
- return
- method, target = line[0], line[1]
- if method.upper() == "CONNECT":
- host, _, port = target.rpartition(":")
- port = int(port or 443)
- rest_out = b""
- else:
- u = target.split("://", 1)[-1].split("/", 1)[0]
- host, _, port = u.partition(":")
- port = int(port or 80)
- rest_out = buf
- ok, info = check_target(host, port=port)
- if not ok:
- c.sendall(b"HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\n\r\n")
- return
- up = socket.create_connection((info, port), timeout=20)
- if method.upper() == "CONNECT":
- c.sendall(b"HTTP/1.1 200 Connection established\r\n\r\n")
- else:
- up.sendall(rest_out)
- t = threading.Thread(target=pipe, args=(up, c), daemon=True)
- t.start()
- pipe(c, up)
- except Exception:
- pass
- finally:
- c.close()
- def main():
- s = socket.socket()
- s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
- s.bind(("0.0.0.0", int(os.environ.get("PROXY_PORT", "3128"))))
- s.listen(128)
- while True:
- conn, _ = s.accept()
- threading.Thread(target=handle, args=(conn,), daemon=True).start()
- if __name__ == "__main__":
- main()
- PYEOF
- }
- # Echo the docker run argv fragments the sandbox container needs when the
- # allowlist is on: one token per line (network override first, then proxy env).
- _sandbox_egress_container_args() {
- local net="$1" proxy="http://${CONTAINER_NAME}-egress:${EGRESS_PROXY_PORT}"
- printf '%s\n' "--network=$net"
- local v
- for v in HTTP_PROXY HTTPS_PROXY http_proxy https_proxy; do
- printf '%s\n' "--env" "$v=$proxy"
- done
- printf '%s\n' "--env" "NO_PROXY=localhost,127.0.0.1"
- }
- # Echo the docker run argv for the proxy sidecar (test-visible, no side effects).
- _sandbox_egress_proxy_args() {
- local script="$1" hosts="$2"
- printf '%s\n' run --detach --name "${CONTAINER_NAME}-egress" \
- --network=bridge --cap-drop=ALL --security-opt=no-new-privileges:true \
- --read-only --pids-limit=128 --memory=256m \
- --env "ALLOW_HOSTS=$hosts" --env "BLOCKED_CIDRS=$EGRESS_BLOCKED_CIDRS" \
- --env "PROXY_PORT=$EGRESS_PROXY_PORT" \
- --volume "$script:/egress-proxy.py:ro" \
- --entrypoint python3 "$SANDBOX_IMAGE" /egress-proxy.py
- }
- # Create the internal network and proxy sidecar. Echoes the internal network name.
- _sandbox_egress_up() {
- local hosts="$1" net="${CONTAINER_NAME}-internal" script
- local dir="${LOKI_RUN_TMP:-}" rec troot
- if [[ -z "$dir" ]]; then
- # Resolve the temp root first (macOS TMPDIR ends in a slash), so the dir is T/loki-egress.X and matches at teardown.
- # mktemp -d picks an unpredictable name with mode 700, so a planted symlink in a shared temp dir is never followed.
- troot="$(cd "${TMPDIR:-/tmp}" 2>/dev/null && pwd -P)" || return 1
- dir="$(mktemp -d "${troot}/loki-egress.XXXXXXXX")" || return 1
- rec="$(_sandbox_egress_record)"
- if ! { mkdir -p "$(dirname "$rec")" && chmod 700 "$(dirname "$rec")" && printf '%s\n' "$dir" > "$rec"; }; then rm -rf -- "$dir"; return 1; fi
- fi
- chmod 700 "$dir" || return 1
- script="$dir/egress-proxy.py"
- _sandbox_egress_proxy_source > "$script"
- docker rm -f "${CONTAINER_NAME}-egress" >/dev/null 2>&1 || true
- docker network rm "$net" >/dev/null 2>&1 || true
- docker network create --internal "$net" >/dev/null || return 1
- local -a pargs=()
- while IFS= read -r line; do pargs+=("$line"); done < <(_sandbox_egress_proxy_args "$script" "$(echo "$hosts" | paste -sd, -)")
- docker "${pargs[@]}" >/dev/null || return 1
- docker network connect --alias "${CONTAINER_NAME}-egress" "$net" "${CONTAINER_NAME}-egress" >/dev/null || return 1
- printf '%s\n' "$net"
- }
- # Host-side state file naming the fallback egress dir for this container. It lives outside PROJECT_DIR on purpose:
- # the project is mounted rw at /workspace, so a record there could be forged by the agent.
- _sandbox_egress_record() { printf '%s\n' "${LOKI_SANDBOX_STATE_DIR:-${HOME}/.loki/sandbox-state}/egress-${CONTAINER_NAME}.dir"; }
- _sandbox_egress_down() {
- local rec fb root
- rec="$(_sandbox_egress_record)"
- if [[ -f "$rec" && ! -L "$rec" ]]; then
- IFS= read -r fb < "$rec" || fb=""
- root="$(cd "${TMPDIR:-/tmp}" 2>/dev/null && pwd -P)" || root=""
- # Remove only a real, own, unlinked loki-egress.XXXXXXXX directory that sits directly in the resolved temp root.
- if [[ -n "$root" && "$(dirname -- "$fb")" == "$root" ]] \
- && [[ "$(basename -- "$fb")" =~ ^loki-egress\.[A-Za-z0-9]{8}$ ]] \
- && [[ -d "$fb" && ! -L "$fb" && -O "$fb" && "$(cd "$fb" 2>/dev/null && pwd -P)" == "$fb" ]]; then
- rm -rf -- "$fb"
- fi
- rm -f -- "$rec"
- fi
- docker rm -f "${CONTAINER_NAME}-egress" >/dev/null 2>&1 || true
- docker network rm "${CONTAINER_NAME}-internal" >/dev/null 2>&1 || true
- }
- start_sandbox() {
- # Parse arguments: extract flags before positional args
- local prd_path=""
- local provider="${LOKI_PROVIDER:-claude}"
- local no_mounts=false
- local -a custom_mounts=()
- while [[ $# -gt 0 ]]; do
- case "$1" in
- --mount)
- if [[ -n "${2:-}" ]]; then
- custom_mounts+=("$2")
- shift 2
- else
- log_error "--mount requires HOST:CONTAINER:MODE argument"
- return 1
- fi
- ;;
- --no-mounts)
- no_mounts=true
- shift
- ;;
- --provider)
- if [[ -n "${2:-}" ]]; then
- provider="$2"
- shift 2
- else
- shift
- fi
- ;;
- *)
- # First non-flag argument is the PRD path
- if [[ -z "$prd_path" ]]; then
- prd_path="$1"
- fi
- shift
- ;;
- esac
- done
- # Set up signal handler to cleanup on Ctrl+C
- trap cleanup_container INT TERM
- check_docker
- validate_project_dir || return 1
- ensure_image
- # Check port availability (unified dashboard/API port)
- if ! check_port_available "$DASHBOARD_PORT"; then
- log_error "Port $DASHBOARD_PORT is already in use"
- log_info "Set LOKI_DASHBOARD_PORT to use a different port"
- return 1
- fi
- # Warn about missing API keys
- warn_missing_api_keys "$provider"
- # Warn about network=none implications
- if [[ "$SANDBOX_NETWORK" == "none" ]]; then
- log_warn "Network disabled (--network=none)"
- log_warn " - Git remote operations will fail"
- log_warn " - Package installations will fail"
- log_warn " - API calls to AI providers will fail"
- fi
- # Check if already running
- if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_warn "Sandbox already running: $CONTAINER_NAME"
- log_info "Use 'loki sandbox status' to check or 'loki sandbox stop' to stop"
- return 0
- fi
- # Clean up any stopped container with same name
- docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
- local network_arg="--network=$SANDBOX_NETWORK" egress_args=() egress_ok=1
- if [[ -n "$SANDBOX_ALLOW_HOSTS" ]]; then
- local egress_hosts egress_net
- egress_hosts=$(_sandbox_egress_hosts) || return 1
- if [[ "$SANDBOX_NETWORK" != "bridge" ]]; then
- log_error "LOKI_SANDBOX_ALLOW_HOSTS requires LOKI_SANDBOX_NETWORK=bridge (got $SANDBOX_NETWORK)"
- return 1
- fi
- egress_net=$(_sandbox_egress_up "$egress_hosts") || { _sandbox_egress_down; log_error "Failed to start egress proxy"; return 1; }
- # Any return after this point (refusal, failed docker run) must tear the proxy and network down.
- egress_ok=0
- trap '[[ "${egress_ok:-1}" == 1 ]] || _sandbox_egress_down; trap - RETURN' RETURN
- while IFS= read -r line; do egress_args+=("$line"); done < <(_sandbox_egress_container_args "$egress_net")
- network_arg="${egress_args[0]}"
- fi
- log_info "Starting sandbox container..."
- log_info " Image: $SANDBOX_IMAGE"
- log_info " Project: $PROJECT_DIR"
- log_info " Provider: $provider"
- log_info " Network: $SANDBOX_NETWORK"
- log_info " CPUs: $SANDBOX_CPUS"
- log_info " Memory: $SANDBOX_MEMORY"
- # Build docker run command
- local docker_args=(
- "run"
- "--name" "$CONTAINER_NAME"
- "--detach"
- "--interactive"
- "--tty"
- # Resource limits
- "--cpus=$SANDBOX_CPUS"
- "--memory=$SANDBOX_MEMORY"
- "--memory-swap=$SANDBOX_MEMORY" # Disable swap
- "--pids-limit=256"
- # Security hardening
- "--security-opt=no-new-privileges:true"
- "--cap-drop=ALL"
- "--cap-add=CHOWN"
- # SETUID/SETGID intentionally omitted: container runs as non-root (UID 1000)
- # and should not be able to change UID/GID, which would undermine isolation
- # Network
- "$network_arg"
- )
- if [[ ${#egress_args[@]} -gt 1 ]]; then
- docker_args+=("${egress_args[@]:1}")
- log_info " Egress: allowlist only ($(echo "$egress_hosts" | paste -sd, -)); private and link-local ranges blocked"
- fi
- # Add seccomp profile if available
- local seccomp_profile="$SKILL_DIR/autonomy/seccomp-sandbox.json"
- if [[ -f "$seccomp_profile" ]]; then
- docker_args+=("--security-opt" "seccomp=$seccomp_profile")
- log_info " Seccomp: enabled"
- fi
- # A5+: workspace-mount allowlist enforcement (opt-in via LOKI_ALLOWED_PATHS).
- #
- # The workspace bind-mount below is where provider-driven agent file writes
- # actually land (the agent writes inside /workspace, which is $PROJECT_DIR on
- # the host). Unlike the custom --mount surface, this mount was previously
- # always made writable regardless of LOKI_ALLOWED_PATHS, so an allowlist that
- # did not include the project dir was silently ignored for the main write
- # surface.
- #
- # When LOKI_ALLOWED_PATHS is set and the workspace itself is OUTSIDE the
- # allowlist, fail closed: refuse to start rather than bind-mount it writable.
- # This is genuinely enforceable because docker mount mode (rw vs ro) and which
- # host paths get bound are decided HERE, before the container exists -- the
- # kernel enforces the resulting mount, not a wrapper check. We refuse rather
- # than auto-downgrade to :ro because a read-only workspace cannot be written
- # by the agent at all and would silently produce a no-op build; refusing makes
- # the misconfiguration visible. (Allowlisted extra paths are still mounted via
- # the custom --mount surface above, which also enforces the allowlist.)
- #
- # When LOKI_ALLOWED_PATHS is empty (default), _sandbox_path_within_allowed
- # returns 0 unconditionally, so this is byte-identical to prior behavior.
- if ! _sandbox_path_within_allowed "$PROJECT_DIR"; then
- log_error "Workspace is outside LOKI_ALLOWED_PATHS, refusing to mount it writable: $PROJECT_DIR"
- log_error " Add the project directory to LOKI_ALLOWED_PATHS, or unset LOKI_ALLOWED_PATHS to disable path enforcement."
- return 1
- fi
- # Mount project directory
- if [[ "$SANDBOX_READONLY" == "true" ]]; then
- docker_args+=("--volume" "$PROJECT_DIR:/workspace:ro")
- # Need a writable .loki directory - copy existing state to a temp dir so we
- # do not start with an empty volume (which would lose config/state).
- # Security: created with mktemp -d (unpredictable name, no symlink/predictable
- # race) and restricted to mode 700 since it holds a copy of the project's .loki
- # state. Lifecycle: this directory is bind-mounted into a DETACHED container
- # (see "--detach" in docker_args) that outlives this function, so an EXIT trap
- # would delete it out from under the running container. Instead, its path is
- # recorded on the container via a docker label and removed by stop_sandbox()
- # right before the container is removed. The label survives across separate
- # script invocations, which a $$-derived name would not.
- local _loki_state_tmp
- _loki_state_tmp="$(mktemp -d "${TMPDIR:-/tmp}/loki-sandbox-state.XXXXXX")"
- chmod 700 "$_loki_state_tmp"
- if [[ -d "$PROJECT_DIR/.loki" ]]; then
- cp -a "$PROJECT_DIR/.loki/." "$_loki_state_tmp/" 2>/dev/null || true
- fi
- docker_args+=("--volume" "$_loki_state_tmp:/workspace/.loki:rw")
- docker_args+=("--label" "loki.state_dir=$_loki_state_tmp")
- else
- docker_args+=("--volume" "$PROJECT_DIR:/workspace:rw")
- fi
- # Config self-protection: mount critical .loki/ paths as read-only
- # Prevents agents from corrupting council state, config, or audit trail
- if [[ -d "$PROJECT_DIR/.loki/council" ]]; then
- docker_args+=("--volume" "$PROJECT_DIR/.loki/council:/workspace/.loki/council:ro")
- fi
- if [[ -f "$PROJECT_DIR/.loki/config.yaml" ]]; then
- docker_args+=("--volume" "$PROJECT_DIR/.loki/config.yaml:/workspace/.loki/config.yaml:ro")
- fi
- # Mount git config (read-only) - mount to /home/loki since container runs as user loki
- if [[ -f "$HOME/.gitconfig" ]]; then
- docker_args+=("--volume" "$HOME/.gitconfig:/home/loki/.gitconfig:ro")
- fi
- # SSH agent forwarding (more secure than mounting .ssh directory)
- # Only forward if SSH agent is available
- if [[ -n "${SSH_AUTH_SOCK:-}" ]]; then
- docker_args+=("--volume" "$SSH_AUTH_SOCK:/ssh-agent:ro")
- docker_args+=("--env" "SSH_AUTH_SOCK=/ssh-agent")
- elif [[ -d "$HOME/.ssh" ]]; then
- # Fallback: mount only known_hosts and public keys (NOT private keys)
- if [[ -f "$HOME/.ssh/known_hosts" ]]; then
- docker_args+=("--volume" "$HOME/.ssh/known_hosts:/home/loki/.ssh/known_hosts:ro")
- fi
- log_warn "SSH agent not available. Git operations may require manual auth."
- fi
- # Mount GitHub CLI config (read-only) - mount to /home/loki since container runs as user loki
- if [[ -d "$HOME/.config/gh" ]]; then
- docker_args+=("--volume" "$HOME/.config/gh:/home/loki/.config/gh:ro")
- fi
- # Apply Docker credential mount presets (additive on top of defaults above)
- if [[ "$no_mounts" != "true" ]] && [[ "${LOKI_NO_DOCKER_MOUNTS:-}" != "true" ]]; then
- resolve_docker_mounts
- if [[ ${#RESOLVED_MOUNTS[@]} -gt 0 ]]; then
- docker_args+=("${RESOLVED_MOUNTS[@]}")
- fi
- fi
- # Apply custom --mount arguments
- local custom_mount
- for custom_mount in "${custom_mounts[@]+"${custom_mounts[@]}"}"; do
- if [[ -n "$custom_mount" ]]; then
- IFS=':' read -ra mount_parts <<< "$custom_mount"
- local c_host="${mount_parts[0]:-}"
- local c_container="${mount_parts[1]:-}"
- local c_mode="${mount_parts[2]:-ro}"
- if [[ -n "$c_host" ]] && [[ -n "$c_container" ]]; then
- # Safe tilde expansion (no eval) - SC2088 disabled: intentional literal match
- # shellcheck disable=SC2088
- if [[ "$c_host" == "~/"* ]]; then
- c_host="$HOME/${c_host#\~/}"
- elif [[ "$c_host" == "~" ]]; then
- c_host="$HOME"
- fi
- if [[ -e "$c_host" ]]; then
- # A5: when LOKI_ALLOWED_PATHS is set, reject a custom mount
- # whose host path is outside the allowlist. This is a write
- # surface run.sh/sandbox.sh genuinely controls (we decide what
- # to bind into the container), so the allowlist is enforced
- # here rather than only documented.
- if ! _sandbox_path_within_allowed "$c_host"; then
- log_error "Custom mount host path is outside LOKI_ALLOWED_PATHS, refusing to mount: $c_host"
- return 1
- fi
- docker_args+=("--volume" "${c_host}:${c_container}:${c_mode}")
- log_info " Custom mount: $c_host -> $c_container ($c_mode)"
- else
- log_warn "Custom mount source does not exist: $c_host"
- fi
- else
- log_warn "Invalid mount format: $custom_mount (expected HOST:CONTAINER[:MODE])"
- fi
- fi
- done
- # Pass API keys as environment variables (more secure than mounting files)
- if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then
- docker_args+=("--env" "ANTHROPIC_API_KEY")
- fi
- if [[ -n "${OPENAI_API_KEY:-}" ]]; then
- docker_args+=("--env" "OPENAI_API_KEY")
- fi
- if [[ -n "${GOOGLE_API_KEY:-}" ]]; then
- docker_args+=("--env" "GOOGLE_API_KEY")
- fi
- if [[ -n "${GITHUB_TOKEN:-}" ]]; then
- docker_args+=("--env" "GITHUB_TOKEN")
- fi
- if [[ -n "${GH_TOKEN:-}" ]]; then
- docker_args+=("--env" "GH_TOKEN")
- fi
- # Loki configuration
- docker_args+=(
- "--env" "LOKI_PROVIDER=$provider"
- "--env" "LOKI_SANDBOX_MODE=true"
- "--env" "LOKI_NOTIFICATIONS=false"
- "--env" "LOKI_DASHBOARD=true"
- )
- # Expose ports
- docker_args+=(
- "--publish" "$API_PORT:57374"
- )
- # Expose additional ports for testing (e.g., LOKI_EXTRA_PORTS="3000:3000,8080:8080")
- if [[ -n "${LOKI_EXTRA_PORTS:-}" ]]; then
- IFS=',' read -ra EXTRA_PORTS <<< "$LOKI_EXTRA_PORTS"
- for port_mapping in "${EXTRA_PORTS[@]}"; do
- docker_args+=("--publish" "$port_mapping")
- done
- fi
- # Working directory
- docker_args+=("--workdir" "/workspace")
- # Override entrypoint to use bash (Dockerfile has loki as entrypoint)
- docker_args+=("--entrypoint" "/bin/bash")
- # Image and command
- docker_args+=("$SANDBOX_IMAGE")
- # Build loki command
- local loki_cmd="loki start"
- if [[ -n "$prd_path" ]]; then
- # Convert to container path (handle paths with spaces)
- # macOS realpath does not support --relative-to, so use Python fallback
- local relative_prd
- relative_prd=$(realpath --relative-to="$PROJECT_DIR" "$prd_path" 2>/dev/null \
- || python3 -c "import os.path; print(os.path.relpath('$prd_path', '$PROJECT_DIR'))" 2>/dev/null \
- || basename "$prd_path")
- local container_prd="/workspace/${relative_prd}"
- # Quote path to handle spaces
- loki_cmd="$loki_cmd \"$container_prd\""
- fi
- loki_cmd="$loki_cmd --provider $provider"
- docker_args+=("-c" "$loki_cmd")
- # Run container
- local container_id
- container_id=$(docker "${docker_args[@]}") || { log_error "docker run failed"; return 1; }
- egress_ok=1
- log_success "Sandbox started: ${container_id:0:12}"
- log_info ""
- log_info "Access:"
- log_info " Dashboard: http://localhost:$DASHBOARD_PORT"
- log_info " API: http://localhost:$API_PORT"
- if [[ -n "${LOKI_EXTRA_PORTS:-}" ]]; then
- log_info " Extra: $LOKI_EXTRA_PORTS"
- fi
- log_info ""
- log_info "Commands:"
- log_info " loki sandbox logs - View logs"
- log_info " loki sandbox shell - Open shell in container"
- log_info " loki sandbox stop - Stop sandbox"
- log_info ""
- log_info "Testing (when ready):"
- log_info " loki sandbox phase - Check SDLC phase & testing tips"
- log_info " loki sandbox test - Run tests"
- log_info " loki sandbox serve - Start dev server"
- log_info " loki sandbox prompt 'msg' - Send real-time prompt"
- }
- stop_sandbox() {
- check_docker
- if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_info "Stopping sandbox: $CONTAINER_NAME"
- # Read the read-only-mode temp state dir path recorded as a container label
- # (see start_sandbox H5 handling). It is removed once the container is gone.
- local _state_dir=""
- _state_dir=$(docker inspect --format '{{ index .Config.Labels "loki.state_dir" }}' "$CONTAINER_NAME" 2>/dev/null || true)
- # Try graceful stop first (touch STOP file)
- docker exec "$CONTAINER_NAME" touch /workspace/.loki/STOP 2>/dev/null || true
- # Wait for graceful shutdown (check every second for up to 10 seconds)
- local waited=0
- while [ $waited -lt 10 ]; do
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_success "Sandbox stopped gracefully"
- _sandbox_egress_down
- # Remove the temp state dir now that the container has stopped.
- if [[ -n "$_state_dir" ]] && [[ -d "$_state_dir" ]]; then
- rm -rf -- "$_state_dir" 2>/dev/null || true
- fi
- return 0
- fi
- sleep 1
- waited=$((waited+1))
- done
- # Force stop if still running
- log_info "Force stopping container..."
- docker stop --time 5 "$CONTAINER_NAME" 2>/dev/null || true
- docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
- # Remove the temp state dir now that the container is gone.
- if [[ -n "$_state_dir" ]] && [[ -d "$_state_dir" ]]; then
- rm -rf -- "$_state_dir" 2>/dev/null || true
- fi
- _sandbox_egress_down
- log_success "Sandbox stopped"
- else
- _sandbox_egress_down
- log_warn "No running sandbox found"
- fi
- }
- # Send a prompt/directive to the running sandbox
- sandbox_prompt() {
- local prompt="$*"
- # Security check: prompt injection disabled by default
- if [[ "$PROMPT_INJECTION_ENABLED" != "true" ]]; then
- log_error "Prompt injection is disabled for security"
- log_info ""
- log_info "To enable, set LOKI_PROMPT_INJECTION=true"
- log_info " Example: LOKI_PROMPT_INJECTION=true loki sandbox prompt 'your message'"
- log_info ""
- log_warn "WARNING: Only enable in trusted environments"
- return 1
- fi
- if [[ -z "$prompt" ]]; then
- log_error "Usage: loki sandbox prompt <your message>"
- log_info "Example: loki sandbox prompt 'start the dev server and show me the URL'"
- return 1
- fi
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running. Start it first with: loki sandbox start"
- return 1
- fi
- log_info "Sending prompt to sandbox..."
- # Write to HUMAN_INPUT.md inside the container
- # Use heredoc to avoid command injection via single quotes in prompt
- docker exec "$CONTAINER_NAME" bash -c 'cat > /workspace/.loki/HUMAN_INPUT.md' <<< "$prompt"
- log_success "Prompt sent. Loki will process it in the next iteration."
- log_info ""
- log_info "Watch the response with: loki sandbox logs"
- log_info "Or check status with: loki sandbox status"
- }
- # Run a command inside the sandbox and show output
- sandbox_run() {
- local cmd="$*"
- if [[ -z "$cmd" ]]; then
- log_error "Usage: loki sandbox run <command>"
- log_info "Example: loki sandbox run 'npm run dev'"
- return 1
- fi
- # A5: defense-in-depth on the operator-supplied command. The container is the
- # real boundary, but an obviously-destructive command is blocked here before
- # it is dispatched into the sandbox (honors LOKI_BLOCKED_COMMANDS).
- if ! _sandbox_command_allowed "$cmd"; then
- return 1
- fi
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running"
- return 1
- fi
- log_info "Running command in sandbox: $cmd"
- # Trust contract: $cmd is the operator's CLI argv (`loki sandbox run <cmd>`),
- # and the operator is the same person who is running the sandbox container
- # on their own machine. Shell metacharacters are intentionally allowed so
- # the operator can run pipelines, redirects, etc. inside the sandbox -- but
- # the sandbox is the security boundary, not this shell-out. We:
- # 1. set the working dir via `docker exec -w` instead of `cd && $cmd`,
- # so the user command cannot break out of the cd via metacharacters
- # that confuse the wrapper (e.g. unmatched quotes affecting cd);
- # 2. pass `--` to bash so the user string cannot be re-interpreted as a
- # bash flag if it begins with `-`;
- # 3. pass the user string as a separate argv element to bash -lc so it
- # is treated as the script body, not concatenated into a wrapper.
- docker exec -it -w /workspace "$CONTAINER_NAME" bash -lc -- "$cmd"
- }
- # Start a dev server inside the sandbox
- sandbox_serve() {
- local port="${1:-3000}"
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running"
- return 1
- fi
- log_info "Looking for dev server configuration..."
- # Detect project type and start appropriate server
- local serve_cmd=""
- if docker exec "$CONTAINER_NAME" test -f /workspace/package.json; then
- # Check for common dev server scripts
- local has_dev
- has_dev=$(docker exec "$CONTAINER_NAME" jq -r '.scripts.dev // empty' /workspace/package.json 2>/dev/null)
- local has_start
- has_start=$(docker exec "$CONTAINER_NAME" jq -r '.scripts.start // empty' /workspace/package.json 2>/dev/null)
- if [[ -n "$has_dev" ]]; then
- serve_cmd="npm run dev"
- elif [[ -n "$has_start" ]]; then
- serve_cmd="npm start"
- fi
- elif docker exec "$CONTAINER_NAME" test -f /workspace/requirements.txt; then
- # Python project
- if docker exec "$CONTAINER_NAME" test -f /workspace/manage.py; then
- serve_cmd="python manage.py runserver 0.0.0.0:$port"
- elif docker exec "$CONTAINER_NAME" test -f /workspace/app.py; then
- serve_cmd="python app.py"
- fi
- fi
- if [[ -z "$serve_cmd" ]]; then
- log_warn "Could not auto-detect dev server"
- log_info "Try: loki sandbox run 'your-dev-command'"
- return 1
- fi
- log_success "Starting dev server: $serve_cmd"
- log_info ""
- # Warn if the dev server port was not published when the container started
- local port_published
- port_published=$(docker port "$CONTAINER_NAME" "$port" 2>/dev/null || true)
- if [[ -z "$port_published" ]]; then
- log_warn "Port $port is NOT published to the host."
- log_warn "The dev server will run inside the container but will not be accessible from localhost."
- log_warn "To fix, restart the sandbox with: LOKI_EXTRA_PORTS='$port:$port' loki sandbox start"
- log_info ""
- fi
- log_info "Access the app at:"
- log_info " http://localhost:$port"
- log_info ""
- log_info "Press Ctrl+C to stop the server (sandbox continues running)"
- docker exec -it "$CONTAINER_NAME" bash -c "cd /workspace && $serve_cmd"
- }
- # Run tests inside the sandbox
- sandbox_test() {
- local test_type="${1:-all}"
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running"
- return 1
- fi
- log_info "Running tests in sandbox..."
- # Detect project type and test command
- local test_cmd=""
- if docker exec "$CONTAINER_NAME" test -f /workspace/package.json; then
- case "$test_type" in
- unit)
- test_cmd=$(docker exec "$CONTAINER_NAME" jq -r '.scripts["test:unit"] // .scripts.test // "npm test"' /workspace/package.json 2>/dev/null)
- ;;
- integration)
- test_cmd=$(docker exec "$CONTAINER_NAME" jq -r '.scripts["test:integration"] // empty' /workspace/package.json 2>/dev/null)
- [[ -z "$test_cmd" ]] && test_cmd="npm run test:integration"
- ;;
- e2e)
- if docker exec "$CONTAINER_NAME" test -d /workspace/node_modules/.bin/playwright; then
- test_cmd="npx playwright test"
- elif docker exec "$CONTAINER_NAME" test -d /workspace/node_modules/.bin/cypress; then
- test_cmd="npx cypress run"
- else
- test_cmd=$(docker exec "$CONTAINER_NAME" jq -r '.scripts["test:e2e"] // empty' /workspace/package.json 2>/dev/null)
- fi
- ;;
- all|*)
- test_cmd=$(docker exec "$CONTAINER_NAME" jq -r '.scripts.test // "npm test"' /workspace/package.json 2>/dev/null)
- ;;
- esac
- elif docker exec "$CONTAINER_NAME" test -f /workspace/requirements.txt; then
- case "$test_type" in
- unit)
- test_cmd="pytest tests/unit/ -v"
- ;;
- integration)
- test_cmd="pytest tests/integration/ -v"
- ;;
- e2e)
- test_cmd="pytest tests/e2e/ -v"
- ;;
- all|*)
- test_cmd="pytest -v"
- ;;
- esac
- elif docker exec "$CONTAINER_NAME" test -f /workspace/Cargo.toml; then
- test_cmd="cargo test"
- elif docker exec "$CONTAINER_NAME" test -f /workspace/go.mod; then
- test_cmd="go test ./..."
- fi
- if [[ -z "$test_cmd" ]] || [[ "$test_cmd" == "null" ]]; then
- log_warn "Could not auto-detect test command"
- log_info "Supported test types: unit, integration, e2e, all"
- log_info "Or run: loki sandbox run 'your-test-command'"
- return 1
- fi
- log_success "Running: $test_cmd"
- docker exec -it "$CONTAINER_NAME" bash -c "cd /workspace && $test_cmd"
- }
- # Check SDLC phase and suggest testing
- sandbox_phase() {
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running"
- return 1
- fi
- # Get current phase from orchestrator state
- local phase
- phase=$(docker exec "$CONTAINER_NAME" bash -c \
- "python3 -c \"import json; print(json.load(open('/workspace/.loki/state/orchestrator.json')).get('currentPhase', 'UNKNOWN'))\" 2>/dev/null" \
- || echo "UNKNOWN")
- echo ""
- echo -e "${BOLD}Current SDLC Phase: ${CYAN}$phase${NC}"
- echo ""
- case "$phase" in
- BOOTSTRAP|DISCOVERY|ARCHITECTURE)
- log_info "Phase $phase - No testing required yet"
- log_info "Testing begins in DEVELOPMENT phase"
- ;;
- INFRASTRUCTURE)
- log_info "Phase $phase - Infrastructure testing"
- echo " Recommended:"
- echo " loki sandbox run 'docker-compose up -d'"
- echo " loki sandbox run 'terraform plan'"
- ;;
- DEVELOPMENT)
- log_info "Phase $phase - Development testing recommended"
- echo ""
- echo " Run dev server:"
- echo " loki sandbox serve"
- echo ""
- echo " Run unit tests:"
- echo " loki sandbox test unit"
- echo ""
- echo " Manual testing:"
- echo " loki sandbox shell"
- ;;
- QA)
- log_info "Phase $phase - Full testing required"
- echo ""
- echo " Run all tests:"
- echo " loki sandbox test all"
- echo ""
- echo " Run specific test suites:"
- echo " loki sandbox test unit"
- echo " loki sandbox test integration"
- echo " loki sandbox test e2e"
- echo ""
- echo " Interactive testing:"
- echo " loki sandbox serve"
- echo " loki sandbox shell"
- echo ""
- echo " To report issues during testing:"
- echo " loki sandbox prompt 'Found bug: describe the bug here'"
- ;;
- DEPLOYMENT)
- log_info "Phase $phase - Smoke testing"
- echo ""
- echo " Run smoke tests:"
- echo " loki sandbox run 'npm run test:smoke'"
- echo ""
- echo " Check deployment status:"
- echo " loki sandbox run 'curl -s http://localhost:3000/health'"
- ;;
- GROWTH|*)
- log_info "Phase $phase - Continuous testing"
- echo ""
- echo " Run regression tests:"
- echo " loki sandbox test all"
- echo ""
- echo " Performance testing:"
- echo " loki sandbox run 'npx k6 run tests/load.js'"
- ;;
- esac
- echo ""
- }
- # Expose additional ports (for development testing)
- sandbox_expose() {
- local port="${1:-}"
- if [[ -z "$port" ]]; then
- log_error "Usage: loki sandbox expose <port>"
- log_info "Example: loki sandbox expose 8080"
- return 1
- fi
- check_docker
- if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_error "Sandbox is not running"
- log_info "Note: Ports must be exposed when starting the sandbox"
- log_info "Set LOKI_EXTRA_PORTS='$port:$port' before running 'loki sandbox start'"
- return 1
- fi
- log_warn "Cannot expose ports on running container"
- log_info ""
- log_info "To expose additional ports, restart sandbox with:"
- echo ""
- echo " LOKI_EXTRA_PORTS='$port:$port' loki sandbox start"
- echo ""
- log_info "Or access via sandbox shell:"
- echo " loki sandbox shell"
- echo " # Then run your server inside the container"
- }
- sandbox_status() {
- check_docker
- if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_success "Sandbox is running: $CONTAINER_NAME"
- echo ""
- docker ps --filter "name=$CONTAINER_NAME" --format "table {{.ID}}\t{{.Status}}\t{{.Ports}}"
- echo ""
- # Try to get loki status
- log_info "Loki Status:"
- docker exec "$CONTAINER_NAME" loki status 2>/dev/null || log_warn "Could not get loki status"
- else
- log_warn "Sandbox is not running"
- # Check for stopped container
- if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_info "Stopped container exists. Use 'loki sandbox start' to restart."
- fi
- fi
- }
- sandbox_logs() {
- local lines="${1:-100}"
- check_docker
- if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- docker logs --tail "$lines" -f "$CONTAINER_NAME"
- else
- log_error "Sandbox is not running"
- exit 1
- fi
- }
- sandbox_shell() {
- check_docker
- if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
- log_info "Opening shell in sandbox..."
- docker exec -it "$CONTAINER_NAME" bash
- else
- log_error "Sandbox is not running"
- exit 1
- fi
- }
- sandbox_build() {
- check_docker
- build_sandbox_image
- }
- show_help() {
- echo -e "${BOLD}Loki Mode Sandbox${NC}"
- echo ""
- echo "Usage: loki sandbox <command> [options]"
- echo ""
- echo "Commands:"
- echo " start [PRD] Start sandbox with optional PRD"
- echo " stop Stop running sandbox"
- echo " status Check sandbox status"
- echo " logs [N] View last N log lines (default: 100)"
- echo " shell Open bash shell in sandbox"
- echo " build Build/rebuild sandbox image"
- echo " cleanup Remove orphaned sandbox worktrees/branches"
- echo ""
- echo "Interactive Commands (while sandbox is running):"
- echo " prompt <msg> Send a prompt/directive to Loki in real-time"
- echo " run <cmd> Run a command inside the sandbox"
- echo " serve [port] Auto-detect and start dev server (default port: 3000)"
- echo ""
- echo "Testing Commands:"
- echo " test [type] Run tests (type: unit, integration, e2e, all)"
- echo " phase Check SDLC phase and get testing suggestions"
- echo " expose <port> Show how to expose additional ports"
- echo ""
- echo "Mode Options:"
- echo " --docker-desktop Force Docker Desktop Sandbox (microVM isolation)"
- echo " --docker Force Docker container sandbox (seccomp isolation)"
- echo " --worktree Force git worktree sandbox (soft isolation)"
- echo " --auto Auto-detect best mode (default)"
- echo ""
- echo "Sandbox Modes (priority order):"
- echo " Docker Desktop - microVM isolation via 'docker sandbox' (Docker Desktop 4.58+)"
- echo " (default) Hypervisor-based VM, private Docker daemon, pre-installed CLIs"
- echo " Docker Container - Container isolation with seccomp, dropped capabilities,"
- echo " (fallback) resource limits, network control (Dockerfile.sandbox)"
- echo " Worktree - Git worktree isolation (last resort if no Docker)"
- echo " (last resort) Warning: No filesystem/network/process isolation"
- echo ""
- echo "Environment Variables:"
- echo " LOKI_SANDBOX_IMAGE Docker image (default: loki-mode:sandbox)"
- echo " LOKI_SANDBOX_NETWORK Network mode: bridge, none, host (default: bridge)"
- echo " LOKI_SANDBOX_ALLOW_HOSTS Egress allowlist, comma-separated hostnames (default: unset)"
- echo " LOKI_SANDBOX_CPUS CPU limit (default: 2)"
- echo " LOKI_SANDBOX_MEMORY Memory limit (default: 4g)"
- echo " LOKI_SANDBOX_READONLY Mount project read-only (default: false)"
- echo " LOKI_SANDBOX_CLEANUP Auto-cleanup worktree on stop (default: true)"
- echo " LOKI_EXTRA_PORTS Expose extra ports (e.g., '3000:3000,8080:8080')"
- echo " LOKI_PROMPT_INJECTION Enable real-time prompts (default: false)"
- echo ""
- echo "Security Features (Docker mode):"
- echo " - Seccomp profile restricts syscalls"
- echo " - No new privileges flag"
- echo " - Dropped capabilities"
- echo " - Resource limits (CPU, memory, PIDs)"
- echo " - API keys passed as env vars (not mounted)"
- echo " - Prompt injection DISABLED by default (enterprise security)"
- echo ""
- echo "Examples:"
- echo " loki sandbox start # Start (auto-detect: Desktop > Docker > Worktree)"
- echo " loki sandbox start --docker-desktop ./prd.md # Force Docker Desktop microVM"
- echo " loki sandbox start --docker ./prd.md # Force Docker container mode"
- echo " loki sandbox start --worktree # Force worktree mode"
- echo " loki sandbox prompt 'start the app and show URL' # Send prompt"
- echo " loki sandbox serve # Start dev server"
- echo " loki sandbox test # Run all tests"
- echo " loki sandbox test unit # Run unit tests only"
- echo " loki sandbox phase # Check phase, get testing tips"
- echo " loki sandbox run 'npm test' # Run custom command"
- echo " loki sandbox cleanup # Remove old worktrees"
- }
- #===============================================================================
- # Main
- #===============================================================================
- main() {
- local command="${1:-help}"
- shift || true
- # Parse mode option
- local mode="auto"
- local args=()
- while [[ $# -gt 0 ]]; do
- case "$1" in
- --docker-desktop) mode="docker-desktop"; shift ;;
- --docker) mode="docker"; shift ;;
- --worktree) mode="worktree"; shift ;;
- --auto) mode="auto"; shift ;;
- *) args+=("$1"); shift ;;
- esac
- done
- # Detect sandbox mode for commands that need it
- # For stop/status/etc, read persisted mode first so we stop the correct sandbox type
- local sandbox_mode=""
- local _mode_file="${PROJECT_DIR}/.loki/sandbox/mode"
- case "$command" in
- start)
- sandbox_mode=$(detect_sandbox_mode "$mode") || exit 1
- # Persist the detected mode so stop/status use the same mode
- mkdir -p "$(dirname "$_mode_file")"
- echo "$sandbox_mode" > "$_mode_file"
- ;;
- stop|status|prompt|shell|logs|run|serve|test|phase)
- # Read persisted mode if no explicit mode override was given
- if [[ "$mode" == "auto" ]] && [[ -f "$_mode_file" ]]; then
- sandbox_mode=$(cat "$_mode_file" 2>/dev/null || true)
- fi
- # Fall back to detection if no persisted mode
- if [[ -z "$sandbox_mode" ]]; then
- sandbox_mode=$(detect_sandbox_mode "$mode") || exit 1
- fi
- ;;
- esac
- case "$command" in
- start)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- start_docker_desktop_sandbox ${args[@]+"${args[@]}"}
- elif [[ "$sandbox_mode" == "docker" ]]; then
- start_sandbox ${args[@]+"${args[@]}"}
- else
- start_worktree_sandbox ${args[@]+"${args[@]}"}
- fi
- ;;
- stop)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- stop_docker_desktop_sandbox
- elif [[ "$sandbox_mode" == "docker" ]]; then
- stop_sandbox
- else
- stop_worktree_sandbox
- fi
- ;;
- status)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_status
- elif [[ "$sandbox_mode" == "docker" ]]; then
- sandbox_status
- else
- worktree_sandbox_status
- fi
- ;;
- logs)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_logs ${args[@]+"${args[@]}"}
- else
- sandbox_logs ${args[@]+"${args[@]}"}
- fi
- ;;
- shell)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_shell
- else
- sandbox_shell
- fi
- ;;
- build)
- sandbox_build
- ;;
- prompt)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_prompt ${args[@]+"${args[@]}"}
- elif [[ "$sandbox_mode" == "docker" ]]; then
- sandbox_prompt ${args[@]+"${args[@]}"}
- else
- worktree_sandbox_prompt ${args[@]+"${args[@]}"}
- fi
- ;;
- run)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_run ${args[@]+"${args[@]}"}
- else
- sandbox_run ${args[@]+"${args[@]}"}
- fi
- ;;
- cleanup)
- cleanup_worktrees
- ;;
- serve)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_run "npm start 2>/dev/null || python3 -m http.server ${args[0]:-3000}"
- else
- sandbox_serve ${args[@]+"${args[@]}"}
- fi
- ;;
- test)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_run "npm test 2>/dev/null || python3 -m pytest 2>/dev/null || echo 'No test runner found'"
- else
- sandbox_test ${args[@]+"${args[@]}"}
- fi
- ;;
- phase)
- if [[ "$sandbox_mode" == "docker-desktop" ]]; then
- docker_desktop_sandbox_run "bash ${PROJECT_DIR}/autonomy/loki status"
- else
- sandbox_phase
- fi
- ;;
- expose)
- sandbox_expose ${args[@]+"${args[@]}"}
- ;;
- help|--help|-h)
- show_help
- ;;
- *)
- log_error "Unknown command: $command"
- show_help
- exit 1
- ;;
- esac
- }
- main "$@"
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.