Skip to content
Back to search

firmware-archaeology-toolset

Project-oriented CLI for finding, retrieving, and analyzing firmware artifacts across vendor sites, web archives, code forges, and registries.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/nozominetworks--firmware-archaeology-toolset/badge.svg)](/listings/nozominetworks--firmware-archaeology-toolset)

HTML

<a href="/listings/nozominetworks--firmware-archaeology-toolset"><img src="/api/listings/nozominetworks--firmware-archaeology-toolset/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

The normal FAT runtime has no third-party dependencies. JavaScript-heavy scraper or Wayback parsers can opt into the Playwright environment described in the plugin guides.

git clone https://github.com/NozomiNetworks/firmware-archaeology-toolset.git
cd firmware-archaeology-toolset
python3 fat.py --help

Create a project, set keywords, then run any plugin:

python3 fat.py project device-research --keywords "vendor sdk,vendor platform"
python3 fat.py github search
python3 fat.py github status
python3 fat.py github select 1,3
python3 fat.py github download selected
python3 fat.py report

Every plugin shares the same command surface: `status`, `options`, `search`, `select`, `clear-selection`, `download`, and `set`.

The `wayback` plugin adds `post` to run the parser post hook and `clear` to wipe snapshots, selections, history, and downloaded files.

All commands accept an optional project name prefix to run commands temporarily into a specific project, different than the selected one.

python3 fat.py device-research github status
python3 fat.py device-research report

This repository contains the generic Wayback engine and the `httpbin` test parser. External vendor plugins are loaded from `~/.fat/plugins/` by default. Configure another directory in `~/.fat/config.json`:

{"plugin_dir": "/opt/fat/plugins"}

Outputs land in `output/report.md` (human-readable report) and `output/downloads/` (artifacts).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 234 lines
  1. # Firmware Archaeology Toolset
  2. [![CI](https://github.com/NozomiNetworks/firmware-archaeology-toolset/actions/workflows/tests.yml/badge.svg)](https://github.com/NozomiNetworks/firmware-archaeology-toolset/actions/workflows/tests.yml)
  3. FAT is a stateful, project-oriented tool for artifact hunting: find likely
  4. firmware images, SDKs, container images, and other downloadable artifacts across
  5. public sources, triage them, and retrieve them, including archived copies from
  6. the Wayback Machine.
  7. Discovery and retrieval stay in one place. Projects, keywords, findings,
  8. selections, and download history persist in a single `.fat.json` state file, so
  9. you can run one tool at a time and pick up where you left off.
  10. ![FAT command-line help](docs/images/fat.png)
  11. ## Getting started
  12. The normal FAT runtime has no third-party dependencies. JavaScript-heavy scraper
  13. or Wayback parsers can opt into the Playwright environment described in the
  14. plugin guides.
  15. ```bash
  16. git clone https://github.com/NozomiNetworks/firmware-archaeology-toolset.git
  17. cd firmware-archaeology-toolset
  18. python3 fat.py --help
  19. ```
  20. Create a project, set keywords, then run any plugin:
  21. ```bash
  22. python3 fat.py project device-research --keywords "vendor sdk,vendor platform"
  23. python3 fat.py github search
  24. python3 fat.py github status
  25. python3 fat.py github select 1,3
  26. python3 fat.py github download selected
  27. python3 fat.py report
  28. ```
  29. Every plugin shares the same command surface: `status`, `options`, `search`,
  30. `select`, `clear-selection`, `download`, and `set`.
  31. The `wayback` plugin adds `post` to run the parser post
  32. hook and `clear` to wipe snapshots, selections, history, and downloaded
  33. files.
  34. All commands accept an optional project name prefix to run commands temporarily
  35. into a specific project, different than the selected one.
  36. ```bash
  37. python3 fat.py device-research github status
  38. python3 fat.py device-research report
  39. ```
  40. This repository contains the generic Wayback engine
  41. and the `httpbin` test parser. External vendor plugins are
  42. loaded from `~/.fat/plugins/` by default. Configure
  43. another directory in `~/.fat/config.json`:
  44. ```json
  45. {"plugin_dir": "/opt/fat/plugins"}
  46. ```
  47. Outputs land in `output/report.md` (human-readable report) and
  48. `output/downloads/` (artifacts).
  49. ## Leads
  50. Record investigation leads in project state so they
  51. survive across runs and agent sessions:
  52. ```bash
  53. python3 fat.py project lead add https://vendor.example/support/model \
  54. --source portal --title "support page" --note "lists fw 3.2.1"
  55. python3 fat.py project lead list
  56. python3 fat.py project lead set 1 status acted
  57. python3 fat.py project lead rm 2
  58. ```
  59. The `status` field can assume the following values:
  60. - `new`: recorded, not examined
  61. - `checked`: examined, still pending
  62. - `acted`: used as input to a FAT action
  63. - `dead`: ruled out
  64. The `source` field instead is among:
  65. - `portal` for vendor sites
  66. - `web` for searches or dorks
  67. - `writeup` for blogs, talks and forums
  68. - `repo` for code repositories
  69. - `image` for container images
  70. - `archive` for archived snaphots
  71. - `user` for other, user-provided sources
  72. Duplicate URLs are rejected.
  73. ## Basic workflows
  74. Search a code forge or registry (same flow for `github`, `gitlab`,
  75. `dockerhub`):
  76. ```bash
  77. python3 fat.py github set per-query 50
  78. python3 fat.py github set min-match-keywords 2
  79. python3 fat.py github set max-requests 25
  80. python3 fat.py github set max-enrichments 8
  81. python3 fat.py github set cache-ttl 3600
  82. python3 fat.py github search
  83. ```
  84. Searches are paginated and budgeted. FAT walks each provider's result pages
  85. round-robin within a per-run request budget, and reports a clearly marked
  86. partial result when the budget ends a query early, so that a big search degrades
  87. gracefully instead of failing.
  88. Transient errors retry with exponential backoff (honoring Retry-After, up to 4
  89. attempts), hard rate limits stop the affected path cleanly, and every page
  90. fetched, failure, and partial query is recorded in .fat.json and
  91. output/report.md.
  92. Scrape a vendor download site after building the vendor parser:
  93. ```bash
  94. python3 fat.py scraper set parser <external-parser>
  95. python3 fat.py scraper set website https://vendor.example/downloads
  96. python3 fat.py scraper search
  97. python3 fat.py scraper select 1,2
  98. python3 fat.py scraper download selected
  99. ```
  100. Enumerate and retrieve archived artifacts with the Wayback plugin:
  101. ```bash
  102. python3 fat.py wayback set parser <external-parser>
  103. python3 fat.py wayback set target-url https://vendor.example/download
  104. python3 fat.py wayback search snapshot
  105. python3 fat.py wayback select 1,2
  106. python3 fat.py wayback search firmware
  107. python3 fat.py wayback select 2:1,2:2
  108. python3 fat.py wayback download selected
  109. ```
  110. ## Extending and contributing
  111. - [Contributing guide](CONTRIBUTING.md)
  112. - [Plugin author guide](plugins/README.md)
  113. - [Wayback parser guide](plugins/wayback/PARSER_HOWTO.md)
  114. - [Plugin API reference](docs/plugin-api/README.md)
  115. - Agent playbooks: [investigation](skills/fat-investigation/SKILL.md) and [extension guide](skills/fat-extension/SKILL.md)
  116. Drop a top-level plugin module or package into `plugins/`, expose
  117. `PLUGIN_CLASS`, and implement the expected methods. Plugins are discovered
  118. automatically at runtime.
  119. Vendor scraper and Wayback parser modules are fetched from the configured
  120. external plugin directory. See the [plugin author guide](plugins/README.md) for
  121. the contracts.
  122. ## Testing
  123. The regular suite runs offline. The source-plugin integration checks try the
  124. live services first and fall back to small API-shaped fixtures when an upstream
  125. service is unavailable or rate-limited.
  126. The Wayback integration suite exercises the live archive first and automatically
  127. falls back to a recorded local mirror (`tests/wayback_mirror.py`) when
  128. web.archive.org is unreachable or throttling; the backend used per test is
  129. printed in the run summary. Force a Wayback backend with
  130. `FAT_WAYBACK_INTEGRATION_MODE=live|mirror` (default `auto`)
  131. For source-plugin integration checks, use
  132. `FAT_SURFACE_INTEGRATION_MODE=live|fixture` (default `auto`) to force the live
  133. service or deterministic fixture backend. To make the complete integration run
  134. deterministic, select both fixture backends:
  135. ```bash
  136. FAT_SURFACE_INTEGRATION_MODE=fixture \
  137. FAT_WAYBACK_INTEGRATION_MODE=mirror \
  138. python3 -m unittest discover -s tests -p 'integration_*.py' -v
  139. ```
  140. ```bash
  141. python3 -m pip install --require-hashes --only-binary=:all: -r requirements-test.txt
  142. python3 -m coverage run --branch -m unittest discover -s tests -p 'test_*.py' -v
  143. python3 -m coverage run --branch -a -m unittest discover -s tests -p 'integration_*.py' -v
  144. python3 -m coverage report --precision=2 --fail-under=80
  145. ```
  146. The browser-rendering integration check is separate because it installs the
  147. hash-locked Playwright package and its matching Chromium runtime:
  148. ```bash
  149. ./scripts/run_browser_tests_container.sh
  150. ```
  151. To install all locked development dependencies, including the Playwright browser
  152. runtime, run:
  153. ```bash
  154. ./scripts/install_dependencies.sh
  155. ```
  156. Run it from a virtual environment when the host Python is externally managed:
  157. ```bash
  158. python3 -m venv .venv
  159. . .venv/bin/activate
  160. ./scripts/install_dependencies.sh
  161. ```
  162. Use `./scripts/install_dependencies.sh --browser` for browser support only.
  163. The Chromium version is selected by the pinned Playwright release.
  164. Before opening a pull request, run the local quality gate. It regenerates the
  165. committed API reference with the pinned documentation environment, runs the
  166. containerized checks, and reports whether the branch is ready:
  167. ```bash
  168. ./scripts/prepare-pr.sh
  169. ```
  170. CI runs the unit, integration, browser integration, and docs jobs on every
  171. pull request.
  172. ## Repository map
  173. - `fat.py`: CLI entry point
  174. - `fatlib/`: shared project, HTTP, discovery, and report helpers
  175. - `plugins/`: plugin base and implementations (`dockerhub`, `github`, `gitlab`, `scraper`, `wayback`)
  176. - `docs/`: generated plugin API reference
  177. - `skills/`: agent playbooks for operating and extending FAT
  178. - `tests/`: offline unit tests and live integration tests
  179. - `scripts/`: docs build and container test scripts

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.