Skip to content
Back to search

cli

CLI and Agent Skill for Firecrawl - Add scrape, search, and browsing capabilities to your AI agents

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/firecrawl/cli with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/firecrawl/cli'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/firecrawl--cli/badge.svg)](/listings/firecrawl--cli)

HTML

<a href="/listings/firecrawl--cli"><img src="/api/listings/firecrawl--cli/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

npm install -g firecrawl-cli

Or set up everything in one command (install CLI globally, authenticate, and add skills across all detected coding editors):

npx -y firecrawl-cli@latest init -y --browser

- `-y` runs setup non-interactively - `--browser` opens the browser for Firecrawl authentication automatically - skills install globally to every detected AI coding agent by default

Setup Skills, Workflows, and MCP

If you are using an AI coding agent like Claude Code, you can also install skill groups manually — one command per family:

firecrawl setup core       # scrape/search/crawl/interact primitives + index skills ("skills" is an alias)
firecrawl setup build      # app-integration skills for the Firecrawl API
firecrawl setup workflows  # end-to-end recipes (lead gen, deep research, ...)

Or install a single skill by name — the `firecrawl-` prefix is optional:

firecrawl setup developer-index
firecrawl setup seo-audit

If no API key is found afterwards, an interactive terminal offers a browser login (pass `--browser` to log in without the prompt); non-interactive runs never block — they print a hint and the skills walk agents through setup on first use.

These install globally across all detected coding editors by default. Use `--agent <agent>` to scope any of them to one editor.

Scope setup to a single harness

By default skills route to every detected harness. To install and route to only one harness in a single command:

# One-shot from the install script (non-interactive, no TTY needed)
curl -fsSL https://firecrawl.dev/install.sh | bash -s -- --agent openclaw

# Or, once the CLI is installed
firecrawl init --agent openclaw          # only OpenClaw
firecrawl setup skills --agent openclaw
firecrawl setup workflows --agent openclaw

Any args after `bash -s --` are forwarded to `firecrawl init`, so you can add things like `--skip-auth` for a skills-only, no-login setup. When you run `firecrawl init` interactively without `--agent`, it shows a checkbox of your detected harnesses (all selected by default) so you can pick a subset.

| Harness | `--agent` value | | ------------ | --------------- | | Claude Code | `claude-code` | | Codex | `codex` | | Cursor | `cursor` | | Windsurf | `windsurf` | | OpenCode | `opencode` | | OpenClaw | `openclaw` | | OpenHands | `openhands` | | Hermes Agent | `hermes-agent` |

Agent skills

The init command installs the **CLI skills** by default and offers the **workflow skills** as optional extras, into AI coding agents (Cursor, Claude Code, Windsurf, etc.):

- **CLI skills** — teach agents how to use the Firecrawl CLI for live web work (search, scrape, interact, map, crawl, agent). Installed by default. - **Workflow skills** — teach agents how to produce Firecrawl-powered deliverables such as research briefs, SEO audits, QA reports, lead lists, knowledge bases, and design-system extraction. Interactive multi-select during init.

All skill families live in the [`firecrawl/skills`](https://github.com/firecrawl/skills) catalog — including the **build skills** for integrating Firecrawl into application code:

npx skills add firecrawl/skills

> Contributing skills? CLI skills (including the research/developer index skills) → PR this repo (`skills/`). Build/SDK skills → PR the [`firecrawl`](https://github.com/firecrawl/firecrawl) monorepo (`skills/`). Workflow skills → PR [`firecrawl/firecrawl-workflows`](https://github.com/firecrawl/firecrawl-workflows). The catalog ([`firecrawl/skills`](https://github.com/firecrawl/skills)) is read-only — never PR it directly.

To reinstall skills manually:

firecrawl setup core
firecrawl setup build
firecrawl setup workflows

To install the Firecrawl MCP server into your editors (Cursor, Claude Code, VS Code, etc.):

firecrawl setup mcp

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,337 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # 🔥 Firecrawl CLI
  2. Command-line interface for Firecrawl, the context API for AI agents. Search, scrape, parse, crawl, map, and interact with the web, search research papers, developer sources, and government sources, and run agent jobs directly from your terminal.
  3. ## Installation
  4. ```bash
  5. npm install -g firecrawl-cli
  6. ```
  7. Or set up everything in one command (install CLI globally, authenticate, and install shared skills for your detected coding editors):
  8. ```bash
  9. npx -y firecrawl-cli@latest init -y --browser
  10. ```
  11. - `-y` runs setup non-interactively
  12. - `--browser` opens the browser for Firecrawl authentication automatically
  13. - skills install globally in the shared skills directory and are made available to detected AI coding agents by default
  14. ### Setup Skills, Workflows, and MCP
  15. If you are using an AI coding agent like Claude Code, you can also install skill groups manually — one command per family:
  16. ```bash
  17. firecrawl setup core # scrape/search/crawl/interact primitives + index skills ("skills" is an alias)
  18. firecrawl setup build # app-integration skills for the Firecrawl API
  19. firecrawl setup workflows # end-to-end recipes (lead gen, deep research, ...)
  20. ```
  21. Or install a single skill by name — the `firecrawl-` prefix is optional:
  22. ```bash
  23. firecrawl setup developer-index
  24. firecrawl setup seo-audit
  25. ```
  26. If no API key is found afterwards, an interactive terminal offers a browser login (pass `--browser` to log in without the prompt); non-interactive runs never block — they print a hint and the skills walk agents through setup on first use.
  27. Default installs use the shared `~/.agents/skills` directory and the coding editors detected by Firecrawl. The npx installer always includes Codex as the shared-directory target, even when Codex is not detected, preserving shared files and agent links. If no editors are detected, only the shared directory is populated. The npx path detects supported global targets, including Kiro and custom configuration locations such as `CLAUDE_CONFIG_DIR` and `XDG_CONFIG_HOME`. Project-only targets are excluded. Use `--agent <agent>` to target a specific editor.
  28. #### Scope setup to a single harness
  29. By default skills route to every detected harness. To install and route to only
  30. one harness in a single command:
  31. ```bash
  32. # One-shot from the install script (non-interactive, no TTY needed)
  33. curl -fsSL https://firecrawl.dev/install.sh | bash -s -- --agent openclaw
  34. # Or, once the CLI is installed
  35. firecrawl init --agent openclaw # only OpenClaw
  36. firecrawl setup skills --agent openclaw
  37. firecrawl setup workflows --agent openclaw
  38. ```
  39. Any args after `bash -s --` are forwarded to `firecrawl init`, so you can add
  40. things like `--skip-auth` for a skills-only, no-login setup. When you run
  41. `firecrawl init` interactively without `--agent`, it shows a checkbox of your
  42. detected harnesses (all selected by default) so you can pick a subset.
  43. | Harness | `--agent` value |
  44. | ------------ | --------------- |
  45. | Claude Code | `claude-code` |
  46. | Codex | `codex` |
  47. | Cursor | `cursor` |
  48. | Windsurf | `windsurf` |
  49. | OpenCode | `opencode` |
  50. | OpenClaw | `openclaw` |
  51. | OpenHands | `openhands` |
  52. | Hermes Agent | `hermes-agent` |
  53. ### Agent skills
  54. The init command installs the **CLI skills** by default and offers the **workflow skills** as optional extras, into AI coding agents (Cursor, Claude Code, Windsurf, etc.):
  55. - **CLI skills** — teach agents how to use the Firecrawl CLI for live web work (search, scrape, interact, map, crawl, agent) and Alexandria data providers. Installed by default.
  56. - **Workflow skills** — teach agents how to produce Firecrawl-powered deliverables such as research briefs, SEO audits, QA reports, lead lists, knowledge bases, and design-system extraction. Interactive multi-select during init.
  57. All skill families live in the [`firecrawl/skills`](https://github.com/firecrawl/skills) catalog — including the **build skills** for integrating Firecrawl into application code:
  58. ```bash
  59. npx skills add firecrawl/skills
  60. ```
  61. > Contributing skills? CLI skills (including the research/developer index skills) → PR this repo (`skills/`). Build/SDK skills → PR the [`firecrawl`](https://github.com/firecrawl/firecrawl) monorepo (`skills/`). Workflow skills → PR [`firecrawl/firecrawl-workflows`](https://github.com/firecrawl/firecrawl-workflows). The catalog ([`firecrawl/skills`](https://github.com/firecrawl/skills)) is read-only — never PR it directly.
  62. The Firecrawl Claude Code and Cursor plugins are released separately and do not include these CLI skills. Changes here still reach the skill catalog.
  63. To reinstall skills manually:
  64. ```bash
  65. firecrawl setup core
  66. firecrawl setup build
  67. firecrawl setup workflows
  68. ```
  69. ### CLI updates
  70. Global npm, pnpm, bun, and yarn installations, plus standalone binaries on macOS
  71. and Linux, check for updates in the background. Commands do not wait for an
  72. update, and the updater never writes to stdout. The next invocation uses the
  73. updated release, with a one-time notice on an interactive terminal.
  74. For global package installs, the installed package is a stable launcher. The
  75. updater uses npm to install a new release into a separate directory under
  76. `~/.firecrawl/versions/`, verifies it, then atomically switches the current
  77. version pointer. It never rewrites the global installation or removes files
  78. used by running commands. Managed versions are shared within a Node major
  79. version, so a patch upgrade reuses the saved release. The launcher checks the
  80. release's Node requirements before selecting it. npm must be available on PATH
  81. for automatic package updates; otherwise use your package manager's manual
  82. update command. Install scripts are disabled. Dependencies resolve to versions
  83. published no later than the release itself, so the waiting period covers them
  84. too.
  85. Each global installation has its own version selection. Maintenance retains
  86. the current and previous release for each installation, plus any older release
  87. still used by a running CLI process. It removes other inactive versions and
  88. staging directories older than 24 hours whose installer has exited. Files owned
  89. by another machine on shared storage are retained conservatively.
  90. Reinstalling with npm resets the launcher's selection to that package,
  91. including an explicit downgrade. Disable automatic updates before reinstalling
  92. if you want to stay on the selected version. Disabling updates alone keeps the
  93. currently selected release.
  94. Standalone binaries download their release archive, verify its SHA-256 checksum
  95. and reported version, then atomically replace the executable. The previous
  96. binary is retained beside it with a `.previous` suffix.
  97. A newly observed release waits at least 24 hours before unattended installation.
  98. The worker rechecks that it is still the latest release after this delay; a
  99. changed candidate starts a new waiting period. Version checks normally refresh
  100. about once a day. Automatic updates never cross a major version or install a
  101. prerelease. Prerelease installations remain pinned.
  102. To update immediately, bypassing the waiting period, including to a new major
  103. version:
  104. ```bash
  105. firecrawl update
  106. ```
  107. To restore the previous version without the network:
  108. ```bash
  109. firecrawl update --rollback
  110. ```
  111. For global package installs, the launcher handles rollback without loading the
  112. active release, so it works even when that release cannot start. Rollback
  113. disables automatic updates so the same release is not immediately
  114. installed again. The first managed-package rollback returns to the original
  115. package-manager installation; later updates can roll back to the previous
  116. managed version. Moving the registry's latest tag back does not automatically
  117. downgrade an existing installation.
  118. Control automatic updates with:
  119. ```bash
  120. firecrawl update --disable-auto
  121. firecrawl update --enable-auto
  122. export FIRECRAWL_NO_AUTO_UPDATE=1
  123. ```
  124. Automatic updates are also off when `CI` is set, unless it is `0` or `false`.
  125. `FIRECRAWL_NO_UPDATE_CHECK=1` disables both automatic updates and notices.
  126. Explicit updates still work with these settings. Failures leave the active
  127. installation unchanged and back off before retrying. Details are recorded in
  128. `~/.firecrawl/auto-update.log`.
  129. Homebrew and other system packages, project-local installs, package runners
  130. such as npx, and standalone Windows binaries retain their selected release and
  131. show the appropriate manual update instruction. Enabling automatic updates on
  132. these installations reports that they are externally managed. Disabling updates
  133. saves the shared user preference from any installation.
  134. Non-writable or differently owned standalone binaries are not replaced.
  135. Package updates use npm's registry configuration. Binary version checks honor
  136. `npm_config_registry`; binary downloads use GitHub releases, or the mirror set
  137. with `FIRECRAWL_RELEASES_URL`.
  138. ### Skill updates
  139. Interactive CLI sessions (except `init`, `setup`, and the `-y`/`--yes` init
  140. shorthand) show a boxed notice when installed Firecrawl skills differ from the
  141. current catalog. The check includes supporting reference files and runs
  142. independently of CLI releases. Installs record which catalog skills existed, so
  143. the notice announces skills published later, once, in families whose umbrella
  144. skill is installed. Skills skipped during setup are not offered again.
  145. Each affected family gets one command: `firecrawl setup core`,
  146. `firecrawl setup build`, or `firecrawl setup workflows`. These commands install
  147. the entire current family, including previously skipped skills and
  148. new skills added since your CLI was released. When installing through npx, failed
  149. catalog API discovery falls back to this CLI release's known names with a warning.
  150. Native installs discover skills directly from the Git repository and require that
  151. repository to be available. An explicit refresh replaces installed files, including
  152. local edits; the notice itself never changes them.
  153. Checks cover global installations recorded in `~/.agents/.skill-lock.json` with
  154. files in `~/.agents/skills`. They cache catalog results for 20 hours, repeat the
  155. notice at most once every 12 hours, and retry failed checks after an hour.
  156. Non-interactive runs stay quiet. Set `FIRECRAWL_NO_UPDATE_CHECK=1` to disable both
  157. CLI and skill update notices.
  158. ### MCP and web defaults
  159. To install the Firecrawl MCP server into your editors (Cursor, Claude Code, VS Code, etc.):
  160. ```bash
  161. firecrawl setup mcp
  162. ```
  163. With a key saved by `firecrawl login`, export `FIRECRAWL_API_KEY` and pass
  164. `--agent <agent>` (or `--agent all`), or add `--keyless`: setup never writes a
  165. stored key into client configs.
  166. Use `--agent <agent>` for one client, or `--agent all --global` to configure
  167. Claude Code, VS Code, Codex, OpenCode, Cursor, Gemini CLI, Windsurf,
  168. Antigravity, Amp, Grok Build, Zed, Hermes Agent, and OpenClaw in one run. The
  169. `--agent` values for the newer clients are `amp` (or `ampcode`, `amp-code`),
  170. `grok` (or `grok-build`, `grok-cli`), and `zed`. Amp, Grok Build, and Zed are written to their user settings, so they do not take
  171. `--project`: Amp to `~/.config/amp/settings.json` (or an existing
  172. `settings.jsonc` when there is no `settings.json`), Grok Build to
  173. `~/.grok/config.toml`, and Zed to `~/.config/zed/settings.json`
  174. (`$XDG_CONFIG_HOME/zed/settings.json` on Linux when it is set,
  175. `%APPDATA%\Zed\settings.json` on Windows). Clients that cannot be
  176. configured on this machine (for example OpenClaw when the `openclaw` command is
  177. not installed) are skipped and listed in the summary; the command only fails
  178. when a client it tried to configure fails. When `FIRECRAWL_API_KEY` is set,
  179. Antigravity and Zed are skipped because they have no verified way to reference
  180. the key from the environment; add `--keyless` to include them.
  181. To make Firecrawl the default web provider for supported AI agents:
  182. ```bash
  183. firecrawl setup defaults # or: firecrawl make default
  184. ```
  185. This disables native web fetch/search where supported so agents route web work
  186. through Firecrawl. When run interactively, it asks harness by harness (Claude
  187. Code, Codex) so you can choose exactly which to change. Use `-y` to skip the
  188. picker and apply to all, or `--agent` to target one:
  189. ```bash
  190. firecrawl setup defaults --agent codex # only Codex
  191. firecrawl setup defaults -y # all harnesses, no prompts
  192. ```
  193. To undo those config changes (also interactive, harness by harness):
  194. ```bash
  195. firecrawl setup defaults --undo # pick which to restore
  196. firecrawl setup defaults --undo --agent claude
  197. ```
  198. ## Quick Start
  199. Just run a command - the CLI will prompt you to authenticate if needed:
  200. ```bash
  201. firecrawl https://example.com
  202. ```
  203. ## Authentication
  204. On first run, you'll be prompted to authenticate:
  205. ```
  206. 🔥 firecrawl cli
  207. Search, scrape, and interact with the web
  208. Welcome! To get started, authenticate with your Firecrawl account.
  209. 1. Login with browser (recommended)
  210. 2. Enter API key manually
  211. Tip: You can also set FIRECRAWL_API_KEY environment variable
  212. Enter choice [1/2]:
  213. ```
  214. ### Authentication Methods
  215. ```bash
  216. # Interactive (prompts automatically when needed)
  217. firecrawl
  218. # Browser login
  219. firecrawl login
  220. # Direct API key
  221. firecrawl login --api-key fc-your-api-key
  222. # Environment variable
  223. export FIRECRAWL_API_KEY=fc-your-api-key
  224. # Per-command API key
  225. firecrawl scrape https://example.com --api-key fc-your-api-key
  226. ```
  227. ### Self-hosted / Local Development
  228. For self-hosted Firecrawl instances or local development, use the `--api-url` option:
  229. ```bash
  230. # Use a local Firecrawl instance (no API key required)
  231. firecrawl --api-url http://localhost:3002 scrape https://example.com
  232. # Or set via environment variable
  233. export FIRECRAWL_API_URL=http://localhost:3002
  234. firecrawl scrape https://example.com
  235. # Self-hosted with API key
  236. firecrawl --api-url https://firecrawl.mycompany.com --api-key fc-xxx scrape https://example.com
  237. ```
  238. When using a custom API URL (anything other than `https://api.firecrawl.dev`), authentication is automatically skipped, allowing you to use local instances without an API key.
  239. ---
  240. ## Commands
  241. ### `scrape` - Scrape URLs
  242. Extract content from any webpage. Pass multiple URLs to scrape them concurrently -- each result is saved to `.firecrawl/` automatically.
  243. ```bash
  244. # Basic usage (outputs markdown)
  245. firecrawl https://example.com
  246. firecrawl scrape https://example.com
  247. # Get raw HTML
  248. firecrawl https://example.com --html
  249. firecrawl https://example.com -H
  250. # Multiple formats (outputs JSON)
  251. firecrawl https://example.com --format markdown,links,images
  252. # Save to file
  253. firecrawl https://example.com -o output.md
  254. firecrawl https://example.com --format json -o data.json --pretty
  255. # Multiple URLs (scraped concurrently, each saved to .firecrawl/)
  256. firecrawl scrape https://firecrawl.dev https://firecrawl.dev/blog https://docs.firecrawl.dev
  257. ```
  258. #### Scrape Options
  259. | Option | Description |
  260. | -------------------------- | ------------------------------------------------------ |
  261. | `-f, --format <formats>` | Output format(s), comma-separated |
  262. | `-H, --html` | Shortcut for `--format html` |
  263. | `-S, --summary` | Shortcut for `--format summary` |
  264. | `--only-main-content` | Extract only main content (the default) |
  265. | `--full-page` | Keep the whole page, including navs and footers |
  266. | `--wait-for <ms>` | Wait time before scraping (for JS-rendered content) |
  267. | `--screenshot` | Take a screenshot |
  268. | `--full-page-screenshot` | Take a full page screenshot |
  269. | `--include-tags <tags>` | Only include specific HTML tags |
  270. | `--exclude-tags <tags>` | Exclude specific HTML tags |
  271. | `--max-age <milliseconds>` | Maximum age of cached content in milliseconds |
  272. | `--lockdown` | Enable lockdown mode for the scrape |
  273. | `--redact-pii` | Redact personally identifiable information from output |
  274. | `--schema <json>` | JSON schema for structured extraction |
  275. | `--schema-file <path>` | Path to JSON schema file for structured extraction |
  276. | `--actions <json>` | JSON actions array to run during scrape |
  277. | `--actions-file <path>` | Path to JSON actions file |
  278. | `--proxy <proxy>` | Proxy mode for scraping (for example, `auto`, `basic`) |
  279. | `-o, --output <path>` | Save output to file |
  280. | `--json` | Output as JSON format |
  281. | `--pretty` | Pretty print JSON output |
  282. | `--timing` | Show request timing info |
  283. #### Available Formats
  284. | Format | Description |
  285. | ---------------- | ---------------------------- |
  286. | `markdown` | Clean markdown (default) |
  287. | `html` | Cleaned HTML |
  288. | `rawHtml` | Original HTML |
  289. | `links` | All links on the page |
  290. | `images` | All images on the page |
  291. | `screenshot` | Screenshot as base64 |
  292. | `summary` | AI-generated summary |
  293. | `json` | Structured JSON extraction |
  294. | `changeTracking` | Track changes on the page |
  295. | `attributes` | Page attributes and metadata |
  296. | `branding` | Brand identity extraction |
  297. #### Examples
  298. ```bash
  299. # Extract only main content as markdown
  300. firecrawl https://blog.example.com --only-main-content
  301. # Wait for JS to render, then scrape
  302. firecrawl https://spa-app.com --wait-for 3000
  303. # Get all links from a page
  304. firecrawl https://example.com --format links
  305. # Screenshot + markdown
  306. firecrawl https://example.com --format markdown --screenshot
  307. # Extract specific elements only
  308. firecrawl https://example.com --include-tags article,main
  309. # Exclude navigation and ads
  310. firecrawl https://example.com --exclude-tags nav,aside,.ad
  311. ```
  312. ---
  313. ### `search` - Search the web
  314. Search the web with query-relevant highlights and optionally scrape content from search results.
  315. ```bash
  316. # Basic search
  317. firecrawl search "firecrawl"
  318. # Limit results
  319. firecrawl search "AI news" --limit 10
  320. # Search news sources
  321. firecrawl search "tech startups" --sources news
  322. # Search images
  323. firecrawl search "landscape photography" --sources images
  324. # Multiple sources
  325. firecrawl search "machine learning" --sources web,news,images
  326. # Filter by category (research-affiliated websites, PDFs, developer index, gov index)
  327. firecrawl search "transformer architecture" --categories research
  328. firecrawl search "machine learning" --categories pdf,research
  329. # Note: --categories research narrows *web* results to research-affiliated
  330. # websites. To search papers themselves, use `firecrawl research search-papers`.
  331. # Developer search: public repositories, GitHub issues, merged PRs, READMEs, and docs
  332. firecrawl search "axum middleware ordering" --categories developer
  333. # Government search: US government sources (cannot be combined with other categories)
  334. firecrawl search "California data breach notification statute" --categories gov
  335. # Time-based search
  336. firecrawl search "AI announcements" --tbs qdr:d # Past day
  337. firecrawl search "tech news" --tbs qdr:w # Past week
  338. # Location-based search
  339. firecrawl search "restaurants" --location "San Francisco,California,United States"
  340. firecrawl search "local news" --country DE
  341. # Search and scrape results
  342. firecrawl search "firecrawl tutorials" --scrape
  343. firecrawl search "API documentation" --scrape --scrape-formats markdown,links
  344. # Output as pretty JSON
  345. firecrawl search "AI data tools"
  346. ```
  347. #### Search Options
  348. | Option | Description |
  349. | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  350. | `--limit <n>` | Maximum results (default: 5, max: 100) |
  351. | `--sources <sources>` | Comma-separated: `web`, `images`, `news`, `alexandria` (default: web). With an API key, `alexandria` is always added with domain matching; `--no-domain-tools` turns domain matching off |
  352. | `--categories <categories>` | Comma-separated: `research` (research-affiliated websites -- for papers use [`research search-papers`](#research---search-research-papers)), `pdf`, `developer`, `gov` (cannot be combined with other categories) |
  353. | `--tbs <value>` | Time filter: `qdr:h` (hour), `qdr:d` (day), `qdr:w` (week), `qdr:m` (month), `qdr:y` (year) |
  354. | `--location <location>` | Geo-targeting (e.g., "Germany", "San Francisco,California,United States") |
  355. | `--country <code>` | ISO country code (default: US) |
  356. | `--timeout <ms>` | Timeout in milliseconds (default: 60000) |
  357. | `--highlights` | Query-relevant highlights for web and news when available (default) |
  358. | `--no-highlights` | Keep the original search snippets |
  359. | `--ignore-invalid-urls` | Exclude URLs invalid for other Firecrawl endpoints |
  360. | `--scrape` | Enable scraping of search results |
  361. | `--scrape-formats <formats>` | Scrape formats when `--scrape` enabled (default: markdown) |
  362. | `--only-main-content` | Include only the main content when scraping (the API default) |
  363. | `--full-page` | Keep the whole page when scraping, including navs and footers |
  364. | `-o, --output <path>` | Save to file |
  365. | `--json` | Output as compact JSON |
  366. #### Examples
  367. ```bash
  368. # Research a topic with recent results
  369. firecrawl search "React Server Components" --tbs qdr:m --limit 10
  370. # Search public developer sources
  371. firecrawl search "web data library" --categories developer --limit 20
  372. # Search and get full content
  373. firecrawl search "firecrawl documentation" --scrape --scrape-formats markdown --json -o results.json
  374. # Find research papers -- use the paper index, not the website filter
  375. firecrawl research search-papers "large language models" --json
  376. # Narrow web results to research-affiliated websites (not the paper index)
  377. firecrawl search "large language models" --categories research --json
  378. # Answer a programming question from public repositories, GitHub issues, merged PRs, READMEs, and docs
  379. firecrawl search "tokio select cancellation safety" --categories developer --json
  380. # Search with location targeting
  381. firecrawl search "best coffee shops" --location "Berlin,Germany" --country DE
  382. # Get news from the past week
  383. firecrawl search "AI startups funding" --sources news --tbs qdr:w --limit 15
  384. ```
  385. ---
  386. ### `developer` - Search developer sources
  387. Search an index built for coding agents: public repositories, GitHub issues, merged pull requests, repository READMEs, and curated documentation sites. Use it for a programming question: code behaviour, a library or framework, an API contract, an error message, or a known bug.
  388. The CLI intentionally keeps this agent-facing surface lean: it accepts only the query and result count. Express repository, source, result-kind, language, topic, license, and other scoping intent in the query text; semantic retrieval handles the scoping. For advanced filters, use the [Developer Index REST API](https://docs.firecrawl.dev/features/developer).
  389. ```bash
  390. firecrawl developer "axum middleware ordering"
  391. ```
  392. #### Options
  393. | Option | Description |
  394. | --------------------- | ---------------------------------------------------------- |
  395. | `--limit <n>` | Number of results (default: 10, max: 100) |
  396. | `-o, --output <path>` | Save to file |
  397. | `--json` | Output full response, including citations and index status |
  398. | `--pretty` | Pretty print JSON output |
  399. #### Examples
  400. ```bash
  401. # Investigate a known bug
  402. firecrawl developer "tokio spawn_blocking panics thread limit" --limit 10
  403. # Put repository and evidence-kind intent directly in the semantic query
  404. firecrawl developer "tokio select cancellation safety in tokio-rs/tokio issues and merged pull requests"
  405. # Keep the full response, including passages, citations, and licenses
  406. firecrawl developer "tokio select cancellation safety" --json -o results.json
  407. ```
  408. ---
  409. ### `gov` - Search the Firecrawl Government Index
  410. Search the Government Index: primary law and regulatory material from US federal, state, and local government sources, including statutes, regulations, codes, court opinions, and other government publications.
  411. For the request and response schema, see the [Government Index REST API](https://docs.firecrawl.dev/features/gov).
  412. ```bash
  413. firecrawl gov "food labeling requirements for allergens"
  414. ```
  415. #### Options
  416. | Option | Description |
  417. | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  418. | `--limit <n>` | Number of results (default: 10, max: 100) |
  419. | `--jurisdiction <code>` | Keep only pages of one jurisdiction: `us` for federal, or `us-` plus a two-letter state or territory code, such as `us-ca` or `us-dc` |
  420. | `--court <id>` | Keep only pages labeled for one court, by CourtListener court id, such as `scotus`, `ca9`, or `nysd` |
  421. | `--court-type <type>` | Keep only pages from one type of court: `federal-supreme`, `federal-appellate`, `federal-district`, `federal-bankruptcy`, `federal-special`, `military`, `state-supreme`, `state-appellate`, `state-trial`, `state-special`, or `state-attorney-general` |
  422. | `--cites <citation>` | Keep only pages that cite one law or case, as a printed citation such as `"42 U.S.C. § 1983"` or `"5 U.S. 137"`, or a citation id such as `us:usc:42:1983` |
  423. | `--domain <domain>` | Keep only pages from one site, such as `irs.gov` |
  424. | `-o, --output <path>` | Save to file |
  425. | `--json` | Output the raw response as JSON |
  426. | `--pretty` | Pretty print JSON output |
  427. Filters combine with AND. Filters keep only pages the index has labeled, so a filtered search can leave out relevant pages, and no results does not mean none exist. Court labels are most complete for federal courts and case-law collections.
  428. #### Examples
  429. ```bash
  430. # Find state statutes on a topic
  431. firecrawl gov "California data breach notification statute" --limit 10
  432. # Narrow to one court
  433. firecrawl gov "qualified immunity excessive force" --court ca9
  434. # Keep only pages that cite a statute
  435. firecrawl gov "civil rights claims against police officers" --cites "42 U.S.C. § 1983"
  436. # Keep the raw response
  437. firecrawl gov "FDA food labeling regulations" --json -o results.json
  438. ```
  439. ---
  440. ### `research` - Search research papers
  441. Search Firecrawl's research paper index: roughly 43M abstracts, around 90% biomedical (PubMed, bioRxiv, medRxiv) plus arXiv. Use this for biomedical, clinical, and scientific literature rather than scraping PubMed, bioRxiv, or Google Scholar by hand.
  442. This is **not** the same thing as `firecrawl search --categories research`, which only narrows ordinary web results to research-affiliated websites.
  443. ```bash
  444. # Find papers by topic (start here)
  445. firecrawl research search-papers "CRISPR base editing off-target effects" --limit 20
  446. # Full metadata for one paper, by any supported id form
  447. firecrawl research inspect-paper pmid:40953549
  448. # Expand along the citation graph from your strongest hits
  449. firecrawl research related-papers pmcid:PMC12530322 --intent "in vivo delivery"
  450. # Read full-text passages to verify a specific claim
  451. firecrawl research read-paper doi:10.1016/j.neunet.2025.108095 --question "What was the sample size?"
  452. # Curated location, function and GO terms for one protein, with cited paper ids
  453. firecrawl research lookup-protein RPUSD3
  454. # A gene symbol in another organism
  455. firecrawl research lookup-protein Prkcb --organism "Mus musculus"
  456. ```
  457. Paper ids accept `pmid:`, `pmcid:`, `doi:`, and `arxiv:` forms, plus canonical `paperId` values returned by search.
  458. `lookup-protein` prints data from UniProtKB/Swiss-Prot, copyright [UniProt Consortium](https://www.uniprot.org), under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
  459. #### Subcommands
  460. | Subcommand | Description |
  461. | ----------------------------- | --------------------------------------------------------------------------- |
  462. | `search-papers <query>` | Semantic (HyDE) search over paper abstracts. The primary entry point. |
  463. | `inspect-paper <paperId>` | Canonical metadata: title, abstract, authors, categories, source ids, dates |
  464. | `related-papers <seedIds...>` | Citation-graph expansion from seed papers, ranked against `--intent` |
  465. | `read-paper <paperId>` | Best-matching in-body full-text passages for a `--question` |
  466. | `lookup-protein <protein>` | Curated UniProt location, function and GO terms for one protein |
  467. #### `search-papers` Options
  468. | Option | Description |
  469. | --------------------------- | -------------------------------------------------------------------------------------- |
  470. | `--limit <n>` | Number of results (default: 40) |
  471. | `--authors <authors>` | Comma-separated author substring filter(s); all must match |
  472. | `--categories <categories>` | arXiv-style taxonomy labels (e.g. `cs.LG,cs.IR`). Biomedical records do not use these. |
  473. | `--from <date>` | Inclusive lower bound on created/updated date (YYYY-MM-DD) |
  474. | `--to <date>` | Inclusive upper bound on created/updated date (YYYY-MM-DD) |
  475. | `--sort <order>` | `relevance` (default) or `date_desc` (newest of the most relevant matches) |
  476. | `-o, --output <path>` | Save to file |
  477. | `--json` | Output as compact JSON |
  478. | `--pretty` | Pretty print JSON output |
  479. #### Tips
  480. - Run several distinct framings of the same question rather than one query -- recall improves markedly.
  481. - Use `search-papers` to find anchors, `related-papers` to expand, then `read-paper` to verify a candidate before including it.
  482. ---
  483. ### `feedback` - Send endpoint job feedback
  484. Send concise feedback for a completed v2 `search`, `scrape`, `parse`, or `map`
  485. job. For search-result quality, `search-feedback` is still the most guided
  486. command; `feedback` is the generic endpoint/job surface.
  487. ```bash
  488. firecrawl feedback scrape 0193f6c5-1234-7890-abcd-1234567890ab \
  489. --rating partial \
  490. --issues missing_markdown \
  491. --tags docs \
  492. --note "The pricing table was missing from the markdown output." \
  493. --url https://example.com/pricing \
  494. --page-numbers 1
  495. ```
  496. Keep notes and metadata small. Do not send raw scrape or parse outputs as
  497. feedback.
  498. Set `FIRECRAWL_NO_ENDPOINT_FEEDBACK=1` to make `firecrawl feedback` skip
  499. endpoint feedback calls silently.
  500. #### Feedback Options
  501. | Option | Description |
  502. | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
  503. | `--rating <rating>` | Required: `good`, `partial`, or `bad` |
  504. | `--issues <codesOrJson>` | Comma-separated issue codes or JSON array |
  505. | `--tags <codesOrJson>` | Comma-separated tags or JSON array |
  506. | `--note <text>` | Short human-readable feedback |
  507. | `--valuable-sources <json>` | JSON array of `{url, reason}` entries |
  508. | `--valuable-results <positionsOrJson>` | Search only: every useful result as `source:position`, e.g. `web:1,news:2`, or a JSON array of `{source, position, reason?}` |
  509. | `--missing-content <json>` | JSON array of `{topic, description}` entries |
  510. | `--query-suggestions <text>` | Search/query improvement notes |
  511. | `--url <url>` | Relevant URL for scrape or parse feedback |
  512. | `--page-numbers <numbersOrJson>` | Comma-separated page numbers or JSON array |
  513. | `--metadata <json>` | Small JSON object with extra context |
  514. | `--metadata-file <path>` | Path to small metadata JSON object |
  515. | `--silent` | Suppress output for background agent calls |
  516. ---
  517. ### `map` - Discover all URLs on a website
  518. Quickly discover all URLs on a website without scraping content.
  519. ```bash
  520. # List all URLs (one per line)
  521. firecrawl map https://example.com
  522. # Output as JSON
  523. firecrawl map https://example.com --json
  524. # Search for specific URLs
  525. firecrawl map https://example.com --search "blog"
  526. # Limit results
  527. firecrawl map https://example.com --limit 500
  528. ```
  529. #### Map Options
  530. | Option | Description |
  531. | --------------------------- | --------------------------------- |
  532. | `--limit <n>` | Maximum URLs to discover |
  533. | `--search <query>` | Filter URLs by search query |
  534. | `--sitemap <mode>` | `include`, `skip`, or `only` |
  535. | `--include-subdomains` | Include subdomains |
  536. | `--ignore-query-parameters` | Dedupe URLs with different params |
  537. | `--timeout <seconds>` | Request timeout |
  538. | `--json` | Output as JSON |
  539. | `-o, --output <path>` | Save to file |
  540. #### Examples
  541. ```bash
  542. # Find all product pages
  543. firecrawl map https://shop.example.com --search "product"
  544. # Get sitemap URLs only
  545. firecrawl map https://example.com --sitemap only
  546. # Save URL list to file
  547. firecrawl map https://example.com -o urls.txt
  548. # Include subdomains
  549. firecrawl map https://example.com --include-subdomains --limit 1000
  550. ```
  551. ---
  552. ### `crawl` - Crawl an entire website
  553. Crawl multiple pages from a website.
  554. ```bash
  555. # Start a crawl (returns job ID)
  556. firecrawl crawl https://example.com
  557. # Wait for crawl to complete
  558. firecrawl crawl https://example.com --wait
  559. # With progress indicator
  560. firecrawl crawl https://example.com --wait --progress
  561. # Check crawl status
  562. firecrawl crawl <job-id>
  563. # Limit pages
  564. firecrawl crawl https://example.com --limit 100 --max-depth 3
  565. ```
  566. #### Crawl Options
  567. | Option | Description |
  568. | ------------------------------ | ------------------------------------------------------------------------------- |
  569. | `--wait` | Wait for crawl to complete |
  570. | `--progress` | Show progress while waiting |
  571. | `--limit <n>` | Maximum pages to crawl |
  572. | `--max-depth <n>` | Maximum crawl depth |
  573. | `--include-paths <paths>` | Only crawl matching paths |
  574. | `--exclude-paths <paths>` | Skip matching paths |
  575. | `--sitemap <mode>` | `include`, `skip`, or `only` |
  576. | `--allow-subdomains` | Include subdomains |
  577. | `--allow-external-links` | Follow external links |
  578. | `--crawl-entire-domain` | Crawl entire domain |
  579. | `--ignore-query-parameters` | Treat URLs with different params as same |
  580. | `--delay <ms>` | Delay between requests in milliseconds, 1 to 60000 (sent to the API in seconds) |
  581. | `--max-concurrency <n>` | Max concurrent requests |
  582. | `--scrape-options <json>` | JSON scrape options passed to each page |
  583. | `--scrape-options-file <path>` | Path to scrape options JSON file |
  584. | `--webhook <url-or-json>` | Webhook URL or configuration |
  585. | `--cancel` | Cancel an active crawl job by job ID |
  586. | `--timeout <seconds>` | Timeout when waiting |
  587. | `--poll-interval <seconds>` | Status check interval |
  588. #### Examples
  589. ```bash
  590. # Crawl blog section only
  591. firecrawl crawl https://example.com --include-paths /blog,/posts
  592. # Exclude admin pages
  593. firecrawl crawl https://example.com --exclude-paths /admin,/login
  594. # Crawl with rate limiting
  595. firecrawl crawl https://example.com --delay 1000 --max-concurrency 2
  596. # Deep crawl with high limit
  597. firecrawl crawl https://example.com --limit 1000 --max-depth 10 --wait --progress
  598. # Save results
  599. firecrawl crawl https://example.com --wait -o crawl-results.json --pretty
  600. ```
  601. ---
  602. ### `batch-scrape` - Scrape a list of URLs
  603. Scrape many known URLs as one async job. Unlike multi-URL `scrape`, every requested format is kept and results come back as JSON.
  604. ```bash
  605. # Start a batch scrape (returns job ID)
  606. firecrawl batch-scrape https://example.com https://example.org
  607. # Read URLs from a file (one per line; blank lines and # comments ignored)
  608. firecrawl batch-scrape --urls-file urls.txt
  609. # Wait for results
  610. firecrawl batch-scrape --urls-file urls.txt --format markdown,links --wait --progress
  611. # Check status (returns documents scraped so far)
  612. firecrawl batch-scrape <job-id>
  613. # Cancel a job
  614. firecrawl batch-scrape <job-id> --cancel
  615. ```
  616. #### Batch Scrape Options
  617. | Option | Description |
  618. | ------------------------------ | ----------------------------------------------- |
  619. | `--urls-file <path>` | File with one URL per line |
  620. | `-f, --format <formats>` | Output format(s), comma-separated |
  621. | `--only-main-content` | Extract only main content (the default) |
  622. | `--full-page` | Keep the whole page, including navs and footers |
  623. | `--wait-for <ms>` | Wait time before scraping each page |
  624. | `--max-age <milliseconds>` | Maximum age of cached content in milliseconds |
  625. | `--schema <json>` | JSON schema for extraction (adds `json` format) |
  626. | `--schema-file <path>` | Path to JSON schema file |
  627. | `--prompt <text>` | Extraction prompt (adds `json` format) |
  628. | `--scrape-options <json>` | JSON scrape options applied to every URL |
  629. | `--scrape-options-file <path>` | Path to scrape options JSON file |
  630. | `--wait` | Wait for the job to complete |
  631. | `--progress` | Show progress while waiting |
  632. | `--timeout <seconds>` | Timeout when waiting |
  633. | `--poll-interval <seconds>` | Status check interval (default: 2) |
  634. | `--webhook <url-or-json>` | Webhook URL or configuration |
  635. | `--max-concurrency <n>` | Max concurrent scrapes for this job |
  636. | `--ignore-invalid-urls` | Skip invalid URLs instead of failing the job |
  637. | `--status` | Check status of a job by job ID |
  638. | `--cancel` | Cancel an active job by job ID |
  639. | `-o, --output <path>` | Save output to file |
  640. | `--pretty` | Pretty print JSON output |
  641. #### Examples
  642. ```bash
  643. # Extract the same fields from every page
  644. firecrawl batch-scrape --urls-file products.txt --wait \
  645. --schema '{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"string"}}}' \
  646. -o products.json --pretty
  647. # Notify a webhook instead of waiting
  648. firecrawl batch-scrape --urls-file urls.txt --webhook https://example.com/hooks/firecrawl
  649. ```
  650. ---
  651. ### `credit-usage` - Check your credits
  652. ```bash
  653. # Show credit usage
  654. firecrawl credit-usage
  655. # Output as JSON
  656. firecrawl credit-usage --json --pretty
  657. ```
  658. ---
  659. ### `agent` - AI-powered web data extraction
  660. Run an AI agent that autonomously browses and extracts structured data from the web based on natural language prompts.
  661. > **Note:** Agent tasks typically take **2 to 5 minutes** to complete, and sometimes longer for complex extractions. Use sparingly and consider `--max-credits` to limit costs.
  662. ```bash
  663. # Basic usage (returns job ID immediately)
  664. firecrawl agent "Find the pricing plans for Firecrawl"
  665. # Wait for completion
  666. firecrawl agent "Extract all product names and prices from this store" --wait
  667. # Focus on specific URLs
  668. firecrawl agent "Get the main features listed" --urls https://example.com/features
  669. # Use structured output with JSON schema
  670. firecrawl agent "Extract company info" --schema '{"type":"object","properties":{"name":{"type":"string"},"employees":{"type":"number"}}}'
  671. # Load schema from file
  672. firecrawl agent "Extract product data" --schema-file ./product-schema.json --wait
  673. # Check status of an existing job
  674. firecrawl agent <job-id>
  675. firecrawl agent <job-id> --wait
  676. ```
  677. #### Agent Options
  678. | Option | Description |
  679. | ------------------------------ | --------------------------------------------------------------------------------------------------------- |
  680. | `--urls <urls>` | Comma-separated URLs to focus extraction on |
  681. | `--model <model>` | `spark-2` (default). `spark-1-mini` and `spark-1-pro` are deprecated and run `spark-2` |
  682. | `--effort <level>` | Reasoning effort: `low`, `medium`, or `high` |
  683. | `--schema <json>` | JSON schema for structured output (inline JSON string) |
  684. | `--schema-file <path>` | Path to JSON schema file for structured output |
  685. | `--max-credits <number>` | Maximum credits to spend (job fails if exceeded) |
  686. | `--webhook <url-or-json>` | Webhook URL or configuration |
  687. | `--thread <threadId>` | Continue an existing thread with this prompt as the next turn |
  688. | `--mode <mode>` | `extract` returns structured data; `chat` returns a text message |
  689. | `--alexandria` | Let the agent call Alexandria providers (implied by the flags below); `--no-alexandria` keeps it off them |
  690. | `--toolkits <slugs>` | Comma-separated provider slugs the agent may use (up to 5; default: the whole catalog) |
  691. | `--max-calls <n>` | Most provider calls the agent may make this turn (1-30) |
  692. | `--require-approval` | Stop for approval before any paid provider call (needs `--mode chat`) |
  693. | `--approve <approvalId>` | Approve the previous turn's pending approval (needs `--thread`) |
  694. | `--call-ids <ids>` | With `--approve`: comma-separated call IDs to approve (default: all) |
  695. | `--always` | With `--approve`: stop asking for the rest of the thread |
  696. | `--decline <approvalId>` | Decline the previous turn's pending approval (needs `--thread`) |
  697. | `--on-terms-required <action>` | When a provider needs data terms the team has not accepted: `skip` it (default) or `ask` |
  698. | `--status` | Check status of existing agent job |
  699. | `--cancel` | Cancel an active agent job by job ID |
  700. | `--wait` | Wait for agent to complete before returning results |
  701. | `--poll-interval <seconds>` | Polling interval in seconds when waiting (default: 5) |
  702. | `--timeout <seconds>` | Timeout in seconds when waiting (default: no timeout) |
  703. | `-o, --output <path>` | Save output to file |
  704. | `--json` | Output as JSON format |
  705. | `--pretty` | Pretty print JSON output |
  706. #### Examples
  707. ```bash
  708. # Research task with timeout
  709. firecrawl agent "Find the top 5 competitors of Notion and their pricing" --wait --timeout 300
  710. # Extract data with cost limit
  711. firecrawl agent "Get all blog post titles and dates" --urls https://blog.example.com --max-credits 100 --wait
  712. # Spend more reasoning on a complex extraction
  713. firecrawl agent "Extract detailed technical specifications" --effort high --wait --json --pretty
  714. # Save structured results to file
  715. firecrawl agent "Extract contact information" --schema-file ./contact-schema.json --wait --json -o contacts.json --pretty
  716. # Check job status without waiting
  717. firecrawl agent abc123-def456-... --json
  718. # Poll a running job until completion
  719. firecrawl agent abc123-def456-... --wait --poll-interval 10
  720. ```
  721. #### Alexandria providers
  722. A run uses Alexandria providers only when it starts with an Alexandria flag. A follow-up turn (`--thread`) keeps the previous turn's setting for every Alexandria flag it omits.
  723. ```bash
  724. # Let the agent use two providers
  725. firecrawl agent "Find the head of sales at example.com" --toolkits apollo,crunchbase --wait
  726. # Ask before any paid provider call, then answer the approval on the next turn
  727. firecrawl agent "Find the head of sales at example.com" --mode chat --require-approval --wait
  728. firecrawl agent "Go ahead" --thread <thread-id> --mode chat --approve <approval-id> --wait
  729. ```
  730. A run that stops for approval prints its approval ID and the exact `--approve` / `--decline` follow-up commands. With `--json`, read `pendingApproval` and `exchange` from the output.
  731. ---
  732. ### `interact` - Interact with scraped pages
  733. Scrape a page, then interact with it in a live browser session using natural language or code. No manual session management required.
  734. ```bash
  735. # 1. Scrape a page first
  736. firecrawl scrape https://example.com
  737. # 2. Interact with it
  738. firecrawl interact "Click the pricing tab"
  739. firecrawl interact "Fill in the email field with test@example.com"
  740. firecrawl interact "Extract the pricing table"
  741. # 3. Code execution (Playwright)
  742. firecrawl interact -c "await page.title()"
  743. firecrawl interact -c "print(await page.title())" --python
  744. # 4. Stop the session
  745. firecrawl interact stop
  746. ```
  747. #### Interact Options
  748. | Option | Description |
  749. | ---------------------- | ---------------------------------------------- |
  750. | `-p, --prompt <text>` | AI prompt (alternative to positional argument) |
  751. | `-c, --code <code>` | Code to execute in the browser sandbox |
  752. | `-s, --scrape-id <id>` | Scrape job ID (default: last scrape) |
  753. | `--python` | Execute code as Python/Playwright |
  754. | `--node` | Execute code as Node.js/Playwright (default) |
  755. | `--bash` | Execute code as Bash |
  756. | `--timeout <seconds>` | Timeout in seconds (1-300, default: 30) |
  757. | `-o, --output <path>` | Save output to file |
  758. | `--json` | Output as JSON format |
  759. #### Profiles
  760. Use `--profile` on the scrape to persist browser state across scrapes:
  761. ```bash
  762. # Login and save state
  763. firecrawl scrape "https://app.example.com/login" --profile my-app
  764. firecrawl interact "Fill in email and click login"
  765. # Come back authenticated later
  766. firecrawl scrape "https://app.example.com/dashboard" --profile my-app
  767. firecrawl interact "Extract the dashboard data"
  768. ```
  769. ---
  770. ### `monitor` - Watch pages for changes
  771. Create monitors when the goal is ongoing change detection, alerting, or repeated checks over time. A monitor runs recurring scrapes or crawls, diffs each result against the last retained snapshot, and can notify webhooks or email recipients.
  772. ```bash
  773. firecrawl monitor create --name "Blog" \
  774. --goal "Notify me when a new post is published" \
  775. --schedule "every 30 minutes" \
  776. --page https://example.com/blog \
  777. --email alerts@example.com
  778. firecrawl monitor create --name "Product pages" \
  779. --goal "Notify me when pricing, docs, or changelog content changes" \
  780. --schedule "every 30 minutes" \
  781. --scrape-urls https://example.com/pricing,https://example.com/docs,https://example.com/changelog
  782. firecrawl monitor create --name "Docs webhook" \
  783. --goal "Notify me when docs content changes" \
  784. --schedule "every 30 minutes" \
  785. --page https://example.com/docs \
  786. --webhook-url https://example.com/webhook \
  787. --webhook-events monitor.page,monitor.check.completed
  788. firecrawl monitor list --limit 20
  789. firecrawl monitor run <monitorId>
  790. firecrawl monitor checks <monitorId>
  791. firecrawl monitor check <monitorId> <checkId> --page-status changed
  792. firecrawl monitor update <monitorId> --state paused
  793. firecrawl monitor delete <monitorId>
  794. ```
  795. #### Monitor Options
  796. | Option | Description |
  797. | ------------------------- | --------------------------------------------------- |
  798. | `--name <name>` | Monitor name |
  799. | `--goal <goal>` | What changes this monitor should look for |
  800. | `--cron <expression>` | Cron schedule (e.g. `*/30 * * * *`) |
  801. | `--schedule <text>` | Natural-language schedule (e.g. `every 30 minutes`) |
  802. | `--timezone <tz>` | Schedule timezone (default: `UTC`) |
  803. | `--page <url>` | Single page URL to scrape on each check |
  804. | `--scrape-urls <list>` | Comma-separated page URLs to scrape on each check |
  805. | `--crawl-url <url>` | Root URL for a crawl target |
  806. | `--webhook-url <url>` | Webhook destination |
  807. | `--webhook-events <list>` | Comma-separated webhook events |
  808. | `--email <list>` | Comma-separated email recipients |
  809. | `--retention-days <n>` | Snapshot retention window |
  810. For advanced targets such as JSON-mode `changeTracking`, pass a JSON payload:
  811. ```bash
  812. firecrawl monitor create monitor.json
  813. cat monitor.json | firecrawl monitor create
  814. ```
  815. ---
  816. ### `config` - Configure settings
  817. ```bash
  818. # Configure with custom API URL
  819. firecrawl config --api-url https://firecrawl.mycompany.com
  820. firecrawl config --api-url http://localhost:3002 --api-key fc-xxx
  821. ```
  822. ### `view-config` - View current configuration
  823. ```bash
  824. # View current configuration and authentication status
  825. firecrawl view-config
  826. ```
  827. Shows authentication status and stored credentials location.
  828. ---
  829. ### `login` / `logout`
  830. ```bash
  831. # Login
  832. firecrawl login
  833. firecrawl login --method browser
  834. firecrawl login --method manual
  835. firecrawl login --api-key fc-xxx
  836. # Login to self-hosted instance
  837. firecrawl login --api-url https://firecrawl.mycompany.com
  838. firecrawl login --api-url http://localhost:3002 --api-key fc-xxx
  839. # Logout
  840. firecrawl logout
  841. ```
  842. ---
  843. ## Global Options
  844. These options work with any command:
  845. | Option | Description |
  846. | --------------------- | ------------------------------------------------------ |
  847. | `--status` | Show version, auth, concurrency, and credits |
  848. | `-k, --api-key <key>` | Use specific API key |
  849. | `--api-url <url>` | Use custom API URL (for self-hosted/local development) |
  850. | `-V, --version` | Show version |
  851. | `-h, --help` | Show help |
  852. ### Check Status
  853. ```bash
  854. firecrawl --status
  855. ```
  856. ```
  857. 🔥 firecrawl cli v1.20.0
  858. ● Authenticated via stored credentials
  859. Concurrency: 0/100 jobs (parallel scrape limit)
  860. Credits: 500,000 / 1,000,000 (50% left this cycle)
  861. ```
  862. ---
  863. ## Output Handling
  864. ### Stdout vs File
  865. ```bash
  866. # Output to stdout (default)
  867. firecrawl https://example.com
  868. # Pipe to another command
  869. firecrawl https://example.com | head -50
  870. # Save to file
  871. firecrawl https://example.com -o output.md
  872. # JSON output
  873. firecrawl https://example.com --format links --pretty
  874. ```
  875. ### Format Behavior
  876. - **Single format**: Outputs raw content (markdown text, HTML, etc.)
  877. - **Multiple formats**: Outputs JSON with all requested data
  878. ```bash
  879. # Raw markdown output
  880. firecrawl https://example.com --format markdown
  881. # JSON output with multiple formats
  882. firecrawl https://example.com --format markdown,links,images
  883. ```
  884. ---
  885. ## Tips & Tricks
  886. ### Scrape multiple URLs
  887. ```bash
  888. # Just pass multiple URLs -- results are saved to .firecrawl/
  889. firecrawl scrape https://firecrawl.dev https://firecrawl.dev/blog https://docs.firecrawl.dev
  890. # Many URLs, other formats, or JSON extraction: run one batch job
  891. firecrawl batch-scrape --urls-file urls.txt --format markdown,links --wait -o results.json
  892. ```
  893. ### Combine with other tools
  894. ```bash
  895. # Extract links and process with jq
  896. firecrawl https://example.com --format links | jq '.links[].url'
  897. # Convert to PDF (with pandoc)
  898. firecrawl https://example.com | pandoc -o document.pdf
  899. # Search within scraped content
  900. firecrawl https://example.com | grep -i "keyword"
  901. ```
  902. ### CI/CD Usage
  903. ```bash
  904. # Set API key via environment
  905. export FIRECRAWL_API_KEY=${{ secrets.FIRECRAWL_API_KEY }}
  906. firecrawl crawl https://docs.example.com --wait -o docs.json
  907. # Use self-hosted instance
  908. export FIRECRAWL_API_URL=${{ secrets.FIRECRAWL_API_URL }}
  909. firecrawl scrape https://example.com -o output.md
  910. ```
  911. ---
  912. ## Telemetry
  913. The CLI collects anonymous usage data during authentication to help improve the product:
  914. - CLI version, OS, and Node.js version
  915. - Detect development tools (e.g., Cursor, VS Code, Claude Code)
  916. **No command data, URLs, or file contents are collected via the CLI.**
  917. To disable telemetry, set the environment variable:
  918. ```bash
  919. export FIRECRAWL_NO_TELEMETRY=1
  920. ```
  921. Requests from the CLI are tagged `integration: cli`. A tool that wraps the CLI can tag them as its own by setting `FIRECRAWL_INTEGRATION`: the CLI sends the value, trimmed, as the `X-Integration` header on every API request (a value with control or non-ASCII characters cannot be a header and is not sent), and the API attributes the request to it when it is one of its integration names or any value starting with an underscore. Any other value is ignored by the API and never makes a request fail.
  922. ```bash
  923. export FIRECRAWL_INTEGRATION=_my-tool
  924. ```
  925. Supported API requests carry an origin of `cli@<version>`. When a coding agent runs the CLI, the origin also names the agent, for example `cli.codex@<version>`; it is detected from environment variables the agent sets in the commands it runs. Calls routed through the SDK include the agent only after the pinned SDK supports the origin option. To send plain `cli@<version>` instead:
  926. ```bash
  927. export FIRECRAWL_CLI_DISABLE_AGENT_ORIGIN=1
  928. ```
  929. ---
  930. ## Experimental
  931. Experimental commands live under `firecrawl experimental` (alias: `firecrawl x`).
  932. ### `download` - Bulk Site Download
  933. Combines `map` + `scrape` to save a site as local files under `.firecrawl/`.
  934. ```bash
  935. firecrawl x download https://docs.firecrawl.dev
  936. firecrawl x download https://docs.firecrawl.dev --screenshot --limit 20 -y
  937. firecrawl x download https://docs.firecrawl.dev --include-paths "/features,/sdks" -y
  938. ```
  939. ### Workflow Skills
  940. The old experimental AI workflow commands have moved to the
  941. [`firecrawl/skills`](https://github.com/firecrawl/skills) catalog
  942. (`skills/workflows/`). Workflow skills infer from the user's request first and only ask
  943. short clarifying questions when required inputs are missing. Install them with:
  944. ```bash
  945. firecrawl setup workflows
  946. ```
  947. ---
  948. ## Documentation
  949. For more details, visit the [Firecrawl Documentation](https://docs.firecrawl.dev).
  950. ### Alexandria tool shorthand
  951. Use a provider/capability in place of a URL. The explicit `--alexandria` form remains supported:
  952. ```bash
  953. firecrawl scrape benzinga/news/search --options '{"pageSize":10}'
  954. firecrawl scrape firecrawl-research-index/read --options '{"paperId":"123","query":"methodology","k":4}'
  955. firecrawl scrape --alexandria benzinga/news/search --options '{"pageSize":10}'
  956. firecrawl scrape https://example.com
  957. ```
  958. URLs (including domains, IP addresses and localhost) continue to scrape websites. Tool addresses go directly to Alexandria, which validates the provider and capability; they never fall back to URL scraping. There is no extra catalog lookup. Bare names such as `firecrawl scrape amazon` fail locally with a suggested website URL and directions to `firecrawl list`. Suggestions are not verified or executed. Mixing URLs and tools in one command is rejected.
  959. ### Alexandria provider terms
  960. When a provider returns `THIRD_PARTY_DATA_TERMS_REQUIRED`, an organization admin must
  961. accept that provider's terms in the Firecrawl dashboard, at the link the error returns
  962. (`requiresAction.url`) or https://www.firecrawl.dev/app/settings?tab=data-sources.
  963. The CLI does not accept terms. To read the current provider agreement and metadata first:
  964. ```bash
  965. npx firecrawl-cli@latest alexandria terms show benzinga --pretty
  966. ```
  967. Agents must give the user the dashboard link and must not retry the provider until the
  968. terms are accepted. Reading terms does not accept them, and a failed lookup is never
  969. consent. After an admin accepts them, rerun the original provider command; its normal
  970. credits apply.
  971. ### Alexandria receipts and retries
  972. Alexandria execution JSON includes an additive `receipt`: `creditsUsed` is actual reported usage (missing means unknown), `requestId` is the client idempotency identity, and `operationId`/`operationType` identify the server scrape. Existing response fields remain available. IDs, reported credits, and available retry delays print to stderr.
  973. Use the same request ID to recover pending or uncertain execution. Completed results, including failures, replay under the same ID; a deliberate new execution needs a new ID and may charge again. Never automatically rotate an uncertain ID. Structured failures preserve available status, code, action and retry metadata.
  974. ### Alexandria session feedback
  975. Report the outcome of a session, missing provider coverage, or capability issues:
  976. ```bash
  977. firecrawl alexandria feedback --rating partial \
  978. --url https://example.com \
  979. --requested-functionality "Find records and download their attachments" \
  980. --objective "Compare contract requirements across agencies before bidding" \
  981. --rationale "Found summaries but could not retrieve attachments" --json
  982. ```
  983. The optional `--objective` is the underlying goal of the session: what you or your user were ultimately trying to accomplish, beyond the single website. No job ID is required. It must be sent within 20 minutes of the team's most recent Alexandria search, discovery, or execution; later submissions are rejected with `FEEDBACK_WINDOW_EXPIRED`. Each submission refunds 1 credit, up to 10 credits per website and 100 credits per team each UTC day; past either cap, submissions are still recorded and return `websiteCapReached` or `dailyCapReached`. Optional `--provider-feedback` and `--capability-feedback` accept JSON arrays; see `firecrawl alexandria feedback --help` for their fields and issue codes. Capability issue codes are `new_capability_request` (requires `requestedFunctionality`), `missing_capability` (the provider exists but lacks this capability), `insufficient_functionality`, `incorrect_result`, `execution_error`, and `other`. Each eligible provider call prints a `Feedback ref` (for example `c3`) so ratings are attributed to the calls they are about. The CLI keeps up to 20 call tokens for 24 hours in its config directory and sends them with the session rating, then clears them after a successful submission. Rate individual calls with a repeatable `--call-rating c3=good`, or pass tokens yourself with a repeatable `--token <token>` (these replace the stored ones). Existing `feedback` and `search-feedback` commands retain their job-specific behavior. Endpoint feedback opt-out environment variables also apply to this command.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.