Skip to content
Back to search

firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. ๐Ÿ”ฅ

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/firecrawl/firecrawl with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/firecrawl/firecrawl'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/firecrawl--firecrawl/badge.svg)](/listings/firecrawl--firecrawl)

HTML

<a href="/listings/firecrawl--firecrawl"><img src="/api/listings/firecrawl--firecrawl/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Sign up at [firecrawl.dev](https://firecrawl.dev) to get your API key. Try the [playground](https://firecrawl.dev/playground) to test it out.

Search

Search the web and get full content from results.

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

search_result = app.search("firecrawl", limit=5)

<details> <summary><b>Node.js / cURL / CLI</b></summary>

**Node.js**

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});

app.search("firecrawl", { limit: 5 })

**cURL**

curl -X POST 'https://api.firecrawl.dev/v2/search' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
  "query": "firecrawl",
  "limit": 5
}'

**CLI**

firecrawl search "firecrawl" --limit 5

</details>

Output:

[
  {
    "url": "https://firecrawl.dev",
    "title": "Firecrawl",
    "markdown": "Turn websites into..."
  },
  {
    "url": "https://docs.firecrawl.dev",
    "title": "Firecrawl Docs",
    "markdown": "# Getting Started..."
  }
]

Scrape

Get LLM-ready data from any website โ€” markdown, JSON, screenshots, and more.

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

result = app.scrape('firecrawl.dev')

<details> <summary><b>Node.js / cURL / CLI</b></summary>

**Node.js**

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" });

app.scrape('firecrawl.dev')

**cURL**

curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
  "url": "firecrawl.dev"
}'

**CLI**

firecrawl scrape https://firecrawl.dev
firecrawl https://firecrawl.dev --only-main-content

</details>

Output:

# Firecrawl

Firecrawl helps AI agents search, scrape, and interact with the web.

## Features
- Search: Find information across the web
- Scrape: Clean data from any page
- Interact: Click, navigate, and operate pages
- Agent: Autonomous data gathering

Interact

Scrape a page, then interact with it using AI prompts or code.

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

result = app.scrape("https://amazon.com")
scrape_id = result.metadata.scrape_id

app.interact(scrape_id, prompt="Search for 'mechanical keyboard'")
app.interact(scrape_id, prompt="Click the first result")

<details> <summary><b>Node.js / cURL / CLI</b></summary>

**Node.js**

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});

const result = await app.scrape("https://amazon.com");

await app.interact(result.metadata.scrapeId, {
  prompt: "Search for 'mechanical keyboard'"
});
await app.interact(result.metadata.scrapeId, {
  prompt: "Click the first result"
});

**cURL**

# 1. Scrape the page
curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"url": "https://amazon.com"}'

# 2. Interact with the page (use scrapeId from step 1)
curl -X POST 'https://api.firecrawl.dev/v2/scrape/SCRAPE_ID/interact' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"prompt": "Search for mechanical keyboard"}'

**CLI**

firecrawl scrape https://amazon.com
firecrawl interact exec --prompt "Search for 'mechanical keyboard'"
firecrawl interact exec --prompt "Click the first result"

</details>

Output:

{
  "success": true,
  "output": "Keyboard available at $100",
  "liveViewUrl": "https://liveview.firecrawl.dev/..."
}

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md ยท README ยท 933 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <h3 align="center">
  2. <a name="readme-top"></a>
  3. <img
  4. src="https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/firecrawl_logo.png"
  5. height="200"
  6. >
  7. </h3>
  8. <div align="center">
  9. <a href="https://github.com/firecrawl/firecrawl/blob/main/LICENSE">
  10. <img src="https://img.shields.io/github/license/firecrawl/firecrawl" alt="License">
  11. </a>
  12. <a href="https://pepy.tech/project/firecrawl-py">
  13. <img src="https://static.pepy.tech/badge/firecrawl-py" alt="Downloads">
  14. </a>
  15. <a href="https://GitHub.com/firecrawl/firecrawl/graphs/contributors">
  16. <img src="https://img.shields.io/github/contributors/firecrawl/firecrawl.svg" alt="GitHub Contributors">
  17. </a>
  18. <a href="https://firecrawl.dev">
  19. <img src="https://img.shields.io/badge/Visit-firecrawl.dev-orange" alt="Visit firecrawl.dev">
  20. </a>
  21. </div>
  22. <div>
  23. <p align="center">
  24. <a href="https://twitter.com/firecrawl">
  25. <img src="https://img.shields.io/badge/Follow%20on%20X-000000?style=for-the-badge&logo=x&logoColor=white" alt="Follow on X" />
  26. </a>
  27. <a href="https://www.linkedin.com/company/104100957">
  28. <img src="https://img.shields.io/badge/Follow%20on%20LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="Follow on LinkedIn" />
  29. </a>
  30. <a href="https://discord.gg/firecrawl">
  31. <img src="https://img.shields.io/badge/Join%20our%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join our Discord" />
  32. </a>
  33. </p>
  34. </div>
  35. ---
  36. # **๐Ÿ”ฅ Firecrawl**
  37. **Knowledge infrastructure to make your agents smarter. ๐Ÿ”ฅ** Search, scrape, and interact with the web at scale, and reach 550+ data providers through [Firecrawl Alexandria](https://alexandria.firecrawl.dev). Open source and available as a [hosted service](https://firecrawl.dev/?ref=github).
  38. _Pst. Hey, you, join our stargazers :)_
  39. <a href="https://github.com/firecrawl/firecrawl">
  40. <img src="https://img.shields.io/github/stars/firecrawl/firecrawl.svg?style=social&label=Star&maxAge=2592000" alt="GitHub stars">
  41. </a>
  42. ---
  43. ## Why Firecrawl?
  44. - **Industry-leading reliability**: Covers 96% of the web, including JS-heavy pages โ€” no proxy headaches, just clean data ([see benchmarks](https://www.firecrawl.dev/blog/the-worlds-best-web-data-api-v25))
  45. - **Blazingly fast**: P95 latency of 3.4s across millions of pages, built for real-time agents and dynamic apps
  46. - **LLM-ready output**: Clean markdown, structured JSON, screenshots, and more โ€” spend fewer tokens, build better AI apps
  47. - **We handle the hard stuff**: Rotating proxies, orchestration, rate limits, JS-blocked content, and more โ€” zero configuration
  48. - **Agent ready**: Connect Firecrawl to any AI agent or MCP client with a single command
  49. - **Document parsing**: Turn PDFs, DOCX, and other documents into clean Markdown or structured JSON
  50. - **Actions**: Click, scroll, write, wait, and press before extracting content
  51. - **Open source**: Developed transparently and collaboratively โ€” [join our community](https://discord.gg/firecrawl)
  52. ---
  53. ## Feature Overview
  54. **Core Endpoints**
  55. | Feature | Description |
  56. |---------|-------------|
  57. | [**Search**](#search) | Search the web and get full page content from results |
  58. | [**Scrape**](#scrape) | Convert any URL to markdown, HTML, screenshots, or structured JSON |
  59. | [**Interact**](#interact) | Scrape a page, then interact with it using AI prompts or code |
  60. **More**
  61. | Feature | Description |
  62. |---------|-------------|
  63. | [**Parse**](https://docs.firecrawl.dev/features/parse) | Turn PDFs, DOCX, and other documents into clean Markdown or structured JSON |
  64. | [**Agent**](#agent) | Automated data gathering, just describe what you need |
  65. | [**Crawl**](#crawl) | Scrape all URLs of a website with a single request |
  66. | [**Map**](#map) | Discover all URLs on a website instantly |
  67. | [**Batch Scrape**](#batch-scrape) | Scrape thousands of URLs asynchronously |
  68. | [**Monitor**](https://docs.firecrawl.dev/features/monitoring) | Get notified when content on a website changes |
  69. | [**Firecrawl Alexandria**](#alexandria) | Give your agent 550+ data providers and specialized indexes (via Firecrawl Cloud) |
  70. ---
  71. ## Quick Start
  72. Sign up at [firecrawl.dev](https://firecrawl.dev) to get your API key. Try the [playground](https://firecrawl.dev/playground) to test it out.
  73. ### Search
  74. Search the web and get full content from results.
  75. ```python
  76. from firecrawl import Firecrawl
  77. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  78. search_result = app.search("firecrawl", limit=5)
  79. ```
  80. <details>
  81. <summary><b>Node.js / cURL / CLI</b></summary>
  82. **Node.js**
  83. ```javascript
  84. import { Firecrawl } from 'firecrawl';
  85. const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});
  86. app.search("firecrawl", { limit: 5 })
  87. ```
  88. **cURL**
  89. ```bash
  90. curl -X POST 'https://api.firecrawl.dev/v2/search' \
  91. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  92. -H 'Content-Type: application/json' \
  93. -d '{
  94. "query": "firecrawl",
  95. "limit": 5
  96. }'
  97. ```
  98. **CLI**
  99. ```bash
  100. firecrawl search "firecrawl" --limit 5
  101. ```
  102. </details>
  103. Output:
  104. ```json
  105. [
  106. {
  107. "url": "https://firecrawl.dev",
  108. "title": "Firecrawl",
  109. "markdown": "Turn websites into..."
  110. },
  111. {
  112. "url": "https://docs.firecrawl.dev",
  113. "title": "Firecrawl Docs",
  114. "markdown": "# Getting Started..."
  115. }
  116. ]
  117. ```
  118. ### Scrape
  119. Get LLM-ready context from any website โ€” markdown, JSON, screenshots, and more.
  120. ```python
  121. from firecrawl import Firecrawl
  122. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  123. result = app.scrape('firecrawl.dev')
  124. ```
  125. <details>
  126. <summary><b>Node.js / cURL / CLI</b></summary>
  127. **Node.js**
  128. ```javascript
  129. import { Firecrawl } from 'firecrawl';
  130. const app = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" });
  131. app.scrape('firecrawl.dev')
  132. ```
  133. **cURL**
  134. ```bash
  135. curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
  136. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  137. -H 'Content-Type: application/json' \
  138. -d '{
  139. "url": "firecrawl.dev"
  140. }'
  141. ```
  142. **CLI**
  143. ```bash
  144. firecrawl scrape https://firecrawl.dev
  145. firecrawl https://firecrawl.dev --only-main-content
  146. ```
  147. </details>
  148. Output:
  149. ```
  150. # Firecrawl
  151. Firecrawl helps AI agents search, scrape, and interact with the web.
  152. ## Features
  153. - Search: Find information across the web
  154. - Scrape: Clean data from any page
  155. - Interact: Click, navigate, and operate pages
  156. - Agent: Autonomous data gathering
  157. ```
  158. ### Interact
  159. Scrape a page, then interact with it using AI prompts or code.
  160. ```python
  161. from firecrawl import Firecrawl
  162. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  163. result = app.scrape("https://amazon.com")
  164. scrape_id = result.metadata.scrape_id
  165. app.interact(scrape_id, prompt="Search for 'mechanical keyboard'")
  166. app.interact(scrape_id, prompt="Click the first result")
  167. ```
  168. <details>
  169. <summary><b>Node.js / cURL / CLI</b></summary>
  170. **Node.js**
  171. ```javascript
  172. import { Firecrawl } from 'firecrawl';
  173. const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});
  174. const result = await app.scrape("https://amazon.com");
  175. await app.interact(result.metadata.scrapeId, {
  176. prompt: "Search for 'mechanical keyboard'"
  177. });
  178. await app.interact(result.metadata.scrapeId, {
  179. prompt: "Click the first result"
  180. });
  181. ```
  182. **cURL**
  183. ```bash
  184. # 1. Scrape the page
  185. curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
  186. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  187. -H 'Content-Type: application/json' \
  188. -d '{"url": "https://amazon.com"}'
  189. # 2. Interact with the page (use scrapeId from step 1)
  190. curl -X POST 'https://api.firecrawl.dev/v2/scrape/SCRAPE_ID/interact' \
  191. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  192. -H 'Content-Type: application/json' \
  193. -d '{"prompt": "Search for mechanical keyboard"}'
  194. ```
  195. **CLI**
  196. ```bash
  197. firecrawl scrape https://amazon.com
  198. firecrawl interact exec --prompt "Search for 'mechanical keyboard'"
  199. firecrawl interact exec --prompt "Click the first result"
  200. ```
  201. </details>
  202. Output:
  203. ```json
  204. {
  205. "success": true,
  206. "output": "Keyboard available at $100",
  207. "liveViewUrl": "https://liveview.firecrawl.dev/..."
  208. }
  209. ```
  210. ---
  211. ## Power Your Agent
  212. Connect Firecrawl to any AI agent or MCP client in minutes.
  213. ### Skill
  214. Give your agent easy access to real-time web data with one command.
  215. ```bash
  216. npx -y firecrawl-cli@latest init --all --browser
  217. ```
  218. Restart your agent after installing. Works with [Claude Code](https://claude.ai/code), [Antigravity](https://antigravity.google), [OpenCode](https://opencode.ai), and more.
  219. ### MCP
  220. Connect any MCP-compatible client to the web in seconds.
  221. ```json
  222. {
  223. "mcpServers": {
  224. "firecrawl-mcp": {
  225. "command": "npx",
  226. "args": ["-y", "firecrawl-mcp"],
  227. "env": {
  228. "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY"
  229. }
  230. }
  231. }
  232. }
  233. ```
  234. ### Agent Onboarding
  235. Are you an AI agent? Fetch this skill to sign up your user, get an API key, and start building with Firecrawl.
  236. ```bash
  237. curl -s https://firecrawl.dev/agent-onboarding/SKILL.md
  238. ```
  239. See the [Skill + CLI documentation](https://docs.firecrawl.dev/sdks/cli) for all available commands. For MCP, see [firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server).
  240. ---
  241. ## Alexandria
  242. [Firecrawl Alexandria](https://alexandria.firecrawl.dev) is knowledge infrastructure to make your agents smarter: 550+ data providers in one library (250M+ people, 75M+ companies, 41M+ research papers), plus specialized indexes and the live web, through one API, MCP, or CLI. Agents discover a tool alongside their web results, inspect its inputs and pricing, then run it to get structured data. Discovery is free; execution uses the credits listed on the tool.
  243. Install it in your agent from [alexandria.firecrawl.dev](https://alexandria.firecrawl.dev), or see the [Alexandria documentation](https://docs.firecrawl.dev/features/alexandria) for CLI, MCP, SDK, and API usage.
  244. > **Note:** Alexandria runs on [Firecrawl Cloud](https://firecrawl.dev). [Firecrawl Local](apps/local) can use it once you sign in with `firecrawl-local login`. The self-hosted server does not include it.
  245. ---
  246. ## More Endpoints
  247. ### Agent
  248. **The easiest way to get data from the web.** Describe what you need, and our AI agent searches, navigates, and retrieves it. No URLs required.
  249. Agent is the evolution of our `/extract` endpoint: faster, more reliable, and doesn't require you to know the URLs upfront.
  250. ```bash
  251. curl -X POST 'https://api.firecrawl.dev/v2/agent' \
  252. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  253. -H 'Content-Type: application/json' \
  254. -d '{
  255. "prompt": "Find the pricing plans for Notion"
  256. }'
  257. ```
  258. Response:
  259. ```json
  260. {
  261. "success": true,
  262. "data": {
  263. "result": "Notion offers the following pricing plans:\n\n1. Free - $0/month...\n2. Plus - $10/seat/month...\n3. Business - $18/seat/month...",
  264. "sources": ["https://www.notion.so/pricing"]
  265. }
  266. }
  267. ```
  268. #### Agent with Structured Output
  269. Use a schema to get structured data:
  270. ```python
  271. from firecrawl import Firecrawl
  272. from pydantic import BaseModel, Field
  273. from typing import List, Optional
  274. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  275. class Founder(BaseModel):
  276. name: str = Field(description="Full name of the founder")
  277. role: Optional[str] = Field(None, description="Role or position")
  278. class FoundersSchema(BaseModel):
  279. founders: List[Founder] = Field(description="List of founders")
  280. result = app.agent(
  281. prompt="Find the founders of Firecrawl",
  282. schema=FoundersSchema
  283. )
  284. print(result.data)
  285. ```
  286. ```json
  287. {
  288. "founders": [
  289. {"name": "Eric Ciarla", "role": "Co-founder"},
  290. {"name": "Nicolas Camara", "role": "Co-founder"},
  291. {"name": "Caleb Peffer", "role": "Co-founder"}
  292. ]
  293. }
  294. ```
  295. #### Agent with URLs (Optional)
  296. Focus the agent on specific pages:
  297. ```python
  298. result = app.agent(
  299. urls=["https://docs.firecrawl.dev", "https://firecrawl.dev/pricing"],
  300. prompt="Compare the features and pricing information"
  301. )
  302. ```
  303. #### Effort Selection
  304. Set how much reasoning the agent spends on the task:
  305. | Effort | Best For |
  306. |--------|----------|
  307. | `low` | Simple lookups on one site |
  308. | `medium` | Multi-step tasks on a few pages |
  309. | `high` | Deep research, complex navigation, critical data |
  310. ```python
  311. result = app.agent(
  312. prompt="Compare enterprise features across Firecrawl, Apify, and ScrapingBee",
  313. effort="high"
  314. )
  315. ```
  316. Every effort level runs the `spark-2` model. Effort changes the reasoning
  317. budget, not the model.
  318. #### Model
  319. Every agent run executes on `spark-2`, the default, so you don't need to set
  320. `model`. The retired `spark-1-pro` and `spark-1-mini` names are still accepted
  321. for backwards compatibility, but they are deprecated and run `spark-2`.
  322. Learn more about Spark models in our [Agent documentation](https://docs.firecrawl.dev/features/agent).
  323. ### Crawl
  324. Crawl an entire website and get content from all pages.
  325. ```bash
  326. curl -X POST 'https://api.firecrawl.dev/v2/crawl' \
  327. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  328. -H 'Content-Type: application/json' \
  329. -d '{
  330. "url": "https://docs.firecrawl.dev",
  331. "limit": 100,
  332. "scrapeOptions": {
  333. "formats": ["markdown"]
  334. }
  335. }'
  336. ```
  337. Returns a job ID:
  338. ```json
  339. {
  340. "success": true,
  341. "id": "123-456-789",
  342. "url": "https://api.firecrawl.dev/v2/crawl/123-456-789"
  343. }
  344. ```
  345. #### Check Crawl Status
  346. ```bash
  347. curl -X GET 'https://api.firecrawl.dev/v2/crawl/123-456-789' \
  348. -H 'Authorization: Bearer fc-YOUR_API_KEY'
  349. ```
  350. ```json
  351. {
  352. "status": "completed",
  353. "total": 50,
  354. "completed": 50,
  355. "creditsUsed": 50,
  356. "data": [
  357. {
  358. "markdown": "# Page Title\n\nContent...",
  359. "metadata": {"title": "Page Title", "sourceURL": "https://..."}
  360. }
  361. ]
  362. }
  363. ```
  364. **Note:** The [SDKs](#sdks) handle polling automatically for a better developer experience.
  365. ### Map
  366. Discover all URLs on a website instantly.
  367. ```bash
  368. curl -X POST 'https://api.firecrawl.dev/v2/map' \
  369. -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  370. -H 'Content-Type: application/json' \
  371. -d '{"url": "https://firecrawl.dev"}'
  372. ```
  373. Response:
  374. ```json
  375. {
  376. "success": true,
  377. "links": [
  378. {"url": "https://firecrawl.dev", "title": "Firecrawl", "description": "Knowledge infrastructure to make your agents smarter"},
  379. {"url": "https://firecrawl.dev/pricing", "title": "Pricing", "description": "Firecrawl pricing plans"},
  380. {"url": "https://firecrawl.dev/blog", "title": "Blog", "description": "Firecrawl blog"}
  381. ]
  382. }
  383. ```
  384. #### Map with Search
  385. Find specific URLs within a site:
  386. ```python
  387. from firecrawl import Firecrawl
  388. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  389. result = app.map("https://firecrawl.dev", search="pricing")
  390. # Returns URLs ordered by relevance to "pricing"
  391. ```
  392. ### Batch Scrape
  393. Scrape multiple URLs at once:
  394. ```python
  395. from firecrawl import Firecrawl
  396. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  397. job = app.batch_scrape([
  398. "https://firecrawl.dev",
  399. "https://docs.firecrawl.dev",
  400. "https://firecrawl.dev/pricing"
  401. ], formats=["markdown"])
  402. for doc in job.data:
  403. print(doc.metadata.source_url)
  404. ```
  405. ---
  406. ## SDKs
  407. Our SDKs provide a convenient way to use all Firecrawl features and automatically handle polling for async operations.
  408. ### Python
  409. Install the SDK:
  410. ```bash
  411. pip install firecrawl
  412. ```
  413. ```python
  414. from firecrawl import Firecrawl
  415. app = Firecrawl(api_key="fc-YOUR_API_KEY")
  416. # Scrape a single URL
  417. doc = app.scrape("https://firecrawl.dev", formats=["markdown"])
  418. print(doc.markdown)
  419. # Use the Agent for autonomous data gathering
  420. result = app.agent(prompt="Find the founders of Stripe")
  421. print(result.data)
  422. # Crawl a website (automatically waits for completion)
  423. docs = app.crawl("https://docs.firecrawl.dev", limit=50)
  424. for doc in docs.data:
  425. print(doc.metadata.source_url, doc.markdown[:100])
  426. # Search the web
  427. results = app.search("best AI data tools 2024", limit=10)
  428. print(results)
  429. ```
  430. ### Node.js
  431. Install the SDK:
  432. ```bash
  433. npm install firecrawl
  434. ```
  435. ```javascript
  436. import { Firecrawl } from 'firecrawl';
  437. const app = new Firecrawl({ apiKey: 'fc-YOUR_API_KEY' });
  438. // Scrape a single URL
  439. const doc = await app.scrape('https://firecrawl.dev', { formats: ['markdown'] });
  440. console.log(doc.markdown);
  441. // Use the Agent for autonomous data gathering
  442. const result = await app.agent({ prompt: 'Find the founders of Stripe' });
  443. console.log(result.data);
  444. // Crawl a website (automatically waits for completion)
  445. const docs = await app.crawl('https://docs.firecrawl.dev', { limit: 50 });
  446. docs.data.forEach(doc => {
  447. console.log(doc.metadata.sourceURL, doc.markdown.substring(0, 100));
  448. });
  449. // Search the web
  450. const results = await app.search('best AI data tools 2024', { limit: 10 });
  451. results.data.web.forEach(result => {
  452. console.log(`${result.title}: ${result.url}`);
  453. });
  454. ```
  455. ### Go
  456. Install the SDK:
  457. ```bash
  458. go get github.com/firecrawl/firecrawl/apps/go-sdk
  459. ```
  460. ```go
  461. package main
  462. import (
  463. "context"
  464. "fmt"
  465. "log"
  466. firecrawl "github.com/firecrawl/firecrawl/apps/go-sdk"
  467. "github.com/firecrawl/firecrawl/apps/go-sdk/option"
  468. )
  469. func main() {
  470. // Create a client (reads FIRECRAWL_API_KEY from environment)
  471. client, err := firecrawl.NewClient(option.WithAPIKey("fc-YOUR_API_KEY"))
  472. if err != nil {
  473. log.Fatal(err)
  474. }
  475. ctx := context.Background()
  476. // Scrape a single URL
  477. doc, err := client.Scrape(ctx, "https://firecrawl.dev", &firecrawl.ScrapeOptions{
  478. Formats: []string{"markdown"},
  479. })
  480. if err != nil {
  481. log.Fatal(err)
  482. }
  483. fmt.Println(doc.Markdown)
  484. // Use the Agent for autonomous data gathering
  485. agent, err := client.Agent(ctx, &firecrawl.AgentOptions{
  486. Prompt: "Find the founders of Stripe",
  487. })
  488. if err != nil {
  489. log.Fatal(err)
  490. }
  491. fmt.Println(agent.Data)
  492. // Crawl a website (automatically waits for completion)
  493. job, err := client.Crawl(ctx, "https://docs.firecrawl.dev", &firecrawl.CrawlOptions{
  494. Limit: firecrawl.Int(50),
  495. })
  496. if err != nil {
  497. log.Fatal(err)
  498. }
  499. fmt.Printf("Crawled %d pages\n", len(job.Data))
  500. // Search the web
  501. results, err := client.Search(ctx, "best AI data tools 2024", &firecrawl.SearchOptions{
  502. Limit: firecrawl.Int(10),
  503. })
  504. if err != nil {
  505. log.Fatal(err)
  506. }
  507. fmt.Println(results)
  508. }
  509. ```
  510. ### Java
  511. Add the dependency ([Gradle/Maven](https://docs.firecrawl.dev/sdks/java#installation)):
  512. ```groovy
  513. repositories {
  514. mavenCentral()
  515. maven { url 'https://jitpack.io' }
  516. }
  517. dependencies {
  518. implementation 'com.github.firecrawl:firecrawl-java-sdk:2.0'
  519. }
  520. ```
  521. ```java
  522. import dev.firecrawl.client.FirecrawlClient;
  523. import dev.firecrawl.model.*;
  524. FirecrawlClient client = new FirecrawlClient(
  525. System.getenv("FIRECRAWL_API_KEY"), null, null
  526. );
  527. // Scrape a single URL
  528. ScrapeParams scrapeParams = new ScrapeParams();
  529. scrapeParams.setFormats(new String[]{"markdown"});
  530. FirecrawlDocument doc = client.scrapeURL("https://firecrawl.dev", scrapeParams);
  531. System.out.println(doc.getMarkdown());
  532. // Use the Agent for autonomous data gathering
  533. AgentParams agentParams = new AgentParams("Find the founders of Stripe");
  534. AgentResponse start = client.createAgent(agentParams);
  535. AgentStatusResponse result = client.getAgentStatus(start.getId());
  536. System.out.println(result.getData());
  537. // Crawl a website (polls until completion)
  538. CrawlParams crawlParams = new CrawlParams();
  539. crawlParams.setLimit(50);
  540. CrawlStatusResponse job = client.crawlURL("https://docs.firecrawl.dev", crawlParams, null, 10);
  541. for (FirecrawlDocument page : job.getData()) {
  542. System.out.println(page.getMetadata().get("sourceURL"));
  543. }
  544. // Search the web
  545. SearchParams searchParams = new SearchParams("best AI data tools 2024");
  546. searchParams.setLimit(10);
  547. SearchResponse results = client.search(searchParams);
  548. for (SearchResult r : results.getResults()) {
  549. System.out.println(r.getTitle() + ": " + r.getUrl());
  550. }
  551. ```
  552. ### Elixir
  553. Add the dependency:
  554. ```elixir
  555. def deps do
  556. [
  557. {:firecrawl, "~> 1.0"}
  558. ]
  559. end
  560. ```
  561. ```elixir
  562. # Scrape a URL
  563. {:ok, response} = Firecrawl.scrape_and_extract_from_url(
  564. url: "https://firecrawl.dev",
  565. formats: ["markdown"]
  566. )
  567. # Crawl a website
  568. {:ok, response} = Firecrawl.crawl_urls(
  569. url: "https://docs.firecrawl.dev",
  570. limit: 50
  571. )
  572. # Search the web
  573. {:ok, response} = Firecrawl.search_and_scrape(
  574. query: "best AI data tools 2024",
  575. limit: 10
  576. )
  577. # Map URLs
  578. {:ok, response} = Firecrawl.map_urls(url: "https://example.com")
  579. ```
  580. ### Rust
  581. Add the dependency:
  582. ```toml
  583. [dependencies]
  584. firecrawl = "2"
  585. tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
  586. ```
  587. ```rust
  588. use firecrawl::{Client, ScrapeOptions, Format, CrawlOptions};
  589. #[tokio::main]
  590. async fn main() -> Result<(), Box<dyn std::error::Error>> {
  591. let client = Client::new("fc-YOUR_API_KEY")?;
  592. // Scrape a URL
  593. let document = client.scrape("https://firecrawl.dev", None).await?;
  594. println!("{:?}", document.markdown);
  595. // Crawl a website
  596. let options = CrawlOptions {
  597. limit: Some(50),
  598. ..Default::default()
  599. };
  600. let result = client.crawl("https://docs.firecrawl.dev", options).await?;
  601. println!("Crawled {} pages", result.data.len());
  602. // Search the web
  603. let response = client.search("best web scraping tools 2024", None).await?;
  604. println!("{:?}", response.data);
  605. Ok(())
  606. }
  607. ```
  608. ### Ruby
  609. Install the SDK:
  610. ```bash
  611. gem install firecrawl-sdk
  612. ```
  613. ```ruby
  614. require "firecrawl"
  615. client = Firecrawl::Client.new(api_key: "fc-YOUR_API_KEY")
  616. # Scrape a single URL
  617. doc = client.scrape("https://firecrawl.dev",
  618. Firecrawl::Models::ScrapeOptions.new(formats: ["markdown"]))
  619. puts doc.markdown
  620. # Use the Agent for autonomous data gathering
  621. result = client.agent(
  622. Firecrawl::Models::AgentOptions.new(prompt: "Find the founders of Stripe"))
  623. puts result.data
  624. # Crawl a website (automatically waits for completion)
  625. job = client.crawl("https://docs.firecrawl.dev",
  626. Firecrawl::Models::CrawlOptions.new(limit: 50))
  627. job.data.each { |d| puts d.metadata.source_url }
  628. # Search the web
  629. results = client.search("best AI data tools 2024",
  630. Firecrawl::Models::SearchOptions.new(limit: 10))
  631. puts results
  632. ```
  633. ### .NET
  634. Install the SDK:
  635. ```bash
  636. dotnet add package firecrawl-sdk
  637. ```
  638. ```csharp
  639. using Firecrawl;
  640. using Firecrawl.Models;
  641. var client = new FirecrawlClient("fc-YOUR_API_KEY");
  642. // Scrape a single URL
  643. var doc = await client.ScrapeAsync("https://firecrawl.dev",
  644. new ScrapeOptions { Formats = new List<object> { "markdown" } });
  645. Console.WriteLine(doc.Markdown);
  646. // Crawl a website (automatically waits for completion)
  647. var job = await client.CrawlAsync("https://docs.firecrawl.dev",
  648. new CrawlOptions { Limit = 50 });
  649. Console.WriteLine($"Crawled {job.Data.Count} pages");
  650. // Search the web
  651. var results = await client.SearchAsync("best AI data tools 2024",
  652. new SearchOptions { Limit = 10 });
  653. Console.WriteLine(results);
  654. ```
  655. ### PHP
  656. Install the SDK:
  657. ```bash
  658. composer require firecrawl/firecrawl-sdk
  659. ```
  660. ```php
  661. <?php
  662. use Firecrawl\Client\FirecrawlClient;
  663. use Firecrawl\Models\ScrapeOptions;
  664. use Firecrawl\Models\CrawlOptions;
  665. use Firecrawl\Models\SearchOptions;
  666. $client = FirecrawlClient::create(apiKey: 'fc-YOUR_API_KEY');
  667. // Scrape a single URL
  668. $doc = $client->scrape('https://firecrawl.dev', ScrapeOptions::with(
  669. formats: ['markdown'],
  670. ));
  671. echo $doc->getMarkdown();
  672. // Crawl a website (automatically waits for completion)
  673. $job = $client->crawl('https://docs.firecrawl.dev', CrawlOptions::with(limit: 50));
  674. foreach ($job->getData() as $page) {
  675. echo $page->getMetadata()['sourceURL'] . "\n";
  676. }
  677. // Search the web
  678. $results = $client->search('best AI data tools 2024', SearchOptions::with(limit: 10));
  679. print_r($results);
  680. ```
  681. ---
  682. ## Integrations
  683. **Agents & AI Tools**
  684. - [Firecrawl Skills Catalog](https://github.com/firecrawl/skills) โ€” install with `npx skills add firecrawl/skills`
  685. - [Firecrawl CLI](https://docs.firecrawl.dev/sdks/cli)
  686. - [Firecrawl MCP](https://github.com/firecrawl/firecrawl-mcp-server)
  687. The build skills (integrating Firecrawl into product code) are authored in this repo under [`skills/`](./skills) and mirrored into the catalog by CI. Contributing skills? CLI skills (including the research/developer index skills) โ†’ PR [`firecrawl/cli`](https://github.com/firecrawl/cli). Build/SDK skills โ†’ PR this repo (`skills/`). Workflow skills โ†’ PR [`firecrawl/firecrawl-workflows`](https://github.com/firecrawl/firecrawl-workflows). The catalog ([`firecrawl/skills`](https://github.com/firecrawl/skills)) is read-only โ€” never PR it directly.
  688. **Platforms**
  689. - [Lovable](https://docs.lovable.dev/integrations/firecrawl)
  690. - [Zapier](https://zapier.com/apps/firecrawl/integrations)
  691. - [n8n](https://n8n.io/integrations/firecrawl/)
  692. [View all integrations โ†’](https://www.firecrawl.dev/integrations)
  693. **Missing your favorite tool?** [Open an issue](https://github.com/firecrawl/firecrawl/issues) and let us know!
  694. ---
  695. ## Resources
  696. - [Documentation](https://docs.firecrawl.dev)
  697. - [API Reference](https://docs.firecrawl.dev/api-reference/introduction)
  698. - [Playground](https://firecrawl.dev/playground)
  699. - [Changelog](https://firecrawl.dev/changelog)
  700. ---
  701. ## Open Source vs Cloud
  702. Firecrawl is open source under the AGPL-3.0 license. The cloud version at [firecrawl.dev](https://firecrawl.dev) includes additional features:
  703. ![Open Source vs Cloud](https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/open-source-cloud.png)
  704. To run locally, see the [Contributing Guide](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md). To self-host, see [Self-Hosting Guide](https://docs.firecrawl.dev/contributing/self-host).
  705. ---
  706. ## Contributing
  707. We love contributions! Please read our [Contributing Guide](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md) before submitting a pull request.
  708. ### Contributors
  709. <a href="https://github.com/firecrawl/firecrawl/graphs/contributors">
  710. <img alt="contributors" src="https://contrib.rocks/image?repo=firecrawl/firecrawl"/>
  711. </a>
  712. ---
  713. ## License
  714. This project is primarily licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). The SDKs and some UI components are licensed under the MIT License. See the LICENSE files in specific directories for details.
  715. ---
  716. **It is the sole responsibility of end users to respect websites' policies when scraping.** Users are advised to adhere to applicable privacy policies and terms of use. By default, Firecrawl respects robots.txt directives. By using Firecrawl, you agree to comply with these conditions.
  717. <p align="right" style="font-size: 14px; color: #555; margin-top: 20px;">
  718. <a href="#readme-top" style="text-decoration: none; color: #007bff; font-weight: bold;">
  719. โ†‘ Back to Top โ†‘
  720. </a>
  721. </p>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If itโ€™s helped, pass it on.