[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/agent.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/agent)
[Get Active Crawls\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
[Get Agent Status\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
⌘I
---
# Advanced Scraping Guide | Firecrawl
[Skip to main content](https://docs.firecrawl.dev/advanced-scraping-guide#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Get Started
Advanced Scraping Guide
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Get Started
* [Quickstart](https://docs.firecrawl.dev/introduction)
* [MCP Server](https://docs.firecrawl.dev/mcp-server)
* [Migrating from v1 to v2](https://docs.firecrawl.dev/migrate-to-v2)
* [Rate Limits](https://docs.firecrawl.dev/rate-limits)
* [Advanced Scraping Guide](https://docs.firecrawl.dev/advanced-scraping-guide)
##### New Features
* Agent
##### Standard Features
* Scrape
* [Search](https://docs.firecrawl.dev/features/search)
* [Map](https://docs.firecrawl.dev/features/map)
* [Crawl](https://docs.firecrawl.dev/features/crawl)
##### Webhooks
* [Overview](https://docs.firecrawl.dev/webhooks/overview)
* [Event Types](https://docs.firecrawl.dev/webhooks/events)
* [Security](https://docs.firecrawl.dev/webhooks/security)
* [Testing](https://docs.firecrawl.dev/webhooks/testing)
##### Developer Guides
* [Full-Stack Templates](https://docs.firecrawl.dev/developer-guides/examples)
* LLM SDKs and Frameworks
* Cookbooks
* MCP Setup Guides
* Common Sites
* Workflow Automation
* Advanced Guides
##### Use Cases
* [Overview](https://docs.firecrawl.dev/use-cases/overview)
* [AI Platforms](https://docs.firecrawl.dev/use-cases/ai-platforms)
* [Lead Enrichment](https://docs.firecrawl.dev/use-cases/lead-enrichment)
* [SEO Platforms](https://docs.firecrawl.dev/use-cases/seo-platforms)
* [Deep Research](https://docs.firecrawl.dev/use-cases/deep-research)
* View more
##### Contributing
* [Open Source vs Cloud](https://docs.firecrawl.dev/contributing/open-source-or-cloud)
* [Running locally](https://docs.firecrawl.dev/contributing/guide)
* [Self-hosting](https://docs.firecrawl.dev/contributing/self-host)
On this page
* [Basic scraping with Firecrawl](https://docs.firecrawl.dev/advanced-scraping-guide#basic-scraping-with-firecrawl)
* [Scraping PDFs](https://docs.firecrawl.dev/advanced-scraping-guide#scraping-pdfs)
* [Scrape options](https://docs.firecrawl.dev/advanced-scraping-guide#scrape-options)
* [Formats (formats)](https://docs.firecrawl.dev/advanced-scraping-guide#formats-formats)
* [Full page content vs main content (onlyMainContent)](https://docs.firecrawl.dev/advanced-scraping-guide#full-page-content-vs-main-content-onlymaincontent)
* [Include tags (includeTags)](https://docs.firecrawl.dev/advanced-scraping-guide#include-tags-includetags)
* [Exclude tags (excludeTags)](https://docs.firecrawl.dev/advanced-scraping-guide#exclude-tags-excludetags)
* [Wait for page readiness (waitFor)](https://docs.firecrawl.dev/advanced-scraping-guide#wait-for-page-readiness-waitfor)
* [Freshness and cache (maxAge)](https://docs.firecrawl.dev/advanced-scraping-guide#freshness-and-cache-maxage)
* [Request timeout (timeout)](https://docs.firecrawl.dev/advanced-scraping-guide#request-timeout-timeout)
* [PDF parsing (parsers)](https://docs.firecrawl.dev/advanced-scraping-guide#pdf-parsing-parsers)
* [Actions (actions)](https://docs.firecrawl.dev/advanced-scraping-guide#actions-actions)
* [Action Execution Notes](https://docs.firecrawl.dev/advanced-scraping-guide#action-execution-notes)
* [Advanced Action Examples](https://docs.firecrawl.dev/advanced-scraping-guide#advanced-action-examples)
* [Example Usage](https://docs.firecrawl.dev/advanced-scraping-guide#example-usage)
* [JSON extraction via formats](https://docs.firecrawl.dev/advanced-scraping-guide#json-extraction-via-formats)
* [Extract endpoint](https://docs.firecrawl.dev/advanced-scraping-guide#extract-endpoint)
* [Crawling multiple pages](https://docs.firecrawl.dev/advanced-scraping-guide#crawling-multiple-pages)
* [Check Crawl Job](https://docs.firecrawl.dev/advanced-scraping-guide#check-crawl-job)
* [Pagination/Next URL](https://docs.firecrawl.dev/advanced-scraping-guide#pagination%2Fnext-url)
* [Crawl prompt and params preview](https://docs.firecrawl.dev/advanced-scraping-guide#crawl-prompt-and-params-preview)
* [Crawler options](https://docs.firecrawl.dev/advanced-scraping-guide#crawler-options)
* [includePaths](https://docs.firecrawl.dev/advanced-scraping-guide#includepaths)
* [excludePaths](https://docs.firecrawl.dev/advanced-scraping-guide#excludepaths)
* [maxDiscoveryDepth](https://docs.firecrawl.dev/advanced-scraping-guide#maxdiscoverydepth)
* [limit](https://docs.firecrawl.dev/advanced-scraping-guide#limit)
* [crawlEntireDomain](https://docs.firecrawl.dev/advanced-scraping-guide#crawlentiredomain)
* [allowExternalLinks](https://docs.firecrawl.dev/advanced-scraping-guide#allowexternallinks)
* [allowSubdomains](https://docs.firecrawl.dev/advanced-scraping-guide#allowsubdomains)
* [delay](https://docs.firecrawl.dev/advanced-scraping-guide#delay)
* [scrapeOptions](https://docs.firecrawl.dev/advanced-scraping-guide#scrapeoptions)
* [Example Usage](https://docs.firecrawl.dev/advanced-scraping-guide#example-usage-2)
* [Mapping website links](https://docs.firecrawl.dev/advanced-scraping-guide#mapping-website-links)
* [Usage](https://docs.firecrawl.dev/advanced-scraping-guide#usage)
* [Map Options](https://docs.firecrawl.dev/advanced-scraping-guide#map-options)
* [search](https://docs.firecrawl.dev/advanced-scraping-guide#search)
* [limit](https://docs.firecrawl.dev/advanced-scraping-guide#limit-2)
* [sitemap](https://docs.firecrawl.dev/advanced-scraping-guide#sitemap)
* [includeSubdomains](https://docs.firecrawl.dev/advanced-scraping-guide#includesubdomains)
This guide will walk you through the different endpoints of Firecrawl and how to use them fully with all its parameters.
[](https://docs.firecrawl.dev/advanced-scraping-guide#basic-scraping-with-firecrawl)
Basic scraping with Firecrawl
----------------------------------------------------------------------------------------------------------------------
To scrape a single page and get clean markdown content, you can use the `/scrape` endpoint.
Python
JavaScript
cURL
Copy
Ask AI
# pip install firecrawl-py
from firecrawl import Firecrawl
firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")
doc = firecrawl.scrape("https://firecrawl.dev")
print(doc.markdown)
[](https://docs.firecrawl.dev/advanced-scraping-guide#scraping-pdfs)
Scraping PDFs
--------------------------------------------------------------------------------------
Firecrawl supports PDFs. Use the `parsers` option (e.g., `parsers: ["pdf"]`) when you want to ensure PDF parsing.
[](https://docs.firecrawl.dev/advanced-scraping-guide#scrape-options)
Scrape options
----------------------------------------------------------------------------------------
When using the `/scrape` endpoint, you can customize scraping with the options below.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#formats-formats)
Formats (`formats`)
* **Type**: `array`
* **Strings**: `["markdown", "links", "html", "rawHtml", "summary", "images"]`
* **Object formats**:
* JSON: `{ type: "json", prompt, schema }`
* Screenshot: `{ type: "screenshot", fullPage?, quality?, viewport? }`
* Change tracking: `{ type: "changeTracking", modes?, prompt?, schema?, tag? }` (requires `markdown`)
* **Default**: `["markdown"]`
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#full-page-content-vs-main-content-onlymaincontent)
Full page content vs main content (`onlyMainContent`)
* **Type**: `boolean`
* **Description**: By default the scraper returns only the main content. Set to `false` to return full page content.
* **Default**: `true`
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#include-tags-includetags)
Include tags (`includeTags`)
* **Type**: `array`
* **Description**: HTML tags/classes/ids to include in the scrape.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#exclude-tags-excludetags)
Exclude tags (`excludeTags`)
* **Type**: `array`
* **Description**: HTML tags/classes/ids to exclude from the scrape.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#wait-for-page-readiness-waitfor)
Wait for page readiness (`waitFor`)
* **Type**: `integer`
* **Description**: Milliseconds of extra wait time before scraping (use sparingly). This waiting time is in addition to Firecrawl’s smart wait feature.
* **Default**: `0`
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#freshness-and-cache-maxage)
Freshness and cache (`maxAge`)
* **Type**: `integer` (milliseconds)
* **Description**: If a cached version of the page is newer than `maxAge`, Firecrawl returns it instantly; otherwise it scrapes fresh and updates the cache. Set `0` to always fetch fresh.
* **Default**: `172800000` (2 days)
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#request-timeout-timeout)
Request timeout (`timeout`)
* **Type**: `integer`
* **Description**: Max duration in milliseconds before aborting.
* **Default**: `30000` (30 seconds)
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#pdf-parsing-parsers)
PDF parsing (`parsers`)
* **Type**: `array`
* **Description**: Control parsing behavior. To parse PDFs, set `parsers: ["pdf"]`.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#actions-actions)
Actions (`actions`)
When using the /scrape endpoint, Firecrawl allows you to perform various actions on a web page before scraping its content. This is particularly useful for interacting with dynamic content, navigating through pages, or accessing content that requires user interaction.
* **Type**: `array`
* **Description**: Sequence of browser steps to run before scraping.
* **Supported actions**:
* `wait` - Wait for page to load: `{ type: "wait", milliseconds: number }` or `{ type: "wait", selector: string }`
* `click` - Click an element: `{ type: "click", selector: string, all?: boolean }`
* `write` - Type text into a field: `{ type: "write", text: string }` (element must be focused first with click)
* `press` - Press a keyboard key: `{ type: "press", key: string }`
* `scroll` - Scroll the page: `{ type: "scroll", direction: "up" | "down", selector?: string }`
* `screenshot` - Capture screenshot: `{ type: "screenshot", fullPage?: boolean, quality?: number, viewport?: { width: number, height: number } }`
* `scrape` - Scrape sub-element: `{ type: "scrape" }`
* `executeJavascript` - Run JS code: `{ type: "executeJavascript", script: string }`
* `pdf` - Generate PDF: `{ type: "pdf", format?: string, landscape?: boolean, scale?: number }`
Python
Node
cURL
Copy
Ask AI
from firecrawl import Firecrawl
firecrawl = Firecrawl(api_key='fc-YOUR-API-KEY')
doc = firecrawl.scrape('https://example.com', {
'actions': [\
{ 'type': 'wait', 'milliseconds': 1000 },\
{ 'type': 'click', 'selector': '#accept' },\
{ 'type': 'scroll', 'direction': 'down' },\
{ 'type': 'click', 'selector': '#q' },\
{ 'type': 'write', 'text': 'firecrawl' },\
{ 'type': 'press', 'key': 'Enter' },\
{ 'type': 'wait', 'milliseconds': 2000 }\
],
'formats': ['markdown']
})
print(doc.markdown)
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#action-execution-notes)
Action Execution Notes
* **Write action**: You must first focus the element using a `click` action before using `write`. The text is typed character by character to simulate keyboard input.
* **Scroll selector**: If you want to scroll a specific element instead of the whole page, provide the `selector` parameter to `scroll`.
* **Wait with selector**: You can wait for a specific element to be visible using `wait` with a `selector` parameter, or wait for a fixed duration using `milliseconds`.
* **Actions are sequential**: Actions are executed in order, and Firecrawl waits for page interactions to complete before moving to the next action.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#advanced-action-examples)
Advanced Action Examples
**Taking a screenshot:**
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://example.com",
"actions": [\
{ "type": "click", "selector": "#load-more" },\
{ "type": "wait", "milliseconds": 1000 },\
{ "type": "screenshot", "fullPage": true, "quality": 80 }\
]
}'
**Clicking multiple elements:**
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://example.com",
"actions": [\
{ "type": "click", "selector": ".expand-button", "all": true },\
{ "type": "wait", "milliseconds": 500 }\
],
"formats": ["markdown"]
}'
**Generating a PDF:**
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://example.com",
"actions": [\
{ "type": "pdf", "format": "A4", "landscape": false }\
]
}'
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#example-usage)
Example Usage
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H '
Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://docs.firecrawl.dev",
"formats": [\
"markdown",\
"links",\
"html",\
"rawHtml",\
{ "type": "screenshot", "fullPage": true, "quality": 80 }\
],
"includeTags": ["h1", "p", "a", ".main-content"],
"excludeTags": ["#ad", "#footer"],
"onlyMainContent": false,
"waitFor": 1000,
"timeout": 15000,
"parsers": ["pdf"]
}'
In this example, the scraper will:
* Return the full page content as markdown.
* Include the markdown, raw HTML, HTML, links, and a screenshot in the response.
* Include only the HTML tags ``, `
`, ``, and elements with the class `.main-content`, while excluding any elements with the IDs `#ad` and `#footer`.
* Wait for 1000 milliseconds (1 second) before scraping to allow the page to load.
* Set the maximum duration of the scrape request to 15000 milliseconds (15 seconds).
* Parse PDFs explicitly via `parsers: ["pdf"]`.
Here is the API Reference: [Scrape Endpoint Documentation](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
[](https://docs.firecrawl.dev/advanced-scraping-guide#json-extraction-via-formats)
JSON extraction via formats
------------------------------------------------------------------------------------------------------------------
Use the JSON format object in `formats` to extract structured data in one pass:
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://firecrawl.dev",
"formats": [{\
"type": "json",\
"prompt": "Extract the features of the product",\
"schema": {"type": "object", "properties": {"features": {"type": "object"}}, "required": ["features"]}\
}]
}'
[](https://docs.firecrawl.dev/advanced-scraping-guide#extract-endpoint)
Extract endpoint
--------------------------------------------------------------------------------------------
Use the dedicated extract job API when you want asynchronous extraction with status polling.
Node
Python
cURL
Copy
Ask AI
import Firecrawl from '@mendable/firecrawl-js';
const firecrawl = new Firecrawl({ apiKey: 'fc-YOUR-API-KEY' });
// Start extract job
const started = await firecrawl.startExtract({
urls: ['https://docs.firecrawl.dev'],
prompt: 'Extract title',
schema: { type: 'object', properties: { title: { type: 'string' } }, required: ['title'] }
});
// Poll status
const status = await firecrawl.getExtractStatus(started.id);
console.log(status.status, status.data);
[](https://docs.firecrawl.dev/advanced-scraping-guide#crawling-multiple-pages)
Crawling multiple pages
----------------------------------------------------------------------------------------------------------
To crawl multiple pages, use the `/v2/crawl` endpoint.
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/crawl \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://docs.firecrawl.dev"
}'
Returns an id
Copy
Ask AI
{ "id": "1234-5678-9101" }
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#check-crawl-job)
Check Crawl Job
Used to check the status of a crawl job and get its result.
cURL
Copy
Ask AI
curl -X GET https://api.firecrawl.dev/v2/crawl/1234-5678-9101 \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY'
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#pagination/next-url)
Pagination/Next URL
If the content is larger than 10MB or if the crawl job is still running, the response may include a `next` parameter, a URL to the next page of results.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#crawl-prompt-and-params-preview)
Crawl prompt and params preview
You can provide a natural-language `prompt` to let Firecrawl derive crawl settings. Preview them first:
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/crawl/params-preview \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://docs.firecrawl.dev",
"prompt": "Extract docs and blog"
}'
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#crawler-options)
Crawler options
When using the `/v2/crawl` endpoint, you can customize the crawling behavior with:
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#includepaths)
includePaths
* **Type**: `array`
* **Description**: Regex patterns to include.
* **Example**: `["^/blog/.*$", "^/docs/.*$"]`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#excludepaths)
excludePaths
* **Type**: `array`
* **Description**: Regex patterns to exclude.
* **Example**: `["^/admin/.*$", "^/private/.*$"]`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#maxdiscoverydepth)
maxDiscoveryDepth
* **Type**: `integer`
* **Description**: Max discovery depth for finding new URLs.
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#limit)
limit
* **Type**: `integer`
* **Description**: Max number of pages to crawl.
* **Default**: `10000`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#crawlentiredomain)
crawlEntireDomain
* **Type**: `boolean`
* **Description**: Explore across siblings/parents to cover the entire domain.
* **Default**: `false`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#allowexternallinks)
allowExternalLinks
* **Type**: `boolean`
* **Description**: Follow links to external domains.
* **Default**: `false`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#allowsubdomains)
allowSubdomains
* **Type**: `boolean`
* **Description**: Follow subdomains of the main domain.
* **Default**: `false`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#delay)
delay
* **Type**: `number`
* **Description**: Delay in seconds between scrapes.
* **Default**: `undefined`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#scrapeoptions)
scrapeOptions
* **Type**: `object`
* **Description**: Options for the scraper (see Formats above).
* **Example**: `{ "formats": ["markdown", "links", {"type": "screenshot", "fullPage": true}], "includeTags": ["h1", "p", "a", ".main-content"], "excludeTags": ["#ad", "#footer"], "onlyMainContent": false, "waitFor": 1000, "timeout": 15000}`
* **Defaults**: `formats: ["markdown"]`, caching enabled by default (maxAge ~ 2 days)
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#example-usage-2)
Example Usage
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/crawl \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://docs.firecrawl.dev",
"includePaths": ["^/blog/.*$", "^/docs/.*$"],
"excludePaths": ["^/admin/.*$", "^/private/.*$"],
"maxDiscoveryDepth": 2,
"limit": 1000
}'
[](https://docs.firecrawl.dev/advanced-scraping-guide#mapping-website-links)
Mapping website links
------------------------------------------------------------------------------------------------------
The `/v2/map` endpoint identifies URLs related to a given website.
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#usage)
Usage
cURL
Copy
Ask AI
curl -X POST https://api.firecrawl.dev/v2/map \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer fc-YOUR-API-KEY' \
-d '{
"url": "https://docs.firecrawl.dev"
}'
###
[](https://docs.firecrawl.dev/advanced-scraping-guide#map-options)
Map Options
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#search)
search
* **Type**: `string`
* **Description**: Filter links containing text.
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#limit-2)
limit
* **Type**: `integer`
* **Description**: Maximum number of links to return.
* **Default**: `100`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#sitemap)
sitemap
* **Type**: `"only" | "include" | "skip"`
* **Description**: Control sitemap usage during mapping.
* **Default**: `"include"`
####
[](https://docs.firecrawl.dev/advanced-scraping-guide#includesubdomains)
includeSubdomains
* **Type**: `boolean`
* **Description**: Include subdomains of the website.
* **Default**: `true`
Here is the API Reference for it: [Map Endpoint Documentation](https://docs.firecrawl.dev/api-reference/endpoint/map)
Thanks for reading!
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/advanced-scraping-guide.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/advanced-scraping-guide)
[Rate Limits\
\
Previous](https://docs.firecrawl.dev/rate-limits)
[Agent\
\
Next](https://docs.firecrawl.dev/features/agent)
⌘I
---
# Batch Scrape - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Batch Scrape
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Scrape multiple URLs and optionally extract information using an LLM
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/batch/scrape \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"urls": [\
""\
],
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"maxConcurrency": 123,
"ignoreInvalidURLs": true,
"formats": [\
"markdown"\
],
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 172800000,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": true,
"timeout": 30000,
"parsers": [\
"pdf"\
],
"actions": [\
{\
"type": "wait",\
"milliseconds": 2,\
"selector": "#my-element"\
}\
],
"location": {
"country": "US",
"languages": [\
"en-US"\
]
},
"removeBase64Images": true,
"blockAds": true,
"proxy": "auto",
"storeInCache": true,
"zeroDataRetention": false
}
'
200
402
429
500
Copy
Ask AI
{
"success": true,
"id": "",
"url": "",
"invalidURLs": [\
""\
]
}
POST
/
batch
/
scrape
Try it
Scrape multiple URLs and optionally extract information using an LLM
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/batch/scrape \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"urls": [\
""\
],
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"maxConcurrency": 123,
"ignoreInvalidURLs": true,
"formats": [\
"markdown"\
],
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 172800000,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": true,
"timeout": 30000,
"parsers": [\
"pdf"\
],
"actions": [\
{\
"type": "wait",\
"milliseconds": 2,\
"selector": "#my-element"\
}\
],
"location": {
"country": "US",
"languages": [\
"en-US"\
]
},
"removeBase64Images": true,
"blockAds": true,
"proxy": "auto",
"storeInCache": true,
"zeroDataRetention": false
}
'
200
402
429
500
Copy
Ask AI
{
"success": true,
"id": "",
"url": "",
"invalidURLs": [\
""\
]
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Body
application/json
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-urls)
urls
string\[\]
required
The URL to scrape
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-webhook)
webhook
object
A webhook specification object.
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-max-concurrency)
maxConcurrency
integer
Maximum number of concurrent scrapes. This parameter allows you to set a concurrency limit for this batch scrape. If not specified, the batch scrape adheres to your team's concurrency limit.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-ignore-invalid-urls)
ignoreInvalidURLs
boolean
default:true
If invalid URLs are specified in the urls array, they will be ignored. Instead of them failing the entire request, a batch scrape using the remaining valid URLs will be created, and the invalid URLs will be returned in the invalidURLs field of the response.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-formats)
formats
(Markdown · object | Summary · object | HTML · object | Raw HTML · object | Links · object | Images · object | Screenshot · object | JSON · object | Change Tracking · object | Branding · object)\[\]
Output formats to include in the response. You can specify one or more formats, either as strings (e.g., `'markdown'`) or as objects with additional options (e.g., `{ type: 'json', schema: {...} }`). Some formats require specific options to be set. Example: `['markdown', { type: 'json', schema: {...} }]`.
* Markdown
* Summary
* HTML
* Raw HTML
* Links
* Images
* Screenshot
* JSON
* Change Tracking
* Branding
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-only-main-content)
onlyMainContent
boolean
default:true
Only return the main content of the page excluding headers, navs, footers, etc.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-include-tags)
includeTags
string\[\]
Tags to include in the output.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-exclude-tags)
excludeTags
string\[\]
Tags to exclude from the output.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-max-age)
maxAge
integer
default:172800000
Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 2 days.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-headers)
headers
object
Headers to send with the request. Can be used to send cookies, user-agent, etc.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-wait-for)
waitFor
integer
default:0
Specify a delay in milliseconds before fetching the content, allowing the page sufficient time to load. This waiting time is in addition to Firecrawl's smart wait feature.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-mobile)
mobile
boolean
default:false
Set to true if you want to emulate scraping from a mobile device. Useful for testing responsive pages and taking mobile screenshots.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-skip-tls-verification)
skipTlsVerification
boolean
default:true
Skip TLS certificate verification when making requests.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-timeout)
timeout
integer
default:30000
Timeout in milliseconds for the request. Default is 30000 (30 seconds). Maximum is 300000 (300 seconds).
Required range: `x <= 300000`
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-parsers)
parsers
object\[\]
Controls how files are processed during scraping. When "pdf" is included (default), the PDF content is extracted and converted to markdown format, with billing based on the number of pages (1 credit per page). When an empty array is passed, the PDF file is returned in base64 encoding with a flat rate of 1 credit for the entire PDF.
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-actions)
actions
(Wait · object | Screenshot · object | Click · object | Write text · object | Press a key · object | Scroll · object | Scrape · object | Execute JavaScript · object | Generate PDF · object)\[\]
Actions to perform on the page before grabbing the content
* Wait
* Screenshot
* Click
* Write text
* Press a key
* Scroll
* Scrape
* Execute JavaScript
* Generate PDF
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-location)
location
object
Location settings for the request. When specified, this will use an appropriate proxy if available and emulate the corresponding language and timezone settings. Defaults to 'US' if not specified.
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-remove-base64-images)
removeBase64Images
boolean
default:true
Removes all base 64 images from the output, which may be overwhelmingly long. The image's alt text remains in the output, but the URL is replaced with a placeholder.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-block-ads)
blockAds
boolean
default:true
Enables ad-blocking and cookie popup blocking.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-proxy)
proxy
enum
default:auto
Specifies the type of proxy to use.
* basic: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works.
* stealth: Stealth proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Costs up to 5 credits per request.
* auto: Firecrawl will automatically retry scraping with stealth proxies if the basic proxy fails. If the retry with stealth is successful, 5 credits will be billed for the scrape. If the first attempt with basic is successful, only the regular cost will be billed.
Available options:
`basic`,
`stealth`,
`auto`
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-store-in-cache)
storeInCache
boolean
default:true
If true, the page will be stored in the Firecrawl index and cache. Setting this to false is useful if your scraping activity may have data protection concerns. Using some parameters associated with sensitive scraping (e.g. actions, headers) will force this parameter to be false.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#body-zero-data-retention)
zeroDataRetention
boolean
default:false
If true, this will enable zero data retention for this batch scrape. To enable this feature, please contact [\[email protected\]](https://docs.firecrawl.dev/cdn-cgi/l/email-protection#7b131e170b3b1d12091e18091a0c17551f1e0d)
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#response-success)
success
boolean
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#response-id)
id
string
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#response-url)
url
string
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape#response-invalid-urls-one-of-0)
invalidURLs
string\[\] | null
If ignoreInvalidURLs is true, this is an array containing the invalid URLs that were specified in the request. If there were no invalid URLs, this will be an empty array. If ignoreInvalidURLs is false, this field will be undefined.
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/batch-scrape.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/batch-scrape)
[Scrape\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
[Get Batch Scrape Status\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
⌘I
---
# Get Batch Scrape Status - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Get Batch Scrape Status
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get the status of a batch scrape job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"status": "",
"total": 123,
"completed": 123,
"creditsUsed": 123,
"expiresAt": "2023-11-07T05:31:56Z",
"next": "",
"data": [\
{\
"markdown": "",\
"html": "",\
"rawHtml": "",\
"links": [\
""\
],\
"screenshot": "",\
"metadata": {\
"title": "",\
"description": "",\
"language": "",\
"sourceURL": "",\
"keywords": "",\
"ogLocaleAlternate": [\
""\
],\
" ": "",\
"statusCode": 123,\
"error": ""\
}\
}\
]
}
GET
/
batch
/
scrape
/
{id}
Try it
Get the status of a batch scrape job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"status": "",
"total": 123,
"completed": 123,
"creditsUsed": 123,
"expiresAt": "2023-11-07T05:31:56Z",
"next": "",
"data": [\
{\
"markdown": "",\
"html": "",\
"rawHtml": "",\
"links": [\
""\
],\
"screenshot": "",\
"metadata": {\
"title": "",\
"description": "",\
"language": "",\
"sourceURL": "",\
"keywords": "",\
"ogLocaleAlternate": [\
""\
],\
" ": "",\
"statusCode": 123,\
"error": ""\
}\
}\
]
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#parameter-id)
id
string
required
The ID of the batch scrape job
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-status)
status
string
The current status of the batch scrape. Can be `scraping`, `completed`, or `failed`.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-total)
total
integer
The total number of pages that were attempted to be scraped.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-completed)
completed
integer
The number of pages that have been successfully scraped.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-credits-used)
creditsUsed
integer
The number of credits used for the batch scrape.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-expires-at)
expiresAt
string
The date and time when the batch scrape will expire.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-next-one-of-0)
next
string | null
The URL to retrieve the next 10MB of data. Returned if the batch scrape is not completed or if the response is larger than 10MB.
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get#response-data)
data
object\[\]
The data of the batch scrape.
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/batch-scrape-get.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/batch-scrape-get)
[Batch Scrape\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
[Cancel Batch Scrape\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
⌘I
---
# Get Crawl Errors - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Get Crawl Errors
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get the errors of a crawl job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/crawl/{id}/errors \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"errors": [\
{\
"id": "",\
"timestamp": "",\
"url": "",\
"error": ""\
}\
],
"robotsBlocked": [\
""\
]
}
GET
/
crawl
/
{id}
/
errors
Try it
Get the errors of a crawl job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/crawl/{id}/errors \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"errors": [\
{\
"id": "",\
"timestamp": "",\
"url": "",\
"error": ""\
}\
],
"robotsBlocked": [\
""\
]
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors#parameter-id)
id
string
required
The ID of the crawl job
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors#response-errors)
errors
object\[\]
Errored scrape jobs and error details
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors#response-robots-blocked)
robotsBlocked
string\[\]
List of URLs that were attempted in scraping but were blocked by robots.txt
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/crawl-get-errors.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/crawl-get-errors)
[Cancel Crawl\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
[Get Active Crawls\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
⌘I
---
# Get Batch Scrape Errors - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Get Batch Scrape Errors
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get the errors of a batch scrape job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/batch/scrape/{id}/errors \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"errors": [\
{\
"id": "",\
"timestamp": "",\
"url": "",\
"error": ""\
}\
],
"robotsBlocked": [\
""\
]
}
GET
/
batch
/
scrape
/
{id}
/
errors
Try it
Get the errors of a batch scrape job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/batch/scrape/{id}/errors \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"errors": [\
{\
"id": "",\
"timestamp": "",\
"url": "",\
"error": ""\
}\
],
"robotsBlocked": [\
""\
]
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors#parameter-id)
id
string
required
The ID of the batch scrape job
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors#response-errors)
errors
object\[\]
Errored scrape jobs and error details
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors#response-robots-blocked)
robotsBlocked
string\[\]
List of URLs that were attempted in scraping but were blocked by robots.txt
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/batch-scrape-get-errors.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/batch-scrape-get-errors)
[Cancel Batch Scrape\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
[Search\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/search)
⌘I
---
# Crawl Params Preview - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Crawl Params Preview
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Preview crawl parameters generated from natural language prompt
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/crawl/params-preview \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"url": "",
"prompt": ""
}
'
200
400
401
500
Copy
Ask AI
{
"success": true,
"data": {
"url": "",
"includePaths": [\
""\
],
"excludePaths": [\
""\
],
"maxDepth": 123,
"maxDiscoveryDepth": 123,
"crawlEntireDomain": true,
"allowExternalLinks": true,
"allowSubdomains": true,
"sitemap": "skip",
"ignoreQueryParameters": true,
"deduplicateSimilarURLs": true,
"delay": 123,
"limit": 123
}
}
POST
/
crawl
/
params-preview
Try it
Preview crawl parameters generated from natural language prompt
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/crawl/params-preview \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"url": "",
"prompt": ""
}
'
200
400
401
500
Copy
Ask AI
{
"success": true,
"data": {
"url": "",
"includePaths": [\
""\
],
"excludePaths": [\
""\
],
"maxDepth": 123,
"maxDiscoveryDepth": 123,
"crawlEntireDomain": true,
"allowExternalLinks": true,
"allowSubdomains": true,
"sitemap": "skip",
"ignoreQueryParameters": true,
"deduplicateSimilarURLs": true,
"delay": 123,
"limit": 123
}
}
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#what%E2%80%99s-new-in-v2)
What’s New in v2
------------------------------------------------------------------------------------------------------------------------
###
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#crawl-params-preview)
Crawl Params Preview
Copy
Ask AI
{
"url": "https://example.com",
"prompt": "Crawl the entire website but exclude /admin and /api"
}
###
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#response)
Response
Copy
Ask AI
{
"url": "https://example.com",
"prompt": "Crawl the entire website",
"crawlEntireDomain": true,
"excludePaths": ["/admin/.*", "/api/.*"],
"sitemap": "include"
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Body
application/json
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#body-url)
url
string
required
The URL to crawl
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#body-prompt)
prompt
string
required
Natural language prompt describing what you want to crawl
Maximum string length: `10000`
#### Response
200
application/json
Successful response with generated crawl parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview#response-data)
data
object
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/crawl-params-preview.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/crawl-params-preview)
[Get Crawl Status\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
[Cancel Crawl\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
⌘I
---
# Cancel Crawl - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Cancel Crawl
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Cancel a crawl job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v2/crawl/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"status": "cancelled"
}
DELETE
/
crawl
/
{id}
Try it
Cancel a crawl job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v2/crawl/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"status": "cancelled"
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete#parameter-id)
id
string
required
The ID of the crawl job
#### Response
200
application/json
Successful cancellation
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete#response-status)
status
enum
Available options:
`cancelled`
Example:
`"cancelled"`
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/crawl-delete.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/crawl-delete)
[Crawl Params Preview\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
[Get Crawl Errors\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
⌘I
---
# Cancel Batch Scrape - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Cancel Batch Scrape
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Cancel a batch scrape job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v2/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"message": "Batch scrape job successfully cancelled."
}
DELETE
/
batch
/
scrape
/
{id}
Try it
Cancel a batch scrape job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v2/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"message": "Batch scrape job successfully cancelled."
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete#parameter-id)
id
string
required
The ID of the batch scrape job
#### Response
200
application/json
Successful cancellation
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete#response-message)
message
string
Example:
`"Batch scrape job successfully cancelled."`
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/batch-scrape-delete.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/batch-scrape-delete)
[Get Batch Scrape Status\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
[Get Batch Scrape Errors\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
⌘I
---
# Credit Usage - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Account Endpoints
Credit Usage
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get remaining credits for the authenticated team
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/credit-usage \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"data": {
"remainingCredits": 1000,
"planCredits": 500000,
"billingPeriodStart": "2025-01-01T00:00:00Z",
"billingPeriodEnd": "2025-01-31T23:59:59Z"
}
}
GET
/
team
/
credit-usage
Try it
Get remaining credits for the authenticated team
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/credit-usage \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"data": {
"remainingCredits": 1000,
"planCredits": 500000,
"billingPeriodStart": "2025-01-01T00:00:00Z",
"billingPeriodEnd": "2025-01-31T23:59:59Z"
}
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage#response-data)
data
object
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/credit-usage.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/credit-usage)
[Get Extract Status\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
[Historical Credit Usage\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
⌘I
---
# Historical Token Usage - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Account Endpoints
Historical Token Usage
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get historical token usage for the authenticated team (Extract only)
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/token-usage/historical \
--header 'Authorization: Bearer '
200
500
Copy
Ask AI
{
"success": true,
"periods": [\
{\
"startDate": "2025-01-01T00:00:00Z",\
"endDate": "2025-01-31T23:59:59Z",\
"apiKey": "",\
"totalTokens": 1000\
}\
]
}
GET
/
team
/
token-usage
/
historical
Try it
Get historical token usage for the authenticated team (Extract only)
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/token-usage/historical \
--header 'Authorization: Bearer '
200
500
Copy
Ask AI
{
"success": true,
"periods": [\
{\
"startDate": "2025-01-01T00:00:00Z",\
"endDate": "2025-01-31T23:59:59Z",\
"apiKey": "",\
"totalTokens": 1000\
}\
]
}
Returns historical token usage on a month-by-month basis. The endpoint can also breaks usage down by API key optionally.
We’ve simplified billing so that Extract now uses credits, just like all of the other endpoints. Each credit is worth 15 tokens. Reported token usage now includes usage from all endpoints.
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Query Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical#parameter-by-api-key)
byApiKey
boolean
default:false
Get historical token usage by API key
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical#response-periods)
periods
object\[\]
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/token-usage-historical.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/token-usage-historical)
[Token Usage\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
[Queue Status\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
⌘I
---
# Token Usage - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/token-usage#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Account Endpoints
Token Usage
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get remaining tokens for the authenticated team (Extract only)
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/token-usage \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"data": {
"remainingTokens": 1000,
"planTokens": 500000,
"billingPeriodStart": "2025-01-01T00:00:00Z",
"billingPeriodEnd": "2025-01-31T23:59:59Z"
}
}
GET
/
team
/
token-usage
Try it
Get remaining tokens for the authenticated team (Extract only)
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/team/token-usage \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"data": {
"remainingTokens": 1000,
"planTokens": 500000,
"billingPeriodStart": "2025-01-01T00:00:00Z",
"billingPeriodEnd": "2025-01-31T23:59:59Z"
}
}
We’ve simplified billing so that Extract now uses credits, just like all of the other endpoints. Each credit is worth 15 tokens. Reported token usage now includes usage from all endpoints.
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/endpoint/token-usage#response-data)
data
object
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/token-usage.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/token-usage)
[Historical Credit Usage\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
[Historical Token Usage\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
⌘I
---
# Get Crawl Status - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Get Crawl Status
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get the status of a crawl job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/crawl/{id} \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"status": "",
"total": 123,
"completed": 123,
"creditsUsed": 123,
"expiresAt": "2023-11-07T05:31:56Z",
"next": "",
"data": [\
{\
"markdown": "",\
"html": "",\
"rawHtml": "",\
"links": [\
""\
],\
"screenshot": "",\
"metadata": {\
"title": "",\
"description": "",\
"language": "",\
"sourceURL": "",\
"keywords": "",\
"ogLocaleAlternate": [\
""\
],\
" ": "",\
"statusCode": 123,\
"error": ""\
}\
}\
]
}
GET
/
crawl
/
{id}
Try it
Get the status of a crawl job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/crawl/{id} \
--header 'Authorization: Bearer '
200
402
429
500
Copy
Ask AI
{
"status": "",
"total": 123,
"completed": 123,
"creditsUsed": 123,
"expiresAt": "2023-11-07T05:31:56Z",
"next": "",
"data": [\
{\
"markdown": "",\
"html": "",\
"rawHtml": "",\
"links": [\
""\
],\
"screenshot": "",\
"metadata": {\
"title": "",\
"description": "",\
"language": "",\
"sourceURL": "",\
"keywords": "",\
"ogLocaleAlternate": [\
""\
],\
" ": "",\
"statusCode": 123,\
"error": ""\
}\
}\
]
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#parameter-id)
id
string
required
The ID of the crawl job
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-status)
status
string
The current status of the crawl. Can be `scraping`, `completed`, or `failed`.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-total)
total
integer
The total number of pages that were attempted to be crawled.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-completed)
completed
integer
The number of pages that have been successfully crawled.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-credits-used)
creditsUsed
integer
The number of credits used for the crawl.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-expires-at)
expiresAt
string
The date and time when the crawl will expire.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-next-one-of-0)
next
string | null
The URL to retrieve the next 10MB of data. Returned if the crawl is not completed or if the response is larger than 10MB.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get#response-data)
data
object\[\]
The data of the crawl.
Show child attributes
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/crawl-get.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/crawl-get)
[Crawl\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
[Crawl Params Preview\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
⌘I
---
# Get Extract Status - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Extract Endpoints
Get Extract Status
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Get the status of an extract job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/extract/{id} \
--header 'Authorization: Bearer '
200
Copy
Ask AI
{
"success": true,
"data": {},
"status": "completed",
"expiresAt": "2023-11-07T05:31:56Z",
"tokensUsed": 123
}
GET
/
extract
/
{id}
Try it
Get the status of an extract job
cURL
Copy
Ask AI
curl --request GET \
--url https://api.firecrawl.dev/v2/extract/{id} \
--header 'Authorization: Bearer '
200
Copy
Ask AI
{
"success": true,
"data": {},
"status": "completed",
"expiresAt": "2023-11-07T05:31:56Z",
"tokensUsed": 123
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#parameter-id)
id
string
required
The ID of the extract job
#### Response
200 - application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#response-success)
success
boolean
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#response-data)
data
object
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#response-status)
status
enum
The current status of the extract job
Available options:
`completed`,
`processing`,
`failed`,
`cancelled`
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#response-expires-at)
expiresAt
string
[](https://docs.firecrawl.dev/api-reference/endpoint/extract-get#response-tokens-used)
tokensUsed
integer
The number of tokens used by the extract job. Only available if the job is completed.
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/extract-get.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/extract-get)
[Extract\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/extract)
[Credit Usage\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
⌘I
---
# Crawl - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Crawl
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/api-reference/v2-introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/endpoint/search)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/endpoint/map)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
* [POST\
\
Crawl Params Preview](https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/endpoint/crawl-active)
##### Agent Endpoints
* [POST\
\
Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent)
* [GET\
\
Get Agent Status](https://docs.firecrawl.dev/api-reference/endpoint/agent-get)
* [DEL\
\
Cancel Agent](https://docs.firecrawl.dev/api-reference/endpoint/agent-delete)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/endpoint/queue-status)
Crawl multiple URLs based on options
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/crawl \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"url": "",
"prompt": "",
"excludePaths": [\
""\
],
"includePaths": [\
""\
],
"maxDiscoveryDepth": 123,
"sitemap": "include",
"ignoreQueryParameters": false,
"limit": 10000,
"crawlEntireDomain": false,
"allowExternalLinks": false,
"allowSubdomains": false,
"delay": 123,
"maxConcurrency": 123,
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"scrapeOptions": {
"formats": [\
"markdown"\
],
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 172800000,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": true,
"timeout": 30000,
"parsers": [\
"pdf"\
],
"actions": [\
{\
"type": "wait",\
"milliseconds": 2,\
"selector": "#my-element"\
}\
],
"location": {
"country": "US",
"languages": [\
"en-US"\
]
},
"removeBase64Images": true,
"blockAds": true,
"proxy": "auto",
"storeInCache": true
},
"zeroDataRetention": false
}
'
200
402
429
500
Copy
Ask AI
{
"success": true,
"id": "",
"url": ""
}
POST
/
crawl
Try it
Crawl multiple URLs based on options
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v2/crawl \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"url": "",
"prompt": "",
"excludePaths": [\
""\
],
"includePaths": [\
""\
],
"maxDiscoveryDepth": 123,
"sitemap": "include",
"ignoreQueryParameters": false,
"limit": 10000,
"crawlEntireDomain": false,
"allowExternalLinks": false,
"allowSubdomains": false,
"delay": 123,
"maxConcurrency": 123,
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"scrapeOptions": {
"formats": [\
"markdown"\
],
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 172800000,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": true,
"timeout": 30000,
"parsers": [\
"pdf"\
],
"actions": [\
{\
"type": "wait",\
"milliseconds": 2,\
"selector": "#my-element"\
}\
],
"location": {
"country": "US",
"languages": [\
"en-US"\
]
},
"removeBase64Images": true,
"blockAds": true,
"proxy": "auto",
"storeInCache": true
},
"zeroDataRetention": false
}
'
200
402
429
500
Copy
Ask AI
{
"success": true,
"id": "",
"url": ""
}
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#what%E2%80%99s-new-in-v2)
What’s New in v2
--------------------------------------------------------------------------------------------------------------
###
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#tell-crawl-what-you-want)
Tell crawl what you want
Describe what to crawl in plain English:
Copy
Ask AI
{
"url": "https://example.com",
"prompt": "Only crawl blog posts and docs, skip marketing pages"
}
This will map the prompt to a set of crawler settings to perform the crawl.
###
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#improved-sitemap-control)
Improved Sitemap Control
In v1, sitemap usage was a boolean. In v2, the `sitemap` option lets you choose:
* `"include"` (default): Use the sitemap and also discover other pages.
* `"skip"`: Ignore the sitemap entirely.
###
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#new-crawling-options)
New Crawling Options
* `crawlEntireDomain` - Crawl whole domain, not just child pages
* `maxDiscoveryDepth` - Control crawl depth (replaces `maxDepth`)
Copy
Ask AI
{
"url": "https://example.com/features",
"crawlEntireDomain": true,
"maxDiscoveryDepth": 2,
"sitemap": "include"
}
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Body
application/json
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-url)
url
string
required
The base URL to start crawling from
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-prompt)
prompt
string
A prompt to use to generate the crawler options (all the parameters below) from natural language. Explicitly set parameters will override the generated equivalents.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-exclude-paths)
excludePaths
string\[\]
URL pathname regex patterns that exclude matching URLs from the crawl. For example, if you set "excludePaths": \["blog/.\*"\] for the base URL firecrawl.dev, any results matching that pattern will be excluded, such as [https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap](https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap)
.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-include-paths)
includePaths
string\[\]
URL pathname regex patterns that include matching URLs in the crawl. Only the paths that match the specified patterns will be included in the response. For example, if you set "includePaths": \["blog/.\*"\] for the base URL firecrawl.dev, only results matching that pattern will be included, such as [https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap](https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap)
.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-max-discovery-depth)
maxDiscoveryDepth
integer
Maximum depth to crawl based on discovery order. The root site and sitemapped pages has a discovery depth of 0. For example, if you set it to 1, and you set `sitemap: 'skip'`, you will only crawl the entered URL and all URLs that are linked on that page.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-sitemap)
sitemap
enum
default:include
Sitemap mode when crawling. If you set it to 'skip', the crawler will ignore the website sitemap and only crawl the entered URL and discover pages from there onwards.
Available options:
`skip`,
`include`
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-ignore-query-parameters)
ignoreQueryParameters
boolean
default:false
Do not re-scrape the same path with different (or none) query parameters
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-limit)
limit
integer
default:10000
Maximum number of pages to crawl. Default limit is 10000.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-crawl-entire-domain)
crawlEntireDomain
boolean
default:false
Allows the crawler to follow internal links to sibling or parent URLs, not just child paths.
false: Only crawls deeper (child) URLs. → e.g. /features/feature-1 → /features/feature-1/tips ✅ → Won't follow /pricing or / ❌
true: Crawls any internal links, including siblings and parents. → e.g. /features/feature-1 → /pricing, /, etc. ✅
Use true for broader internal coverage beyond nested paths.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-allow-external-links)
allowExternalLinks
boolean
default:false
Allows the crawler to follow links to external websites.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-allow-subdomains)
allowSubdomains
boolean
default:false
Allows the crawler to follow links to subdomains of the main domain.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-delay)
delay
number
Delay in seconds between scrapes. This helps respect website rate limits.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-max-concurrency)
maxConcurrency
integer
Maximum number of concurrent scrapes. This parameter allows you to set a concurrency limit for this crawl. If not specified, the crawl adheres to your team's concurrency limit.
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-webhook)
webhook
object
A webhook specification object.
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-scrape-options)
scrapeOptions
object
Show child attributes
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#body-zero-data-retention)
zeroDataRetention
boolean
default:false
If true, this will enable zero data retention for this crawl. To enable this feature, please contact [\[email protected\]](https://docs.firecrawl.dev/cdn-cgi/l/email-protection#6109040d112107081304021300160d4f050417)
#### Response
200
application/json
Successful response
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#response-success)
success
boolean
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#response-id)
id
string
[](https://docs.firecrawl.dev/api-reference/endpoint/crawl-post#response-url)
url
string
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/endpoint/crawl-post.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/endpoint/crawl-post)
[Map\
\
Previous](https://docs.firecrawl.dev/api-reference/endpoint/map)
[Get Crawl Status\
\
Next](https://docs.firecrawl.dev/api-reference/endpoint/crawl-get)
⌘I
---
# Cancel Batch Scrape - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v1
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Cancel Batch Scrape
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/v1/api-reference/introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/v1/api-reference/introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get-errors)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-active)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/v1-endpoint/map)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/v1-endpoint/search)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/queue-status)
Cancel a batch scrape job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v1/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"message": "Batch scrape job successfully cancelled."
}
DELETE
/
batch
/
scrape
/
{id}
Try it
Cancel a batch scrape job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v1/batch/scrape/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"success": true,
"message": "Batch scrape job successfully cancelled."
}
> Note: A new [v2 version of this API](https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete)
> is now available with improved features and performance.
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete#parameter-id)
id
string
required
The ID of the batch scrape job
#### Response
200
application/json
Successful cancellation
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete#response-success)
success
boolean
Example:
`true`
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete#response-message)
message
string
Example:
`"Batch scrape job successfully cancelled."`
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/v1-endpoint/batch-scrape-delete.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/v1-endpoint/batch-scrape-delete)
[Get Batch Scrape Status\
\
Previous](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get)
[Get Batch Scrape Errors\
\
Next](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get-errors)
⌘I
---
# Open Source vs Cloud | Firecrawl
[Skip to main content](https://docs.firecrawl.dev/contributing/open-source-or-cloud#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v2
English
Search...
⌘KAsk AI
Search...
Navigation
Contributing
Open Source vs Cloud
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Get Started
* [Quickstart](https://docs.firecrawl.dev/introduction)
* [MCP Server](https://docs.firecrawl.dev/mcp-server)
* [Migrating from v1 to v2](https://docs.firecrawl.dev/migrate-to-v2)
* [Rate Limits](https://docs.firecrawl.dev/rate-limits)
* [Advanced Scraping Guide](https://docs.firecrawl.dev/advanced-scraping-guide)
##### New Features
* Agent
##### Standard Features
* Scrape
* [Search](https://docs.firecrawl.dev/features/search)
* [Map](https://docs.firecrawl.dev/features/map)
* [Crawl](https://docs.firecrawl.dev/features/crawl)
##### Webhooks
* [Overview](https://docs.firecrawl.dev/webhooks/overview)
* [Event Types](https://docs.firecrawl.dev/webhooks/events)
* [Security](https://docs.firecrawl.dev/webhooks/security)
* [Testing](https://docs.firecrawl.dev/webhooks/testing)
##### Developer Guides
* [Full-Stack Templates](https://docs.firecrawl.dev/developer-guides/examples)
* LLM SDKs and Frameworks
* Cookbooks
* MCP Setup Guides
* Common Sites
* Workflow Automation
* Advanced Guides
##### Use Cases
* [Overview](https://docs.firecrawl.dev/use-cases/overview)
* [AI Platforms](https://docs.firecrawl.dev/use-cases/ai-platforms)
* [Lead Enrichment](https://docs.firecrawl.dev/use-cases/lead-enrichment)
* [SEO Platforms](https://docs.firecrawl.dev/use-cases/seo-platforms)
* [Deep Research](https://docs.firecrawl.dev/use-cases/deep-research)
* View more
##### Contributing
* [Open Source vs Cloud](https://docs.firecrawl.dev/contributing/open-source-or-cloud)
* [Running locally](https://docs.firecrawl.dev/contributing/guide)
* [Self-hosting](https://docs.firecrawl.dev/contributing/self-host)
Firecrawl is open source available under the [AGPL-3.0 license](https://github.com/mendableai/firecrawl/blob/main/LICENSE)
. To deliver the best possible product, we offer a hosted version of Firecrawl alongside our open-source offering. The cloud solution allows us to continuously innovate and maintain a high-quality, sustainable service for all users. Firecrawl Cloud is available at [firecrawl.dev](https://firecrawl.dev/)
and offers a range of features that are not available in the open source version: 
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/contributing/open-source-or-cloud.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/contributing/open-source-or-cloud)
[Observability & Monitoring\
\
Previous](https://docs.firecrawl.dev/use-cases/observability)
[Running locally\
\
Next](https://docs.firecrawl.dev/contributing/guide)
⌘I
---
# Cancel Crawl - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v1
English
Search...
⌘KAsk AI
Search...
Navigation
Crawl Endpoints
Cancel Crawl
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/v1/api-reference/introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/v1/api-reference/introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get-errors)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-active)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/v1-endpoint/map)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/v1-endpoint/search)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/queue-status)
Cancel a crawl job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v1/crawl/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"status": "cancelled"
}
DELETE
/
crawl
/
{id}
Try it
Cancel a crawl job
cURL
Copy
Ask AI
curl --request DELETE \
--url https://api.firecrawl.dev/v1/crawl/{id} \
--header 'Authorization: Bearer '
200
404
500
Copy
Ask AI
{
"status": "cancelled"
}
> Note: A new [v2 version of this API](https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete)
> is now available with improved features and performance.
#### Authorizations
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete#authorization-authorization)
Authorization
string
header
required
Bearer authentication header of the form `Bearer `, where `` is your auth token.
#### Path Parameters
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete#parameter-id)
id
string
required
The ID of the crawl job
#### Response
200
application/json
Successful cancellation
[](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete#response-status)
status
enum
Available options:
`cancelled`
Example:
`"cancelled"`
[Suggest edits](https://github.com/firecrawl/firecrawl-docs/edit/main/api-reference/v1-endpoint/crawl-delete.mdx)
[Raise issue](https://github.com/firecrawl/firecrawl-docs/issues/new?title=Issue%20on%20docs&body=Path:%20/api-reference/v1-endpoint/crawl-delete)
[Get Crawl Status\
\
Previous](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get)
[Get Crawl Errors\
\
Next](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get-errors)
⌘I
---
# Batch Scrape - Firecrawl Docs
[Skip to main content](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape#content-area)
[Firecrawl Docs home page](https://firecrawl.dev/)
v1
English
Search...
⌘KAsk AI
Search...
Navigation
Scrape Endpoints
Batch Scrape
[Documentation](https://docs.firecrawl.dev/introduction)
[SDKs](https://docs.firecrawl.dev/sdks/overview)
[Integrations](https://www.firecrawl.dev/app)
[API Reference](https://docs.firecrawl.dev/v1/api-reference/introduction)
* [Playground](https://firecrawl.dev/playground)
* [Blog](https://firecrawl.dev/blog)
* [Community](https://discord.gg/gSmWdAkdwd)
* [Changelog](https://firecrawl.dev/changelog)
##### Using the API
* [Introduction](https://docs.firecrawl.dev/v1/api-reference/introduction)
##### Scrape Endpoints
* [POST\
\
Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/scrape)
* [POST\
\
Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape)
* [GET\
\
Get Batch Scrape Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get)
* [DEL\
\
Cancel Batch Scrape](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-delete)
* [GET\
\
Get Batch Scrape Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/batch-scrape-get-errors)
##### Crawl Endpoints
* [POST\
\
Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-post)
* [GET\
\
Get Crawl Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get)
* [DEL\
\
Cancel Crawl](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-delete)
* [GET\
\
Get Crawl Errors](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-get-errors)
* [GET\
\
Get Active Crawls](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-active)
##### Map Endpoints
* [POST\
\
Map](https://docs.firecrawl.dev/api-reference/v1-endpoint/map)
##### Search Endpoints
* [POST\
\
Search](https://docs.firecrawl.dev/api-reference/v1-endpoint/search)
##### Extract Endpoints
* [POST\
\
Extract](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract)
* [GET\
\
Get Extract Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/extract-get)
##### Account Endpoints
* [GET\
\
Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage)
* [GET\
\
Historical Credit Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/credit-usage-historical)
* [GET\
\
Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage)
* [GET\
\
Historical Token Usage](https://docs.firecrawl.dev/api-reference/v1-endpoint/token-usage-historical)
* [GET\
\
Queue Status](https://docs.firecrawl.dev/api-reference/v1-endpoint/queue-status)
Scrape multiple URLs and optionally extract information using an LLM
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v1/batch/scrape \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"urls": [\
""\
],
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"maxConcurrency": 123,
"ignoreInvalidURLs": false,
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 0,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": false,
"timeout": 30000,
"parsePDF": true,
"jsonOptions": {
"schema": {},
"systemPrompt": "",
"prompt": ""
},
"actions": [\
{\
"type": "wait",\
"milliseconds": 2,\
"selector": "#my-element"\
}\
],
"location": {
"country": "US",
"languages": [\
"en-US"\
]
},
"removeBase64Images": true,
"blockAds": true,
"proxy": "basic",
"storeInCache": true,
"formats": [\
"markdown"\
],
"changeTrackingOptions": {
"modes": [\
"git-diff"\
],
"schema": {},
"prompt": "",
"tag": null
},
"zeroDataRetention": false
}
'
200
402
429
500
Copy
Ask AI
{
"success": true,
"id": "",
"url": "",
"invalidURLs": [\
""\
]
}
POST
/
batch
/
scrape
Try it
Scrape multiple URLs and optionally extract information using an LLM
cURL
Copy
Ask AI
curl --request POST \
--url https://api.firecrawl.dev/v1/batch/scrape \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '
{
"urls": [\
""\
],
"webhook": {
"url": "",
"headers": {},
"metadata": {},
"events": [\
"completed"\
]
},
"maxConcurrency": 123,
"ignoreInvalidURLs": false,
"onlyMainContent": true,
"includeTags": [\
""\
],
"excludeTags": [\
""\
],
"maxAge": 0,
"headers": {},
"waitFor": 0,
"mobile": false,
"skipTlsVerification": false,
"timeout": 30000,
"parsePDF": true,
"jsonOptions": {
"schema": {},
"systemPrompt": "",
"prompt": "