Quick Start
Example Request
Example Response
How it works
1
You provide a starting URL
Give Map the URL of the website you want to explore
2
Map discovers URLs
- Reads the website’s sitemap (if available) - Analyzes page links and navigation - Identifies all discoverable URLs on the site
3
Collects metadata for each URL
- Extracts page titles from sitemap or meta tags - Gathers descriptions when available - Associates context with each discovered URL
4
Returns structured URL list
Get a complete list of URLs with titles and descriptions ready for AI
reasoning or crawling
Parameters
Supported input parameters:url - Required
url - Required
sitemap
sitemap
domain_filter
domain_filter
country
country
Usage
Basic map
Map a website using default settings:Sitemap-only mapping
Fast mapping using only the sitemap:Skip sitemap
Map without using sitemap, discovering URLs through page crawling:Include subdomains
Map URLs across all subdomains:Exact domain
Restrict mapping to exact domain only:Geo-targeted mapping
Map with specific country and locale settings:Combined parameters
Map with multiple parameters for precise control:Response Fields
When you use Map, you receive:- URLs with context - Each URL includes optional title and description for AI reasoning
- Complete discovery - Every discoverable page on the site
- Fast results - Most sites mapped in seconds
- Smart filtering - Subdomain control and comprehensive sitemap + link discovery
Example Response
Example Response
Use cases
AI Agent Planning
Give AI agents URL context (titles, descriptions) to intelligently decide
which pages to scrape
Pre-crawl Planning
Discover all URLs before crawling to plan budget and prioritize important
pages
Site Audits
Quick inventory of all pages for SEO audits, content analysis, or quality
checks
Competitive Research
See competitor site structure - product pages, blog posts, landing pages
Real-world examples
AI agent intelligent scraping
AI agent intelligent scraping
Scenario: Your AI agent needs to extract product data from an e-commerce site but should only scrape relevant product pages.How Map helps:
- Discovers all URLs with titles and descriptions
- AI agent reads the context (title: “Laptop Pro 15”, description: “High-performance laptop…”)
- Agent intelligently decides which URLs contain product data vs navigation/legal pages
- Only scrapes relevant product URLs, saving costs and time
E-commerce competitor analysis
E-commerce competitor analysis
Scenario: You want to understand a competitor’s product catalog structure.How Map helps:
- Discover all product category pages
- Find hidden product pages not linked from the homepage
- Identify promotional landing pages
- Map out the entire site structure before detailed scraping
Content research and monitoring
Content research and monitoring
Scenario: You’re tracking content strategy across multiple news sites.How Map helps:
- Get a complete list of articles and sections
- Discover new content categories as they’re added
- Identify archive structures and content organization
- Plan targeted crawling for specific content types
SEO and site structure analysis
SEO and site structure analysis
Scenario: Analyzing website architecture for SEO optimization.How Map helps:
- Visualize complete site hierarchy
- Identify orphaned pages (pages not well-linked)
- Discover deep pages buried in site structure
- Understand internal linking patterns
Pre-crawl planning
Pre-crawl planning
Scenario: You need to scrape data from a large website efficiently.How Map helps:
- Get complete URL inventory before crawling
- Filter for specific sections or page types
- Plan crawl budget and prioritization
- Avoid wasting resources on irrelevant pages
Map vs Crawl
Next steps
API Reference
Explore endpoints, request parameters, and response schemas for the Map API