MCP Web Scraping — Plug Real-World Data into Claude, Cursor & Any AI Agent
Give your AI assistant direct access to 80+ web scraping and data extraction APIs via the Model Context Protocol (MCP). Search Google, scrape Google Maps, monitor reviews, fetch YouTube transcripts, and qualify leads from Claude Desktop, Cursor, Codex, OpenCode, Gemini CLI, VS Code, or Windsurf.
MCP server URL
Streamable-HTTP transport. Authenticate with the X-API-KEY header from your Scrappa dashboard.
https://scrappa.co/mcp
MCP scraping hub
Start with the integration guide, then move into client-specific setup and endpoint walkthroughs for Claude Desktop, Cursor, Google Maps, Google Search, YouTube, and agent use cases.
Maps
Google Maps scraper
simple-search
Find places, enrich business profiles, and pull review data for lead generation and local market research agents.
Search
Google Search scraper
search
Give Claude, Cursor, and other AI agents structured SERP results for rank tracking, research, and competitor monitoring.
Social
YouTube scraper
youtube-transcript
Search videos, fetch metadata, collect comments, and retrieve transcripts for content research workflows.
Client setup
Claude Desktop setup
mcp-remote
Copy-paste the mcp-remote configuration and connect Claude Desktop to Scrappa in under a minute.
Client setup
Cursor MCP setup
cursor
Add Scrappa to Cursor so agent mode can call web scraping tools while you work in the IDE.
Use cases
Agent use cases
agent-workflows
Build lead generation, review monitoring, real estate research, and SERP tracking agents with live web data.
What is MCP?
The Model Context Protocol (MCP) is an open standard that allows AI assistants to connect to external tools and data sources. Scrappa's web scraping MCP server is a single managed remote endpoint — point any MCP-compatible client at the Scrappa MCP URL above, pass your API key, and your AI can:
Supported Clients
Claude Desktop
Anthropic's official desktop app
Claude Code
Terminal-based AI coding assistant
Codex
OpenAI coding agent in the CLI and IDE
OpenCode
Open-source terminal AI coding agent
Gemini CLI
Google's command-line AI assistant
Cursor
AI-powered code editor
VS Code
GitHub Copilot Agent mode
Windsurf
AI-powered code editor
Prerequisites
Get Your API Key
Sign up at scrappa.co and get your API key from the dashboard.
Install Node.js for Bridge-Based Clients
Node.js 18+ is only needed for clients that use mcp-remote: Claude Desktop, Cursor, VS Code, and Windsurf. Codex, OpenCode, and Gemini CLI connect through native remote MCP configuration.
Required before install
Get the API key you will paste into MCP
Create a free Scrappa account, copy the dashboard key, and use it anywhere this guide shows YOUR_SCRAPPA_API_KEY.
Optional sanity check
Test a live request before wiring your client
Open the public playground, run a Google Search or Maps request, and confirm the JSON shape your AI agent will receive through MCP.
Quick Install
Using Claude Code or Gemini CLI? Run the matching command:
Claude Code
claude mcp add scrappa --transport http https://scrappa.co/mcp -H 'X-API-KEY: YOUR_SCRAPPA_API_KEY'Gemini CLI
gemini mcp add --transport http --header 'X-API-KEY: YOUR_SCRAPPA_API_KEY' scrappa https://scrappa.co/mcpDone! Restart your AI client and start using Scrappa tools.
Using a different tool? See the detailed instructions below.
Not ready to edit your MCP config yet?
Run the same Scrappa endpoints in the playground first, then paste your API key into Claude, Codex, Gemini CLI, Cursor, VS Code, or Windsurf.
Detailed Installation
Choose your AI tool and follow the installation instructions.
Add the following to your Claude Desktop configuration file:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"scrappa": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
}
}
}Run this single command in your terminal:
claude mcp add scrappa --transport http https://scrappa.co/mcp -H 'X-API-KEY: YOUR_SCRAPPA_API_KEY'That's it! Restart Claude Code and you're ready to go.
Or add it to your project's .mcp.json file:
{
"mcpServers": {
"scrappa": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
}
}
}Add to your Codex configuration file:
File: ~/.codex/config.toml
[mcp_servers.scrappa]
url = "https://scrappa.co/mcp"
http_headers = { "X-API-KEY" = "YOUR_SCRAPPA_API_KEY" }Add to your OpenCode configuration file:
File: opencode.json in your project root, or ~/.config/opencode/opencode.json for global setup.
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"scrappa": {
"type": "remote",
"url": "https://scrappa.co/mcp",
"enabled": true,
"oauth": false,
"headers": {
"X-API-KEY": "YOUR_SCRAPPA_API_KEY"
}
}
}
}Run this command in your terminal:
gemini mcp add --transport http --header 'X-API-KEY: YOUR_SCRAPPA_API_KEY' scrappa https://scrappa.co/mcp
Or add it to ~/.gemini/settings.json:
{
"mcpServers": {
"scrappa": {
"httpUrl": "https://scrappa.co/mcp",
"headers": {
"X-API-KEY": "YOUR_SCRAPPA_API_KEY"
}
}
}
}Add to your Cursor MCP configuration:
File: ~/.cursor/mcp.json
{
"mcpServers": {
"scrappa": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
}
}
}
Add to your VS Code settings (settings.json):
{
"mcp": {
"servers": {
"scrappa": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
}
}
}
}Add to your Windsurf MCP configuration file:
File: ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"scrappa": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
}
}
}Available Tools
The Scrappa MCP server provides the following tools to your AI assistant:
G Google Search, Maps, Flights & Hotels
search
- Google web SERPs
google-search-light
- Lightweight web search
simple-search
- Google Maps places (simple search)
google-reviews
- Google Maps reviews
google-business-details
- Maps business profile
google-flights-one-way
- One-way flight search
google-flights-round-trip
- Round-trip flights
google-hotels-search
- Hotel listings & prices
google-jobs
- Jobs SERPs
google-news
- Google News results
google-translate-api
- Translate text
google-images
- Search images
google-videos
- Search videos
★ Reviews & Reputation
trustpilot-company-reviews
- Trustpilot ratings & reviews
trustpilot-company-details
- Trustpilot company profile
kununu-reviews
- Kununu employer reviews
trustedshops-reviews
- TrustedShops reviews
jameda-reviews
- Jameda doctor reviews
in Professional & Marketplace
linkedin-profile
- LinkedIn profile data
linkedin-company
- LinkedIn company pages
linkedin-post
- LinkedIn post details
indeed-jobs
- Indeed job listings
vinted-search
- Vinted product search
brave-search
- Brave search results
YT YouTube
youtube-search
- Search videos
youtube-video
- Get video details
youtube-channel
- Channel profile
youtube-comments
- Fetch comments
youtube-transcript
- Transcript segments
youtube-related
- Related videos
youtube-chapters
- Video chapters
youtube-trending
- Trending videos
youtube-suggestions
- Search suggestions
youtube-channel-videos
- Channel uploads
youtube-channel-playlists
- Channel playlists
youtube-channel-community
- Community posts
Usage Examples
Once Scrappa is connected, just ask your AI assistant naturally. These prompts are real web scraping workflows people run today through Claude, Codex, Cursor, and Gemini CLI:
Code examples in TypeScript and Python
MCP is the fastest path for AI agents, but every tool is backed by the same REST API. Use direct calls when you need a scheduled job, backend enrichment pipeline, or test fixture outside the AI client.
TypeScript
const response = await fetch('https://api.scrappa.co/google-maps/simple-search?query=coffee%20shops%20Austin', {
headers: { 'X-API-KEY': process.env.SCRAPPA_API_KEY! }
});
const places = await response.json();
console.log(places);Python
import os
import requests
response = requests.get(
'https://api.scrappa.co/google/search',
params={'query': 'best mcp scraper'},
headers={'X-API-KEY': os.environ['SCRAPPA_API_KEY']},
)
print(response.json())Pricing for MCP usage
MCP calls use the same Scrappa credits as REST API calls. There is no separate MCP plan, no agent-seat minimum, and every account can start with 500 free monthly credits before moving to credit packs or subscriptions on the pricing page.
Scrappa MCP vs alternatives
| Provider | MCP focus | Best fit |
|---|---|---|
| Scrappa | 80+ structured scraping APIs exposed as MCP tools | Agents that need search, maps, reviews, jobs, social, and marketplace data |
| Firecrawl | Website crawl and page extraction MCP | Markdown extraction from known websites |
| Apify | Actor ecosystem via MCP | Teams already maintaining Apify actors |
| Bright Data | Enterprise data collection tooling | Large proxy-heavy operations with enterprise procurement |
Get Your API Key
Sign up for a free Scrappa account to get your API key and start using the MCP integration with your AI assistant.
Get Started FreeRelated resources
Pair the MCP setup guide with the main docs, pricing overview, server comparison, and full API reference while you evaluate production use.
Homepage
Return to the main product overview and marketing entry points.
Quick start docs
Start with account setup, first requests, and core documentation.
Pricing
Check credits, savings, and alternative-page comparisons before rollout.
Best MCP servers
Compare Scrappa with Firecrawl, Apify, Bright Data, and other web scraping MCP options.
API reference
Open the full reference and OpenAPI docs for endpoint-level detail.
Web scraping MCP server troubleshooting
If you are evaluating a web scraping MCP server for Claude or another AI client, the Scrappa MCP URL https://scrappa.co/mcp is a single managed endpoint that fronts every Scrappa scraping API. These are the issues that come up most often during setup.
The MCP server URL must be exact
Use https://scrappa.co/mcp with no trailing slash, no /v1, and no extra path segments. Native remote MCP clients reject mismatched URLs silently and fall back to "no tools".
X-API-KEY header formatting
The header name is X-API-KEY (uppercase). Pass the raw key value with no "Bearer" prefix and no surrounding quotes. Bridge-based clients pass it through --header "X-API-KEY: YOUR_KEY".
mcp-remote bridge requires Node.js 18+
Claude Desktop, Cursor, VS Code, and Windsurf launch the npx mcp-remote bridge against the Scrappa MCP URL. Make sure node --version reports 18 or higher and that npx is on PATH for the user the AI client runs as.
Restart the AI client after editing config
MCP servers are loaded once per client session. After updating claude_desktop_config.json, ~/.codex/config.toml, opencode.json, or ~/.cursor/mcp.json, fully quit and relaunch the client — reloading the window is not enough.
Verify the server is registered
Run "claude mcp list" or "gemini mcp list" to confirm scrappa shows up. Inside the chat, asking "list your tools" should return Scrappa endpoints such as search, trustpilot-company-reviews, linkedin-profile, and youtube-transcript.
Frequently Asked Questions
What is an MCP server and how does it work with AI clients?
MCP (Model Context Protocol) is an open standard that lets AI assistants connect to external tools and data sources. Scrappa's MCP server exposes 80+ web scraping APIs as tools that Claude, Codex, OpenCode, Gemini CLI, Cursor, VS Code, and Windsurf can call directly during conversations.
What is the Scrappa MCP server URL?
The Scrappa MCP server URL is https://scrappa.co/mcp. It is a remote streamable-HTTP MCP endpoint authenticated with the X-API-KEY header from your Scrappa dashboard. Native remote MCP clients (Claude Code, Codex, OpenCode, Gemini CLI) connect to this URL directly, while Claude Desktop, Cursor, VS Code, and Windsurf reach it through the mcp-remote bridge.
Which AI clients support Scrappa's MCP server?
Scrappa's MCP server works with Claude Desktop, Claude Code, Codex, OpenCode, Gemini CLI, Cursor, VS Code with GitHub Copilot, and Windsurf. Any client that supports the Model Context Protocol can connect to Scrappa.
Do I need to install anything to use Scrappa through MCP?
Claude Code and Gemini CLI can add Scrappa with one command. Claude Desktop, Cursor, VS Code, and Windsurf use JSON settings, Codex uses TOML, and OpenCode uses opencode.json. Node.js 18+ is required only for clients that use the mcp-remote bridge.
How many APIs are available through the MCP server?
Scrappa provides 80+ API endpoints across Google Maps, Google Search, YouTube, LinkedIn, Trustpilot, Google Flights, Google Hotels, Indeed, Brave Search, and more. All endpoints are automatically available as MCP tools, including the latest YouTube transcript endpoint added in April 2026.
Can I use Scrappa's MCP server for web scraping in Claude?
Yes. The Scrappa MCP server is purpose-built for web scraping inside Claude, Claude Code, Codex, Cursor, and other MCP clients. Once connected, Claude can search Google, fetch Google Maps reviews, pull Trustpilot or Kununu ratings, scrape LinkedIn profiles, retrieve YouTube transcripts, and translate text — all without leaving the chat window.
Can I use Scrappa's MCP server for free?
You can sign up for a free Scrappa account to get an API key. The MCP server uses the same API credits as the REST API, including the 500 free monthly credits on every account. Check the pricing page for paid plans and credit packs once you outgrow the free tier.
How do I troubleshoot an MCP server that will not connect?
Most issues come from header formatting or stale clients. Confirm the MCP server URL is exactly https://scrappa.co/mcp, that the X-API-KEY header is unquoted on a single line, and that your client has been fully restarted after editing the config. Run "claude mcp list" or "gemini mcp list" to verify the server is registered, and check that mcp-remote-using clients have Node.js 18+ on PATH.
Need Help?
Check out our authentication docs for more details on API keys, or visit the API playground to test endpoints directly.
MCP integration guide last updated .