SerpScrap

SerpScrap retrieves structured search results through a Python package, a command-line interface, a Docker application, and an MCP-compatible gateway. The current release line is 2.0.0-alpha.2.

This documentation has two audiences:

• Users run searches with Python, the CLI, the Docker UI/API, or the MCP server.

• Developers extend providers, services, parsers, tests, and deployment files.

Start here

Choose the interface that matches your workflow:

• Install — install the package, Chrome, and development dependencies.

• Examples — complete Python and CLI examples.

• Command-line use — all important CLI commands and options.

• Docker application — run the API, UI, PostgreSQL, MCP, SearXNG, and Valkey services with Compose.

• MCP server — connect an MCP client and call search/history/configuration tools.

• Result Data — understand result rows, fusion, failures, and JSON output.

• Configuration — configure engines, markets, concurrency, caching, and diagnostics.

User guide

• 1. Install
  • 1.1. ChromeDriver
  • 1.2. Development
  • 1.3. Consent and live provider smoke tests
• 2. Examples
  • 2.1. Python package
    • 2.1.1. Basic search
    • 2.1.2. Multiple queries, pages, and engines
    • 2.1.3. Local SearXNG
    • 2.1.4. Vertical searches
    • 2.1.5. JSON files and URL enrichment
    • 2.1.6. Failures and related searches
    • 2.1.7. Configuration and diagnostics
  • 2.2. Command line
  • 2.3. MCP server
• 3. Command-line use
  • 3.1. Global options
  • 3.2. Search
  • 3.3. Browser check
• 4. Result Data
  • 4.1. Local JSON
  • 4.2. Failures and related keywords
  • 4.3. URL enrichment
  • 4.4. serp_type
• 5. Configuration
  • 5.1. Core settings
  • 5.2. Example
  • 5.3. SearXNG selection
  • 5.4. Proxy sources
  • 5.5. Request policy
• 6. Docker application
  • 6.1. Quick start
  • 6.2. Configuration
  • 6.3. Health and operations
  • 6.4. Persistent data and backups
  • 6.5. Deployment boundaries
• 7. MCP server
  • 7.1. Start the server
  • 7.2. Security and trust boundary
  • 7.3. Configurations for popular model clients
    • 7.3.1. Claude Code
    • 7.3.2. Cursor and other JSON-configured clients
    • 7.3.3. OpenAI Responses API
    • 7.3.4. Anthropic Messages API
  • 7.4. Protocol flow
  • 7.5. Available tools
  • 7.6. Example search call
• 8. Search engines
  • 8.1. Active defaults
    • 8.1.1. SearXNG sources without API keys
  • 8.2. Selecting engines
  • 8.3. Provider safety
  • 8.4. Adding a provider

Developer guide

• Development
  • Repository layout
  • Local environment
  • Validation
  • Consent-flow verification
  • Architecture
  • Documentation changes
  • UI search-result contract
  • Release preparation

Version and support

The alpha release is suitable for evaluation and development. Provider pages can change, access controls can produce partial failures, and live browser checks require Chrome and network access. SerpScrap never bypasses CAPTCHAs, rate limits, consent decisions, or access controls.

Source code and issue tracking are available on GitHub.