Multi-engine web + image search with proxy distribution, circuit breaker, REST API, and metrics
Find a file
amancca 1b56f32c72 Add MCP JSON-RPC endpoint for web_search and image_search tools
Introduces an HTTP-based MCP handler (POST /mcp) that exposes polysearch's
search capabilities to AI agents via the Model Context Protocol. Supports
tools/list, tools/call (web_search, image_search), and initialize methods.

The endpoint shares the existing SearchRunner, proxy pool, and Bearer auth,
so no additional infrastructure or credentials are needed.

💘 Generated with Crush

Assisted-by: Crush:deepseek-v4-flash-free
2026-06-12 04:19:35 +03:00
src Add MCP JSON-RPC endpoint for web_search and image_search tools 2026-06-12 04:19:35 +03:00
.env.example feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
.gitignore feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
API.md feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
benchmark.js feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
config.example.json feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
package-lock.json feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
package.json feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00
README.md feat: initial polysearch - multi-engine search with proxy distribution, circuit breaker, REST API, and metrics 2026-06-05 14:39:11 +03:00

PolySearch

Multi-engine web + image search with smart proxy distribution, circuit breakers, structured AI agent output, and a REST API.

node src/index.js -q "quantum computing" -t both -l 10 -m agent

Features

  • Web + Image search — both result types, one tool
  • Smart proxy distribution — least-used selection per hour, balanced across providers
  • Circuit breaker per proxy — exponential backoff on failure, auto-recovery
  • Multi-provider proxy system — add Webshare, Oxylabs, BrightData in one file each
  • Multi-engine architecture — add Brave, Bing, Google in one file each
  • Per-provider metrics — requests, success rate, latency, hourly usage grouped by provider
  • Dual output modes:
    • human — colorized terminal
    • agent — structured JSON with statistics
  • REST API — single search, batch search, auth with API keys. See API.md

Requirements

Node.js 18+

Quick start

# Single image search
node src/index.js -q "vintage radio"

# Web search
node src/index.js -q "quantum computing" -t web

# Both types, AI agent JSON
node src/index.js -q "spacex starship" -t both -l 10 -m agent

# Show proxy metrics after a search
node src/index.js -q "mars rover" -M

CLI

Flag Long Description Default
-q --query Search query
-t --type web, image, or both image
-l --limit Max results per type 10
-m --mode human or agent human
-p --proxy Single proxy URL override
-c --config Path to config file auto-detect
-M --metrics Dump proxy pool metrics
--serve Start REST API server
--port API server port 9876
--generate-key Generate API key
-h --help Show help

REST API

For AI agent consumption. See API.md for full documentation.

node src/index.js --generate-key   # create an API key
node src/index.js --serve --port 9876  # start the server

Endpoints: GET /health, POST /search, POST /batch, GET /metrics


CLI

Providers are auto-discovered from environment variables:

Provider Env vars Type
Webshare WEBSHARE_API_KEY API-fetched, 10 rotating IPs
Oxylabs OXYLABS_USERNAME, OXYLABS_PASSWORD, OXYLABS_COUNTRY Single datacenter endpoint

Add a new provider by creating a file in src/http/providers/ that calls registerProvider(name, fetcher). The fetcher returns an array of proxy URL strings.

Engine architecture

Engines are registered in src/engines/setup.js. Each engine supports web, image, or both. DuckDuckGo is the default. Add Brave, Bing, or custom engines by implementing the search(query, opts) interface.

Project structure

src/
├── index.js              # CLI + programmatic API + API server dispatch
├── api.js                # REST API server (/search, /batch, /metrics, /health)
├── api-key.js            # Key generation + env storage
├── cli.js                # Argument parsing
├── config.js             # Config loader (json + env + providers)
├── run.js                # Search orchestration + engine fallback
├── engines/
│   ├── base.js           # Abstract engine interface
│   ├── index.js          # Engine registry
│   ├── setup.js          # Built-in engine registration
│   └── duckduckgo.js     # DuckDuckGo (web + image)
├── http/
│   ├── client.js         # Fetch wrapper (proxy, retry, timeout, UA)
│   ├── proxy.js          # Proxy pool (least-used, circuit breaker, metrics)
│   └── providers/
│       ├── index.js      # Provider registry
│       ├── webshare.js   # Webshare.io
│       └── oxylabs.js    # Oxylabs datacenter
├── output/
│   ├── human.js          # Terminal formatting
│   └── agent.js          # JSON formatting
└── utils/
    ├── logger.js         # Pino structured logging
    ├── retry.js          # Exponential backoff + jitter
    └── ua.js             # User-agent pool