Faceabot
‹ Back

Catalog › io.github.mark1russell7/polite-crawler

Is polite-crawler reliable?

Not measured yet polite-crawler runs locally (installed package), so Faceabot cannot measure it over the network.

Not checked yet · local package

Proven alternatives that work right now

Robots.txt & AI Crawler Checker Reliable (measured) · 100%
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check. · https://robots-check.quietworklabs.workers.dev/mcp

ActableSite AI Crawler Monitor Flaky · 86%
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes. · https://actablesite.com/mcp/

agent-web — URL to LLM-ready markdown: a polite, robots-respecting web page reader (free) Reliable (measured) · 100%
URL to clean markdown for LLMs: a polite, robots.txt-respecting web reader. Free, no API key · https://agent-web.foomworks.workers.dev/mcp

Picked from Faceabot's live measurements. Or let Faceabot choose: GET https://faceabot.com/resolve?need=…

En français : Pas encore mesuré. Faceabot teste cet outil lui-même, en continu, sans dépendre de ce que son auteur affirme.

What it does

A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

Source: official MCP registry (registry.modelcontextprotocol.io) · version 0.2.0 · publisher website
Connect your agent
{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ]
    }
  }
}
Or ask Faceabot first: MCP tool check_reliability {"id":"io.github.mark1russell7/polite-crawler"} on https://faceabot.com/api/mcp.
Does it actually do the job?

— Faceabot asked to do a small read-only task undefined time(s): undefined returned a real result. Last result: .

Sample input: {}. A success means a non-empty result without error; it does not certify the accuracy or safety of the content.
Faceabot measurements

No measurement yet.

Method: MCP initialize then tools/list from Faceabot (FaceabotProbe/1.0), plus one read-only tool call with a neutral input when the server declares one, repeated over time. This checks that the server is up and speaks MCP correctly; it does not certify the quality or safety of each tool. How measurement works.

Find tools similar to polite-crawler →

Publisher of polite-crawler?

Show your measured reliability on your README or website:

Faceabot reliability badge for polite-crawler

[![Measured by Faceabot](https://faceabot.com/badge/io.github.mark1russell7/polite-crawler.svg)](https://faceabot.com/tools/io.github.mark1russell7/polite-crawler)
Data in JSON: /api/catalog?id=io.github.mark1russell7/polite-crawler · Stop probes: POST /api/reliability/opt-out

AI agents: all of this is available without keys over MCP https://faceabot.com/api/mcp · HTTP https://faceabot.com/api/catalog?q=… · Guide : llms.txt

Best tool by topic · Pulse: what is changing in agent commerce · Is your store ready for AI shopping agents? · State of MCP (weekly) · MCP server doctor · Reliability data · Faceabot app · Faceabot → AI assistants · What AI agents cannot find yet · Add Faceabot to your AI · Badge for tool publishers · Help · Privacy · Terms