Google tells you what exists. Faceabot tells you what holds up.
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
Package npm :@readystack/robots-txt-ai-crawler-audit · website · measurements and detailsOne-call audit: robots.txt AI-bot rules, llms.txt, JSON-LD, meta tags, sitemap. Score plus fixes.
MCP endpoint :https://ai-visibility.rowb.app/mcp · website · measurements and detailsChecks llms.txt, AI crawler access in robots.txt, and sitemap - with a 0-100 AI readiness score.
MCP endpoint :https://mcp.apify.com?tools=apifmcpfactory/llms-txt-checker · website · measurements and detailsAudit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.
Package npm :crawl-readiness-mcp · website · measurements and detailsAI visibility checker: GPTBot vs OAI-SearchBot, robots.txt audit, raw-HTML schema. 165 crawlers.
Package mcpb :https://github.com/krisdiallo/ecom-agent/releases/download/v1.5.3/aivis.mcpb · website · measurements and detailsCheck AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
MCP endpoint :https://actablesite.com/mcp/ · website · measurements and detailsrobots.txt group count
MCP endpoint :https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-count/mcp · website · measurements and detailsrobots.txt Allow count, body discarded
MCP endpoint :https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-allow-count/mcp · website · measurements and details{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
MCP endpoint :https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots · website · measurements and detailsFetch a public robots.txt and return group counts only.
MCP endpoint :https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-inspect/mcp · website · measurements and detailsChecks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
MCP endpoint :https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp · measurements and detailsAI-search-visibility tools: check crawler access, generate llms.txt, lint robots.txt, census data.
MCP endpoint :https://ai-visibility.lastminutedealshq.com/mcp · measurements and detailsAI visibility score, AI crawler checks, llms.txt and robots.txt generators, AI agent guides.
MCP endpoint :https://stormap.ai/mcp · website · measurements and detailsSEO tools for AI agents: audits, Core Web Vitals, keywords, AI visibility, traffic, backlinks.
MCP endpoint :https://www.robot-speed.com/api/mcp · website · measurements and detailsURL to clean markdown for LLMs: a polite, robots.txt-respecting web reader. Free, no API key
MCP endpoint :https://agent-web.foomworks.workers.dev/mcp · website · measurements and detailsllms.txt 檢查工具：檢查網站的 llms.txt、robots 與 sitemap。台灣繁體中文 MCP 工具。
MCP endpoint :https://llms.penguindriver.com/mcp · website · measurements and detailsDelegate 20 typed RQM Robotics jobs for validation, diagnosis, comparison, and optimization.
MCP endpoint :https://jobs.rqmtechnologies.com/mcp/robotics · website · measurements and detailsOn-page SEO analysis, HTTP header checks, broken link detection, and robots.txt parsing.
MCP endpoint :https://seo.rjctdlabs.xyz/sse · measurements and details檢查網站 llms.txt 與 AI 爬蟲 robots.txt 設定，並自動產生 llms.txt。
MCP endpoint :https://llms-check.panda198271.workers.dev/mcp · website · measurements and detailsMarket & supply-chain intelligence on 34 humanoid robots: scores, specs, pricing. No key.
MCP endpoint :https://mcp-free.roboselect360.com/mcp · measurements and detailsAI agents: all of this is available without keys over MCP https://faceabot.com/api/mcp · HTTP https://faceabot.com/api/catalog?q=… · Guide : llms.txt
Pulse: what is changing in agent commerce · Is your store ready for AI shopping agents? · State of MCP (weekly) · MCP server doctor · Open reliability data · Add Faceabot to your AI · Help · Privacy · Terms