FaceabotFREN
‹ Back

The real catalog of tools for AI agents

Google tells you what exists. Faceabot tells you what holds up.

44,737real tools
29,284callable over HTTPS
19,862measured
5,943proven reliable
Robots.txt & AI Crawler Checker Reliable (measured) · 100% over 4 checks Does the job (tested) · 4/4

robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

MCP endpoint : https://robots-check.quietworklabs.workers.dev/mcp · website · measurements and details
robots-txt-checker Reliable (measured) · 100% over 7 checks

Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).

MCP endpoint : https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp · measurements and details
AI Visibility Checker Not measured yet

AI visibility checker: GPTBot vs OAI-SearchBot, robots.txt audit, raw-HTML schema. 165 crawlers.

Package mcpb : https://github.com/krisdiallo/ecom-agent/releases/download/v1.5.3/aivis.mcpb · website · measurements and details
RobotsGate Reliable (measured) · 100% over 9 checks Does the job (tested) · 9/9

Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

MCP endpoint : https://robotsgate.mike-tusa.workers.dev/mcp · website · measurements and details
AI Visibility Audit — Can AI crawlers read and cite your site? Reliable (measured) · 100% over 43 checks

One-call audit: robots.txt AI-bot rules, llms.txt, JSON-LD, meta tags, sitemap. Score plus fixes.

MCP endpoint : https://ai-visibility.rowb.app/mcp · website · measurements and details
robots-txt-ai-crawler-audit Not measured yet

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

Package npm : @readystack/robots-txt-ai-crawler-audit · website · measurements and details
ActableSite AI Crawler Monitor Flaky · 86% over 7 checks Does the job (tested) · 6/6

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

MCP endpoint : https://actablesite.com/mcp/ · website · measurements and details
llms.txt Checker - AI Readiness Audit Private (key required)

Checks llms.txt, AI crawler access in robots.txt, and sitemap - with a 0-100 AI readiness score.

MCP endpoint : https://mcp.apify.com?tools=apifmcpfactory/llms-txt-checker · website · measurements and details
crawl-readiness Not measured yet

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

Package npm : crawl-readiness-mcp · website · measurements and details
ai-visibility Reliable (measured) · 100% over 43 checks

AI-search-visibility tools: check crawler access, generate llms.txt, lint robots.txt, census data.

MCP endpoint : https://ai-visibility.lastminutedealshq.com/mcp · website · measurements and details
Botcard Being measured · 100% over 1 checks Real task being tested · 1/1

Cited AI crawler registry (training, search or user fetch) plus robots.txt checks. No fetching.

MCP endpoint : https://whosthatbot.com/mcp · website · measurements and details
llms.txt 檢查工具 Reliable (measured) · 100% over 42 checks

檢查網站 llms.txt 與 AI 爬蟲 robots.txt 設定，並自動產生 llms.txt。

MCP endpoint : https://llms-check.panda198271.workers.dev/mcp · website · measurements and details
robotstxt-ai Not measured yet

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

Package npm : robotstxt-ai-mcp · website · measurements and details
Robot Speed Private (key required)

SEO tools for AI agents: audits, Core Web Vitals, keywords, AI visibility, traffic, backlinks.

MCP endpoint : https://www.robot-speed.com/api/mcp · website · measurements and details
polite-crawler Not measured yet

A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

Package npm : polite-crawler-mcp · website · measurements and details
SEOAudit Being measured · 0% over 1 checks

On-page SEO analysis, HTTP header checks, broken link detection, and robots.txt parsing.

MCP endpoint : https://seo.rjctdlabs.xyz/sse · measurements and details
Scrape-LE Not measured yet

Analyse robots.txt content and report whether a path may be crawled.

Package npm : scrape-le-mcp · website · measurements and details
AI Site Scorer Being measured · 50% over 2 checks

Check website AI-readiness: Schema.org, llms.txt, E-E-A-T, robots.txt. Works in Cursor & Claude.

MCP endpoint : https://ai-readiness.mingles.ai/api/mcp/ · website · measurements and details
CRW Web Scraper Not measured yet

Open-source web scraper for AI agents with scrape, crawl, and map tools

Package oci : ghcr.io/fastcrw/crw:latest · website · measurements and details
Page Fetch Reliable (measured) · 100% over 8 checks Does the job (tested) · 8/8

Read one public web page as clean Markdown with its links, after a robots.txt check.

MCP endpoint : https://pagefetch.openkrill.app/mcp · website · measurements and details

AI agents: all of this is available without keys over MCP https://faceabot.com/api/mcp · HTTP https://faceabot.com/api/catalog?q=… · Guide : llms.txt

Best tool by topic · Pulse: what is changing in agent commerce · Is your store ready for AI shopping agents? · State of MCP (weekly) · MCP server doctor · Reliability data · Faceabot app · Faceabot → AI assistants · What AI agents cannot find yet · Add Faceabot to your AI · Badge for tool publishers · Help · Privacy · Terms