The real catalog of tools for AI agents

Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index

Checks llms.txt, AI crawler access in robots.txt, and sitemap - with a 0-100 AI readiness score.

Scan and fix your site's AI discoverability: crawler access, llms.txt, JSON-LD. Free.

Free SEO, GEO, and AEO audits: analyze any page or domain, AI-crawler access, agent readiness.

crawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

Free AI-readiness audit of any URL: AI crawler rules, JS-free text, JSON-LD, llms.txt. Tool catalog.

AEO tools for AI agents: crawler permissions, llms.txt, structured data, AI-readiness audit.

Inspect a site's AI-search readiness: AI crawler access, llms.txt, schema markup, meta directives

Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits.

Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

Enrich, search, assess, and manage threat intelligence through 80+ typed MCP tools.

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

Audit webpage access for AI crawlers. API key and Premium or Partner plan required.

Search or monitor any Flarum-powered forum for discussions, replies, participants, dates, and…

Enrich Google Maps leads, crawl docs to RAG Markdown, transcribe podcasts, check AI crawler access.

Cloudflare Workers MCP server: ai-crawler-policy