Hacker News · AI·· 6 h agoAI score38
Show HN: FloxoLab AI Crawler Access Checker Tests 14 AI Crawlers
Show HN: I built a tool to check website access for 14 AI crawlers
AI summary
A developer has launched the FloxoLab AI Crawler Access Checker, a free tool that tests whether 14 AI crawlers can actually read a given page. It reads robots.txt per RFC 9309, re-requests the URL with each crawler's user agent, and compares status codes and headers against a normal browser request, returning a verdict per crawler. The tool is free with no account and limited to 8 checks per 10 minutes.
Source:Hacker News · AI · floxolab.com