Developer launches free checker for AI crawler access

A developer has released the FloxoLab AI Crawler Access Checker, a free tool that tests whether 14 AI crawlers can actually read a given page. It reads robots.txt per RFC 9309, re-requests the URL with each crawler's user agent, and compares status codes and headers against a normal browser request, returning a verdict per crawler. The tool requires no account and is limited to 8 checks per 10 minutes. It reads raw HTML up to 3 MB and does not follow pages behind authentication; only ports 80 and 443 are checked, and private, local and internal network addresses are refused. According to the developer, checks run on request and reports return only to the user's browser, with no database of URLs or results kept.

Generated from reports · updated 6 h ago

Not enough continuous observations yet to draw a trend.