What We Check
AI search is here. When someone asks ChatGPT "who's the best plumber in Denver?" or Perplexity "compare CRM software for small businesses," can your site be part of the answer? We check the 5 factors that determine your AI visibility.
robots.txt for 11 AI Crawlers
We check your robots.txt for rules targeting GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended, Bingbot (Copilot), CCBot (Common Crawl), Bytespider (TikTok), FacebookBot, Amazonbot, Applebot, and Yandex AI. Each is individually assessed: allowed, blocked, or unaddressed.
llms.txt & llms-full.txt
The emerging standard for AI-readable site summaries. We check if you have an llms.txt file (brief site description for AI) and llms-full.txt (detailed version). These files help AI models understand your site's purpose and content structure.
noai / noimageai Meta Tags
Detection of meta robots tags that specifically block AI training: noai (blocks text training) and noimageai (blocks image training). We show whether these are present and which pages carry them.
JavaScript Rendering Dependency
AI crawlers vary in their ability to render JavaScript. If your key content only appears after JS execution, some AI search engines will see an empty page. We check whether your content is available in the initial HTML response.
JSON-LD Structured Data Depth
AI models use structured data to understand entity relationships. We assess the depth and coverage of your JSON-LD: Organization, Product, FAQ, HowTo, Article, and more. Rich structured data makes your content more useful to AI systems.
What You Learn
AI visibility is a new dimension that most website owners haven't considered. Here are real findings that change how you think about your online presence.
GPTBot and ClaudeBot are both blocked — but Perplexity is allowed
Your robots.txt blocks GPTBot and ClaudeBot but doesn't mention PerplexityBot. This means Perplexity can reference your content in AI search results, but ChatGPT and Claude cannot. This may or may not be intentional — but it's worth a deliberate decision rather than an oversight.
No llms.txt file found
The llms.txt standard is gaining adoption as a way to give AI models a concise summary of your site. Without it, AI systems must infer your site's purpose from page content alone — which often leads to incomplete or inaccurate representations. Creating one takes 15 minutes.
90% of content requires JavaScript to render
Your React-based site delivers a near-empty HTML document that gets populated by JavaScript. GPTBot can execute JS, but many other AI crawlers cannot. Your blog posts, service descriptions, and FAQ content are invisible to roughly half of AI search systems.
Why This Matters
AI-powered search is growing at 40%+ year-over-year. ChatGPT has over 200 million weekly users. Perplexity processes millions of searches daily. Google's AI Overviews now appear on 25%+ of search results. This isn't a future trend — it's current reality, and it's reshaping how people find businesses online.
The rules are different from traditional SEO. Blocking GPTBot might protect your content from AI training, but it also means ChatGPT can't cite your business in response to relevant queries. Allowing all crawlers means your content may be used to train models without attribution. There's no right answer — but there should be a deliberate strategy.
Most websites have no AI visibility strategy at all. Their robots.txt was written before AI crawlers existed. They've never heard of llms.txt. Their JavaScript-heavy sites are invisible to half the AI ecosystem. Our audit gives you a complete picture of your AI visibility posture and helps you make informed, intentional decisions about how AI systems interact with your content.
Related Features
SEO Audit
Traditional SEO and AI visibility are converging. Structured data helps both.
View SEO Audit →Security Audit
Your robots.txt and privacy decisions affect both security posture and AI visibility.
View Security Audit →