Part 1: can AI read your site?
This part is technical, and every item has a clear pass. Finish it first; otherwise the results in part 2 don't mean much.
| Check | How to check | Pass |
|---|---|---|
| robots.txt | Enter your URL in the AI crawler check | OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Googlebot and Bingbot are all allowed |
| Firewall and CDN | Request the homepage with a crawler's User-Agent and look at the status codecurl -s -o /dev/null -w "%{http_code}" -A "OAI-SearchBot/1.3" https://your-site.com/ | 200, the same as in a browser |
| Text in the HTML | See whether the raw HTML contains your textcurl -s https://your-site.com/ | grep -c "a sentence from your homepage" | Prints more than 0; the homepage, pricing page and feature pages all show their text and prices |
| sitemap | Open /sitemap.xml and check that robots.txt has a Sitemap line | Lists every page you want indexed, and every URL returns 200 |
| Structured data | Look for application/ld+json in the homepage's source | Organization (with sameAs pointing to your social accounts) or SoftwareApplication is there |
| Trust pages | About, contact, pricing, terms, privacy policy | All five exist and are linked from the footer |
| llms.txt | Open /llms.txt; if there is none, write one with the llms.txt generator | Plain text that opens directly; lowest priority, not every AI has committed to reading it |
The request test only changes the User-Agent; it can't reproduce a crawler's real IP. Some CDNs recognise crawlers by IP, so results can differ from what the real crawler gets.
Part 2: do AI answers include you?
1. List 10–20 buyer questions
Leave out brand names and write them the way buyers would: "best X tool", "X alternatives", "how to do Y". Use each market's own language.
2. Ask each question 3 times in ChatGPT, Perplexity and Claude
Start a new chat each time with web search on. Asking once tells you nothing: in SparkToro's 2026-01 study, asking ChatGPT or Google's AI search the same question again and again returned the same list of recommendations less than 1% of the time.
3. Record it in one table
One row each: question, which AI, which run, whether you were mentioned, who was recommended, which URLs were cited. Then work out two numbers: answers mentioning you out of all answers, and the competitors recommended most often.
4. Sort the cited URLs
Sort them by type: list articles, comparison pages, forum threads, directories, competitors' sites, your own pages. Pages that list products in your category but not you are the ones to submit to or write to.
5. Put the work in order
First fix whatever failed in part 1; then deal with "the cited pages don't include you"; last, rework how your pages are written. After each batch of changes, run the same questions again and compare the counts.
Numbers from two real evaluations
These are two sites kagentx evaluated, as they were on the day of the evaluation (not after any changes). You can fill in the same columns for your own site:
| Site | Evaluated | Mentioned / runs | Cited pages read | Of which list similar products | Of which mention it | Homepage raw / rendered characters |
|---|---|---|---|---|---|---|
| GammaSlide | 2026-10-26 | 0/21 | 28 | 19 | 0 | 314 / 18,630 |
| MochiStamp | 2026-10-22 | 0/3 | 10 | 10 | 0 | 8,790 / 9,625 |
Crawlers could reach both sites (part 1 mostly passed), yet no AI answer mentioned either of them: the cited pages didn't include them, and GammaSlide's text also waited for scripts. Both rounds asked Claude Sonnet 5.5.
FAQ
Why no single AI visibility score?
AI answers differ every time, and one score hides that. What helps more is how many times you asked and how many answers mentioned you, plus who was recommended and which pages were cited; those turn straight into things to do.
How often should I run it?
After each batch of site changes, to see whether they worked; otherwise once a month is enough. Use the same questions and the same number of runs each time so the results compare.
Part 1 all passes, but AI still doesn't mention me?
That's common. Being readable is only the precondition; whom AI recommends depends mostly on whether the pages it cites include you. The pages you find in step 4 of part 2 are your next work.
Doing it by hand takes too long. Is there a ready-made option?
A kagentx evaluation does both parts: it checks crawlers, rendering, the sitemap, structured data and more, and puts your buyers' questions to ChatGPT, Perplexity and Claude, listing the pages they cite. Basic results usually take 2–3 minutes.
Sources
- OpenAI · Overview of OpenAI Crawlers （checked 2026-11-03）
- Perplexity · Perplexity Crawlers （checked 2026-11-03）
- SparkToro × Gumshoe · AIs are highly inconsistent when recommending brands or products （2026-01, 12 prompts run 60–100 times on each platform）
- llmstxt.org · The /llms.txt file （proposed in 2024）
- kagentx evaluations: the data on the case study pages; each example gives its date.