What is special about Brave Search's crawler?
It has its own index, its crawler does not give a name, and its access follows Googlebot. These are the key points from Brave's official "Brave Search crawler" page (opened and checked 2026-11-09; the page carries no date):
| Question | What the official page says | What it means for you |
|---|---|---|
| Does it have its own crawler? | Yes: like other search engines it discovers new pages and indexes them, partly using information from users who enabled the Web Discovery Project | Your site has to be crawlable by itself; you cannot rely on other engines alone |
| User-Agent | It does not use a distinct User-Agent, to avoid discriminating against sites that only let Google crawl | You cannot recognize it in logs, nor allow or block it on its own |
| Relation to Googlebot | If Googlebot cannot crawl a domain or page, Brave's crawler will not crawl it either | Allowing Googlebot means allowing Brave |
| robots.txt | robots.txt is not used to keep a page from being indexed | To stay out of the index, use noindex |
| noindex | Site owners delete a page with the robots noindex directive; after the page is updated Brave has to re-fetch it before it takes effect | After adding noindex, wait for a re-fetch |
| Submitting URLs | There is a form, described as being for re-fetching existing pages and for reporting pages that no longer exist | How to submit a brand-new URL is not covered on this page |
This page does not mention AI or crawl frequency. For AI crawlers elsewhere, see AI crawler user agents: a side-by-side table.
Which AI products use Brave's search data?
Officially, only Brave's own products are named; nothing else is. Below, what the official sources say is separated from what is unverified:
| Product | Does it use Brave? | Sources |
|---|---|---|
| Brave Leo (the AI assistant in the browser) | Yes: the official blog says Leo's answers can include search results, "powered by the Brave Search API" | Brave blog, 2024-06-13 |
| Answer with AI in Brave Search | Yes: the official blog says AI Grounding already powers this feature | Brave blog, 2025-08-05 |
| "Most of the top 10 LLMs" | Brave says its API gives them real-time search data, without naming them | Brave blog, 2025-08-05; which ones is unverified |
| Mistral Le Chat Deep Research | Only reported second-hand in a blog; we did not find an official statement from Mistral or Brave | Unverified, see the link below |
| Other AI assistants | Not named officially | Not verified |
So do not tailor your work to an AI assistant because it is "said to" use Brave. What is certain is that Brave's own Leo and answer features read Brave's index. For Mistral Le Chat see How to get Mistral Le Chat to recommend my product; for the discussion of ChatGPT's search source see Does ChatGPT use Bing?.
How do I get my site into Brave's index?
Brave offers no "submit for indexing" entry, so follow the steps below to make sure nothing is blocking you:
1. Make sure Googlebot is not blocked
Brave's access follows Googlebot. Run the lines below to see whether robots.txt disallows Googlebot and whether the page has noindex:
# Does robots.txt block Googlebot? Brave's crawler follows Googlebot's access curl -s https://example.com/robots.txt | grep -i -B1 -A3 "googlebot" # Is the page itself marked noindex? curl -s https://example.com/page/ | grep -io '<meta name="robots"[^>]*>' curl -sI https://example.com/page/ | grep -i "x-robots-tag"
2. Do not use robots.txt to control indexing
To keep a page out, use noindex; to have it appear, do not add it. For how to write the rules see How to write robots.txt for AI crawlers.
3. Put the body text straight in the HTML
The reason is in Do AI crawlers render JavaScript?.
4. Keep the sitemap and internal links healthy
The official page does not say whether Brave reads sitemaps, so internal links must also reach every page. For lastmod see Does sitemap lastmod matter to AI crawlers?; for duplicate pages see How do duplicate content and canonical affect AI search?.
5. Submit for re-fetching after changes, when needed
The form on the official page is described as being for re-fetching existing pages. Submit after you have updated the page, and do not submit repeatedly.
How do I know whether Brave has indexed me?
The official page mentions no webmaster tools. What you can do is search "site:yourdomain" on search.brave.com, then search a few of your buyers' questions and see whether your pages appear and where. For how to record results see How do you measure AI visibility? Four metrics and a table you can follow.
For another independent search engine, Kagi (its index sources and its crawler Kagibot), see How do I get Kagi to recommend my product? Kagibot and sources.
FAQ
What is Brave Search's crawler User-Agent?
There is no dedicated name. Brave's official page says its crawler does not use a distinct User-Agent, to avoid discriminating against sites that only let Google crawl. So you cannot recognize it in logs by name.
Can I allow only Brave in robots.txt?
No. The official page says that if Googlebot cannot crawl a domain or page, Brave's crawler will not crawl it either. So Brave's access follows Googlebot, with no separate switch.
How do I get Brave to remove a page from results?
The official page says: use the robots noindex directive; robots.txt cannot be used to prevent indexing. After the page is updated Brave has to re-fetch it before it takes effect, and you can use the form linked from the official page to ask for a re-fetch.
How do I submit a new URL to Brave?
The submission form on the official page is described as being for re-fetching existing pages and reporting pages that no longer exist. How to submit a brand-new URL is not covered on that page, and we do not guess. What you can do is keep the site crawlable by Googlebot, with healthy internal links and a sitemap.
Which AI products use Brave's search API?
Officially, only Brave's own Leo and Answer with AI in Brave Search are named. Brave also says its API gives search data to most of the top 10 large models but does not name them, so we mark every other product unverified.
Sources
- Brave · Brave Search crawler (Checked 2026-11-09; the page carries no date)
- Brave · Leo now incorporates real-time Brave Search results (Published 2024-06-13)
- Brave · Introducing AI Grounding with Brave Search API (Published 2025-08-05, updated 2025-09-01)
- kagentx evaluations: the data on the case study pages; each example gives its date.