Which pages of a command-line tool can AI read?
Usually three: the Homebrew formula page, the GitHub repo page, and the docs page on the official site. On 2026-11-12 we fetched all three for jq with curl (no scripts executed, User-Agent Mozilla/5.0, no AI crawler simulated), and all three return the body text directly:
| Page | HTTP status and size | title and meta description | Visible text (scripts removed) |
|---|---|---|---|
| formulae.brew.sh/formula/jq | 200, 11,096 bytes | title is “Homebrew Formulae: jq”; meta description is the generic “Homebrew’s package index”; has a canonical | About 1,230 characters: install command, one-line description, homepage, license, version, dependencies, download counts |
| github.com/jqlang/jq | 200, about 297 KB | title has the project name and a sentence; meta description is the sentence plus GitHub's generic second half; no canonical tag | About 5,670 characters, including README sections such as Installation |
| jqlang.org (official site) | 200, 11,540 bytes | title is just “jq”; no meta description; canonical points to jqlang.org | About 2,431 characters |
None of the three titles describes the tool itself; only the formula page and the GitHub page have a “what it is” sentence in the body. So write the one-line description as the same sentence in all three places, and make the official site's title and description say it clearly; for the reasoning, see Do AI crawlers render JavaScript?.
Which fields do the Homebrew formula page and JSON API have?
The formula page is server-rendered, and the same data is also available through a public JSON API: formulae.brew.sh/api/formula/jq.json returns 4,462 bytes with no login; the full formula.json is 32,279,544 bytes with 8,730 entries. Fields useful for being found (measured values for jq):
| Field | Where it appears on the page | Value for jq |
|---|---|---|
| desc | First sentence of the body | Lightweight and flexible command-line JSON processor |
| homepage | Links in the body | https://jqlang.github.io/jq/ |
| license | The License line in the body | MIT |
| versions.stable | Current versions | 1.8.2 |
| executables | Binaries | jq |
| dependencies | Depends on | oniguruma 6.9.10 |
| analytics, github_packages_downloads | Download count, install events (last 30 days) | The page shows 280,128 downloads and 63,420 install events; the data source and date are stated on the page |
The formula page's JSON-LD is only a generic WebPage, with no SoftwareApplication; the sitemap has 16,605 URLs (8,731 containing /formula/ and 7,826 containing /cask/) and no lastmod; robots.txt has a single Sitemap line and no disallow rules. All of this was tested on 2026-11-12; source formulae.brew.sh/formula/jq.
What does Homebrew officially require for desc and homepage?
The Homebrew Formula Cookbook (docs.brew.sh, original text read on 2026-11-12) is short on this:
| Field | Official requirement |
|---|---|
| desc | Summarize what the formula does in one line; the formula name is automatically prepended to desc when printed, so don't repeat it; the text gives no character limit |
| homepage | Required; the docs state that formulae without a homepage are not accepted; prefer https; if the site blocks automated requests, you can write browsed: date to skip the automatic check for a year |
class Mytool < Formula desc "Command-line tool that converts CSV files to JSON" homepage "https://mytool.example.com/" url "https://github.com/example/mytool/archive/refs/tags/v1.0.0.tar.gz" license "MIT" end
The above are writing examples. The docs don't say how desc affects search or AI citation; we have no evidence on that, so it is unverified. But desc appears both in the formula page body and in the public JSON, so it is the one sentence readable in both places and worth writing carefully.
How do the GitHub repo page and the official docs page work together?
GitHub's robots.txt (29,821 bytes) puts GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot and Perplexity-User in one group: Crawl-delay: 1, no Disallow: /; what it blocks is /*/tree/, /*/raw/, /*/blame/, /*/archive/ and URLs with ?tab= or ?q=. The repo home page is not blocked while subdirectory pages are, so the README must be fully readable on the repo home page. For how to write the README, see Will AI search cite my GitHub README?.
The official docs page is the only place where you fully control the title, description, canonical and structured data. jq's official site has just “jq” as its title and no meta description, which is something to improve: write it as “tool name — one-line description”. For how to write the docs, see How should developer docs be written so AI can understand and cite them?.
How do you make a command-line tool easier to find, step by step?
1. Write one sentence and use it in all three places
Write the formula's desc, the GitHub repo's About, the official site's meta description and the first sentence above the fold as the same sentence: what it does, who it's for, what it replaces.
2. Give the official homepage and docs page a clear title and description
Make the title “tool name — one-line description” and the description a complete sentence of 120–160 characters; jq's site, whose title is only the tool name, is the counterexample.
3. Put the install command and a minimal example on the official site
brew install, other package managers, and a minimal copyable example. The formula page body has only the install command and no example, so put the example on the official site and in the README.
4. Align the README opening with the official site
Put what the tool does and the install command in the first lines of the README (see Will AI search cite my GitHub README?); for package registry pages (npm, PyPI) see How do packages on npm and PyPI get recommended by AI?.
5. Don't block crawlers by mistake in robots.txt
Check the official site's robots.txt for rules that block AI crawlers by User-Agent; for how to write it, see How do you write robots.txt for AI crawlers?.
6. Test again after a few weeks with the same questions
Ask ChatGPT and Perplexity “is there a command-line tool for X” and “how do I install X with brew”, asking each question 3 times, and note whether you are mentioned and which URLs are cited.
Related guides: How do open source projects get recommended by AI search? Six places beyond the README; for the desktop counterparts see How do Mac apps get recommended by AI? and How do Linux apps get recommended by AI search?.
FAQ
Can crawlers that don't run scripts read a formulae.brew.sh formula page?
Yes. On 2026-11-12 we fetched the jq page with curl (11,096 bytes, no scripts executed); with scripts stripped it has about 1,230 visible characters, including the install command, one-line description, homepage, license, version, dependencies and download counts. We tested only this one page.
Is the formula page's meta description my description?
No. The jq page's meta description is Homebrew's generic “Homebrew’s package index”. Your description appears in the page body and in the JSON API's desc field, not in the meta description. We tested only the jq page; whether other pages are the same is unverified.
How long should desc be?
The Homebrew Formula Cookbook only asks for a one-line summary of what the formula does, with no character limit; the formula name is automatically prepended to desc, so don't repeat the name.
Do I need an official website?
A formula must have a homepage (the docs say formulae without a homepage are not accepted). The official site is where you can write out usage, comparisons and install methods in full; see the steps in this guide.
What is GitHub's robots.txt rule for AI crawlers?
Tested on 2026-11-12: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot and Perplexity-User are in one group with Crawl-delay: 1 and no Disallow: /; paths such as /*/tree/, /*/raw/ and /*/blame/ and URLs with ?tab= are blocked. The repo home page is not blocked.
Do AI answers cite formulae.brew.sh?
We have no citation evidence, so it is unverified. All we can confirm is that it is open to crawlers and the page server-renders the install command.
What should the official docs page contain?
At minimum: what the tool does, how to install it (brew install and other methods), a minimal example, and how it differs from similar tools. For how to write it, see the guide on developer docs.
Should I add SoftwareApplication structured data for a command-line tool?
The formula page's own JSON-LD is only a generic WebPage. You can add it on your own site; for how to write it, see the SoftwareApplication guide. We have no evidence on whether adding it makes AI cite you.
Sources
- Homebrew · formulae.brew.sh: robots.txt, sitemap, formula page and JSON API for jq (Read 2026-11-12)
- Homebrew · Formula Cookbook (desc and homepage) (Read in full 2026-11-12, page has no date)
- GitHub · github.com/robots.txt and the jqlang/jq repository page (Read 2026-11-12)
- jq · jqlang.org home page (Read 2026-11-12)
- kagentx evaluations: the data on the case study pages; each example gives its date.