If my site is built with Webflow or Framer, can AI read the body text?
On 2026-11-11 we fetched the home pages of four public sites with curl (two Webflow, two Framer). All four are server-rendered: with no script execution, the HTML already contains the title, description and body text. After removing scripts, the smallest visible text is framer.university at about 2,600 characters and the largest is pitch.com at about 15,500. We picked these four by the platform marker in the page, only tested the home pages, and they cannot represent your site.
| Site (platform) | Visible text | Title, description, canonical | JSON-LD |
|---|---|---|---|
| relume.io, redirects to relume.ai (Webflow) | About 20,000 characters, two h1 | All present; canonical points to www.relume.ai | 2 blocks |
| finsweet.com (Webflow) | About 11,000 characters, one h1 | All present; canonical points to itself | 1 block |
| framer.university (Framer) | About 2,600 characters, three h1 | All present; canonical points to itself | None |
| pitch.com (Framer) | About 15,500 characters, one h1 | All present; the canonical tag carries Framer's auto-generated marker | 1 block |
Source: the home page HTML we fetched with curl on 2026-11-11. It describes these four home pages only, not every template of the platform.
What do robots.txt and sitemap look like by default?
| Site | robots.txt | sitemap.xml |
|---|---|---|
| relume.io (Webflow) | Only two Sitemap lines, no User-agent rules | 200, 3,706 URLs, plus /resources/sitemap.xml |
| finsweet.com (Webflow) | Only one Sitemap line, pointing to another hostname, wf.finsweet.com | 200, 209 URLs |
| framer.university (Framer) | User-agent * Allow: /, plus a Sitemap line | 200, 992 URLs |
| pitch.com (Framer) | Googlebot and * groups both have an empty Disallow:, plus Sitemap | 200, an index listing 4 child sitemaps |
None of these four sites names an AI crawler such as GPTBot, ClaudeBot or PerplexityBot, so by the text of robots.txt all of them allow AI crawlers. These are the site owners' own settings, not platform defaults. How to write it: How do you write robots.txt for AI crawlers?.
What do the official help docs say?
| Platform | What the official documentation says | Date and status |
|---|---|---|
| Framer | The site automatically includes sitemap.xml and robots.txt; turning off a page's Search engines switch adds a noindex tag and does not change robots.txt | Pages marked updated 2026-09-15 and 2026-09-23; original text read 2026-11-11 |
| Framer (canonical) | The SEO overview page does not mention canonical; on the sample site pitch.com the canonical tag carries data-framer-generated-canonical | Not stated in the official text, not verified; the sample was fetched with curl by us |
| Webflow | Default settings for robots.txt, sitemap, noindex and canonical | Not verified: the help center returned 403 bot verification to both curl and WebFetch, and we did not bypass it |
Framer is explicit: to keep a page out of the index, use the page switch (noindex), not robots.txt, because robots.txt only tells search engines not to crawl. This applies to AI search as well: a page blocked by robots.txt can still have its URL surfaced by links elsewhere.
We could not read Webflow's official explanation. Search result snippets quote it, but those are second-hand, so we do not put them here. You can log in and check the robots.txt and sitemap switches in the SEO section of the site settings.
How does it compare with WordPress?
| Webflow / Framer | WordPress | |
|---|---|---|
| Body text | The home pages of all four sample sites are server-rendered | Depends on the theme and plugins; check with curl yourself |
| robots.txt | Generated by the platform, edited in site settings (Framer's docs say it is included automatically) | Usually a virtual file generated by Yoast or Rank Math; a real file in the root blocks editing |
| sitemap | Generated automatically by the platform; all sample sites have one | wp-sitemap.xml by default, or the SEO plugin's own |
| llms.txt | Not found in the official docs; not verified | Yoast and Rank Math each have a switch |
WordPress details are in How to optimize a WordPress site for AI search; not repeated here.
What should I do now with my Webflow or Framer site?
1. Check the home page with curl
The commands check, in order, the platform marker, the title, the canonical, a sentence from the body, and finally robots.txt. If the body line prints 0, AI crawlers read an empty shell; see Do AI crawlers render JavaScript?.
# Is the text in the raw HTML? Platform marker, title, description, canonical curl -sL -A "Mozilla/5.0" https://your-site.com/ | grep -io 'data-wf-site\|name="generator" content="Framer[^"]*"' curl -sL -A "Mozilla/5.0" https://your-site.com/ | grep -io '<title>[^<]*</title>' curl -sL -A "Mozilla/5.0" https://your-site.com/ | grep -io '<link[^>]*rel="canonical"[^>]*>' curl -sL -A "Mozilla/5.0" https://your-site.com/ | grep -c "a sentence from your homepage" curl -s https://your-site.com/robots.txt
2. Make sure robots.txt does not block unknown crawlers as a group
The sample sites all allow crawling. Check whether your own site settings contain Disallow: / and what your CDN's bot rules do.
3. Make sure the sitemap opens and lists your main pages
The sample sites' sitemaps all return 200. Note that finsweet.com's robots.txt points Sitemap to another hostname; with that setup, make sure that address opens.
4. Use noindex for pages you want hidden, not robots.txt
Framer's docs say so. Webflow also has a matching switch in page settings, but its exact location is not verified; go by the dashboard.
5. Point canonical at the URL you want indexed
Framer generates canonical automatically; check it if you changed the domain or use several domains, see Canonical tags and duplicate content: how to set them up on a multilingual site so pages are not treated as duplicates.
FAQ
Can ChatGPT read a Webflow site?
On the two Webflow sites we tested, the home page HTML directly contains the body text and robots.txt does not block AI crawlers. Whether ChatGPT cites them, we have no evidence and it is not verified.
Is a Framer site rendered with JavaScript so that AI cannot read it?
On the two Framer sites we tested, the home page HTML contains the title, description and body text, with visible text from about 2,600 to about 15,500 characters. Only home pages were tested; check inner pages yourself with curl.
Can I edit Framer's robots.txt?
The official SEO overview page only says robots.txt is included automatically and does not say how to edit it; not verified.
How do I keep a Framer page out of the index?
Officially: turn off Search engines in page settings and republish; Framer adds a noindex tag. It does not change robots.txt.
What do Webflow's official docs say about the default robots.txt?
Not verified. The help center returned 403 bot verification to our fetching and we did not bypass it.
Do I need to submit a Webflow or Framer sitemap myself?
The sample sites' sitemap.xml all return 200 and list pages. Whether to submit it to Search Console and Bing yourself, we did not read in the official docs; we suggest submitting it.
Do these results represent my site?
No. These are the home pages of four public sites, and each owner has their own settings. Run the self-check commands on this page against your own site.
Sources
- Framer · Guide to SEO features and tools (original text read 2026-11-11, page marked updated 2026-09-15)
- Framer · Prevent pages from being indexed by search engines (original text read 2026-11-11, page marked updated 2026-09-23)
- Webflow · robots.txt(Webflow 官网自己的) (fetched 2026-11-11)
- Framer University · robots.txt (fetched 2026-11-11)
- Finsweet · robots.txt (fetched 2026-11-11)
- Pitch · robots.txt (fetched 2026-11-11)
- kagentx evaluations: the data on the case study pages; each example gives its date.