AI Optimization Is Just SEO, Until You Leave Google

Google has been unusually direct about generative AI optimization: there is no such thing, at least not on Google. Their documentation says so in plain language. The interesting part is what happens to that advice the moment your content leaves Google’s index and gets requested by a crawler that works nothing like Googlebot.

Google’s position is documented, and it holds up

This is not an inference from a conference talk. Google’s guide to optimizing for generative AI features sets one eligibility condition: “To be eligible to be shown in generative AI features on Google Search, a page must be indexed and eligible to be shown in Google Search with a snippet.”

The companion AI Features and your website documentation is blunter still: “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” And on the cottage industry of AI-specific file formats: “You don’t need to create new machine readable files, AI text files, or markup to appear in these features. There’s also no special schema.org structured data that you need to add.”

Three tactics commonly sold as generative engine optimization are explicitly ruled out in that same guide. Google says structured data “isn’t required for generative AI search,” that “there’s no requirement to break your content into tiny pieces for AI to better understand it,” and that “you don’t need to write in a specific way just for generative AI search.”

The strongest confirmation is architectural rather than editorial. When Google shipped generative AI performance reports in Search Console in June 2026, the announcement noted that the AI feature data “is included in the overall performance report, where it will continue to be tracked to give site owners an overview of the overall visibility of their site in Google Search.” AI visibility inside Google is not a parallel system with its own qualification path. It is a view onto the pipeline you were already optimizing.

The Google-Extended misunderstanding worth correcting

A recurring mistake in AI access policies is treating Google-Extended as the control for AI Overviews. It is not. Google’s documentation states that AI “is built into Search and integral to how Search functions, which is why robots.txt directives for Googlebot is the control for site owners to manage access to how their sites are crawled for Search.” Google-Extended governs training and grounding in other Google systems.

Read that consequence carefully. There is no robots.txt directive that keeps you in Google’s organic results while removing you from AI Overviews. The available levers are snippet controls: nosnippet, data-nosnippet, max-snippet, and noindex. Blocking Googlebot to opt out of AI features removes you from Search entirely. That is a costly configuration to arrive at by accident.

Why “just do SEO” holds inside Google

Google’s advice works because Googlebot is a browser. Its JavaScript SEO documentation describes processing in three phases, crawling, rendering, and indexing, and confirms that “all pages with a 200 HTTP status code are sent to the rendering queue, no matter whether JavaScript is present on the page.” Once resources allow, “a headless Chromium renders the page and executes the JavaScript.”

Google acknowledges the exact failure mode it is compensating for: “Some JavaScript sites may use the app shell model where the initial HTML does not contain the actual content and Google needs to execute JavaScript before being able to see the actual page content that JavaScript generates.”

So a client-rendered page is survivable on Google. It costs you time in the render queue and it is fragile, which is why a proper technical audit still flags it, but the content can still be rendered and become eligible for indexing. Rendering is not the same as being indexed, and Google promises neither simply because it queued the page. Google absorbs the cost of your rendering choices. That absorption is the entire reason “AI optimization is just SEO” is a defensible statement when the speaker is Google.

Do AI crawlers render JavaScript?

Googlebot executes JavaScript in headless Chromium. Vercel’s December 2024 network analysis found no JavaScript execution from the OpenAI crawlers it tested, GPTBot, OAI-SearchBot and ChatGPT-User, or from Anthropic’s ClaudeBot. Anthropic has since introduced separate Claude-SearchBot and Claude-User identities and does not publicly document whether they render. For non-Google retrieval, the safe assumption is that only the initial HTML response counts, so critical content belongs there.

Where that guidance quietly stops applying

Nobody else absorbs it.

Vercel’s analysis of AI crawler behavior across its network tested the major crawlers and reported that “none of the major AI crawlers currently render JavaScript,” naming OpenAI’s OAI-SearchBot, ChatGPT-User, and GPTBot alongside Anthropic’s ClaudeBot. The crawlers do request JavaScript files, roughly 11.5 percent of ChatGPT’s fetches and 23.8 percent of Claude’s, but requesting a script is not executing it.

Two caveats belong here, and they are why this article ends with a test you run yourself rather than a number to take on faith. That analysis was published in December 2024, and crawler capabilities can change without announcement. Neither OpenAI’s crawler documentation nor Anthropic’s crawler documentation makes any claim about JavaScript rendering in either direction. The absence of a documented rendering pipeline is not proof that none exists, but combined with the observed fetch behavior it is the best evidence publicly available.

What follows if it holds is straightforward. When a non-rendering crawler requests a client-rendered page, it receives the shell: a root div, some script tags, and whatever meta the server sent. That is the document it keeps. Your service descriptions, your proof points, your FAQ answers, and your JSON-LD all sit on the other side of a script execution that never happens.

See the gap for yourself

The interaction below is the whole argument in one control. Pick how a page is rendered, then pick which system is requesting it. The panel shows what that requester actually ends up holding.

The same comparison as static reference, which is also what you get here with JavaScript switched off:

One site, three crawler families, different rules for each

Treating "AI crawlers" as a single category is where most access policies go wrong. Each vendor runs several bots with distinct jobs, and the documentation is explicit that they are governed separately. OpenAI states plainly that "each setting is independent of the others."

The pattern worth internalizing: blocking training does not block retrieval, and blocking retrieval does not block training. A robots.txt that disallows GPTBot and ClaudeBot while leaving OAI-SearchBot and Claude-SearchBot open is a coherent position, signaling that future collection should not feed model training while staying eligible for search retrieval. A robots.txt that blocks all of them because someone pasted a list from a forum post is a decision to be invisible, usually made without anyone realizing a decision occurred. We walked through the auditing side of this in our AI crawler governance checklist.

Bing is the third pipeline, and it reports back

Microsoft's position sits closer to Google's. Its guidance for AI-powered search is about keeping content crawlable, fresh, and fully indexed: complete XML sitemaps, accurate lastmod values, sitemap discovery through robots.txt, and IndexNow so AI systems reference the current version of a page. Their framing is that "keeping your website crawlable, fresh, and fully indexed is more important than ever." The separate Bing Webmaster Guidelines carry the structural half: link every page from at least one other crawlable page, and avoid nofollow and noindex on content you want surfaced.

Bing also does something neither OpenAI nor Anthropic offers: it reports citations back to you. The AI Performance report in Bing Webmaster Tools exposes total citations, average cited pages, the grounding queries that retrieved your content, and page-level citation counts across Copilot and Bing's AI summaries. Combined with Google's Search Console generative AI reports, that gives you measured AI visibility on two of the surfaces that matter and inference on the rest. The measurement gap on the remaining platforms is real, and we covered what can and cannot be measured about AI visibility separately.

Running a machine-accessibility parity audit

The deliverable this all points to is a parity audit: a documented comparison of what your raw HTML contains, what Google's rendered version contains, and what a non-rendering crawler receives. Six checks, in the order they pay off.

The two-minute version

Before commissioning anything, run the cheapest possible test. Open a key service page, disable JavaScript in your browser, and reload. What remains is approximately what a non-rendering crawler retains. If the page collapses to a logo and a nav bar, you have found your bottleneck, and no amount of content work will move it until the delivery changes.

Then check the response your server gives a declared AI crawler, since robots.txt permission and firewall permission are different things:

curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" -s -o body.html -w "%{http_code}
" https://example.com/your-service-page/

Use a full GET rather than a HEAD request, since edge rules routinely treat the two differently. A 200 tells you the URL is reachable over HTTP for that user agent. A 403 or 429 points at something between the crawler and your origin refusing a request your robots.txt invited, which is a common mismatch on sites behind aggressive bot management, though the block could sit at the CDN, the WAF, or the origin itself. Then read body.html: a status code says nothing about whether the response carried usable content, which is what the JavaScript test above measures.

What this changes about priorities

The honest summary is that Google's advice is correct and incomplete, and it is incomplete in a way that favors Google. "Just do good SEO" is accurate guidance for the surface where the crawler renders your JavaScript, absorbs your architectural shortcuts, and reports your AI impressions back to you. It transfers poorly to surfaces where the retriever takes one look at your initial HTML response and moves on.

That reframes the work. Content strategy for AI visibility is downstream of a question most sites have never answered: does the content exist in the response, or only after execution? Sites that render on the server were quietly correct all along and need no migration. Sites running a client-rendered front end have a real project ahead, and it is an engineering project rather than a marketing one.

The reason to treat this as urgent rather than interesting is that the failure is silent. There is no warning in Search Console, no ranking drop, no error report. Rankings can hold steady while your citation share on other surfaces stays at zero, and nothing in your existing dashboards distinguishes "never retrieved" from "retrieved and not chosen." That is the same class of invisible technical problem we unpacked in crawled but currently not indexed, and it responds to the same discipline: verify at the protocol level rather than trusting the interface.

Everything in the parity audit is ordinary technical SEO work. Server-rendered content, clean status codes, coherent robots directives, schema in the initial response, sensible page delivery. What has changed is the consequence of skipping it. These used to be efficiency items that made Google's job faster. They are now eligibility items that decide whether several major answer engines can read you at all, which is exactly why generative engine optimization that skips the technical layer tends to produce activity without visibility.

Sources

• Google Search Central: Optimizing your website for generative AI features on Google Search
• Google Search Central: AI features and your website
• Google Search Central: Understand JavaScript SEO basics
• Google Search Central: Introducing Search generative AI performance reports in Search Console
• OpenAI: Overview of OpenAI Crawlers
• Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?
• Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools
• Bing Webmaster Blog: Keeping content discoverable with sitemaps in AI-powered search
• Vercel: The rise of the AI crawler

Crawler capabilities and vendor documentation change without notice. Statements about JavaScript rendering describe observed behavior reported in third-party network analysis as of the dates cited, not vendor commitments. Verify current behavior against your own server logs before making architectural decisions. Omega Function is an independent consultancy and is not affiliated with Google, OpenAI, Anthropic, or Microsoft.