Congratulations, you've just set up a brand new shop on the high street. You've got the best products in stock, better than any of your competitors. You know you're on to a winner but something feels off. It's as though you've left the front door locked and no one can get in.

That's exactly what's happening with a lot of B2B content right now. It's there, but our shoppers – AI bots such as ChatGPT, Claude, Google AI Overviews and the rest of the cast – are deciding in a fraction of a second whether your site is worth stepping into.

The majority of marketing teams are pouring real effort into optimising content for AI discovery tools, but far fewer have checked whether AI systems can actually reach that content, read it, and cite it. It's a gap that's going to widen, and it'll cost teams pipeline.

The shopper has already changed

Press Gazette and Chartbeat tracked global publisher traffic through 2025 and found Google search referrals dropped by a third in one year. Buyers don't bother clicking through ten blue links to compare vendors any more. AI assistants have taken that role, crafting the shortlist and citing whoever they can reach.

This is our new high street. Human footfall is still important, but focus also needs to be on whether AI can walk straight into your shop and gleefully begin filling its basket.

A stocked shop with a locked door is a sad tale of wasted potential

Most teams assume good content surfaces on its own. Sadly, that's not how it works. AI crawlers don't experience your site the way a person does, and a lot of modern websites are built in a way that quietly locks the door on them.

Vercel and MERJ analysed over 500 million requests from OpenAI's GPTBot and found no evidence it executes JavaScript at all, and the same held for ClaudeBot, PerplexityBot and Bytespider. Only Google's Gemini and Applebot, riding on browser-based crawler infrastructure, render pages the way a human visitor's browser would. If your product pages, pricing or key claims only appear once JavaScript runs client-side, most AI systems are reading an empty shell.

It's tempting to assume the fix is a quick layer of schema markup, tagging everything up so AI knows what it's looking at. Ahrefs tracked 1,885 pages that added JSON-LD schema and compared them against thousands of matched control pages that didn't. The result was no meaningful lift in citations on ChatGPT or Google's AI Mode, and a small decline on AI Overviews. Pages already cited by AI are roughly three times more likely to carry schema than pages that aren't, but that's correlation with sites that are already technically sound, not proof that a tag alone gets the cash register ringing.

Why the gap persists

  • Content and technical infrastructure sit in different teams. Whoever writes the article rarely owns the CMS rendering method or the sitemap, so nobody's checking the two against each other. It's those pesky silos again.
  • Inputting metadata requires cognitive recall. Some CMS platforms keep metadata in a separate UI from the one you use to build the page. A page may look complete, while nobody's confirmed the metadata is actually attached.
  • Traditional SEO habits don't fully transfer. A page can rank well in Google, because Googlebot renders JavaScript, and still be invisible to the AI crawlers that don't.
  • The failure is invisible from the inside. You can't see what a crawler couldn't read. There's no error message when GPTBot gives up on a blank div.

How to close the gap

Far from being a one-off tidy-up, brands need to roll up their sleeves for a spot of ongoing maintenance. You need to keep the door unlocked, the sign lit and the shelves tidy and visible, because AI crawlers, indexes and models all keep changing what they expect to find. That's the thinking behind our technical LLM ingestion audit. We score your site's crawlability, renderable content, schema markup, sitemap and metadata, and page speed, then benchmark you directly against up to three competitors, so you know precisely where the issues lie and what to fix first.

Request your technical LLM ingestion audit

Authentic, helpful content is still king, but let's just make sure it can actually be seen.

→ Explore how your brand can improve its GEO maturity.

Sources: PressGazette, Global publisher Google traffic dropped by a third in 2025, 2026Vercel and MERJ, The rise of the AI crawler, 2024. Ahrefs, We Tracked 1,885 Pages Adding Schema. AI Citations Barely Moved, 2026.