Insights · AI, SEO

There are now 2 internets. Is your business visible in both?

Google is locking its results away from scrapers, so AI tools find the web through Bing, partners and their own crawlers. Here’s how to show up in both.
15 November 2026
8 min read
By Bradley

Ranking on Google no longer means AI tools can see you. Google is closing its results pages to scrapers, so ChatGPT, Copilot and Perplexity are finding the web their own way. The result is 2 internets, and your business needs to show up in both.

I spotted the first sign myself. Searching Google in an incognito window, I now get a bot check before I see results. I’m not alone. Search Engine Roundtable has reported Google testing a “Sign in to verify you’re a human” prompt in Chrome incognito (August 2026) and a “verify your request” screen (September 2026). Google hasn’t announced either. They look like tests, not a new rule. But they fit a pattern.

Key points

  • Google’s results are getting harder for bots to read. JavaScript required since January 2025, the 100 results setting gone since September 2025, and a lawsuit against a company that resells Google results.
  • Your Google rankings have not changed. Google has not wiped or rebuilt its index.
  • AI tools use other sources. Copilot runs on Bing. ChatGPT search uses its own crawler plus outside providers, Bing among them.
  • Bing has a second job now. In Ireland it had 4.2% of searches in September 2026, and it also feeds AI answers.
  • Your own website decides the rest. Your robots.txt, host and firewall control which AI crawlers get in, and they can only read text that is in the HTML.

Google is locking its results away

Over the last 2 years Google has made its results pages much harder for bots to read.

  • January 2025: Google Search started requiring JavaScript. Google said it was to fight bots and spam.
  • September 2025: Google stopped supporting the &num=100 setting. Rank trackers used it to pull 100 results in 1 request.
  • December 2025: Google sued SerpApi, a company that sells Google results to other tools. Google says SerpApi gets around “SearchGuard”, its bot blocking system. A judge dismissed the first complaint in July 2026. Google filed a narrower one in August, and the judge dismissed that too on 9 November 2026. So far the courts have not backed Google, and the case may not be over.
  • 1 January 2027: Google’s Custom Search JSON API closes to existing users. It is already closed to new ones.

Here is what has not happened. Google has not wiped or rebuilt its index. Your Google rankings are where they were. What is changing is who else gets to read Google’s results.

That matters because AI tools have used Google’s results. In August 2025, The Information reported that OpenAI was getting Google data through SerpApi. Neither company confirmed it. Either way, scraping Google is getting harder, slower and more legally risky. So the AI tools lean on other sources.

There are now 2 internets

Google’s internet. Google Search, AI Overviews, AI Mode and Gemini all run on Google’s own index.

Everyone else’s. The other AI tools build answers from a mix of sources:

  • Microsoft Copilot is grounded on Bing.
  • ChatGPT search uses OpenAI’s own crawler (OAI-SearchBot) plus outside providers. Microsoft says its Web IQ service, powered by Bing, is used by ChatGPT. Research by Peec AI (September 2026) found ChatGPT uses at least 8 outside providers, including Bing and Google results bought from data companies.
  • Perplexity and Claude each run their own search crawlers.

These indexes drift apart. A page Google has indexed can be missing from Bing. A site that ranks well on Google can be absent from a Copilot answer. Being visible in one internet doesn’t guarantee the other. At Pixelated Bubble we check our clients in both.

We told clients to ignore Bing. That advice is out of date.

For years I told clients not to worry about Bing. In Ireland, Google had 93.5% of searches in September 2026 and Bing had 4.2% (StatCounter). Google still matters most.

But Bing now does a second job. It feeds Copilot, and it is one of the sources behind ChatGPT search. Bing Webmaster Tools even has an AI Performance report (public preview since February 2026) that shows how often your pages are cited in Copilot and Bing’s AI answers.

Here’s what to do. It takes about 30 minutes.

  1. Compare your index on both. Search site:yourdomain.ie on Google and on Bing. The counts are rough. If Bing shows far fewer pages, you have a gap. For a firm answer, use URL Inspection in each tool.
  2. Check Google Search Console. Confirm your sitemap is submitted and look at the Pages report for anything not indexed.
  3. Set up Bing Webmaster Tools. Sign in and choose Import from Google Search Console. Your sites are verified automatically and your sitemaps come across. Data can take up to 48 hours to appear.
  4. Turn on IndexNow. It tells Bing (and Yandex, Naver, Seznam, Yep and Amazon) the moment a page changes. Google doesn’t use it. On WordPress, Rank Math includes it free in the Instant Indexing module. In Yoast it’s a Premium feature.
  5. Test the answers. Ask ChatGPT, Copilot and Perplexity the questions your customers ask, such as “best plumber in Letterkenny”. Note who gets named and which pages get cited. Repeat monthly.

Google’s lockdown doesn’t stop AI reading your website

This is the part people mix up. Google’s blocking only affects bots scraping Google’s results pages. It does nothing to stop AI crawlers visiting your own website directly. Your robots.txt file decides which bots are welcome. So do your host and your firewall.

Let the search bots in. Keep the heavy crawlers in check.

Each AI company runs separate bots for separate jobs. That gives you control. OpenAI says each setting is independent, so blocking GPTBot does not remove you from ChatGPT search.

BotCompanyJobOur advice
BingbotMicrosoftBing index, CopilotAllow
OAI-SearchBotOpenAIChatGPT searchAllow
Claude-SearchBotAnthropicClaude searchAllow
PerplexityBotPerplexityPerplexity searchAllow
GPTBotOpenAIModel trainingYour choice
ClaudeBotAnthropicModel trainingYour choice
CCBotCommon CrawlTraining datasetsYour choice
BytespiderByteDanceModel trainingConsider blocking

Watch the names. ClaudeBot is Anthropic’s training crawler. The one that powers Claude’s search is Claude-SearchBot. Blocking training bots is a business decision, not a must. It saves server load, but it may also keep you out of what future AI models know.

A starting robots.txt for a WordPress site:

# Search and citation bots: welcome
User-agent: Bingbot
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Allow: /

# Training crawlers: blocked (optional)
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Bytespider
Disallow: /

# Everyone else
User-agent: *
Crawl-delay: 10
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://yourdomain.ie/sitemap_index.xml

3 things to know:

  • Crawl-delay is a hint. Google ignores it. Bing supports it. Anthropic says it respects it “where appropriate”. OpenAI and Perplexity don’t say.
  • robots.txt is a request, not a lock. Bots fetching a page because a user asked (ChatGPT-User, Perplexity-User) may not follow it. In 2025 Cloudflare caught Perplexity crawling with undeclared bots. To enforce limits, use Cloudflare’s AI Crawl Control to allow or block each crawler, and add a rate limiting rule.
  • Your host or firewall may overrule you. Some managed WordPress hosts block AI bots by default. Since 15 September 2026, new Cloudflare domains block training and agent bots by default on pages that show ads. Check your settings so your “allow” actually works.

Then cache everything. A cached page costs your server almost nothing to serve, whoever asks for it.

Why a cached WordPress site has the edge

A December 2024 study by Vercel and MERJ found GPTBot, ClaudeBot and PerplexityBot don’t run JavaScript. Googlebot does. We’ve seen nothing newer that changes this.

That matters. WordPress with LiteSpeed Cache sends every visitor a complete HTML page. Any crawler can read it, and it’s cheap to serve.

Many apps built with tools like Lovable work differently. The page arrives nearly empty and JavaScript fills it in. A crawler that doesn’t run JavaScript can see almost nothing. Lovable has improved this: apps created since May 2026 render on the server. Plenty of older ones have not caught up. When we fetched 271 AI built Irish websites without JavaScript on 15 November 2026, 111 sent back fewer than 25 words. The background is in our State of Irish Websites 2026 report and in what Google and ChatGPT actually see.

If your site was built this way, check it. Open the page source (not the inspector) and see if your text is there.

There’s a cost side too. On Vercel, the AI bot rules are switched off by default. In 2026, people on Vercel’s community forum reported free Hobby accounts paused after heavy GPTBot and ClaudeBot traffic. Crawling isn’t free when you pay per request.

Show up in both internets

Pixelated Bubble is a Google Partner with 12 years in search. We check where your site appears on Google, on Bing and in the AI answers your customers read. Then we fix the gaps. Book a free 15 minute call.

FAQ

Does ChatGPT use Google or Bing?

Neither on its own. ChatGPT search uses OpenAI’s own crawler plus outside providers. Microsoft says its Bing powered Web IQ service is one of them. Research by Peec AI in September 2026 found at least 8 providers.

Should I block GPTBot?

It is your choice. GPTBot collects training data. Blocking it does not remove you from ChatGPT search, which uses a separate bot called OAI-SearchBot.

Do I need Bing Webmaster Tools in Ireland?

Yes, it is worth 30 minutes. Bing had 4.2% of Irish searches in September 2026, but it also feeds Copilot and is one of the sources behind ChatGPT search.

Has Google changed my rankings?

No. Google has not wiped or rebuilt its index. What has changed is how easily other tools can read Google’s results pages.

Can AI crawlers read a JavaScript website?

Often not. A 2024 study by Vercel and MERJ found the main AI crawlers don’t run JavaScript. If your text is not in the page source, they can’t read it.

Last updated: 15 November 2026.

Sources

Free guides you can use today

Short, practical guides on SEO, Google Ads and keeping WordPress safe. Free to download, no fluff.

Ready when you are

Say hello.

Tell us what you need: a new site, more enquiries from ads, or both. We’ll tell you what it will take. A 15 minute call, no pitch deck.

Keep reading

AI, SEO
14 Nov 2026
We swept 225,237 .ie domains and checked every live AI built site we found. 76% have no Google Analytics. 57% show 2 or more AI design tells. Here is what we found, how we did it and what to check on your own site.
Paid Media
11 Nov 2026
Media buying in 60 seconds is James’s series of short, plain English Meta ads fixes for Irish small businesses: what a media buyer does, the four things to get right on Meta, creative, tracking, lead quality, traffic campaigns, and who James is.
Paid Media
11 Nov 2026
James has run Facebook and Instagram ads since 2008, starting in events and nightlife and now working with hospitality, fashion, sport and drinks brands. He thinks in systems: content, tracking, campaigns, website and follow up working together.