Discourage Search Engines in WordPress: What It Does

The website went live some time ago. It looks right, the contact form works, and yet searching for your own business name on Google brings up your Facebook page, a directory listing and a competitor, but not you. Someone suggests checking “that box in the settings”, and you are not sure which box they mean.

They mean the WordPress setting called “Discourage search engines from indexing this site”. It does a sensible job while a site is being built and quiet damage when somebody forgets it after launch. What it actually changes is smaller than most people think, and checking your own site takes a few minutes.

Where the search engine visibility setting is

Go to Settings → Reading. Near the bottom is a row called Search engine visibility with a single checkbox:

Discourage search engines from indexing this site

Under it, WordPress adds a short note: it is up to search engines to honour this request. That sentence is the most important thing on the screen, and it is worth unpacking.

What the checkbox does, and what it does not

When the box is ticked, current versions of WordPress do three things:

1. Every page gets a robots meta tag in its code telling search engines not to index the page and not to follow its links. In the page source it looks like <meta name='robots' content='noindex, nofollow' />.
2. WordPress’s built-in XML sitemap at /wp-sitemap.xml is switched off. (SEO plugins that produce their own sitemaps handle this in their own way.)
3. The At a Glance box on the dashboard home screen shows a reminder: “Search engines discouraged”.

Older versions of WordPress handled this differently, through the robots.txt file. The effect for you is the same: a polite request to stay out of search results.

Here is what it does not do:

• It does not hide the site from people. Anyone with the address can still visit every page.
• It does not stop bots. Well-behaved search engines such as Google and Bing respect the noindex request; scrapers and spam bots ignore it.
• It does not remove pages already in Google instantly. They drop out as Google recrawls them and sees the tag.
• It does not password-protect anything.

The classic mistake after launch

Developers often tick the box while building a site, which is sensible: nobody wants a half-finished site with placeholder text showing up in Google. The trouble is what happens next.

• The site is built on a temporary address and moved. Moving the database moves the setting with it, so the live site arrives already discouraged.
• A staging copy is pushed live. Some hosts’ staging tools tick the box on the staging copy automatically so it stays out of search results. When the staging site is later pushed over the live one, the setting can travel too. Our post on using a staging site covers that workflow.
• Nobody looks at the dashboard home screen. The reminder sits quietly in the At a Glance box, among other boxes, where it is easy to miss.

The result can last months: a working site that search engines have been asked, politely and on every page, to leave alone.

How to check your site in five minutes

1. Look at the dashboard. On the home screen, find the At a Glance box. If it says “Search engines discouraged”, the box is ticked.
2. Confirm in Settings → Reading. Check whether the checkbox is ticked.
3. Check a public page’s source. Log out, or open a private window, visit your home page, and view the page source (most browsers offer this when you right-click the page). Search for “noindex”. If a robots tag containing it appears, something is asking search engines to stay away. If the WordPress box is unticked and you still see it, an SEO plugin setting or a page-level option is adding it instead.
4. Use Google Search Console if you have it set up. Inspecting your home page’s address shows whether Google can index it, and the page indexing report lists pages excluded because of a noindex tag. If you have not connected your site yet, our guide to verifying your site in Google Search Console walks through it.
5. Search Google for site:yourdomain.com, using your own domain. It is a rough check rather than an exact count, but no results at all on an established site is a strong hint.

Turning it off and getting indexed

1. Go to Settings → Reading.
2. Untick Discourage search engines from indexing this site.
3. Press Save Changes.
4. Clear your site’s cache, so visitors and crawlers get pages without the old tag.
5. Check a page’s source again to confirm the noindex tag has gone.
6. In Search Console, submit your sitemap and request indexing for the home page and your most important pages.

Then give it time. Google decides when to crawl and what to include, and nobody can promise a date or a position. For an established business with a real site, pages often start appearing within days or weeks, but that is an expectation, not a rule.

If your SEO plugin shows its own warning about search engine visibility, it should disappear once the WordPress setting is saved. If it does not, look through the plugin’s general settings for a site-wide noindex option.

When to discourage search engines, and when to use something else

It is right for a site under construction, a staging copy, or an internal site that does not need to be found. Even then, pair it with something that keeps people out if the content is private.

Other jobs call for other tools:

• Hiding one page, such as a thank-you page: use your SEO plugin’s per-page robots setting. Our post on keeping a single page out of Google explains it.
• Keeping visitors out during work: use password protection or a maintenance screen. Our post on WordPress maintenance mode covers the safe way to do it.
• Telling crawlers which folders to skip: that is robots.txt, which works differently from noindex. See editing robots.txt in WordPress before you change it.

This checkbox is one of the first things our dashboard tour tells new owners to look at, and it belongs on any launch checklist built from our complete WordPress guide.

Frequently asked questions

Should I tick the box while I redesign my live site?

No. On a site that is already indexed, ticking it asks search engines to drop every page, and getting them back takes time. Build the new design on a staging copy instead, and if visitors must be kept out for a short while, use a maintenance screen, which tells search engines the outage is temporary.

I ticked it by accident on a live site. Will I lose my rankings?

Pages drop out of results only as Google recrawls them and sees the noindex request, so a short mistake usually does limited harm. Untick it, save, clear the cache and request indexing for your key pages in Search Console. Pages that dropped out generally return once they are recrawled, though not always straight away.

Does it stop people visiting my site?

No. The setting only asks search engines not to list your pages. Anyone who types your address, clicks a link in an email or scans a QR code still reaches the site normally. To keep people out, use password protection, a maintenance screen or your host’s access controls instead.

Do other search engines respect the setting?

The major ones, including Bing, honour the noindex robots tag that WordPress adds, much as Google does. Smaller crawlers, data scrapers and spam bots may ignore it entirely, which is why WordPress describes it as a request. Treat it as a way to stay out of search results, never as privacy or security.

Weeks after unticking the box, some sites are still missing from results, and then the cause is usually somewhere else: an SEO plugin option, a redirect, or a canonical tag naming a different address. Looking at what the site really sends to crawlers is the kind of check our WordPress support service can run with you.

Next in this series: How to Disable Comments in WordPress Completely