Skip to main content
Signocore
// signocore seo docs

Sitemaps, indexing and robots.txt

Configure the XML sitemap, noindex rules, robots.txt, IndexNow and llms.txt in Signocore SEO.

Signocore SEO builds your XML sitemap and robots.txt on the fly, decides which pages search engines may index, and can tell search engines about new content the moment you publish it.

XML sitemap

Go to Signocore SEO → Visibility and find the XML Sitemap card. The link at the bottom of the card opens your sitemap. By default it lives at https://example.com/sitemap-index.xml.

Setting Default What it does
Sitemap Index Filename sitemap-index.xml The address of the sitemap index. The name is turned into a URL-friendly slug, and .xml is added if you leave it out.
Sitemap Entry Limit 10000 The most URLs in one sitemap file. Larger sets are split into numbered files, such as post-sitemap-1.xml and post-sitemap-2.xml. Google accepts up to 50,000 URLs or 50 MB per file.

The filename field only appears when your site uses pretty permalinks. With plain permalinks, the sitemap is at https://example.com/?wpsseo-xml-sitemap=true.

WordPress's built-in sitemap is turned off, and requests for /sitemap.xml and /wp-sitemap.xml are redirected to your sitemap with a 301.

Important: If a real sitemap-index.xml file (or whatever name you chose) sits in your site's root folder, the dynamic sitemap cannot load. Signocore SEO shows a warning with Delete file and Remove this warning. Make a backup before you delete anything.

What the sitemap includes

The sitemap index links to one sitemap per content type:

  • Post types: every public post type with published content, for example post-sitemap.xml, page-sitemap.xml and product-sitemap.xml.
  • Taxonomies: every public taxonomy that has terms, for example category-sitemap.xml.
  • Authors: author-sitemap.xml, with authors who have published posts.

These are left out:

  • Post types, taxonomies and archive types you noindex under Noindex Settings (see below).
  • Posts, terms and authors where Hide this page from search engine results is on.
  • Terms without published posts, while Taxonomy Terms is on.
  • WooCommerce system pages such as cart, checkout and my account.

Each URL lists its images: the featured image, product gallery images, images in the content and images uploaded to that post. When your front page shows your latest posts, the home page is added at the top of the page sitemap.

Each URL's date is the time the content was last modified. Sitemaps are cached for up to 24 hours. Saving content in the editor refreshes the affected sitemaps right away, and saving any Signocore SEO settings page refreshes all of them.

Multilingual sites

The sitemap lists content in the language of the request, as filtered by your language plugin. With Polylang set to take the language from the directory name, each language folder gets its own sitemap, for example https://example.com/es/sitemap-index.xml (or https://example.com/language/es/sitemap-index.xml when Polylang keeps /language/ in its addresses), and robots.txt lists all of them.

The browser view

Search engines read the raw XML. When you open a sitemap in a browser, a stylesheet (/sitemap.xsl) shows it as a table instead: the index lists each sitemap with its date, and each sitemap lists its URLs with the number of images and a date. Sitemaps are sent with a noindex header, so the files themselves stay out of search results. Developers can change the accent color with wpsseo_sitemap_accent_color.

Noindex rules

Noindex rules live in two cards on Signocore SEO → Visibility.

Indexing Settings:

Setting Default What it does
Exclude Noindexed Posts Off Hides noindexed posts, all posts of noindexed post types, and the WooCommerce cart, checkout and account pages from your site's own search.
Taxonomy Terms On Leaves terms without published posts out of the sitemap, gives them a noindex robots tag and marks them (auto) noindex in their SEO box.
RSS Feeds Off Blocks feed URLs in robots.txt.

Noindex Settings:

Setting Checked by default
Apply Noindex To Post Types Media
Apply Noindex To Taxonomies Tags, Product tags
Apply Noindex To Archives Author, Date, Search

Checked post types and taxonomies are left out of the sitemap and out of IndexNow, and each of their items shows Hide this page from search engine results (auto) in its SEO box. Checked archive types get a noindex robots tag, and noindexing author archives also removes the author sitemap.

Note: The rules apply right away, including the defaults on a site where Visibility was never saved. Items of a checked post type or taxonomy get a noindex robots tag without being saved one by one.

Noindex a single page

In the SEO box of a post, page, term or author profile, turn on Hide this page from search engine results under Search Engines. The page gets a noindex tag and leaves the sitemap.

When an automatic rule covers an item, the switch is marked (auto). Click Disable auto noindex to exempt that one item from the rule. This cannot be undone. An exempted post loses the noindex tag and is listed in the sitemap, even though its post type is noindexed.

For a front page that shows your latest posts, use Signocore SEO → General → Home Page SEO → Search Results.

Important: When Search engine visibility is checked under Settings → Reading in WordPress, your whole site is hidden. Signocore SEO shows a warning, robots.txt blocks everything and IndexNow stays off.

robots.txt

Signocore SEO generates https://example.com/robots.txt for you. There is no editor; the only option is RSS Feeds under Indexing Settings. On a site without WooCommerce, the file looks like this:

# Robots.txt for Example Company - Automatically generated by the Signocore SEO plugin for WordPress.
# Learn more at https://signocore.com/wordpress-plugins/signocore-seo/

# WordPress
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-json/
Allow: /wp-admin/admin-ajax.php

# Files
Disallow: *.pdf

# Search
Disallow: /?s=*
Disallow: /search/*

# Cloudflare
Disallow: /cdn-cgi/bm/cv/
Disallow: /cdn-cgi/challenge-platform/
Disallow: /cdn-cgi/l/email-protection/

# Unbeneficial Bots
User-agent: Nuclei
User-agent: WikiDo
User-agent: Riddler
User-agent: Zoominfobot
User-agent: Go-http-client
User-agent: Node/simplecrawler
User-agent: CazoodleBot
User-agent: dotbot/1.0
User-agent: Gigabot
User-agent: Barkrowler
User-agent: BLEXBot
User-agent: magpie-crawler
Disallow: /

# Sitemap
Sitemap: https://example.com/sitemap-index.xml

The first line uses your company name. WooCommerce sites get an extra group that blocks WooCommerce's log and upload folders, and RSS Feeds adds a group that blocks feed URLs.

Note: PDF files are always blocked in robots.txt.

As with the sitemap, a real robots.txt file in your root folder stops the generated one, and the plugin offers to delete it. Developers can change the output with wpsseo_robots_txt.

IndexNow

IndexNow lets you tell search engines that a URL has changed. When you save a published post, page or term in the admin, or a scheduled post goes live, Signocore SEO sends its URL to the IndexNow API, Bing, Naver, Seznam, Yandex and Yep. Google is not among them, so it relies on your sitemap.

IndexNow runs when:

  • Your site is visible to search engines.
  • The WordPress environment type is production, which is the default.
  • The key file exists. The plugin creates a key and writes it to a text file in your site's root folder, so that folder must be writable.
  • The item is published and not noindexed.
  • The item was not sent in the last 10 minutes.

To send a URL by hand, open the SEO Health column in the list of posts or terms and click Request Indexing in the details. The same details show Last IndexNow Request. When sending is not possible, you see Instant indexing unavailable with the reason.

llms.txt

Pro: This needs a Signocore SEO Pro license.

Pro publishes https://example.com/llms.txt, a plain text guide for AI assistants. It holds your company name, your home page description, a link to the home page and, for each public content type except media, up to five indexable pages with their SEO titles. Pages are picked by view count when your site records one, otherwise the newest come first.

There are no settings. The Indexing Settings card on Signocore SEO → Visibility shows that it is active and links to the file. The file is cached per language for up to a day and refreshed when you save settings or content.

Submit your sitemap to search engines

robots.txt already points crawlers to your sitemap, but submitting it gives you reports on how it is read.

  1. Verify your site with Google and Bing. See Analytics, verification and scripts.
  2. In Google Search Console, open Sitemaps, enter your sitemap URL and submit it.
  3. In Bing Webmaster Tools, open Sitemaps and submit the same URL.

Submit only the sitemap index. The search engines find the individual sitemaps from there.

Stuck on something the docs don't cover?

Questions go straight to the developer who builds the plugins. Replies usually within a day.

September Sale

€20 off Signocore SEO Pro

Pay €49 instead of €69, one time for unlimited sites. code SEP20