Signocore SEO builds your XML sitemap and robots.txt on the fly, decides which pages search engines may index, and can tell search engines about new content the moment you publish it.
XML sitemap
Go to Signocore SEO → Visibility and find the XML Sitemap card. The link at the bottom of the card opens your sitemap. By default it lives at https://example.com/sitemap-index.xml.
| Setting | Default | What it does |
|---|---|---|
| Sitemap Index Filename | sitemap-index.xml |
The address of the sitemap index. The name is turned into a URL-friendly slug, and .xml is added if you leave it out. |
| Sitemap Entry Limit | 10000 |
The most URLs in one sitemap file. Larger sets are split into numbered files, such as post-sitemap-1.xml and post-sitemap-2.xml. Google accepts up to 50,000 URLs or 50 MB per file. |
The filename field only appears when your site uses pretty permalinks. With plain permalinks, the sitemap is at https://example.com/?wpsseo-xml-sitemap=true.
WordPress's built-in sitemap is turned off, and requests for /sitemap.xml and /wp-sitemap.xml are redirected to your sitemap with a 301.
Important: If a real
sitemap-index.xmlfile (or whatever name you chose) sits in your site's root folder, the dynamic sitemap cannot load. Signocore SEO shows a warning with Delete file and Remove this warning. Make a backup before you delete anything.
What the sitemap includes
The sitemap index links to one sitemap per content type:
- Post types: every public post type with published content, for example
post-sitemap.xml,page-sitemap.xmlandproduct-sitemap.xml. - Taxonomies: every public taxonomy that has terms, for example
category-sitemap.xml. - Authors:
author-sitemap.xml, with authors who have published posts.
These are left out:
- Post types, taxonomies and archive types you noindex under Noindex Settings (see below).
- Posts, terms and authors where Hide this page from search engine results is on, or that an automatic rule hides.
- Terms without published posts, while Taxonomy Terms is on.
- WooCommerce system pages such as cart, checkout and my account, and products with the catalog visibility Hidden.
- Password-protected posts.
- Posts, terms and authors whose Canonical URL points to another page.
Each URL lists its images: the featured image, product gallery images and the images in the content, including images from other sites. Images that are only uploaded to a post, without being used in it, are not listed. When your front page shows your latest posts, the home page is added at the top of the page sitemap.
Each URL's date is the time the content was last modified. Sitemaps are cached for up to 24 hours. Any change to your content refreshes them right away, whether you save in the editor, use Quick Edit, a post is published on schedule or another plugin or tool updates it. Saving any Signocore SEO settings page, or changing your site title, permalinks or front page, refreshes them too.
Multilingual sites
The sitemap lists content in the language of the request, as filtered by your language plugin. With Polylang set to take the language from the directory name, each language folder gets its own sitemap, for example https://example.com/es/sitemap-index.xml (or https://example.com/language/es/sitemap-index.xml when Polylang keeps /language/ in its addresses), and robots.txt lists all of them.
The browser view
Search engines read the raw XML. When you open a sitemap in a browser, a stylesheet (/sitemap.xsl) shows it as a table instead: the index lists each sitemap with its date, and each sitemap lists its URLs with the number of images and a date. Sitemaps are sent with a noindex header, so the files themselves stay out of search results. Developers can change the accent color with wpsseo_sitemap_accent_color.
Noindex rules
Noindex rules live in two cards on Signocore SEO → Visibility.
Indexing Settings:
| Setting | Default | What it does |
|---|---|---|
| Exclude Noindexed Posts | Off | Hides noindexed posts, all posts of noindexed post types, and the WooCommerce cart, checkout and account pages from your site's own search. |
| Taxonomy Terms | On | Leaves terms without published posts out of the sitemap, gives them a noindex robots tag and marks them (auto) noindex in their SEO box. A parent category whose posts all sit in its subcategories counts as having posts. |
| Attachment Pages | Your WordPress setting | Gives every uploaded file its own page. See Attachment pages. |
Noindex Settings:
| Setting | Checked by default |
|---|---|
| Apply Noindex To Post Types | Media |
| Apply Noindex To Taxonomies | Tags, Product tags |
| Apply Noindex To Archives | Author, Date, Search |
Checked post types and taxonomies are left out of the sitemap and out of IndexNow, and each of their items shows Hide this page from search engine results (auto) in its SEO box. Checked archive types get a noindex robots tag, and noindexing author archives also removes the author sitemap.
Note: The rules apply right away, including the defaults on a site where Visibility was never saved. Items of a checked post type or taxonomy get a noindex robots tag without being saved one by one.
Automatic noindex
Besides your Noindex Settings, these pages are hidden from search engines automatically, because they have nothing to offer in search results:
- Terms without published posts, while Taxonomy Terms is on.
- The WooCommerce cart, checkout and my account pages, including their translations on a multilingual site.
- Products with the catalog visibility Hidden, which WooCommerce keeps out of the shop and its search. They are also left out of the sitemap.
- Author archives of users with no published posts and no biography, such as shop customers.
Each of these shows Hide this page from search engine results (auto) in its SEO box. Archives that combine several terms, such as /tag/coffee+tea/, get a noindex tag as well, because they are not the page of any one term.
While an automatic rule applies, saving the item does not store a noindex of its own. When the rule stops applying, for example when an empty category gets its first post, the page can be indexed again without any action from you.
Attachment pages
Attachment Pages in the Indexing Settings card decides whether every uploaded file gets its own page on your site. It is the same setting WordPress uses, and it starts out with the value your site already has. When it is off, WordPress sends visitors of those pages to the file itself.
Attachment pages hold little more than the file, which is why WordPress switches them off on new sites. If you turn them on, keep Media checked under Noindex Settings, as it is by default. With Pro, the redirect manager can send visitors of a file's address to the post the file belongs to while attachment pages are off, see Redirects and the 404 manager.
New content types
When a plugin or theme adds a new public post type or taxonomy, such as events or courses, administrators see the notice Signocore SEO: new content types found. It names the new types, because search engines may index them and they are listed in the sitemap. Click Review visibility settings to noindex them under Noindex Settings if they should not be found. The notice stays until you dismiss it.
Noindex a single page
In the SEO box of a post, page, term or author profile, turn on Hide this page from search engine results under Search Engines. The page gets a noindex tag and leaves the sitemap.
When an automatic rule covers an item, the switch is marked (auto). Click Disable auto noindex to exempt that one item from the rule. This cannot be undone. An exempted post loses the noindex tag and is listed in the sitemap, even though its post type is noindexed.
For a front page that shows your latest posts, use Signocore SEO → General → Home Page SEO → Search Results.
Comment feeds get a noindex header, so search engines can follow them without listing them. The feed address of a category, tag or author that does not exist returns a 404 instead of an empty feed.
Important: When Search engine visibility is checked under Settings → Reading in WordPress, your whole site is hidden. Signocore SEO shows a warning, robots.txt blocks everything and IndexNow stays off.
robots.txt
Signocore SEO generates https://example.com/robots.txt for you. There is no editor. Instead, the robots.txt card on Signocore SEO → Visibility has these options, and the link at the bottom of the card opens the file:
| Setting | Default | What it does |
|---|---|---|
| RSS Feeds | Off | Blocks feed URLs. |
| PDF Files | On | Asks search engines not to crawl PDF files. Turn it off when your PDFs, such as manuals or price lists, should be found in search. Sites that allowed PDF indexing in an earlier version start with it off. |
| REST API | Off | Asks search engines not to crawl /wp-json/. WordPress already keeps these addresses out of search results, and blocks that load content through the REST API, such as WooCommerce product lists, need it to show up in Google. |
| AI Training | Off | Asks AI companies not to use your content to train their models. |
| AI Search and Answers | Off | Asks AI assistants and AI search engines not to read your pages for their answers. |
Blocking AI crawlers
The two AI settings are separate, because they have different effects:
- AI Training blocks the crawlers that collect content for AI models: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, meta-externalagent, Amazonbot and MistralAI-Training. Your pages stay in regular search. Google-Extended only controls Gemini, so blocking it does not affect Google Search or AI Overviews.
- AI Search and Answers blocks the crawlers behind AI search and assistants such as ChatGPT, Claude and Perplexity: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, meta-webindexer, meta-externalfetcher, Amzn-SearchBot, Amzn-User, DuckAssistBot, MistralAI-Index and MistralAI-User. Your pages are then no longer cited in their answers. Assistants that open a page because a user asked for it may ignore this.
robots.txt is a request, not a lock: well-behaved crawlers follow it, others do not. Developers can change the lists with wpsseo_ai_crawlers.
What the file contains
With the default settings, on a site without WooCommerce, the file looks like this:
# Robots.txt for Example Company - Automatically generated by the Signocore SEO plugin for WordPress. # Learn more at https://signocore.com/wordpress-plugins/signocore-seo/ # WordPress User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php # Files Disallow: /*.pdf$ # Search Disallow: /?s=* Disallow: /search/* # Cloudflare Disallow: /cdn-cgi/bm/cv/ Disallow: /cdn-cgi/challenge-platform/ Disallow: /cdn-cgi/l/email-protection/ # Unbeneficial Bots User-agent: Nuclei User-agent: WikiDo User-agent: Riddler User-agent: Zoominfobot User-agent: Go-http-client User-agent: Node/simplecrawler User-agent: CazoodleBot User-agent: dotbot/1.0 User-agent: Gigabot User-agent: Barkrowler User-agent: BLEXBot User-agent: magpie-crawler Disallow: / # Sitemap Sitemap: https://example.com/sitemap-index.xml
The first line uses your company name. WooCommerce sites get rules that block WooCommerce's log and upload folders, RSS Feeds adds a group that blocks feed URLs, and each AI setting adds a group with its crawlers before # Unbeneficial Bots.
Rules that WordPress and other plugins add to robots.txt are kept. Their rules for all crawlers join the User-agent: * group at the top, without duplicates, and their groups for other crawlers and their Sitemap: lines are added as they are.
As with the sitemap, a real robots.txt file in your root folder stops the generated one, and the plugin offers to delete it. Developers can change the output with wpsseo_robots_txt.
IndexNow
IndexNow lets you tell search engines that a URL has changed. When you save a published post, page or term in the admin, or a scheduled post goes live, Signocore SEO sends its URL to IndexNow, which shares it with every participating search engine, such as Bing and Yandex. Google does not take part, so it relies on your sitemap. URLs from one save, such as a bulk edit, are sent together in one request.
When a published item is unpublished, moved to the trash or gets a new address, its old address is sent too, so search engines revisit it and drop or update it. The new address of a moved item is sent right away, even within the 10-minute limit below.
IndexNow runs when:
- Your site is visible to search engines.
- The WordPress environment type is production, which is the default.
- The key file exists. The plugin creates a key and writes it to a text file in your site's root folder, so that folder must be writable.
- The item is published and indexable: not noindexed by you or by an automatic rule, not password-protected, and its Canonical URL does not point to another page.
- The item was not sent in the last 10 minutes.
To send a URL by hand, open the SEO Health column in the list of posts or terms and click Request Indexing in the details. The same details show Last IndexNow Request. When sending is not possible, you see Instant indexing unavailable instead. It names the reason when your site is hidden from search engines or the environment is not production; for the other conditions above, check the list. If IndexNow turns down a request you send by hand, the message says why, for example when it cannot verify your key file.
llms.txt
Pro: This needs a Signocore SEO Pro license.
Pro publishes https://example.com/llms.txt, a plain text guide for AI assistants. It holds your company name, your home page description, a link to the home page and, for each public content type except media, up to five indexable pages. Each page is listed with its title as the link text, followed by its meta description. Pages are picked by view count when your site records one, otherwise the newest come first. Password-protected posts, the WooCommerce system pages and pages whose Canonical URL points to another page are left out.
There are no settings. The Indexing Settings card on Signocore SEO → Visibility shows that it is active and links to the file. The file is cached per language for up to a day and refreshed when you save settings or content.
Submit your sitemap to search engines
robots.txt already points crawlers to your sitemap, but submitting it gives you reports on how it is read.
- Verify your site with Google and Bing. See Analytics, verification and scripts.
- In Google Search Console, open Sitemaps, enter your sitemap URL and submit it.
- In Bing Webmaster Tools, open Sitemaps and submit the same URL.
Submit only the sitemap index. The search engines find the individual sitemaps from there.