# Sitemap Monitoring Your XML sitemap tells search engines which pages to crawl. If a deploy removes half of your sitemap URLs, or a sitemap file stops loading, search engines can stop finding your pages. Usually, nobody sees this. Sitemap Monitoring reads your sitemaps once a day. It compares each check with the previous check and tells you what changed. It also checks each sitemap file against the official limits and rules. ## What it answers [#what-it-answers] * Are my sitemaps healthy, and do all sitemap files load? * Which URLs were added to or removed from my sitemaps, and when? * Do removed URLs still get Google clicks? * Do my sitemap files break an official limit, for example 50,000 URLs or 50 MB? * Which sitemap URLs are broken, redirected, noindex or not in Google's index? * Did Googlebot visit the new URLs in my sitemap? ## How it works [#how-it-works] Sitemap Monitoring needs no setup. It starts when you add a domain. 1. **VitalSentinel finds your sitemaps.** See [How sitemaps are found](#how-sitemaps-are-found). 2. **VitalSentinel reads every sitemap file once a day.** The first check starts right after you add the domain. On a large site, it can take some time. 3. **VitalSentinel compares each check with the previous check.** It records the URLs that are new, removed, back in the sitemap or updated. 4. **VitalSentinel checks each file** against the official limits and recommendations. See [Limits and recommendations](#limits-and-recommendations). Sitemap Monitoring is available on every plan, with no limits. You do not need to verify the domain. The module has no settings. ### How sitemaps are found [#how-sitemaps-are-found] VitalSentinel finds your sitemaps automatically, in these places: | Found in | Meaning | | --------------------- | ---------------------------------------------------------------------------------------- | | **robots.txt** | A `Sitemap:` line in your robots.txt | | **Search Console** | A sitemap that you submitted in [Google Search Console](/features/google-search-console) | | **Common location** | A standard address, for example `/sitemap.xml` | | **Listed in *index*** | A sitemap that a sitemap index lists | You cannot add a sitemap by hand. To add a sitemap, add a `Sitemap:` line to your robots.txt, or submit the sitemap in Google Search Console. VitalSentinel looks again every day. If a sitemap file returns "not found" for 7 days in a row, VitalSentinel stops checking it. Its status then shows **No longer checked**. ### Removed URLs are confirmed [#removed-urls-are-confirmed] Sitemap Monitoring does not report a URL as removed by mistake: * If a sitemap file cannot be downloaded, VitalSentinel keeps its URLs. They show as **Held – file not read** and do not count as removed until the file downloads again. * If a file suddenly loses many URLs, VitalSentinel downloads it again one hour later. The removals count only if the second download confirms them. ### Data from other modules [#data-from-other-modules] Sitemap Monitoring does not open your pages. It uses data from other modules to show more about each URL: | Data | Source | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------- | | HTTP status, redirects and noindex | [Site Health Monitoring](/site-health/introduction) and [Page Monitoring](/page-monitoring/introduction) checks | | Whether Google indexed the URL | [Indexing Monitoring](/features/indexing-monitoring) | | Googlebot visits | [Server Log Monitoring](/server-logs/introduction) | | Google clicks | [Google Search Console](/features/google-search-console) | A column or a card that needs a source that you did not connect shows a link to set it up. ## First check [#first-check] While the first check runs, the page shows **Checking your sitemaps for the first time**. The page updates automatically. After the first check, the page shows **First check done**, with the number of URLs and sitemaps that VitalSentinel found. Changes show from the next check. If VitalSentinel cannot find or read a sitemap, the page shows one of these messages: | Message | What to do | | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **No sitemap found** | Add a `Sitemap:` line to robots.txt, or put a sitemap at `/sitemap.xml`. VitalSentinel looks again every day. | | **We couldn't download your sitemap** | Make sure that the sitemap opens in a browser. The message shows the reason, for example "403 (access denied)". If a firewall blocks the VitalSentinel crawler, see [Monitoring IPs](/others/monitoring-ips#how-to-allowlist). | ## Where to find it [#where-to-find-it] Open a domain and select **Sitemap Monitoring** in the **Monitoring** group of the sidebar. The page has three tabs: **Overview**, **URLs** and **Files**. ## Overview [#overview] The **Overview** tab tells you if something is wrong and what changed. ### Sitemap health [#sitemap-health] The top card shows the time of the last check and the next check. It gives a verdict: **Your sitemaps look healthy** or **Your sitemaps need attention**. **Top problems** lists what to fix first, each with a link to the details: | Problem | What to do | | ---------------------------------------------------------------------- | ----------------------------------------------- | | Removed URLs that still got Google clicks in the last 90 days | Put them back in the sitemap, or redirect them. | | A large drop: many URLs left your sitemaps | Make sure that the removal was on purpose. | | Sitemap files stopped downloading | Make sure that the files load. | | Your server or firewall blocks the VitalSentinel crawler | Click **Let our crawler in** for the steps. | | Sitemap URLs that redirect, return 404, return an error or are noindex | Fix the pages, or remove them from the sitemap. | | A file breaks an official limit or rule | Click **See which files**. | **Recommendations** lists things that are not errors but are worth a fix. The list is closed by default. Click it to open it. For example, sitemap URLs whose canonical points to a different page. List the canonical URL instead. ### URLs over time [#urls-over-time] This chart shows the number of URLs in your sitemaps, and the new and removed URLs on each check. An amber dot marks a check in which some files failed. Click a day to see what changed on it. ### What changed [#what-changed] This card lists the changes between two checks. By default, it compares the last check with the check before it. To compare other checks, select the dates in **From** and **To**. Highlighted days in the calendar have a check. The tiles show the totals for each type of change. Click a tile to filter the list: | Change | Meaning | | -------------------------- | --------------------------------------------------------- | | **New** | The URL is in your sitemaps for the first time. | | **Removed** | The URL is no longer in any sitemap. | | **Back in sitemap** | The URL left your sitemaps before and is back now. | | **Updated** | Its last modified date moved forward. | | **Date went backwards** | Its last modified date moved to an earlier date. | | **Date now given** | Your sitemap now gives a last modified date for it. | | **Date no longer given** | Your sitemap no longer gives a last modified date for it. | | **In a different sitemap** | The URL is still listed, but in a different sitemap file. | Some URLs also show a chip: | Chip | Meaning | | --------------------------------- | ------------------------------------------------------------------------------------------------ | | **Held – file not read** | Its sitemap file did not download. VitalSentinel keeps the URL and does not count it as removed. | | **Sitemaps disagree on the date** | Two sitemaps give different dates for this URL. It counts as updated only when they agree. | | **Date in the future** | Its last modified date is more than one day after the check. | | **Keeps coming and going** | The URL left and came back, or its date changed back and forth, 3 or more times in 14 days. | Select **Still getting clicks** to see the removed URLs that Google still sends visitors to. This list needs Google Search Console. The **What to do** column tells you the next step: | What to do | Meaning | | ------------------------- | ---------------------------------------------------------------------------------------------- | | **Moved – nothing to do** | The URL redirects to its new address. | | **Check the redirect** | The URL redirects. Make sure that it goes to the closest page. | | **Add it back** | The page still works and gets clicks. Add it back to the sitemap if the removal was a mistake. | | **Redirect it** | The page is gone but still gets clicks. Redirect it to the closest page. | | **Check the page** | VitalSentinel does not know if the page still works. Open it to check. | ### Google's index and Googlebot visits [#googles-index-and-googlebot-visits] * **Google's index** shows how many of the inspected sitemap URLs are not in Google's index, and why. It needs [Indexing Monitoring](/features/indexing-monitoring). * **Googlebot visits** shows when Googlebot last read a sitemap, and how many new URLs from the last 30 days Googlebot did not visit yet. It needs [Server Log Monitoring](/server-logs/introduction). ### Files that could not be downloaded [#files-that-could-not-be-downloaded] Click **See which files** on a download problem to open **Sitemap files that could not be downloaded**. The panel shows each file, why it failed and when it last downloaded. If you use Server Log Monitoring, it also shows whether Googlebot can read the file. If only the VitalSentinel crawler is blocked, select **Your firewall** to see the steps to let the crawler in. ## URLs [#urls] The **URLs** tab lists every URL in your sitemaps. Select a view: | View | What it shows | Needs | | ------------------------ | ---------------------------------------------------------------------------------------- | ---------------------- | | **All URLs** | Every URL in your sitemaps | - | | **Broken or redirected** | Sitemap URLs that redirect, return 404, return an error or are noindex on the last check | Site Health Monitoring | | **Not in Google** | Inspected sitemap URLs that Google did not index, with Google's reason | Indexing Monitoring | | **Not visited yet** | URLs added in the last 30 days that Googlebot did not visit yet | Server Log Monitoring | A sitemap must list only pages that load and can be indexed. Fix each broken page, or remove it from the sitemap. The table shows **URL**, **Sitemap file**, **Our crawl**, **In Google's index**, **Googlebot visit** and **Google clicks (last 90 days)**. Each column shows only when its source is connected. Use the filter bar at the bottom of the screen to narrow the list. Click a URL to open its panel. It shows the sitemaps that list the URL, its canonical and redirect, its Google clicks, its history in your sitemaps, and links to other modules. ## Files [#files] The **Files** tab lists every sitemap file. | Column | What it shows | | ------------------ | -------------------------------------------------------------------------------------------- | | **Sitemap file** | The address of the file | | **Status** | **Read**, **Has errors**, **Couldn't download**, **Not read today** or **No longer checked** | | **URLs** | The number of URLs in the file, or the number of sitemaps in an index | | **Found in** | Where VitalSentinel found the file | | **Last read OK** | The last successful download | | **Googlebot read** | When Googlebot last read the file. Needs Server Log Monitoring | A file can also show a chip for its limit findings: **Over limit**, **Error**, **Close to limit** or **Recommendation**. Click a file to open its panel. It shows the status of the file and its limit findings, with the rule and how to fix it. It also shows the URLs over time, the sitemaps that the file lists, and its downloads. ## Limits and recommendations [#limits-and-recommendations] VitalSentinel checks each sitemap file against the rules of Google, Bing and sitemaps.org. | Level | Meaning | Where it shows | | ------------------ | ------------------------------------------------------------------------------- | ------------------- | | **Over limit** | The file breaks an official limit. Search engines can ignore all or part of it. | **Top problems** | | **Error** | The file breaks an official rule. | **Top problems** | | **Close to limit** | The file is at 90% or more of an official limit. | **Top problems** | | **Recommendation** | A good practice, not an official rule. | **Recommendations** | **Limits:** * **Over 50,000 URLs**: a sitemap file can list up to 50,000 URLs. * **Over 50 MB**: a sitemap file can be up to 50 MB uncompressed, also when it is gzipped. * **Index lists over 50,000 sitemaps**: a sitemap index can list up to 50,000 sitemaps. * **Over 1,000 images on a page**: an entry can list up to 1,000 images. * **Over 1,000 news entries**: a news sitemap can list up to 1,000 news entries. **Errors:** * **Index inside an index**: a sitemap index can list only sitemap files. * **URLs of 2,048 characters or more**: each URL must be shorter than 2,048 characters. * **Not full URLs**: each URL must start with `http://` or `https://`. * **Entries with two `` tags**: each entry must have exactly one `` tag. * **Empty sitemap**: a sitemap must list at least one URL. * **URLs on another host or protocol**: URLs must use the same protocol and host as the sitemap. * **URLs above the sitemap's folder**: URLs must be in the folder of the sitemap or below it. This rule does not apply if you submitted the sitemap in Search Console or named it in robots.txt. * **Index lists sitemaps on another host** and **Index lists sitemaps above its folder**: the same rules for sitemap indexes. * **Wrong namespace** and **Not UTF-8**: the file must use the sitemaps.org namespace and UTF-8 encoding. * **Dates not in W3C format**: `lastmod` dates must use the W3C Datetime format. * **News entries missing required tags**, **News language is not a code**, **Videos missing required tags** and **Video values out of range**: rules for news and video sitemaps. * **hreflang without the page itself**, **Unusable hreflang codes** and **hreflang links that are not full URLs**: rules for hreflang in sitemaps. **Close to limit:** **Close to 50,000 URLs**, **Close to 50 MB** and **Index close to 50,000 sitemaps**. **Recommendations:** * **More than 10,000 URLs in one file** and **Larger than 10 MB**: VitalSentinel recommends smaller files. This is stricter than the official limit. * **Large file sent uncompressed**: send large files with gzip. * **Dates in the future**, **No dates** and **Invalid priority or changefreq**: give accurate `lastmod` dates. Google and Bing use them. * **News older than two days**: news sitemaps must list only recent articles. * **The same URL twice**: list each URL one time. This is common practice, not an official rule. * **robots.txt names no sitemap**, **No sitemap submitted in Search Console** and **Many sitemaps without an index**: rules for your whole site. ## History [#history] Sitemap Monitoring keeps 12 months of history on every plan, including Free. See [Data & Privacy](/others/data-privacy#retention). ## Alerts and other places [#alerts-and-other-places] * [Sitemap Monitoring alerts](/alerts/alert-types#sitemap-monitoring-alerts) tell you when a sitemap file fails, a sitemap shrinks or grows, or removed URLs still get clicks. * Site Health Monitoring opens [issues](/site-health/issue-types#issues-from-other-modules) from sitemap data, for example **Sitemap files unreadable**. * The [Server Log Monitoring Sitemap report](/server-logs/intelligence#sitemap) compares your sitemap with what Googlebot fetched. * The [public API](/api/endpoints#sitemap-monitoring) and the [MCP server](/mcp/tools#technical-seo-sitemaps) can read Sitemap Monitoring data. ## Related [#related] robots.txt changes and sitemap conflicts Which pages Google indexed