Retry incomplete sitemap discovery
The scanner could not finish sitemap discovery. It has not established that the sitemap is missing.
Aditya R. Singh maintains Let The Bots In, a technical AI-readiness scanner and reference site.
What does this finding mean?
The report identifies this check as sitemap-unknown in Access. The category carries up to 28 of the 100 rubric points. This is a category weight, not an automatic deduction for this finding. Read the scan detail for the measured state, overlapping signals and any score cap.
Inspect the advertised sitemap URL and child indexes before editing site configuration.
Why does this matter for AI retrieval?
A retrieval system needs a usable HTTP response before it can read the page. robots.txt policy and server enforcement are separate observations. ChatGPT, Claude and Perplexity have distinct crawler controls; Google Search AI features depend on Google crawling and indexing.
How do I check and resolve it?
- Check the sitemap URL in robots.txt and the response at /sitemap.xml. A sitemap index must point to valid child sitemaps.
- Publish XML containing canonical, public URLs. Replace the example date with the actual substantive update date, or omit lastmod.
- Return application/xml or text/xml with HTTP 200. Exclude redirects, login pages and noindex routes.
Adapt the example to the reported issue and the actual route. Placeholder names, URLs and dates must be replaced with facts from your site.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/guide/</loc>
<lastmod>2026-10-18</lastmod>
</url>
</urlset> Verify the result
Fetch the sitemap and a sample of listed pages. Each loc should resolve to the intended public canonical URL. Rescan to check discovery.
Open the related tool or technical guide, then run a new scan of the affected URL. Keep unknown checks marked unknown until you have a complete response.
Copy a prompt for your coding agent
Paste this into Codex, Claude Code or Cursor from your own repository. Include the scan's exact URL and detail so the agent can verify the observation.
Investigate the scanner finding "Retry incomplete sitemap discovery" (sitemap-unknown) in this repository.
Observation: The scanner could not finish sitemap discovery. It has not established that the sitemap is missing.
Required change: Inspect the advertised sitemap URL and child indexes before editing site configuration.
Inspect the existing route, shared layouts and configuration first. Confirm the finding against the actual response before editing. Preserve intentional access restrictions, existing content and unrelated changes. Use only factual identity and date values. Make the smallest appropriate fix, verify the deployed HTML or resource response and run the relevant project checks. Explain any measurement that remains unknown. Do not promise search rankings or AI citations.
Reference: https://letthebots.in/fixes/sitemap-unknown/ Check your site's technical readiness
Scan a public URL for crawler rules, delivered HTML and page metadata. The report distinguishes measured issues from checks it could not complete.
Check my site →