Remove an unintended noindex directive

What does this finding mean?

The report identifies this check as meta-robots-noindex in Access. The category carries up to 28 of the 100 rubric points. This is a category weight, not an automatic deduction for this finding. Read the scan detail for the measured state, overlapping signals and any score cap.

Remove noindex only if the route is meant to appear in public search. Check HTTP headers as well as HTML.

Why does this matter for AI retrieval?

A retrieval system needs a usable HTTP response before it can read the page. robots.txt policy and server enforcement are separate observations. ChatGPT, Claude and Perplexity have distinct crawler controls; Google Search AI features depend on Google crawling and indexing.

How do I check and resolve it?

1. Inspect both meta robots tags and the HTTP X-Robots-Tag header. Removing a tag in a template does not override a restrictive response header.
2. Confirm the route should be public. Keep intentional noindex or snippet restrictions on private, internal or licensed material.
3. For public pages intended for search, remove the specific unwanted restriction at its source. Recheck CDN and security-plugin header rules.

Adapt the example to the reported issue and the actual route. Placeholder names, URLs and dates must be replaced with facts from your site.

Copy code

<!-- For a public page you want indexed and quoted: -->
<meta name="robots" content="index, follow" />
<!-- Restrict a specific section if needed: -->
<div data-nosnippet>Text excluded from supported snippets</div>

Verify the result

Fetch the final response and inspect headers plus HTML. Request reindexing through the relevant webmaster tool where available; an immediate scan cannot confirm index updates.

Open the related tool or technical guide, then run a new scan of the affected URL. Keep unknown checks marked unknown until you have a complete response.

Copy a prompt for your coding agent

Paste this into Codex, Claude Code or Cursor from your own repository. Include the scan's exact URL and detail so the agent can verify the observation.

Copy prompt for coding agent

Investigate the scanner finding "Remove an unintended noindex directive" (meta-robots-noindex) in this repository.
Observation: The response tells supporting search engines not to index this page.
Required change: Remove noindex only if the route is meant to appear in public search. Check HTTP headers as well as HTML.
Inspect the existing route, shared layouts and configuration first. Confirm the finding against the actual response before editing. Preserve intentional access restrictions, existing content and unrelated changes. Use only factual identity and date values. Make the smallest appropriate fix, verify the deployed HTML or resource response and run the relevant project checks. Explain any measurement that remains unknown. Do not promise search rankings or AI citations.
Reference: https://letthebots.in/fixes/meta-robots-noindex/