Understand an unofficial noai directive

What does this finding mean?

The report identifies this check as noai-directive in Agent diagnostics. This diagnostic is outside the six scored categories. This is a category weight, not an automatic deduction for this finding. Read the scan detail for the measured state, overlapping signals and any score cap.

Keep it if it expresses your preference, and use documented operator controls for a specific crawl opt-out.

Why does this matter for AI retrieval?

These observations describe optional formats and declared content-use preferences. They earn no score points. Support varies by operator; do not assume every engine reads an optional file or honors an unofficial directive.

How do I check and resolve it?

1. Inspect both meta robots tags and the HTTP X-Robots-Tag header. Removing a tag in a template does not override a restrictive response header.
2. Confirm the route should be public. Keep intentional noindex or snippet restrictions on private, internal or licensed material.
3. For public pages intended for search, remove the specific unwanted restriction at its source. Recheck CDN and security-plugin header rules.

Adapt the example to the reported issue and the actual route. Placeholder names, URLs and dates must be replaced with facts from your site.

Copy code

<!-- For a public page you want indexed and quoted: -->
<meta name="robots" content="index, follow" />
<!-- Restrict a specific section if needed: -->
<div data-nosnippet>Text excluded from supported snippets</div>

Verify the result

Fetch the final response and inspect headers plus HTML. Request reindexing through the relevant webmaster tool where available; an immediate scan cannot confirm index updates.

Open the related tool or technical guide, then run a new scan of the affected URL. Keep unknown checks marked unknown until you have a complete response.

Copy a prompt for your coding agent

Paste this into Codex, Claude Code or Cursor from your own repository. Include the scan's exact URL and detail so the agent can verify the observation.

Copy prompt for coding agent

Investigate the scanner finding "Understand an unofficial noai directive" (noai-directive) in this repository.
Observation: The page declares noai. It is not a standard access restriction shared by all major crawlers.
Required change: Keep it if it expresses your preference, and use documented operator controls for a specific crawl opt-out.
Inspect the existing route, shared layouts and configuration first. Confirm the finding against the actual response before editing. Preserve intentional access restrictions, existing content and unrelated changes. Use only factual identity and date values. Make the smallest appropriate fix, verify the deployed HTML or resource response and run the relevant project checks. Explain any measurement that remains unknown. Do not promise search rankings or AI citations.
Reference: https://letthebots.in/fixes/noai-directive/