Scanner finding guides

Find the meaning, remediation steps and verification commands for every scanner finding. Passing checks and unknown measurements have explanations too; they do not always call for a code change.

The six category weights total 100 points. Individual findings can overlap, and some are unscored diagnostics. Numbered parser warnings and resource errors link to stable family guides.

How do I use a finding guide?

1. Match the finding ID in your scan to a guide below.
2. Read the measured state before changing code. A passed or unknown check may need no change.
3. Apply the relevant remediation and verify the actual response.
4. Rescan the exact URL and keep measurement limits with the result.

Access 28 points

Why the score has a ceilingscore-cappedAn access condition limits the total even when content categories score well.When the scanner cannot fetch a pagepage-unreachableThe scanner did not receive usable page HTML. Its network is not a vendor crawler network.When robots.txt could not be checkedrobots-unknownA request failed or was incomplete, so the scanner cannot infer a crawler policy.Publish a readable robots.txtno-robots-txtNo usable robots.txt was detected. Missing rules do not automatically mean crawlers are blocked.Allow the intended search crawlersblocked-search-botsA matching robots.txt rule disallows one or more search crawlers for the scanned path.Verify permitted search crawler rulessearch-bots-accessibleThe parsed robots.txt policy permits the checked search crawlers at this path.Review blocked browsing agentsblocked-user-botsA rule targets one or more user-invoked browsing identities. Operator compliance differs.Keep training and search policy separateblocked-training-botsTraining-related crawler rules disallow collection. That can be an intentional policy.Resolve robots.txt parser warningsrobots-warningThe scanner found syntax or grouping that needs review. Numbered warning IDs share this guide.Investigate a bot-header response blockserver-block-detectedA browser-like request received content while a request with a crawler header received a block. Both came from our infrastructure.Resolve an uncertain server comparisonserver-block-uncertainThe limited request comparison did not establish a reliable crawler-specific block.Interpret a successful header probeserver-block-passThe limited probe did not detect a crawler-header block. It does not verify access from a vendor network.Remove an unintended noindex directivemeta-robots-noindexThe response tells supporting search engines not to index this page.Retry incomplete sitemap discoverysitemap-unknownThe scanner could not finish sitemap discovery. It has not established that the sitemap is missing.Publish and advertise an XML sitemapno-sitemapNo usable sitemap was found through the checked locations.Maintain a discovered sitemapsitemap-foundThe scanner found a usable XML sitemap.Review snippet restrictionsnosnippetA nosnippet or max-snippet:0 instruction restricts supported search snippets and Google AI features.Fix an HTML fallback at a resource URLsoft-200A resource endpoint returned HTTP 200 with an invalid body, often the website app shell. Resource-specific IDs share this guide.

Readability 14 points

Close the rendered-content gapjs-content-gapA browser render exposed much more text than the raw response.Serve main content before JavaScriptjs-only-pageThe initial HTML appears to contain little usable content and relies on client rendering.Maintain readable initial HTMLcontent-readableThe scanner found useful text without requiring browser execution.Check whether the page answers its questionthin-contentThe response contains very little useful text. A short utility page may still be complete.Verify a possible access restrictionpossible-paywallA weak gate signal was found. A mention of subscriptions can trigger it without an actual paywall.Handle an intentional paywallpaywall-detectedThe scanner found stronger evidence that the main text requires payment.Separate public content from a login routelogin-gate-detectedThe fetched page appears to require a login for its useful content.

Structured data 16 points

Add accurate page identity markupno-structured-dataThe scanner found no supported structured-data representation in the response.Review microdata or RDFa before adding JSON-LDnon-jsonld-schemaStructured data exists outside JSON-LD. Valid microdata or RDFa can already describe the page.Maintain the detected JSON-LDhas-json-ldAt least one JSON-LD block was found. Presence alone does not establish semantic correctness.Identify the actual site publishermissing-organization-schemaThe expected Organization node was not detected for this page context.Connect the site-level WebSite entitymissing-website-schemaA WebSite node was not detected in the supplied markup.Choose schema for this page typemissing-schema-typeThe report recommends a type relevant to the detected page purpose.Fix the reported structured-data issueschema-issueThe schema parser reported a specific issue. Numbered issue IDs share this guide.Maintain the schema audit resultschema-audit-cleanThe checked schema principles produced no reported violations. This is a bounded audit.Say only what the page saysschema-content-matchSome structured-data facts do not match the visible page content.Connect your entitiesschema-entity-graphThe structured entities lack fields or references needed to identify their relationships.Use the narrowest true typeschema-specific-typeThe schema type is too generic for the main content this page presents.Make answers quotableschema-direct-answersThe scanner found FAQ answers that need clearer standalone wording or a length review.Parse cleanlyschema-validityThe scanner found a JSON syntax, schema value or reference problem.Declare each thing onceschema-single-sourceMultiple schema sources declare the same entity or disagree about its facts.

Authority and entity 18 points

Complete identity and accountability evidenceidentity-evidenceThe report lists the name, URL, description and accountability evidence it found or missed.Identify the publisher or authorno-entity-schemaThe markup does not identify a Person or Organization.Match the byline to the author entitypartial-author-attributionA named byline and Person schema were not both detected where author attribution applies.Verify existing sameAs linksentity-profilesSupported external profile links were detected. Presence does not confirm ownership.Verify a Wikidata search candidatewikidata-candidateA name search returned a possible entity. It earns no confirmed-match points.

Extractability 14 points

Write a descriptive document titlemissing-titleThe response has no usable title element.Shorten or clarify the document titletitle-lengthThe title falls outside this rubric's preferred length range. That range is not a search-engine rule.Add a main heading for the documentmissing-h1No H1 was detected in the delivered HTML.Clarify the page heading outlinemultiple-h1Multiple H1 elements make the scanner's primary title signal ambiguous.Describe the document in metadatamissing-meta-descriptionNo meta description was detected in the response head.Repair skipped heading levelsweak-heading-structureThe detected heading sequence skips levels or lacks a useful section structure.Maintain a clear page outlineextractability-solidThe checked title and heading signals passed this rubric.Put the answer near its headingno-quotable-formatsThe scanner did not find the answer-oriented formats it checks. Some page types do not need them.Finish the useful answer structurepartial-quotable-formatsSome answer-oriented signals were detected while others were absent.Keep answers understandable out of contextquotable-formatsThe scanner found answer-oriented formatting.

Freshness and hygiene 10 points

Retry an optional llms.txt checkllms-unknownThe optional file request did not produce a reliable observation.Check the content of llms.txthas-llms-txtA usable text resource was detected at /llms.txt. Presence is not full format validation.Decide whether llms.txt would helpmissing-llms-txtNo usable optional llms.txt file was detected. There is no score deduction for its absence.Correct invalid or conflicting datesdate-contradictionsThe scanner found invalid calendar values, future dates or inconsistent publication and update ordering.Show real dates on dated contentno-date-signalsAn article or documentation page has no valid date signal. Evergreen homepages are exempt.Maintain truthful document datesdate-signals-presentA parseable date was detected. It does not prove the content is current.Set one canonical document URLmissing-canonicalThe scanner did not find a usable canonical URL.Deliver the document over HTTPSnot-httpsThe scanned URL uses HTTP.Investigate slow response timingslow-responseThe measured request was slow. One timing sample is not a site performance benchmark and earns no points.Interpret a fast response samplefast-responseThis request returned within the diagnostic timing threshold. It adds no score points.Use accurate sitemap update datesno-sitemap-lastmodThe sitemap lacks the optional lastmod value checked by this diagnostic.

Agent diagnostics unscored

Review restrictive content-use preferencescontent-signal-restrictiveThe live file declares search=no or ai-input=no. These are preferences rather than a server access barrier.Maintain declared content-use preferencescontent-signal-presentThe scanner found Content-Signal declarations. Operator support varies.Find the source of managed robots.txt linescloudflare-managed-robotsThe response includes a Cloudflare-managed section. Your repository file may not be the whole response.Understand an unofficial noai directivenoai-directiveThe page declares noai. It is not a standard access restriction shared by all major crawlers.Maintain optional Markdown negotiationmarkdown-negotiationA request asking for Markdown received a Markdown representation.Decide whether to offer Markdownno-markdown-negotiationNo Markdown representation was confirmed. HTML remains sufficient for this score.Maintain an optional full-text indexllms-full-presentThe optional /llms-full.txt resource was detected.Decide whether to publish llms-full.txtno-llms-fullThe optional full-text companion was not detected. It earns no points.

Aditya R. Singh maintains Let The Bots In, a technical AI-readiness scanner and reference site.