Check the content of llms.txt
A usable text resource was detected at /llms.txt. Presence is not full format validation.
Aditya R. Singh maintains Let The Bots In, a technical AI-readiness scanner and reference site.
What does this finding mean?
The report identifies this check as has-llms-txt in Freshness and hygiene. The category carries up to 10 of the 100 rubric points. This is a category weight, not an automatic deduction for this finding. Read the scan detail for the measured state, overlapping signals and any score cap.
Check the heading, summary and canonical document links in the validator.
Why does this matter for AI retrieval?
Canonical URLs and valid dates describe which document this is and when it changed. HTTPS protects delivery. These are observable document signals, not a measurement of how often ChatGPT, Claude, Perplexity or Gemini visits.
How do I check and resolve it?
- Decide whether a compact Markdown index would help tools using your documentation. llms.txt is optional and earns no points.
- Serve /llms.txt as UTF-8 plain text or Markdown. Start with one H1, a concise blockquote summary and Markdown links grouped under H2 headings.
- Use canonical public URLs and check each link. Keep secrets, authentication tokens and private document titles out of the public file.
Adapt the example to the reported issue and the actual route. Placeholder names, URLs and dates must be replaced with facts from your site.
# Actual site name
> A short description of the site and who it helps.
## Documentation
- [Crawler access](https://example.com/guide/): Configure and test public crawler rules.
- [About](https://example.com/about/): Who maintains this site. Verify the result
Fetch /llms.txt directly. Confirm HTTP 200 and a text body rather than an HTML fallback. Run the validator for format diagnostics; it does not establish adoption by an AI search engine.
Open the related tool or technical guide, then run a new scan of the affected URL. Keep unknown checks marked unknown until you have a complete response.
Copy a prompt for your coding agent
Paste this into Codex, Claude Code or Cursor from your own repository. Include the scan's exact URL and detail so the agent can verify the observation.
Investigate the scanner finding "Check the content of llms.txt" (has-llms-txt) in this repository.
Observation: A usable text resource was detected at /llms.txt. Presence is not full format validation.
Required change: Check the heading, summary and canonical document links in the validator.
Inspect the existing route, shared layouts and configuration first. Confirm the finding against the actual response before editing. Preserve intentional access restrictions, existing content and unrelated changes. Use only factual identity and date values. Make the smallest appropriate fix, verify the deployed HTML or resource response and run the relevant project checks. Explain any measurement that remains unknown. Do not promise search rankings or AI citations.
Reference: https://letthebots.in/fixes/has-llms-txt/ Check your site's technical readiness
Scan a public URL for crawler rules, delivered HTML and page metadata. The report distinguishes measured issues from checks it could not complete.
Check my site →