The method, in full

How SI Ready grades a site

SI Ready grades a site on 24 public checks worth 100 points. Site checks count once; page checks count the share of the pages read that pass. A site passes with 80 points or more, a home page over HTTPS and an llms.txt. Every check, its points, the crawlers it looks for and the sources are below, and the grader uses these same numbers.

CheckPointsWhy it mattersThe fix
Crawler access
Answer engine crawlers16Search and answer crawlers need robots.txt to let them in, or the site cannot be found and quoted in answers.Allow these crawlers in robots.txt: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Googlebot, Bingbot.
Training crawlers10These crawlers decide whether pages can shape future models. Blocking them is a choice; SI Ready counts it so you know.To let models learn from your pages, allow in robots.txt: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot. Keep them blocked if that is your choice.
HTTPS4Answer engines and browsers trust pages served over HTTPS.Serve the site over HTTPS and send http to https.
llms.txt
llms.txt6/llms.txt is a short Markdown map of the site for language models, proposed at llmstxt.org.Publish /llms.txt. Start from the one SI Ready made below from your own pages.
llms.txt format6The proposal asks for an H1 with the name, a short summary in a blockquote, and sections of links.Give /llms.txt an H1 (# Name), a summary line (> ...) and at least one ## section of - [title](url) links.
llms-full.txt3/llms-full.txt carries the text of the main pages in one file, for tools that read everything at once.Publish /llms-full.txt with the text of your main pages.
llms files in robots.txt2A line in robots.txt that points to llms.txt and llms-full.txt helps crawlers find them.Add two comment lines to robots.txt: # LLM summary: https://example.com/llms.txt and # Full text for LLMs: https://example.com/llms-full.txt
Sitemap and status
XML sitemap5A sitemap lists every page, so crawlers miss none.Publish a valid XML sitemap at /sitemap.xml.
Sitemap in robots.txt2The Sitemap line in robots.txt is how most crawlers find the sitemap.Add a line to robots.txt: Sitemap: https://example.com/sitemap.xml
Sitemap dates2A lastmod date on every address tells crawlers what changed and when.Give every address in the sitemap a lastmod date.
Pages answer 2004The pages a sitemap lists should answer at once, with no redirect and no error.List only final addresses in the sitemap and fix these pages.
Page signals
One H13One main heading says what the page is about.Give each page exactly one H1.
Title length3Titles of 10 to 60 characters show in full in results and answers.Write a title of 10 to 60 characters.
Meta description3A description of 50 to 160 characters is the summary engines show and quote.Write a meta description of 50 to 160 characters.
Canonical address3A canonical link tells engines which address is the real one.Add a canonical link with the full address of the page.
Indexable2A noindex in the robots meta tag or the X-Robots-Tag header keeps a page out of every index.Remove noindex from pages that should be found.
Open Graph3og:title, og:description and og:image make the card people see when the page is shared.Add og:title, og:description and og:image.
X card2twitter:card sets how a shared link looks on X.Add a twitter:card meta tag, for example summary_large_image.
Structured data and answers
JSON-LD5Structured data in JSON-LD tells engines exactly what a page is. It has to parse.Add JSON-LD to every page and make sure it is valid JSON.
Breadcrumbs3BreadcrumbList data shows where an inner page sits in the site.Add a BreadcrumbList to the JSON-LD of inner pages.
FAQ data2Pages that ask and answer questions can say so with FAQPage data.Add FAQPage JSON-LD to pages with two or more questions.
Language versions3hreflang links must name the page itself and point both ways, or engines ignore them.Make every hreflang link point back from the other page, and name the page's own language too.
Answer first5A page that opens with a clear answer is easier to quote.Open each page with a paragraph of 20 words or more right after the H1, with a first sentence of 40 words or fewer.
Image alt text3Alt text describes images to engines and to people who cannot see them.Give every image an alt attribute (empty for pure decoration).
Total100

The bar

80 points or more, the home page served over HTTPS, and an llms.txt in place. A check that does not apply to any page read (no images, no questions, one language) counts in full.

Page checks

SI Ready reads the home page, the address you pasted, and up to 20 pages spread evenly through the sitemap, plus up to five language versions to see whether they point back. A page check scores the share of those pages that pass it.

Crawler access

The 13 crawlers SI Ready looks for

Each name is the robots.txt token the company documents. SI Ready reads the rules for each one, and the * rules when a crawler has none of its own, and asks whether it may read the home page.

robots.txt tokenCompanyKindDocumentation
OAI-SearchBotOpenAISearch and answersOpenAI
ChatGPT-UserOpenAIFetches for a personOpenAI
GPTBotOpenAITrainingOpenAI
Claude-SearchBotAnthropicSearch and answersAnthropic
Claude-UserAnthropicFetches for a personAnthropic
ClaudeBotAnthropicTrainingAnthropic
PerplexityBotPerplexitySearch and answersPerplexity
Perplexity-UserPerplexityFetches for a personPerplexity
GooglebotGoogleSearch and answersGoogle
Google-ExtendedGoogleTrainingGoogle
BingbotMicrosoft BingSearch and answersMicrosoft Bing
Applebot-ExtendedAppleTrainingApple
CCBotCommon CrawlTrainingCommon Crawl
How it reads

A careful crawler

Its own name

Every request says SIReadyBot/1.0 (+https://sicto.si/ready/), so a site owner can see who asked.

robots.txt first

Beyond the home page and the standard files, it reads only what robots.txt allows for SIReadyBot, or for * when there is no rule of its own.

Small and slow on purpose

25 pages at most, 2 MB and 8 seconds per page, three redirects at most, six requests at a time, one check per domain per day.

Public addresses only

Only http and https on the standard ports. Every address a name resolves to must be public, and every redirect is checked again.

Nothing kept but findings

No page text is stored: only the scores, the fix list and the llms.txt made from titles and descriptions.

The same rules for everyone

The six SI sites are graded by the same method and appear on the list only if they truly pass.

Questions

About the method

Why does blocking a training crawler cost points?

Because SI Ready reports what the site lets in. Training crawlers are worth 10 of 100 points and are scored apart from search and answer crawlers, so a site can block them as a choice and still pass.

Is llms.txt a standard?

It is a proposal published at llmstxt.org in 2024, not a web standard. SI Ready checks it because answer engine tools read it, and it gives you one made from your own pages.

How is answer first measured?

One of the first two paragraphs after the H1 has 20 words or more, and its first sentence has 40 words or fewer. Chinese, Japanese and Korean text counts about two characters a word.

Will the method change?

When engines change what they read. Every change is dated on this page and in the sicto.si changelog, and old results stay as they were graded.

SI Ready

Check a site

80 points or more, the home page served over HTTPS, and an llms.txt in place. A check that does not apply to any page read (no images, no questions, one language) counts in full.