Leaderboard
Who is ready for agents, and who is not
Ranked by the CrawlIndex Score. Only fully measured sites appear here: a partial assessment is renormalised over fewer points, so putting it in the same column would compare two different scales.
Most agent-ready
| Rank | Domain | Score | Answer-surface crawlers | Stack |
|---|---|---|---|---|
| 2935 | railway.app | Score 100 out of 100, grade A | all allowed | unidentified |
| -- | crawlindex.org | Score 100 out of 100, grade A | all allowed | Next.js . Vercel |
| 2888 | crazyegg.com | Score 96 out of 100, grade A | all allowed | Cloudflare |
| 43 | wordpress.org | Score 95 out of 100, grade A | all allowed | WordPress |
| 94 | samsung.com | Score 95 out of 100, grade A | all allowed | Adobe Experience Manager . Akamai |
| 99 | wordpress.com | Score 95 out of 100, grade A | all allowed | WordPress |
| 142 | reg.ru | Score 95 out of 100, grade A | all allowed | Nuxt |
| 358 | zendesk.com | Score 95 out of 100, grade A | all allowed | Next.js . Cloudflare |
| 434 | wp.com | Score 95 out of 100, grade A | all allowed | WordPress |
| 500 | foxnews.com | Score 95 out of 100, grade A | all allowed | Fastly |
| 603 | onetrust.com | Score 95 out of 100, grade A | all allowed | Adobe Experience Manager . Cloudflare |
| 614 | klaviyo.com | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Cloudflare |
| 640 | branch.io | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 654 | stackadapt.com | Score 95 out of 100, grade A | all allowed | WordPress |
| 664 | intercom.io | Score 95 out of 100, grade A | all allowed | Sanity . Amazon CloudFront |
| 773 | singular.net | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Cloudflare |
| 774 | gumgum.com | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 962 | netlify.app | Score 95 out of 100, grade A | all allowed | Astro . Netlify |
| 1048 | pendo.io | Score 95 out of 100, grade A | all allowed | Next.js . Vercel |
| 1099 | cloudinary.com | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 1112 | usercentrics.eu | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 1301 | hostinger.com | Score 95 out of 100, grade A | all allowed | Nuxt . Cloudflare |
| 1377 | haberler.com | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 1381 | auth0.com | Score 95 out of 100, grade A | all allowed | Contentful . Cloudflare |
| 1413 | utorrent.com | Score 95 out of 100, grade A | all allowed | Gatsby . Amazon CloudFront |
| 1414 | edna.ru | Score 95 out of 100, grade A | all allowed | WordPress |
| 1445 | protothema.gr | Score 95 out of 100, grade A | all allowed | unidentified |
| 1856 | livechatinc.com | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Akamai |
| 1889 | musicdel.ir | Score 95 out of 100, grade A | all allowed | WordPress |
| 1892 | bitdefender.com | Score 95 out of 100, grade A | all allowed | Adobe Experience Manager . Cloudflare |
| 1971 | intercom.com | Score 95 out of 100, grade A | all allowed | Sanity . Amazon CloudFront |
| 2044 | iubenda.com | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Bunny |
| 2139 | primis.tech | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Amazon CloudFront |
| 2651 | woocommerce.com | Score 95 out of 100, grade A | all allowed | WooCommerce |
| 2659 | yotpo.com | Score 95 out of 100, grade A | all allowed | WordPress . Amazon CloudFront |
| 2693 | cutt.ly | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 2696 | nsw.gov.au | Score 95 out of 100, grade A | all allowed | Drupal . Amazon CloudFront |
| 2752 | zapier.com | Score 95 out of 100, grade A | all allowed | Contentful . Amazon CloudFront |
| 2757 | pubnative.net | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 2850 | cookieyes.com | Score 95 out of 100, grade A | all allowed | Drupal . Cloudflare |
| 3034 | intercomcdn.com | Score 95 out of 100, grade A | all allowed | Next.js . Amazon CloudFront |
| 3059 | duosecurity.com | Score 95 out of 100, grade A | all allowed | Amazon CloudFront |
| 3096 | drpc.org | Score 95 out of 100, grade A | all allowed | Next.js . Cloudflare |
| 3201 | dnsimple.com | Score 95 out of 100, grade A | all allowed | unidentified |
| 3267 | rtbhouse.com | Score 95 out of 100, grade A | all allowed | Next.js . Google Cloud / GFE |
| 3372 | demandbase.com | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 3388 | iterable.com | Score 95 out of 100, grade A | all allowed | WordPress . Fastly |
| 3486 | domaintools.com | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 3590 | brightspace.com | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 3749 | dinahosting.com | Score 95 out of 100, grade A | all allowed | unidentified |
| 3854 | similarweb.com | Score 95 out of 100, grade A | all allowed | WordPress . Amazon CloudFront |
| 3957 | helpshift.com | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 4142 | livechat.com | Score 95 out of 100, grade A | all allowed | HubSpot CMS . Akamai |
| 4190 | sardine.ai | Score 95 out of 100, grade A | all allowed | Sanity . Vercel |
| 4230 | razer.com | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 4251 | netlify.com | Score 95 out of 100, grade A | all allowed | Astro . Netlify |
| 4257 | giantpanda.com | Score 95 out of 100, grade A | all allowed | Next.js . Vercel |
| 4273 | sketchup.com | Score 95 out of 100, grade A | all allowed | Contentful . Netlify |
| 4349 | symphony.com | Score 95 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 4411 | ip.sb | Score 95 out of 100, grade A | all allowed | Cloudflare |
| 4417 | popsci.com | Score 95 out of 100, grade A | all allowed | WordPress |
| 4490 | tanium.com | Score 95 out of 100, grade A | all allowed | Sanity . Cloudflare |
| 4532 | picsart.com | Score 95 out of 100, grade A | all allowed | Next.js . Cloudflare |
| 4546 | ably.io | Score 95 out of 100, grade A | all allowed | Gatsby . Cloudflare |
| 4609 | adspower.net | Score 95 out of 100, grade A | all allowed | Amazon CloudFront |
| 4840 | cyberfolks.pl | Score 95 out of 100, grade A | all allowed | WordPress |
| 4875 | kayak.com | Score 95 out of 100, grade A | all allowed | Fastly |
| 4333 | nanit.com | Score 93 out of 100, grade A | all allowed | Shopify . Cloudflare |
| 2 | cloudflare.com | Score 92 out of 100, grade A | all allowed | Astro . Cloudflare |
| 521 | dynatrace.com | Score 92 out of 100, grade A | all allowed | Nuxt . Amazon CloudFront |
| 592 | atlassian.com | Score 92 out of 100, grade A | all allowed | Amazon CloudFront |
| 1528 | casinoplus.com.ph | Score 92 out of 100, grade A | all allowed | Cloudflare |
| 1796 | pewresearch.org | Score 92 out of 100, grade A | all allowed | WordPress |
| 2168 | datto.com | Score 92 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 2246 | asana.com | Score 92 out of 100, grade A | all allowed | Next.js . Netlify |
| 2500 | urbanairship.com | Score 92 out of 100, grade A | all allowed | WordPress . Fastly |
| 2858 | webflow.com | Score 92 out of 100, grade A | all allowed | Webflow . Cloudflare |
| 3179 | mountain.com | Score 92 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 3192 | egnyte.com | Score 92 out of 100, grade A | all allowed | Drupal . Akamai |
| 3543 | razorpay.com | Score 92 out of 100, grade A | all allowed | Framer . Amazon CloudFront |
| 3900 | complianz.io | Score 92 out of 100, grade A | all allowed | HubSpot CMS . Cloudflare |
| 4604 | bittorrent.com | Score 92 out of 100, grade A | all allowed | Amazon CloudFront |
| 4791 | salesmartly.com | Score 92 out of 100, grade A | all allowed | Next.js . Amazon CloudFront |
| 46 | digicert.com | Score 91 out of 100, grade A | all allowed | Adobe Experience Manager . Fastly |
| 140 | dropbox.com | Score 91 out of 100, grade A | all allowed | unidentified |
| 158 | one.one | Score 91 out of 100, grade A | all allowed | Next.js . Varnish |
| 415 | dell.com | Score 91 out of 100, grade A | all allowed | Adobe Experience Manager . Akamai |
| 590 | wix.com | Score 91 out of 100, grade A | all allowed | Wix . Fastly |
| 600 | squarespace.com | Score 91 out of 100, grade A | all allowed | Squarespace |
| 697 | bitrix24.ru | Score 91 out of 100, grade A | all allowed | unidentified |
| 887 | frontiersin.org | Score 91 out of 100, grade A | all allowed | Nuxt . Azure Front Door |
| 1508 | sondakika.com | Score 91 out of 100, grade A | all allowed | Cloudflare |
| 1556 | therapservices.net | Score 91 out of 100, grade A | all allowed | WordPress . Cloudflare |
| 1754 | rapid7.com | Score 91 out of 100, grade A | all allowed | Next.js . Cloudflare |
| 1802 | one.com | Score 91 out of 100, grade A | all allowed | Next.js . Varnish |
| 1828 | gdemoideti.ru | Score 91 out of 100, grade A | all allowed | Next.js |
| 2110 | instructure.com | Score 91 out of 100, grade A | all allowed | Drupal . Fastly |
| 2194 | revenuecat.com | Score 91 out of 100, grade A | all allowed | Sanity . Vercel |
| 2282 | useinsider.com | Score 91 out of 100, grade A | all allowed | HubSpot CMS . Cloudflare |
| 2306 | cookiebot.com | Score 91 out of 100, grade A | all allowed | WordPress . Cloudflare |
Least agent-ready
| Rank | Domain | Score | Answer-surface crawlers | Stack |
|---|---|---|---|---|
| 25 | amazon.comrefused GPTBot | Score 8 out of 100, grade F | 10 blocked | Akamai |
| 501 | amazon.co.jprefused GPTBot | Score 8 out of 100, grade F | 10 blocked | Akamai |
| 834 | amazon.com.brrefused GPTBot | Score 8 out of 100, grade F | 10 blocked | Akamai |
| 3508 | amazon.aerefused GPTBot | Score 8 out of 100, grade F | 10 blocked | Akamai |
| 4096 | amazon.com.trrefused GPTBot | Score 8 out of 100, grade F | 10 blocked | Akamai |
| 56 | tiktok.com | Score 15 out of 100, grade F | 11 blocked | unidentified |
| 317 | amazon.co.uk | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 382 | amazon.de | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 612 | amazon.fr | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 642 | amazon.ca | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 666 | amazon.in | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 707 | amazon.co.za | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 728 | amazon.it | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 734 | amazon.es | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 791 | amazon.com.au | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 1035 | amazon.com.mx | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 1187 | amazon.eg | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 3389 | amazon.se | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 3546 | amazon.nl | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 4154 | amazon.sa | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 4163 | amazon.pl | Score 18 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 264 | launchpad.netrefused GPTBot | Score 19 out of 100, grade F | 9 blocked | unidentified |
| 2908 | imageshack.us | Score 19 out of 100, grade F | 11 blocked | Varnish |
| 4820 | imageshack.com | Score 19 out of 100, grade F | 11 blocked | Varnish |
| 1279 | dw.com | Score 25 out of 100, grade F | 9 blocked | Akamai |
| 1689 | manhwaweb.comrefused GPTBot | Score 25 out of 100, grade F | 5 blocked | Cloudflare |
| 1746 | bpexch.liverefused GPTBot | Score 25 out of 100, grade F | 5 blocked | Cloudflare |
| 531 | usatoday.com | Score 27 out of 100, grade F | 11 blocked | unidentified |
| 1314 | f95zone.torefused GPTBot | Score 27 out of 100, grade F | 9 blocked | Cloudflare |
| 2094 | themoviedb.org | Score 27 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 2679 | tmdb.org | Score 27 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 3487 | stuff.co.nz | Score 28 out of 100, grade F | 8 blocked | Fastly |
| 3520 | politico.eurefused GPTBot | Score 28 out of 100, grade F | 11 blocked | WordPress . Cloudflare |
| 4102 | mainichi.jprefused GPTBot | Score 28 out of 100, grade F | 10 blocked | Amazon CloudFront |
| 4129 | nrk.norefused GPTBot | Score 28 out of 100, grade F | 10 blocked | SvelteKit |
| 1002 | theconversation.com | Score 29 out of 100, grade F | 11 blocked | Next.js . Fastly |
| 1432 | iltalehti.fi | Score 29 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 2013 | vildey.cfdrefused GPTBot | Score 29 out of 100, grade F | 5 blocked | Cloudflare |
| 2032 | blacktoon413.comrefused GPTBot | Score 29 out of 100, grade F | 5 blocked | Cloudflare |
| 453 | cnbc.comrefused GPTBot | Score 30 out of 100, grade F | 9 blocked | Akamai |
| 602 | dailymail.co.ukrefused GPTBot | Score 30 out of 100, grade F | 11 blocked | Akamai |
| 693 | dailymail.comrefused GPTBot | Score 30 out of 100, grade F | 11 blocked | Akamai |
| 2894 | faz.netrefused GPTBot | Score 30 out of 100, grade F | 10 blocked | Nuxt |
| 3984 | vaticannews.varefused GPTBot | Score 30 out of 100, grade F | 5 blocked | Cloudflare |
| 4871 | hochi.news | Score 30 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 1370 | mercadolibre.comrefused GPTBot | Score 31 out of 100, grade F | 6 blocked | Alibaba Cloud CDN |
| 1472 | ekstrabladet.dk | Score 31 out of 100, grade F | 11 blocked | Fastly |
| 2098 | ign.comrefused GPTBot | Score 31 out of 100, grade F | 10 blocked | WordPress . Fastly |
| 2714 | tds.net | Score 31 out of 100, grade F | 5 blocked | Varnish |
| 3422 | nzherald.co.nz | Score 31 out of 100, grade F | 11 blocked | Akamai |
| 188 | cnn.com | Score 32 out of 100, grade F | 10 blocked | Fastly |
| 2029 | blacktoon414.comrefused GPTBot | Score 32 out of 100, grade F | 5 blocked | Cloudflare |
| 4691 | channel4.com | Score 32 out of 100, grade F | 8 blocked | Akamai |
| 1503 | dogdrip.netrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 1773 | blacktoon410.comrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | unidentified |
| 1824 | wikihow.com | Score 33 out of 100, grade F | 9 blocked | unidentified |
| 1902 | tukrd.comrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 2021 | filmyzilla45.comrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 2028 | moviezwap.onlrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 2034 | filmyzilla40.comrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 2035 | filmyzilla43.comrefused GPTBot | Score 33 out of 100, grade F | 5 blocked | Cloudflare |
| 3214 | infojobs.net | Score 33 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 4923 | dr.dkrefused GPTBot | Score 33 out of 100, grade F | 11 blocked | Next.js |
| 286 | nature.com | Score 34 out of 100, grade F | 7 blocked | Fastly |
| 816 | nbcnews.com | Score 34 out of 100, grade F | 11 blocked | Next.js . Akamai |
| 1223 | pimpbunny.comrefused GPTBot | Score 34 out of 100, grade F | 8 blocked | Cloudflare |
| 1817 | zdnet.comrefused GPTBot | Score 34 out of 100, grade F | 10 blocked | unidentified |
| 1930 | sportfilm800.comrefused GPTBot | Score 34 out of 100, grade F | 5 blocked | Next.js . Cloudflare |
| 119 | snapchat.com | Score 35 out of 100, grade F | 5 blocked | Next.js . Google Cloud / GFE |
| 1274 | franceinfo.fr | Score 35 out of 100, grade F | 11 blocked | unidentified |
| 2916 | asahi.com | Score 35 out of 100, grade F | 11 blocked | unidentified |
| 3113 | crunchbase.com | Score 35 out of 100, grade F | 11 blocked | Angular . Cloudflare |
| 1079 | bfmtv.com | Score 36 out of 100, grade F | 11 blocked | Amazon CloudFront |
| 1774 | isaidub.gururefused GPTBot | Score 36 out of 100, grade F | 5 blocked | Cloudflare |
| 2016 | isaidub.ceorefused GPTBot | Score 36 out of 100, grade F | 5 blocked | Cloudflare |
| 2446 | rockstargames.com | Score 36 out of 100, grade F | 4 blocked | Akamai |
| 2782 | chicagotribune.com | Score 37 out of 100, grade F | 11 blocked | WordPress |
| 2930 | today.com | Score 37 out of 100, grade F | 11 blocked | Next.js . Akamai |
| 3431 | chron.com | Score 37 out of 100, grade F | 6 blocked | Fastly |
| 3523 | nydailynews.com | Score 37 out of 100, grade F | 11 blocked | WordPress |
| 3811 | aol.derefused GPTBot | Score 37 out of 100, grade F | 5 blocked | Next.js |
| 4910 | yr.no | Score 37 out of 100, grade F | 8 blocked | unidentified |
| 5049 | mercurynews.com | Score 37 out of 100, grade F | 11 blocked | WordPress |
| 154 | nytimes.comrefused GPTBot | Score 38 out of 100, grade F | 11 blocked | WordPress . Fastly |
| 207 | bbc.com | Score 38 out of 100, grade F | 9 blocked | Next.js . Fastly |
| 1349 | gst.gov.inrefused GPTBot | Score 38 out of 100, grade F | all allowed | unidentified |
| 2093 | mashable.comrefused GPTBot | Score 38 out of 100, grade F | 10 blocked | Cloudflare |
| 2191 | mirror.co.uk | Score 38 out of 100, grade F | 6 blocked | Amazon CloudFront |
| 2382 | julydns.comrefused GPTBot | Score 38 out of 100, grade F | all allowed | Alibaba Cloud CDN |
| 2552 | mparticle.com | Score 38 out of 100, grade F | 8 blocked | Next.js . Vercel |
| 2939 | thehindu.comrefused GPTBot | Score 38 out of 100, grade F | 6 blocked | Cloudflare |
| 3091 | metro.co.ukrefused GPTBot | Score 38 out of 100, grade F | 11 blocked | WordPress |
| 3263 | lifehacker.comrefused GPTBot | Score 38 out of 100, grade F | 10 blocked | Cloudflare |
| 3767 | judns.comrefused GPTBot | Score 38 out of 100, grade F | all allowed | Alibaba Cloud CDN |
| 4465 | 20minutes.fr | Score 38 out of 100, grade F | 10 blocked | Cloudflare |
| 4566 | oxfordjournals.orgrefused GPTBot | Score 38 out of 100, grade F | all allowed | unidentified |
| 567 | telegraph.co.uk | Score 39 out of 100, grade F | 10 blocked | Adobe Experience Manager . Fastly |
| 1967 | moviesda31.comrefused GPTBot | Score 39 out of 100, grade F | 5 blocked | Cloudflare |
| 2005 | moviesda33.comrefused GPTBot | Score 39 out of 100, grade F | 5 blocked | Cloudflare |
| 2036 | moviesda32.comrefused GPTBot | Score 39 out of 100, grade F | 5 blocked | Cloudflare |
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Agent readiness leaderboard." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.