FriendlyCrawler Disallow tracking and usage
Websites using FriendlyCrawler Disallow by countrymarket share
32%
🇺🇸US12%
🇩🇪DE5.1%
🇬🇧GB3.7%
🇫🇷FR2.3%
🇨🇦CA1.9%
🇳🇴NO1.8%
🇳🇱NL1.8%
🇪🇸ESWebsites using FriendlyCrawler Disallow by industrymarket share · top 6 industries
News & Weather
Print & Publishing
SaaS & Cloud
Shopping
Technology & Computing
Movies & TV
0%13%25%38%50%
Download report
We track 9,271 websites using FriendlyCrawler Disallow, along with their country, traffic, industry and platforms. There are 2,922 websites using FriendlyCrawler Disallow in United States.
Filter by country
🇺🇸 United States2,922 sites
🇩🇪 Germany1,127 sites
🇬🇧 United Kingdom474 sites
🇫🇷 France347 sites
🇨🇦 Canada215 sites
🇳🇴 Norway175 sites
🇳🇱 Netherlands168 sites
🇪🇸 Spain164 sites
🇸🇪 Sweden118 sites
🇮🇹 Italy115 sites
🇨🇭 Switzerland110 sites
🇦🇺 Australia97 sites
🇫🇮 Finland86 sites
🇵🇱 Poland77 sites
🇦🇹 Austria73 sites
🇧🇪 Belgium62 sites
🇧🇷 Brazil60 sites
🇦🇷 Argentina52 sites
🇮🇳 India41 sites
🇨🇿 Czechia31 sites
🇸🇮 Slovenia30 sites
🇩🇰 Denmark28 sites
🇯🇵 Japan24 sites
🇿🇦 South Africa21 sites
🇳🇿 New Zealand21 sites
🇰🇷 Korea, Republic of20 sites
🇬🇷 Greece19 sites
🇭🇺 Hungary19 sites
🇵🇹 Portugal17 sites
🇨🇳 China16 sites
🇮🇩 Indonesia16 sites
🇲🇽 Mexico14 sites
🇻🇳 Viet Nam13 sites
🇹🇷 Turkey12 sites
🇷🇴 Romania12 sites
🇷🇺 Russian Federation12 sites
🇮🇪 Ireland12 sites
🇻🇪 Venezuela, Bolivarian Republic of11 sites
🇨🇱 Chile10 sites
🇸🇬 Singapore9 sites
🇦🇪 United Arab Emirates7 sites
🇱🇹 Lithuania7 sites
🇱🇺 Luxembourg6 sites
🇪🇪 Estonia6 sites
🇪🇬 Egypt5 sites
🇵🇭 Philippines5 sites
🇹🇼 Taiwan, Province of China5 sites
🇵🇷 Puerto Rico5 sites
🇭🇰 Hong Kong4 sites
🇧🇩 Bangladesh4 sites
🇷🇸 Serbia4 sites
🇸🇦 Saudi Arabia4 sites
🇲🇾 Malaysia4 sites
🇰🇪 Kenya4 sites
🇲🇹 Malta4 sites
🇮🇱 Israel4 sites
🇮🇸 Iceland4 sites
🇹🇭 Thailand3 sites
🇨🇾 Cyprus3 sites
🇲🇦 Morocco3 sites
🇧🇸 Bahamas3 sites
🇵🇪 Peru3 sites
🇳🇬 Nigeria3 sites
🇧🇬 Bulgaria3 sites
🇨🇷 Costa Rica2 sites
🇦🇩 Andorra2 sites
🇸🇰 Slovakia2 sites
🇺🇦 Ukraine2 sites
🇹🇹 Trinidad and Tobago2 sites
🇬🇹 Guatemala2 sites
🇷🇪 Réunion2 sites
🇨🇲 Cameroon2 sites
🇶🇦 Qatar2 sites
🇨🇮 Côte d'Ivoire2 sites
🇮🇷 Iran, Islamic Republic of2 sites
🇱🇻 Latvia2 sites
🇮🇲 Isle of Man2 sites
🇻🇺 Vanuatu1 sites
🇻🇮 Virgin Islands, U.S.1 sites
🇲🇳 Mongolia1 sites
🇲🇬 Madagascar1 sites
🇹🇬 Togo1 sites
🇲🇵 Northern Mariana Islands1 sites
🇳🇦 Namibia1 sites
🇵🇰 Pakistan1 sites
🇬🇺 Guam1 sites
🇬🇭 Ghana1 sites
🇩🇴 Dominican Republic1 sites
🇧🇾 Belarus1 sites
🇱🇾 Libya1 sites
🇮🇴 British Indian Ocean Territory1 sites
🇺🇿 Uzbekistan1 sites
🇦🇬 Antigua and Barbuda1 sites
🇫🇯 Fiji1 sites
🇺🇾 Uruguay1 sites
🇺🇬 Uganda1 sites
🇸🇯 Svalbard and Jan Mayen1 sites
🇱🇰 Sri Lanka1 sites
🇪🇨 Ecuador1 sites
About FriendlyCrawler Disallow
Sites blocking the FriendlyCrawler crawler in robots.txt.
Categories
Websites using FriendlyCrawler Disallow
showing 25 of 9,271| Website | Country | Traffic | Industry |
|---|---|---|---|
 amazon.co.jp | 🇯🇵JP | Top 500 | Shopping |
| 🇫🇷FR | Top 500 | News & Weather | |
 amazon.co.uk | 🇬🇧GB | Top 500 | Entertainment |
 amazon.fr | 🇫🇷FR | Top 500 | Logistics & Shipping |
 amazon.de | 🇩🇪DE | Top 500 | Logistics & Shipping |
| 🇺🇸US | Top 500 | Entertainment | |
| 🇺🇸US | Top 500 | News & Weather | |
| 🇺🇸US | Top 500 | News & Weather | |
| 🇺🇸US | Top 500 | Movies & TV | |
| 🇬🇧GB | Top 500 | Print & Publishing | |
| 🇦🇺AU | Top 500 | Software Development | |
| 🇫🇷FR | Top 1K | News & Weather | |
| 🇺🇸US | Top 1K | News & Weather | |
 amazon.it | 🇮🇹IT | Top 1K | Shopping |
| 🇩🇪DE | Top 1K | News & Weather | |
 amazon.es | 🇪🇸ES | Top 1K | Technology & Computing |
 amazon.com.au | 🇦🇺AU | Top 1K | Shopping |
| 🇺🇸US | Top 1K | Technology & Computing | |
| 🇩🇪DE | Top 1K | News & Weather | |
 amazon.ca | 🇨🇦CA | Top 1K | Logistics & Shipping |
| 🇺🇸US | Top 1K | News & Weather | |
 amazon.in | 🇮🇳IN | Top 1K | Shopping |
| 🇺🇸US | Top 1K | Print & Publishing | |
| 🇮🇹IT | Top 1K | News & Weather | |
| 🇺🇸US | Top 1K | Gaming |
Showing 25 of 9,271 websitesView detailed list →
