FirecrawlAgent Disallow tracking and usage
Websites using FirecrawlAgent Disallow by countrymarket share
28%
🇺🇸US15%
🇩🇪DE3.9%
🇬🇧GB3.2%
🇫🇷FR2.9%
🇨🇦CA2.8%
🇳🇴NO2.4%
🇪🇸ES2.1%
🇳🇱NLWebsites using FirecrawlAgent Disallow by industrymarket share · top 6 industries
News & Weather
Print & Publishing
Technology & Computing
Shopping
Non-profit & Charity
Education
0%13%25%38%50%
Download report
We track 5,271 websites using FirecrawlAgent Disallow, along with their country, traffic, industry and platforms. There are 1,494 websites using FirecrawlAgent Disallow in United States.
Filter by country
🇺🇸 United States1,494 sites
🇩🇪 Germany803 sites
🇬🇧 United Kingdom204 sites
🇫🇷 France170 sites
🇨🇦 Canada151 sites
🇳🇴 Norway150 sites
🇪🇸 Spain128 sites
🇳🇱 Netherlands109 sites
🇧🇷 Brazil99 sites
🇮🇹 Italy62 sites
🇨🇭 Switzerland60 sites
🇩🇰 Denmark57 sites
🇦🇺 Australia53 sites
🇮🇳 India51 sites
🇦🇹 Austria42 sites
🇫🇮 Finland41 sites
🇵🇱 Poland36 sites
🇲🇨 Monaco36 sites
🇧🇪 Belgium35 sites
🇨🇿 Czechia19 sites
🇸🇪 Sweden18 sites
🇵🇹 Portugal17 sites
🇳🇿 New Zealand12 sites
🇹🇷 Turkey12 sites
🇲🇽 Mexico12 sites
🇮🇩 Indonesia11 sites
🇿🇦 South Africa11 sites
🇯🇵 Japan11 sites
🇦🇷 Argentina10 sites
🇨🇳 China9 sites
🇬🇷 Greece9 sites
🇷🇺 Russian Federation9 sites
🇱🇹 Lithuania8 sites
🇭🇰 Hong Kong8 sites
🇦🇪 United Arab Emirates8 sites
🇨🇱 Chile8 sites
🇦🇽 Åland Islands8 sites
🇭🇺 Hungary7 sites
🇸🇮 Slovenia6 sites
🇺🇦 Ukraine6 sites
🇸🇬 Singapore5 sites
🇨🇴 Colombia5 sites
🇻🇳 Viet Nam5 sites
🇧🇬 Bulgaria5 sites
🇲🇾 Malaysia5 sites
🇪🇪 Estonia4 sites
🇸🇦 Saudi Arabia4 sites
🇪🇬 Egypt4 sites
🇷🇴 Romania4 sites
🇵🇭 Philippines4 sites
🇨🇷 Costa Rica4 sites
🇵🇰 Pakistan4 sites
🇹🇼 Taiwan, Province of China4 sites
🇮🇷 Iran, Islamic Republic of3 sites
🇰🇷 Korea, Republic of3 sites
🇮🇸 Iceland3 sites
🇲🇦 Morocco3 sites
🇮🇪 Ireland3 sites
🇶🇦 Qatar3 sites
🇳🇬 Nigeria3 sites
🇱🇺 Luxembourg3 sites
🇩🇴 Dominican Republic3 sites
🇸🇰 Slovakia3 sites
🇭🇷 Croatia3 sites
🇵🇪 Peru3 sites
🇦🇩 Andorra2 sites
🇪🇨 Ecuador2 sites
🇵🇷 Puerto Rico2 sites
🇭🇳 Honduras2 sites
🇧🇭 Bahrain2 sites
🇨🇮 Côte d'Ivoire2 sites
🇨🇾 Cyprus2 sites
🇲🇹 Malta2 sites
🇹🇭 Thailand2 sites
🇨🇲 Cameroon2 sites
🇦🇿 Azerbaijan2 sites
🇵🇾 Paraguay2 sites
🇮🇲 Isle of Man2 sites
🇭🇹 Haiti1 sites
🇰🇭 Cambodia1 sites
🇺🇬 Uganda1 sites
🇬🇭 Ghana1 sites
🇰🇪 Kenya1 sites
🇫🇯 Fiji1 sites
🇮🇱 Israel1 sites
🇯🇴 Jordan1 sites
🇲🇰 North Macedonia1 sites
🇦🇬 Antigua and Barbuda1 sites
🇷🇼 Rwanda1 sites
🇹🇳 Tunisia1 sites
🇸🇩 Sudan1 sites
🇽🇰 Kosovo1 sites
🇺🇿 Uzbekistan1 sites
🇮🇴 British Indian Ocean Territory1 sites
🇧🇾 Belarus1 sites
🇧🇿 Belize1 sites
🇧🇴 Bolivia, Plurinational State of1 sites
🇬🇹 Guatemala1 sites
🇳🇮 Nicaragua1 sites
🇵🇦 Panama1 sites
🇸🇻 El Salvador1 sites
🇺🇾 Uruguay1 sites
🇻🇪 Venezuela, Bolivarian Republic of1 sites
🇬🇪 Georgia1 sites
🇱🇻 Latvia1 sites
🇲🇸 Montserrat1 sites
🇦🇴 Angola1 sites
🇧🇩 Bangladesh1 sites
🇯🇲 Jamaica1 sites
About FirecrawlAgent Disallow
Sites blocking the FirecrawlAgent crawler in robots.txt.
Categories
Websites using FirecrawlAgent Disallow
showing 25 of 5,271| Website | Country | Traffic | Industry |
|---|---|---|---|
| 🇫🇷FR | Top 500 | News & Weather | |
| 🇬🇧GB | Top 500 | News & Weather | |
| 🇬🇧GB | Top 500 | News & Weather | |
| 🇺🇸US | Top 500 | News & Weather | |
| 🇬🇧GB | Top 500 | Mental Health | |
| 🇫🇷FR | Top 1K | News & Weather | |
| 🇬🇧GB | Top 1K | Software Development | |
| 🇺🇸US | Top 1K | Print & Publishing | |
| 🇩🇪DE | Top 1K | News & Weather | |
| 🇨🇳CN | Top 1K | Technology & Computing | |
| 🇺🇸US | Top 1K | Technology & Computing | |
| 🇺🇸US | Top 1K | Technology & Computing | |
| 🇩🇪DE | Top 1K | News & Weather | |
| 🇺🇸US | Top 1K | News & Weather | |
| 🇫🇷FR | Top 1K | News & Weather | |
| 🇦🇺AU | Top 1K | Entertainment | |
| 🇺🇸US | Top 1K | Entertainment | |
| 🇺🇸US | Top 5K | News & Weather | |
| - | Top 5K | Print & Publishing | |
| 🇺🇸US | Top 5K | News & Weather | |
| - | Top 5K | Entertainment | |
| 🇺🇸US | Top 5K | Print & Publishing | |
| 🇩🇪DE | Top 5K | News & Weather | |
| 🇳🇱NL | Top 5K | News & Weather | |
| 🇺🇸US | Top 5K | Music |
Showing 25 of 5,271 websitesView detailed list →
