Magpie-Crawler Disallow

Magpie-Crawler Disallow tracking and usage

Websites using Magpie-Crawler Disallow by countrymarket share
23%
🇺🇸US
13%
🇬🇧GB
7.4%
🇩🇪DE
4.8%
🇪🇸ES
3.4%
🇮🇹IT
3.3%
🇫🇷FR
2%
🇨🇦CA
1.7%
🇳🇱NL
Websites using Magpie-Crawler Disallow by industrymarket share · top 6 industries
News & Weather
38%
Technology & Computing
16%
Shopping
12%
Real Estate
12%
Print & Publishing
11%
Entertainment
11%
0%10%20%30%40%
Download report

We track 2,727 websites using Magpie-Crawler Disallow, along with their country, traffic, industry and platforms. There are 627 websites using Magpie-Crawler Disallow in United States.

Filter by country
🇺🇸 United States627 sites
🇬🇧 United Kingdom352 sites
🇩🇪 Germany203 sites
🇪🇸 Spain130 sites
🇮🇹 Italy93 sites
🇫🇷 France90 sites
🇨🇦 Canada55 sites
🇳🇱 Netherlands47 sites
🇦🇺 Australia46 sites
🇵🇱 Poland45 sites
🇷🇺 Russian Federation42 sites
🇸🇪 Sweden35 sites
🇧🇷 Brazil31 sites
🇨🇭 Switzerland27 sites
🇮🇳 India27 sites
🇩🇰 Denmark26 sites
🇸🇦 Saudi Arabia21 sites
🇬🇷 Greece19 sites
🇦🇹 Austria16 sites
🇧🇪 Belgium14 sites
🇮🇩 Indonesia14 sites
🇯🇵 Japan14 sites
🇫🇮 Finland13 sites
🇵🇹 Portugal11 sites
🇹🇷 Turkey11 sites
🇭🇺 Hungary9 sites
🇳🇿 New Zealand8 sites
🇦🇷 Argentina7 sites
🇨🇳 China7 sites
🇱🇹 Lithuania7 sites
🇦🇪 United Arab Emirates6 sites
🇨🇴 Colombia6 sites
🇨🇿 Czechia6 sites
🇱🇻 Latvia5 sites
🇷🇴 Romania5 sites
🇻🇳 Viet Nam5 sites
🇳🇴 Norway5 sites
🇻🇪 Venezuela, Bolivarian Republic of5 sites
🇲🇦 Morocco4 sites
🇲🇨 Monaco4 sites
🇵🇰 Pakistan4 sites
🇿🇦 South Africa4 sites
🇵🇭 Philippines4 sites
🇪🇪 Estonia4 sites
🇮🇪 Ireland4 sites
🇸🇬 Singapore4 sites
🇲🇽 Mexico4 sites
🇹🇼 Taiwan, Province of China3 sites
🇮🇷 Iran, Islamic Republic of3 sites
🇹🇭 Thailand3 sites
🇸🇰 Slovakia3 sites
🇵🇪 Peru3 sites
🇵🇷 Puerto Rico3 sites
🇧🇩 Bangladesh2 sites
🇪🇨 Ecuador2 sites
🇭🇷 Croatia2 sites
🇳🇦 Namibia2 sites
🇹🇳 Tunisia2 sites
🇰🇷 Korea, Republic of2 sites
🇪🇬 Egypt2 sites
🇩🇴 Dominican Republic2 sites
🇧🇾 Belarus1 sites
🇱🇾 Libya1 sites
🇺🇦 Ukraine1 sites
🇭🇰 Hong Kong1 sites
🇻🇮 Virgin Islands, U.S.1 sites
🇦🇼 Aruba1 sites
🇲🇩 Moldova, Republic of1 sites
🇷🇸 Serbia1 sites
🇰🇼 Kuwait1 sites
🇧🇬 Bulgaria1 sites
🇧🇲 Bermuda1 sites
🇬🇭 Ghana1 sites
🇨🇮 Côte d'Ivoire1 sites
🇦🇿 Azerbaijan1 sites
🇨🇾 Cyprus1 sites
🇲🇹 Malta1 sites
🇸🇮 Slovenia1 sites
🇸🇳 Senegal1 sites
🇸🇯 Svalbard and Jan Mayen1 sites
🇲🇾 Malaysia1 sites
🇯🇪 Jersey1 sites
🇦🇩 Andorra1 sites
🇸🇴 Somalia1 sites

About Magpie-Crawler Disallow

This website has found the Disallow rules for Magpie-Crawler.

Categories
Robots.txtWeb Standards
Official websitewww.brandwatch.com/magpie-crawler↗

Websites using Magpie-Crawler Disallow

showing 25 of 2,727
WebsiteCountryTrafficIndustry
bbc.co.uk🇬🇧GBTop 500News & Weather
bbc.com🇬🇧GBTop 500News & Weather
nytimes.com🇺🇸USTop 500News & Weather
cnn.com🇺🇸USTop 500News & Weather
criteo.com🇫🇷FRTop 500Marketing & Advertising
weather.com🇺🇸USTop 500Movies & TV
bild.de🇩🇪DETop 1KNews & Weather
theatlantic.com🇺🇸USTop 1KPrint & Publishing
repubblica.it🇮🇹ITTop 1KNews & Weather
elpais.com🇪🇸ESTop 1KNews & Weather
aol.com🇺🇸USTop 1KSoftware Development
cnet.com🇺🇸USTop 1KEntertainment
gazeta.pl🇵🇱PLTop 5KMarketing & Advertising
buzzfeed.com🇺🇸USTop 5KTechnology & Computing
radiofrance.fr🇫🇷FRTop 5KMovies & TV
20min.ch🇨🇭CHTop 5KNews & Weather
nypost.com🇺🇸USTop 5KNews & Weather
marca.com🇪🇸ESTop 5KNews & Weather
e-recht24.de🇩🇪DETop 5KLegal Services
aol.de🇩🇪DETop 5KTechnology & Computing
mos.ru🇷🇺RUTop 5KLaw & Government
technologyreview.com🇺🇸USTop 5KPrint & Publishing
mapquest.com🇺🇸USTop 5KBusiness & Finance
kompas.com🇮🇩IDTop 5KNews & Weather
sport.es🇪🇸ESTop 5KNews & Weather
Showing 25 of 2,727 websitesView detailed list →

Alternatives

same category
HTML5 DocType43.42MsitesFacebook Redirect16.88MsitesInstagram10.95MsitesOrganization Schema6.77MsitesLinkedIn5.8MsitesEvents Page4.12MsitesMembers Page3.53MsitesGoogle AJAX Feed API3.04Msites