Scrapy Disallow

Scrapy Disallow tracking and usage

Websites using Scrapy Disallow by countrymarket share
21%
🇺🇸US
17%
🇨🇦CA
8.3%
🇮🇳IN
4.2%
🇬🇧GB
4.2%
🇦🇺AU
4.2%
🇻🇳VN
4.2%
🇩🇪DE
4.2%
🇧🇲BM
Websites using Scrapy Disallow by industrymarket share · top 6 industries
News & Weather
25%
Entertainment
25%
Software Development
25%
Technology & Computing
8.3%
Medical & Healthcare
8.3%
Telecommunications
8.3%
0%6.3%13%19%25%
Download report

We track 24 websites using Scrapy Disallow, along with their country, traffic, industry and platforms. There are 5 websites using Scrapy Disallow in United States.

Filter by country
🇺🇸 United States5 sites
🇨🇦 Canada4 sites
🇮🇳 India2 sites
🇬🇧 United Kingdom1 sites
🇦🇺 Australia1 sites
🇻🇳 Viet Nam1 sites
🇩🇪 Germany1 sites
🇧🇲 Bermuda1 sites
🇰🇭 Cambodia1 sites
🇧🇷 Brazil1 sites
🇸🇦 Saudi Arabia1 sites

About Scrapy Disallow

This website has found the Disallow rules for Scrapy.

Official websitescrapy.org

Websites using Scrapy Disallow

showing 25 of 24
WebsiteCountryTrafficIndustry
amazon.de🇩🇪DETop 500Logistics & Shipping
bbc.co.uk🇬🇧GBTop 500News & Weather
bbc.com🇬🇧GBTop 500News & Weather
nytimes.com🇺🇸USTop 500News & Weather
cnn.com🇺🇸USTop 500News & Weather
weather.com🇺🇸USTop 500Movies & TV
launchpad.net🇬🇧GBTop 500Mental Health
businessinsider.com🇺🇸USTop 500News & Weather
wired.com🇺🇸USTop 1KTechnology & Computing
bild.de🇩🇪DETop 1KNews & Weather
amazon.ca🇨🇦CATop 1KLogistics & Shipping
theverge.com🇺🇸USTop 1KNews & Weather
theatlantic.com🇺🇸USTop 1KPrint & Publishing
amazon.in🇮🇳INTop 1KShopping
dailymail.com🇺🇸USTop 1KPrint & Publishing
repubblica.it🇮🇹ITTop 1KNews & Weather
lefigaro.fr🇫🇷FRTop 1KNews & Weather
meta.com🇺🇸USTop 1KSoftware Development
aol.com🇺🇸USTop 1KSoftware Development
theconversation.com🇦🇺AUTop 1KEntertainment
cnet.com🇺🇸USTop 1KEntertainment
dailymail.co.uk🇬🇧GBTop 1KNews & Weather
furaffinity.net-Top 5KFine Art
gazeta.pl🇵🇱PLTop 5KMarketing & Advertising
vg.no🇳🇴NOTop 5KNews & Weather
Showing 25 of 24 websitesView detailed list →