User-agent: Scrapy Disallow: /energyheritage/ Disallow: /EnergyHeritage/