User-agent: * Disallow: /ads/ Disallow: /adlg/ Disallow: /images/ Disallow: /grafics/ Disallow: /tnails/ Disallow: /logos/ Disallow: /links/ Disallow: /immageimmo/ Disallow: /imagelocs/ Disallow: /imagelocannee/ Disallow: /immoimmageimmo/ Disallow: /immoimagelocs/ Disallow: /immoimagelocannee/ # blocks any URLs that end with .xls Disallow: /*.xls$ Disallow: /customerror*.* Disallow: /*error*.* Disallow: /*print*.* Disallow: /*geolocalise*.* User-agent: SemrushBot User-agent: MJ12bot User-agent: WebCopier+v2.5 User-agent: WebCopier+v3.0 User-agent: WebCopier User-agent: pompos Disallow: /