Robots.txt ile Botları Engelleme

Sitenize ve SEO‘nuza zarar verebilecek bazı kötü botları, rakiplerin sitenizi analiz etmesini önleyecek bazı analiz botlarını engellemek için Robots.txt listesi hazırladık;

User-agent: asterias
Disallow: /
User-agent: bdbrandprotect
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: BotRightHere
Disallow: /
User-agent: BPImageWalker
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: CazoodleBot
Disallow: /
User-agent: Cegbfeieh
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: ConveraCrawler
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: discobot
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: Download Ninja
Disallow: /
User-agent: Fasterfox
Disallow: /
User-agent: Fetch
Disallow: /
User-agent: findlinks
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: grub
Disallow: /
User-agent: heritrix
Disallow: /
User-agent: hloader
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: MLBot
Disallow: /
User-agent: moget
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: NaverBot
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: SEOENGBot
Disallow: /
User-agent: ShopWiki
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: Templeton
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: TightTwatBot
Disallow: /
User-agent: Titan
Disallow: /
User-agent: TurnitinBot
Disallow: /
User-agent: TwengaBot
Disallow: /
User-agent: Twiceler
Disallow: /
User-agent: Updownerbot
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: VCI
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebZIP
Disallow: /
User-agent: wget
Disallow: /
User-agent: woriobot
Disallow: /
User-agent: Xenu
Disallow: /
User-agent: YACYBIT
Disallow: /
User-agent: Yeti
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Zealbot
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: ZyBorg
Disallow: /
User-agent: ranksignals
Disallow: /

Rakiplerin sitenizi analiz etmesini engellemek ve web sitesinin kopyalanmasını önlemek için hazırladığımız robots.txt tam liste için aşağıdaki kaynağı inceleyebilirsiniz.