Robots.txt ile Botları Engelleme

K

kukumavnet

#1
Sitenize ve SEO‘nuza zarar verebilecek bazı kötü botları, rakiplerin sitenizi analiz etmesini önleyecek bazı analiz botlarını engellemek için Robots.txt listesi hazırladık;

User-agent: asterias
Disallow: /
User-agent: bdbrandprotect
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: BotRightHere
Disallow: /
User-agent: BPImageWalker
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: CazoodleBot
Disallow: /
User-agent: Cegbfeieh
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: ConveraCrawler
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: discobot
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: Download Ninja
Disallow: /
User-agent: Fasterfox
Disallow: /
User-agent: Fetch
Disallow: /
User-agent: findlinks
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: grub
Disallow: /
User-agent: heritrix
Disallow: /
User-agent: hloader
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: MLBot
Disallow: /
User-agent: moget
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: NaverBot
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: SEOENGBot
Disallow: /
User-agent: ShopWiki
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: Templeton
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: TightTwatBot
Disallow: /
User-agent: Titan
Disallow: /
User-agent: TurnitinBot
Disallow: /
User-agent: TwengaBot
Disallow: /
User-agent: Twiceler
Disallow: /
User-agent: Updownerbot
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: VCI
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebZIP
Disallow: /
User-agent: wget
Disallow: /
User-agent: woriobot
Disallow: /
User-agent: Xenu
Disallow: /
User-agent: YACYBIT
Disallow: /
User-agent: Yeti
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Zealbot
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: ZyBorg
Disallow: /
User-agent: ranksignals
Disallow: /

Rakiplerin sitenizi analiz etmesini engellemek ve web sitesinin kopyalanmasını önlemek için hazırladığımız robots.txt tam liste için aşağıdaki kaynağı inceleyebilirsiniz.
 
Bize Katılın!

Bu sayfada şu an yalnızca sınırlı sayıda yorum görüyorsunuz. Aşağıdaki butona tıklayarak konunun tam haline erişebilirsiniz. Konuya Git