PDA

Zobacz pełną wersję : Poprawne blokowanie BADbotów crawlerówcraw .htaccess Joomla 3.7



Saluno
09-07-2017, 11:03
Witam.

Chciałem zablokować skutecznie roboty, crawlery zbierające linki prowadzące do strony m.in dla MOZ, SEOprofiler, Ahrefs etc.
Niestety nie znam dobrze zasad tworzenia takiego pliku.

Wiem, że z moim "nie jest wszystko OK" bo pomimo zapisania na serwerze pliku w wersji jak poniżej - dalej młócą moją www.
Ostatnio przez dwa miesiące działał, ale "coś zmieniłem" i znów zbierają dane. :(
Zapewne coś na poziomie: [NC,OR] [NC] i może nawet banał, ale jednak.

Może powinno być

RewriteCond %{HTTP_USER_AGENT} .*360Spider* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*80legs* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Abonti* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ZyBorg* [NC]
RewriteRule ^.* - [F,L]

RewriteBase /

Bardzo proszę o spojrzenie i podpowiedzenie ŁOPATOLIGICZNIE co zmienić, aby plik spełniał funkcję, którą potrzebuję.

Jakby ktoś mógł pomóc, to uprzejmie proszę o skopiowanie/wklejenie całej zawartości dokumentu, ale już bez błędu - w odpowiedzi.
Czyli łopatologiczniej się nie da.
I jakieś info co było źle i dlaczego.

Na 100% wielu się to potem przyda.

Serwer Home.pl
Działało, więc błąd jest nie w parametrach serwera, ale zapisie.


+++++POCZĄTEK_ZAWARTOŚCI_PLIKU_.HTACCES

##
# @package Joomla
# @copyright Copyright (C) 2005 - 2016 Open Source Matters. All rights reserved.
# @license GNU General Public License version 2 or later; see LICENSE.txt
##

##
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that disallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
##

## No directory listings
IndexIgnore *

## Can be commented out if causes errors, see notes above.
Options +FollowSymlinks
Options -Indexes

## Mod_rewrite in use.

RewriteEngine On
rewritecond %{http_host} ^www.matyjudo.pl [nc]
rewriterule ^(.*)$ http://matyjudo.pl/$1 [r=301,nc]

RewriteCond %{HTTP_USER_AGENT} .*360Spider* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*80legs* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Abonti* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Aboundex* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Acunetix* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ADmantX* [NC]
RewriteCond %{HTTP_USER_AGENT} .*AhrefsBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*AIBOT* [NC]
RewriteCond %{HTTP_USER_AGENT} .*AiHitBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Aipbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Alexibot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Alligator* [NC]
RewriteCond %{HTTP_USER_AGENT} .*AllSubmitter* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Anarchie* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Apexoo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ASPSeek* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Asterias* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Attach* [NC]
RewriteCond %{HTTP_USER_AGENT} .*autoemailspider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BackDoorBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BacklinkCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BackStreet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BackWeb* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Badass* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bandit* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BatchFTP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Battleztar\ Bazinga* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BBBike* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BDFetch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BetaBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bigfoot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bitacle* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Black\ Hole* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BlackWidow* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BLEXBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Blow* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BlowFish* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Boardreader* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bolt* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BotALot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Brandprotect* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Brandwatch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bubing* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Buddy* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BuiltBotTough* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BuiltWith* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Bullseye* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BunnySlippers* [NC]
RewriteCond %{HTTP_USER_AGENT} .*BuzzSumo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Calculon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CATExplorador* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CazoodleBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CCBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Cegbfeieh* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CheeseBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CherryPicker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ChinaClaw* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Chlooe* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Claritybot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Cliqzbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Cogentbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Collector* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Copier* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CopyRightCheck* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Copyscape* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Cosmos* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Craftbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CrazyWebCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Crescent* [NC]
RewriteCond %{HTTP_USER_AGENT} .*CSHttp* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Curious* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Custo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DBLBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Demon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Deusu* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Devil* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DIIbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Dirbuster* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Disco* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Discobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Discoverybot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DittoSpyder* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DomainAppender* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DomainCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DomainSigmaCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Dotbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Download\ Demon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Download\ Devil* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Download\ Wonder* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Dragonfly* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Drip* [NC]
RewriteCond %{HTTP_USER_AGENT} .*DTS\ Agent* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EasyDL* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ebingbong* [NC]
RewriteCond %{HTTP_USER_AGENT} .*eCatch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ECCP/1.0* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ecxi* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EirGrabber* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EMail\ Collector* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EMail\ Extractor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EMail\ Siphon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EMail\ Wolf* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EroCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Evil* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Exabot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Express\ WebPictures* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Extractor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ExtractorPro* [NC]
RewriteCond %{HTTP_USER_AGENT} .*EyeNetIE* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ezooms* [NC]
RewriteCond %{HTTP_USER_AGENT} .*FHscan* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Fimap* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Findxbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Firefox/7.0* [NC]
RewriteCond %{HTTP_USER_AGENT} .*FlashGet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Flunky* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Foobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Freeuploader* [NC]
RewriteCond %{HTTP_USER_AGENT} .*FrontPage* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Fyrebot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GalaxyBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Genieo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Getintent* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GetRight* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GetWeb* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Gigablast* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Gigabot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Go-Ahead-Got-It* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Gotit* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GoZilla* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Go!Zilla* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Grabber* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GrabNet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Grafula* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GrapeFX* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GrapeshotCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GridBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GSiteCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GT::WWW* [NC]
RewriteCond %{HTTP_USER_AGENT} .*GuzzleHttp* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HaosouSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Harvest* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Havij* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Heritrix* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Hloader* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HMView* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HTMLparser* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HTTP::Lite* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HTTrack* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Humanlinks* [NC]
RewriteCond %{HTTP_USER_AGENT} .*HybridBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Iblog* [NC]
RewriteCond %{HTTP_USER_AGENT} .*IDBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Id-search* [NC]
RewriteCond %{HTTP_USER_AGENT} .*IlseBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Image\ Fetch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Image\ Stripper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Image\ Sucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Indy\ Library* [NC]
RewriteCond %{HTTP_USER_AGENT} .*InfoNaviRobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*InfoTekies* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Intelliseek* [NC]
RewriteCond %{HTTP_USER_AGENT} .*InterGET* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Internet\ Ninja* [NC]
RewriteCond %{HTTP_USER_AGENT} .*InternetSeer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*internetVista\ monitor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ip-web-crawler.com* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ips-agent* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Iria* [NC]
RewriteCond %{HTTP_USER_AGENT} .*IRLbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Iskanie* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JamesBOT* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Jbrofuzz* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JennyBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JetCar* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JikeSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JOC\ Web\ Spider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Joomla* [NC]
RewriteCond %{HTTP_USER_AGENT} .*JustView* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Jyxobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Kenjin\ Spider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Keyword\ Density* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Lanshanbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Larbin* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LeechFTP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LeechGet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LexiBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Lftp* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LibWeb* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Libwhisker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Lightspeedsystems* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Likse* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Linkdex* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Linkdexbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinkextractorPro* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinkpadBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinkScan* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinksManager* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinkWalker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinqiaMetadataDownloaderBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinqiaRSSBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LinqiaScrapeBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Lipperhey* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Litemage_walker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Lmspider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LNSpiderguy* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ltx71* [NC]
RewriteCond %{HTTP_USER_AGENT} .*lwp-request* [NC]
RewriteCond %{HTTP_USER_AGENT} .*LWP::Simple* [NC]
RewriteCond %{HTTP_USER_AGENT} .*lwp-trivial* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Magnet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mag-Net* [NC]
RewriteCond %{HTTP_USER_AGENT} .*magpie-crawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mail.ru* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Majestic12* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MarkMonitor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MarkWatch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Masscan* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mass\ Downloader* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mata\ Hari* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Meanpathbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MegaIndex.ru* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Metauri* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MFC_Tear_Sample* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Microsoft\ Data\ Access* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Microsoft\ URL\ Control* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MIDown\ tool* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MIIxpc* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mister\ PiX* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MJ12bot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Mojeek* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MSFrontPage* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MSIE\ 6.0* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MSIECrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Msrabot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*MS\ Web\ Services\ Client\ Protocol* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Musobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Name\ Intelligence* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nameprotect* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Navroad* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NearSite* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Needle* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nessus* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NetAnts* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Netcraft* [NC]
RewriteCond %{HTTP_USER_AGENT} .*netEstate\ NE\ Crawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NetLyzer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NetMechanic* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NetSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nettrack* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Net\ Vampire* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NetZIP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NextGenSearchBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nibbler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NICErsPRO* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Niki-bot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nikto* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NimbleCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ninja* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nmap* [NC]
RewriteCond %{HTTP_USER_AGENT} .*NPbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Nutch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Octopus* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Offline\ Explorer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Offline\ Navigator* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Openfind* [NC]
RewriteCond %{HTTP_USER_AGENT} .*OpenLinkProfiler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Openvas* [NC]
RewriteCond %{HTTP_USER_AGENT} .*OrangeBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*OrangeSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*OutfoxBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PageAnalyzer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Page\ Analyzer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PageGrabber* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Page\ Grabber* [NC]
RewriteCond %{HTTP_USER_AGENT} .*page\ scorer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PageScorer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Panscient* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Papa\ Foto* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pavuk* [NC]
RewriteCond %{HTTP_USER_AGENT} .*pcBrowser* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PECL::HTTP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PeoplePal* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PHPCrawl* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Picscout* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Picsearch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pimonster* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pi-Monster* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pixray* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PleaseCrawl* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pockey* [NC]
RewriteCond %{HTTP_USER_AGENT} .*POE-Component-Client-HTTP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Probethenet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ProPowerBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ProWebWalker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Proximic* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Psbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Pump* [NC]
RewriteCond %{HTTP_USER_AGENT} .*PyCurl* [NC]
RewriteCond %{HTTP_USER_AGENT} .*QueryN\ Metasearch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Qwantify* [NC]
RewriteCond %{HTTP_USER_AGENT} .*RankActiveLinkBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Rankivabot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*RealDownload* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Reaper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Recorder* [NC]
RewriteCond %{HTTP_USER_AGENT} .*RedesScrapy* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ReGet* [NC]
RewriteCond %{HTTP_USER_AGENT} .*RepoMonkey* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Ripper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*RocketCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Rogerbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SalesIntelligent* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SBIder* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ScanAlert* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Scanbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Scrapy* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Screaming* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Screaming\ Frog\ SEO\ Spider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ScreenerBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Searchestate* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SearchmetricsBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Semrush* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SemrushBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SEOkicks* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SEOkicks-Robot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SEOlyticsCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Seomoz* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SEOprofiler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SEOstats* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Siphon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SISTRIX* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SISTRIX\ Crawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sitebeam* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SiteExplorer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Siteimprove* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SiteLockSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SiteSnagger* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SiteSucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Site\ Sucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sitevigil* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Slackbot-LinkExpanding* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SlySearch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SmartDownload* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Snake* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Snapbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Snoopy* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SocialRankIOBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sogou\ web\ spider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sosospider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SpaceBison* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Spammen* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SpankBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Spanner* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Spbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Spinn3r* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SputnikBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SpyFu* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sqlmap* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sqlworm* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sqworm* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Steeler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Stripper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Sucuri* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SuperBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SuperHTTP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Surfbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*SurveyBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Suzuran* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Swiftbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Szukacz* [NC]
RewriteCond %{HTTP_USER_AGENT} .*T0PHackTeam* [NC]
RewriteCond %{HTTP_USER_AGENT} .*T8Abot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*tAkeOut* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Teleport* [NC]
RewriteCond %{HTTP_USER_AGENT} .*TeleportPro* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Telesoft* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Telesphoreo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Telesphorep* [NC]
RewriteCond %{HTTP_USER_AGENT} .*The\ Intraformant* [NC]
RewriteCond %{HTTP_USER_AGENT} .*TheNomad* [NC]
RewriteCond %{HTTP_USER_AGENT} .*TightTwatBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Titan* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Toata* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Toweyabot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Trendictionbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*True_Robot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Turingos* [NC]
RewriteCond %{HTTP_USER_AGENT} .*TurnitinBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Turnitin\ Bot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Turnitin\ Robot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*TwengaBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Twice* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Typhoeus* [NC]
RewriteCond %{HTTP_USER_AGENT} .*URLy.Warning* [NC]
RewriteCond %{HTTP_USER_AGENT} .*URLy\ Warning* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Vacuum* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Vagabondo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*VB\ Project* [NC]
RewriteCond %{HTTP_USER_AGENT} .*VCI* [NC]
RewriteCond %{HTTP_USER_AGENT} .*VeriCiteCrawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*VidibleScraper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*VoidEYE* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Voil* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Voltron* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WallpapersHD* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WASALive-Bot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WBSearchBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Webalta* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebAuto* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Auto* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebBandit* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Bandit* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebCollage* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Collage* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebCopier* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Copier* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WEBDAV* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WEBDAV\ Client* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebEnhancer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Enhancer* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebFetch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Fetch* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebFuck* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Fuck* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebGo\ IS* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebImageCollector* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Image\ Collector* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebLeacher* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebmasterWorldForumBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*webmeup-crawler* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebPix* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Pix* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebReaper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Reaper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebSauger* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Sauger* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Webshag* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebsiteExtractor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Website\ Extractor* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebsiteQuester* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Website\ Quester* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Webster* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebStripper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Stripper* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebSucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Sucker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebWhacker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Web\ Whacker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WebZIP* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WeSEE* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Whack* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Whacker* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Whatweb* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Widow* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WinHTTrack* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WiseGuys\ Robot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WISENutbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Wonderbot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Woobot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Wotbox* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Wprecon* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WPScan* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WWW-Collector-E* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WWW-Mechanize* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WWW::Mechanize* [NC]
RewriteCond %{HTTP_USER_AGENT} .*WWWOFFLE* [NC]
RewriteCond %{HTTP_USER_AGENT} .*x22Mozilla* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Xaldon_WebSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Xaldon\ WebSpider* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Xenu* [NC]
RewriteCond %{HTTP_USER_AGENT} .*YoudaoBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Zade* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Zermelo* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Zeus* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Zgrab* [NC]
RewriteCond %{HTTP_USER_AGENT} .*Zitebot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ZmEu* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ZumBot* [NC]
RewriteCond %{HTTP_USER_AGENT} .*ZyBorg* [NC]
RewriteRule ^.* - [F,L]

RewriteBase /

## Begin - Rewrite rules to block out some common exploits.
# If you experience problems on your site block out the operations listed below
# This attempts to block the most common type of exploit `attempts` to Joomla!
#
# Block out any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a ********** tag in URL.
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL.
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL.
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
## End - Rewrite rules to block out some common exploits.

## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
## End - Custom redirects

##
# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root).
##

# RewriteBase /

## Begin - Joomla! core SEF Section.
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
## End - Joomla! core SEF Section.

+++++KONIEC_ZAWARTOŚCI_PLIKU_.HTACCES

adam.lachut
09-07-2017, 19:40
Może powinno być
RewriteCond %{HTTP_USER_AGENT} .*360Spider* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*80legs* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Abonti* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ZyBorg* [NC]
RewriteRule ^.* - [F,L]

Dokładnie tak powinno być.
Domyślnie warunki reguły traktowane są jako iloczyn logiczny (koniunkcja, AND): reguła zadziała jeżeli wszystkie warunki będę spełnione, czyli UserAgent musiałby jednocześnie zawierać wszystkie sygnatury.
Jeżeli dodasz OR (ornext), to warunki reguły będą traktowane jako suma logiczne (alternatywa, OR): reguła zadziała jeśli chociaż jeden warunek będzie spełniony.

Dodatkowo możesz się zastanowić czy nie dopisać warunku:

RewriteCond %{HTTP_USER_AGENT} ^$

który oznacza 'pusty' UserAgent

A.

Saluno
10-07-2017, 15:19
Wielkie dzięki. Na pewno trochę osób z Twojej pomocy skorzysta na przestrzeni lat. :up:

Jeżeli chodzi o pustego UserAgenta, znam temat, ale tu znów nie wiedziałem jak to ubrać w kod.
Czy może to być zrobione tak jak poniżej?

Podobno to najlepsze wyjście bo:
PUSTE POLE REFERENCJI I NAZWY AGENTA - CHRONI PRZED WYKLUCZENIEM UŻYTKOWNIKÓW UŻUWAJĄCYCH STARYCH ANTYWIRUSÓW.
Co sądzisz?

Jeżeli mógłbyś jeszcze z tym jednym pomóc ŁOPATOLOGICZNIE.

RewriteEngine On
rewritecond %{http_host} ^www.ubezpieczeniapzu.org (http://www.ubezpieczeniapzu.org) [nc]
rewriterule ^(.*)$ http:/ /ubezpieczenia.org/$1 [r=301,nc]

RewriteCond %{HTTP_USER_AGENT} .*360Spider* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*80legs* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Abonti* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ZyBorg* [NC]
RewriteCond %{HTTP_REFERER} ^-?$ [NC]
RewriteCond %{HTTP_USER_AGENT} ^-?$ [NC]
RewriteRule ^.* - [F,L]

RewriteBase /