My site is overcrawled with bots, as discussed here
https://www.8theme.com/topic/site-crashing-due-to-fileers-being-attacked-by-bots/#post-456271
my hosting suggest blocking filters in robots.txt file and/or .htaccess. hhey shared this general code for robots.txt
User-agent: *
Disallow: /*?*filter_*&filter_*
Disallow: /*?*filter_*&*shop_view=
Disallow: /*?*filter_*&*per_page=
Disallow: /*?*filter_*&*query_type_*
Disallow: /*?*query_type_*&*filter_*
Disallow: /*?*min_price=
Disallow: /*?*max_price=
Disallow: /*?*add-to-cart=
Disallow: /*?*add-to-wishlist=
and this one for .htaccess
# BEGIN – Block UA & QUERY requests for WooCommerce filters and add-to-cart
RewriteEngine On
# Specify UA (User Agent) – Googlebot hammer
RewriteCond %{REQUEST_METHOD} ^(GET|POST)$
RewriteCond %{HTTP_USER_AGENT} (Googlebot) [NC]
# Specify patterns (QUERY_STRING) for filters and add-to-cart
RewriteCond %{QUERY_STRING} (add-to-cart|filter_color|filter_size|filter_brand|min_price|max_price) [NC]
# Return 429 and stop
RewriteRule “^.*$” – [R=429,L]
# END – Block UA & URL QUERY requests for WooCommerce filters and add-to-cart
is this ok or do I need to adjust it for xstore?
