Our support will answer all your general questions here.
Moderator: Rock
TA2
Posts: 46 Joined: Sat Mar 15, 2014 5:14 pm
Location: Web
Post
by TA2 » Sun Jan 03, 2016 9:16 am
Hello.
I am wondering of an effective way to possibly use TE3 to block China and also it's search engine traffic from Baidu (CN)
http://www.baidu.com .
From the information I have looked up about this. I have seen that this code should be used in your robots.txt file .. But It may not work:
Code: Select all
User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /
Is there any other way to block incoming traffic and direct to a 403 page using TE3? If so can you please outline the procedure?
I would like to just block China traffic all together, even from trades, and its traffic from m.baidu.com and baidu.com SE.
Thank you for any information that may help. Here are some informational links about the subject:
https://www.webmasterworld.com/search_e ... 475767.htm
http://webmasters.stackexchange.com/que ... du-spiders
Rock
Posts: 226 Joined: Tue Dec 16, 2014 6:11 pm
Post
by Rock » Mon Jan 04, 2016 8:16 am
Hello, TA2.
Placing the following code in your robots.txt file should do the job:
Code: Select all
User-agent: Baiduspider
Disallow: /
User-agent: Baiduspider
Disallow: /*?*
User-agent: Baiduspider+
Disallow: /
User-agent: Baiduspider+
Disallow: /*?*
User-agent: baiduspider
Disallow: /
User-agent: baiduspider
Disallow: /*?*
Another option would be to use .htaccess. Add user agents' IP ranges to the following rule:
Code: Select all
order allow,deny
allow from all
# Block access to Baiduspider
deny from
Best regards and let me know if it works,
Rock
I am a moderator on this forum and I handle part of TE support. ICQ: 651089602
TA2
Posts: 46 Joined: Sat Mar 15, 2014 5:14 pm
Location: Web
Post
by TA2 » Mon Jan 04, 2016 11:05 am
Thank you, Rock! I have added to my robots files and I'll check for results after they accumulate. I may also add the code in the htaccess file if need be. As I said - I want to block all China traffic. Not just Baidu's SE. I also want to block Vietnam and send them both to my 403 page as directed in the TE3 app to redirect blocked traffic.
It would be a nice addition to TE3 to have the option to block Country IP Blocks or Domain names.
I have set up specific 403 pages for all traffic being blocked. Not entirely sure it is all working correctly though, yet.
Thank you again.
Rock
Posts: 226 Joined: Tue Dec 16, 2014 6:11 pm
Post
by Rock » Mon Jan 04, 2016 3:23 pm
Thank you for the feature request - we'll look into it and try to come up with something in the future
In the meantime either the robots.txt or the .htaccess solution should block unwanted user agents.
Rock
I am a moderator on this forum and I handle part of TE support. ICQ: 651089602