Page 1 of 1

Block China - Baidu.com with TE3?

Posted: Sun Jan 03, 2016 9:16 am
by TA2
Hello.

I am wondering of an effective way to possibly use TE3 to block China and also it's search engine traffic from Baidu (CN) http://www.baidu.com.

From the information I have looked up about this. I have seen that this code should be used in your robots.txt file .. But It may not work:

Code: Select all

User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /
Is there any other way to block incoming traffic and direct to a 403 page using TE3? If so can you please outline the procedure?

I would like to just block China traffic all together, even from trades, and its traffic from m.baidu.com and baidu.com SE.

Thank you for any information that may help. Here are some informational links about the subject:
https://www.webmasterworld.com/search_e ... 475767.htm
http://webmasters.stackexchange.com/que ... du-spiders

Re: Block China - Baidu.com with TE3?

Posted: Mon Jan 04, 2016 8:16 am
by Rock
Hello, TA2.

Placing the following code in your robots.txt file should do the job:

Code: Select all

User-agent: Baiduspider
Disallow: /

User-agent: Baiduspider
Disallow: /*?*

User-agent: Baiduspider+
Disallow: /

User-agent: Baiduspider+
Disallow: /*?*

User-agent: baiduspider
Disallow: /

User-agent: baiduspider
Disallow: /*?*
Another option would be to use .htaccess. Add user agents' IP ranges to the following rule:

Code: Select all

order allow,deny 
allow from all
# Block access to Baiduspider 
deny from  
Best regards and let me know if it works,
Rock

Re: Block China - Baidu.com with TE3?

Posted: Mon Jan 04, 2016 11:05 am
by TA2
Thank you, Rock! I have added to my robots files and I'll check for results after they accumulate. I may also add the code in the htaccess file if need be. As I said - I want to block all China traffic. Not just Baidu's SE. I also want to block Vietnam and send them both to my 403 page as directed in the TE3 app to redirect blocked traffic.

It would be a nice addition to TE3 to have the option to block Country IP Blocks or Domain names.

I have set up specific 403 pages for all traffic being blocked. Not entirely sure it is all working correctly though, yet.

Thank you again.

Re: Block China - Baidu.com with TE3?

Posted: Mon Jan 04, 2016 3:23 pm
by Rock
Thank you for the feature request - we'll look into it and try to come up with something in the future :)

In the meantime either the robots.txt or the .htaccess solution should block unwanted user agents.

Rock