Search Engine Spiders Distorting Stats
Posted: Fri Sep 07, 2012 7:31 pm
Hey
I have brought this up a couple times in the past but wanted to make a thread because I'm not sure if this was ever resolved. Basically, I have TE on some smaller sites which receive relatively speaking a large amount of search engine traffic. So the bookmarkers slot for traffic will have huge numbers sometimes, but when I look at the ips they will be listed as Google or other spiders usually. So sometimes I'll have anywhere between 20-50% bookmarker traffic which will generate clicks, go to trades, but is not actual real surfers. So all my traffic stats get terribly distorted and nothing is even remotely correct, this I have confirmed with by simply trading with my own sites and seeing the Out hits aren't anywhere close to the in hits on the 2nd site.
Any idea what I can do to fix this? I wouldn't want to block the spiders as I would worry about my SE traffic, but I don't want trade expert tracking them either.
Also to toss out an example, I just stuck TE on a site with ~500 traffic a day, mostly search engine traffic. I plan on growing it up some, but also like just having more stats that TE offers. Right now I am getting ~280 bookmarker hits an hour, which will be roughly 6k in bookmarker/spider traffic a day. So for every 10 hits I send to trades, 9 won't actually be surfers, making this pretty useless.
I have brought this up a couple times in the past but wanted to make a thread because I'm not sure if this was ever resolved. Basically, I have TE on some smaller sites which receive relatively speaking a large amount of search engine traffic. So the bookmarkers slot for traffic will have huge numbers sometimes, but when I look at the ips they will be listed as Google or other spiders usually. So sometimes I'll have anywhere between 20-50% bookmarker traffic which will generate clicks, go to trades, but is not actual real surfers. So all my traffic stats get terribly distorted and nothing is even remotely correct, this I have confirmed with by simply trading with my own sites and seeing the Out hits aren't anywhere close to the in hits on the 2nd site.
Any idea what I can do to fix this? I wouldn't want to block the spiders as I would worry about my SE traffic, but I don't want trade expert tracking them either.
Also to toss out an example, I just stuck TE on a site with ~500 traffic a day, mostly search engine traffic. I plan on growing it up some, but also like just having more stats that TE offers. Right now I am getting ~280 bookmarker hits an hour, which will be roughly 6k in bookmarker/spider traffic a day. So for every 10 hits I send to trades, 9 won't actually be surfers, making this pretty useless.