Spiders and Bots...?

[Kherr]

What's with these spiders and bots always being logged in as registered users Trout? I want to know and I'm sure others do too...

Baidu [Spider], Google [Bot], Yahoo [Bot]

Why don't they show up in the users list? Also, what's their purpose...? I don't like the term "spider." That throws a red flag in my book...

*EDIT*

I've done a little research on these bots/spiders, and if you let them continue to index your site, they're going to rack up your bandwidth. I'd disallow them if I were you. Hope I've found out what they were on my own... >.>

[Jesse]

I think they are suppose to be there. Don't they collect and send info that helps people find your site or something? I don't know, but I'm sure this page that I'm too lazy to read through does:

http://en.wikipedia.org/wiki/Web_crawler

Whatever the case may be, I'm sure HT has it under control, being the bad arse uber geek he is and all.

[Kherr]

Alright, I was right then. Anyway, I'm just stating now I guess that I'm worried about his bandwidth. They index the site by accessing every page, that's in turn racking up his bandwidth alottance from his ISP. That needs to be curbed, quickly if he wants to save money... BUT, on the flip side, yes, it's good that they make this site easier to find. That's a very good thing. IDK what to do about them, but the Baidu one can be disabled, do we really need CHINESE traffic? No, not really... (they're a little good at hacking...) :p

[Guest]

Lol yes all the spiders are for is indexing. The only reason to ban them would be if they are taking TOO MUCH bandwidth which I dont know if they are or not. I highly doubt it though. I agree with removing the Baidu one or an even better idea would be to hide them from the whos online part. They arent users, just crawlers.
I was a crawler once. Very interesting

[Hot Trout]

The spiders are indexing all this stuff for the search engines. Until they sart using too much bandwidth I will continue to let them do their job as in doind so they should help internet users find this site more easily.