Skip to main content.
home | support | download

Back to List Archive

Re: User-Agent: Mozilla/2.0 and Mozilla/5.0 (compatible

From: Peter Karman <peter(at)>
Date: Wed Jan 10 2007 - 15:11:37 GMT
James scribbled on 1/10/07 8:48 AM:

> Thanks, I'll check out the link.  By the way, I did take the time to
> read up on UTF-8 and user agents.  But since there is a plethora of
> information and since you guys are the experts, I am asking you
> because I figure that you will be able to speed up my learning and/or
> point me to some information that you are already aware of.  That's
> why novices seek out help from experts in forums and discussion
> groups, right?  

well, we're the "experts" at swish, but as to why Google, Yahoo or Ask do things 
in a certain way, you're tempting fate. I'm glad Thomas was able to answer the 
Google bot question; I had no idea they'd switched to a Gecko-based spider.

>> and in regard to the user agent question: I believe that one reason bots
>> identify themselves as particular user agents is because they want to receive
>> the same responses that the server would hand out to those non-bot agents.
> So, is this a real benefit to the Swish-e Spider and how would it be
> accomplished?

there's no benefit to identifying the Swish-e spider as anything other than what 
it is, unless you're trying to fool a robots.txt file or something, in which 
case, that's just plain naughty.

Peter Karman  .  .  peter(at)
Received on Wed Jan 10 07:11:37 2007