James scribbled on 1/10/07 8:48 AM:
> Thanks, I'll check out the link. By the way, I did take the time to
> read up on UTF-8 and user agents. But since there is a plethora of
> information and since you guys are the experts, I am asking you
> because I figure that you will be able to speed up my learning and/or
> point me to some information that you are already aware of. That's
> why novices seek out help from experts in forums and discussion
> groups, right?
well, we're the "experts" at swish, but as to why Google, Yahoo or Ask do things
in a certain way, you're tempting fate. I'm glad Thomas was able to answer the
Google bot question; I had no idea they'd switched to a Gecko-based spider.
>> and in regard to the user agent question: I believe that one reason bots
>> identify themselves as particular user agents is because they want to receive
>> the same responses that the server would hand out to those non-bot agents.
> So, is this a real benefit to the Swish-e Spider and how would it be
there's no benefit to identifying the Swish-e spider as anything other than what
it is, unless you're trying to fool a robots.txt file or something, in which
case, that's just plain naughty.
Peter Karman . http://peknet.com/ . peter(at)not-real.peknet.com
Received on Wed Jan 10 07:11:37 2007