Skip to main content.
home | support | download

Back to List Archive

Re: Disallow in Robots.txt

From: Bill Moseley <moseley(at)>
Date: Tue Jan 09 2007 - 13:34:22 GMT
On Tue, Jan 09, 2007 at 02:19:51AM -0800, James wrote:
> Brad,
> It appears that you were right, in part, after all.
> changed: $server->{agent} ||= 'swish-e spider 2.2';
> to: $server->{agent} ||= 'MyTitle';
> changed: $ua->agent( "SwishSpider" );
> To: $ua->agent( "MyTitle" );
> It took a bit of poking around, but I think I got it.  I'll begin testing on
> it to see that it actually works as expected.

Only a bit: -> documentation -> -> configuration options

But doesn't get you very much info:

    # agent

    This optional key sets the name of the spider.

But it does basically answer your question.

    "is there a way to declare what bot I am?"

Which is:

    agent   => 'MyAgent',

And, if you don't top-post the list archives will be more friendly to others.

Bill Moseley

Unsubscribe from or help with the swish-e list:

Help with Swish-e:
Received on Tue Jan 9 05:34:27 2007