Skip to main content.
home | support | download

Back to List Archive


From: David VanHook <dvanhook(at)>
Date: Thu Jul 24 2003 - 19:20:56 GMT
Sorry, can't find this in the docs or discussion list anyplace:

I need to set up robots.txt to disallow everybody except SWISH from
spidering my server.  I'm using called through S-prog.  What is
the user-agent name for the SWISH spider?

Here's what I thought robots.txt would look like -- but this makes SWISH
abort as soon as it starts spidering:


User-agent: *
Disallow: /

What's the appropriate "name" for when run via S-prog?

Thanks --

David VanHook
Received on Thu Jul 24 19:21:12 2003