Skip to main content.
home | support | download

Back to List Archive

Re: Any way to restrict URLs with http method since you can't use FileRules?

From: Ron Klatchko <ron(at)>
Date: Thu Apr 29 1999 - 21:06:26 GMT
My original design assumed you would control that with robots.txt.  You can
use the useragent name of swishspider.


At 01:49 PM 4/29/99 -0700, wrote:
>Any way to restrict URLs when using http method since you can't use
>FileRules Directive?
>I'd like to be able to stop indexing of certian areas. My guess is that
>I will have to modify
>the swishspider perl program. Has anyone else done this already.
>-John Leth-Nissen
>Gulfstream Aerospace Corp.
          Ron Klatchko - Manager, Advanced Technology Group           
           UCSF Library and Center for Knowledge Management           
Received on Thu Apr 29 14:03:01 1999