Skip to main content.
home | support | download

Back to List Archive

RE: Directive to "NOT INDEX" a particular directory

From: Bill Moseley <moseley(at)>
Date: Thu Dec 04 2003 - 16:03:51 GMT
On Thu, Dec 04, 2003 at 07:37:45AM -0800, Aaron Bazar wrote:
> What about "FileRules dirname" ?

FileRules will not work when spidering with (e.g. using -S

For that you should use a test_url function in the spider config file.

See the documentation.

Bill Moseley
Received on Thu Dec 4 16:04:26 2003