Skip to main content.
home | support | download

Back to List Archive

RE: Server or documet limit on spider.pl

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Wed Jan 07 2004 - 18:24:44 GMT
On Wed, Jan 07, 2004 at 10:01:12AM -0800, Lee Thompson wrote:
> I saw that happen when the spider ran into a page requiring a username
> and password.

It shouldn't, though.  It should prompt for a username and password, and
if none is provided after some number of seconds it will continue
skipping that file.

I get a mail every day from a cron job with such a message.


-- 
Bill Moseley
moseley@hank.org
Received on Wed Jan 7 18:24:53 2004