Skip to main content.
home | support | download

Back to List Archive

Re: URL-fixing with callback routines for spider.pl

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Thu May 19 2005 - 18:57:19 GMT
On Thu, May 19, 2005 at 09:07:34AM -0700, koszalekopalek wrote:
> Btw, any pointers on why the server is not happy
> with use_cookies => 1, ?

I'm not sure, but I'd like to find out -- just in case something isn't
working like it should.

You can do this:

   SPIDER_DEBUG=headers ./spider.pl test.conf >/dev/null

and see the headers.  You can also use debug => 'headers', in your
config.

One thing I'm seeing is the Referer: header is the new $uri set in the
filter_content() callback.  Your server isn't looking at the Referer
header, is it?

-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu
Received on Thu May 19 11:57:19 2005