Skip to main content.
home | support | download

Back to List Archive

Re: Callback Functions For Indexing

From: andy rosbrook <andy_rosbrook(at)not-real.hotmail.com>
Date: Fri Jan 27 2006 - 17:46:21 GMT
Well i just want to know after each URL in spider.config weather the 
spidering was a success or a failure. I know i could just check for a 
complete index.swish-e but this doesnt allow me to capture any error 
messages.

Ill take a look at grabbin STDERR though, thanks.


>From: Bill Moseley <moseley@hank.org>
>Reply-To: moseley@hank.org
>To: Multiple recipients of list <swish-e@sunsite3.berkeley.edu>
>Subject: [SWISH-E] Re: Callback Functions For Indexing
>Date: Fri, 27 Jan 2006 07:21:45 -0800 (PST)
>
>On Fri, Jan 27, 2006 at 06:30:56AM -0800, andy rosbrook wrote:
> > Is there anyway to use a callback function to catch errors when 
>spidering
> > websites with spider.pl?
> >
> > I am currently spidering only a few small sites at a time and need a way 
>of
> > knowing weather the spider successfully indexed the site or not, is this
> > possible? if so is there a way of grabbing the error message into perl?
>
>Not sure what you mean.  You want to know if any file returned a
>non-200 status?  Or if swish-e indexed any words?
>
>
>IPC::Open3 will capture stderr and stdout.
>
>--
>Bill Moseley
>moseley@hank.org
>
>Unsubscribe from or help with the swish-e list:
>    http://swish-e.org/Discussion/
>
>Help with Swish-e:
>    http://swish-e.org/current/docs
>    swish-e@sunsite.berkeley.edu
>

_________________________________________________________________
Are you using the latest version of MSN Messenger? Download MSN Messenger 
7.5 today! http://messenger.msn.co.uk
Received on Fri Jan 27 09:46:24 2006